A model refuses to answer. The job throws. Somewhere downstream a service reads the empty response and has to decide: is this video safe, or not?
That decision gets made either way. The only question is whether you made it on purpose.
Most moderation write-ups talk about the model as if it were the system. It is the least interesting part. The model is a function you call; it has good days. The system is everything around it, and the system is where the mistakes live. A classifier that returns "unsafe" with high confidence is easy to handle. A provider that times out, a file that is corrupt, a model that declines the request entirely, these are the cases that decide whether your platform ships harmful content or blocks a creator's perfectly good upload for no reason.
So the first thing I did when we built moderation for a creator platform was stop treating absence of an answer as an answer.
Three ways to get nothing back
Pull apart what "the model didn't say it was safe" can actually mean, and you get at least three unrelated events wearing the same costume.
The first is a real finding: the model looked at the content and judged it against policy. That is a verdict. It is evidence. You can act on it.
The second is a refusal. The model, or the guardrails wrapped around it, declined to process the request. This happens, and it tells you almost nothing about the content. A refusal is a statement about the model's willingness to answer, not about what the file contains. Treating it as "probably bad" means punishing creators for the provider's caution. Treating it as "probably fine" means waving through exactly the material you were most worried about.
The third is an outage. The provider is down, the network dropped, the job died halfway. Nothing was inspected at all.
These three look identical if you only check whether a safety label came back. They are completely different if you ask a better question: did the safety check actually run? That single distinction is the spine of the whole design.
One contract, and it never lies about what ran
Every item that goes through the pipeline comes out with the same shape. Not "a label if we have one." The same keys, every time. A decision, approve or reject. A reason, drawn from a small fixed set: it passed, it broke policy, it failed the technical spec, or something went wrong in processing. And, crucially, a field for each check that says whether it was evaluated at all.
That last field is the one people forget to build, and it is the one that matters. When the safety stage does not run, the result does not quietly omit it and let a downstream reader assume the best. It says, in as many words, this check did not happen. A rejection for a corrupt file and a rejection for a genuine policy violation are both rejections, but they carry different reasons and different "evaluated" flags, and anyone reading the record can tell them apart forever.
This is the difference between a system that holds an opinion and a system that keeps a record. I wanted a record. If a creator disputes a block, I can tell them precisely why: your file broke policy, or your file could not be read, or our provider refused and we held your upload until we could look properly. Those are three different conversations, and a system that collapses them into one is a system that will eventually accuse someone of something it never checked.
Fail closed, but do not record a finding you did not make
Here is the rule that sounds contradictory until you hold both halves at once. Content that cannot be inspected does not get published. And content that cannot be inspected is not marked as a violation.
Fail closed is not optional on a platform where the content is the product. If I cannot clear a file, it does not go out. Uninspected media is not innocent; it is unknown, and unknown stays behind the gate. That protects the audience.
But the gate closing is not the same as the model speaking. When I hold a video because the provider refused, the record says the processing failed, and it says the safety check was never evaluated. It does not say "violation." I get the safe outcome, the content stays down, without manufacturing evidence I do not have. The protection and the honesty are separate concerns, and I keep them separate. A platform that logs every outage as a policy breach will, within a quarter, have a moderation history that is mostly fiction, and it will make its next decisions on top of that fiction.
The Moderation Lab
An interactive admission line: an upload passes a probe and a spec gate before any model call, and the final stamp records a decision, a reason, and whether each check actually ran.
The Moderation Lab
An interactive admission line: an upload passes a probe and a spec gate before any model call, and the final stamp records a decision, a reason, and whether each check actually ran.
An interactive admission line: an upload passes a probe and a spec gate before any model call, and the final stamp records a decision, a reason, and whether each check actually ran.
Interactive figure: The Moderation Lab. An interactive admission line: an upload passes a probe and a spec gate before any model call, and the final stamp records a decision, a reason, and whether each check actually ran.
Set the provider to refuse and watch the upload get held with a processing reason, not a safety verdict: the gate closes without the record claiming a finding.
Fig. 1The Moderation Lab. Set the provider to refuse and watch the upload get held with a processing reason, not a safety verdict: the gate closes without the record claiming a finding. Open the full exhibit
Cheap before expensive, retry before terminal
Two more rules fall out of taking the failure cases seriously, and both happen to save money.
Cheap checks come first. Before I spend a paid model call on a file, I read its technical facts and hold them against a spec: duration, resolution, the things that are true of the bytes regardless of content. A file that fails the spec is rejected right there, and I never paid for a classification I did not need. Most of what you reject, you can reject for free. The expensive judgement is reserved for files that have earned it by being plausible in the first place.
And a transient failure is not a verdict either. When the provider times out, the job does not resolve to "reject" on the spot. It goes to a retry path and gets another attempt, because a timeout is a statement about the network, not the content. Only after retries are exhausted does the system reach a terminal state, and even then the terminal state is an honest processing failure that held the file, not a fabricated finding against it. Giving up is a decision too, and I wanted it to take more than one bad second to get there.
None of this is about the model being clever. It is about refusing to let silence speak for the content. A refusal is the model declining to testify. An outage is the courtroom losing power. Neither is a guilty verdict, and a system that can't tell the difference isn't moderating. It's guessing, and writing the guesses down as fact.
Exhibit 06 · Circo
Result contract
decision
Waiting
reason
Waiting
safety.evaluated
Waiting
confidence
Waiting
// waiting for the result
An upload waits at the intake. Choose what it contains and how the model provider behaves.
What happens, step by step
1
A cheap probe reads the file's technical facts: duration, resolution, frame rate, audio.
2
The facts pass the spec gate.
3
The multimodal model classifies the video: nothing of concern, with a confidence.
4
Dead-letter retry · succeedsskipped for this input
The job is retried through the dead-letter queue and the second attempt returns: nothing of concern.
5
One result, same keys every time: APPROVED, PASSED. Every check ran and passed.