"A human reviews all AI output" is an easy line to put in a policy. It is much harder to do well, because nobody tells the human what reviewing actually means.
Why review goes wrong
Checking AI output is harder than checking a coworker's draft, for one specific reason: it reads well. The grammar is clean, the tone is confident, and the structure looks finished. The usual signs that something is off — awkward wording, gaps, hedging — are not there. So the reviewer skims, it looks fine, and it goes out.
The fix is not "read more carefully." It is knowing what to look for and checking those things on purpose.
Decide how much checking the work needs
Not every output needs the same effort. Sort it by what happens if it is wrong.
| Stakes | Examples | Level of check |
|---|---|---|
| Low | Internal brainstorm, rough outline, meeting agenda | Read it once. Fix what is obviously off. |
| Medium | Routine customer email, internal summary, social post | Full check of facts, names, numbers, and tone |
| High | Anything about a person's job, money, health, or legal rights; anything sent to a regulator, a court, or a client as a deliverable | Full check, every source opened, and a second reviewer |
Write this down for your team. Otherwise everyone sets their own level, and that level tends to slide as people get comfortable with the tool.
The five things to check
- Facts and figures. Every number, date, name, title, and quote. AI tools can produce specifics that look sourced and are not. If you cannot trace a figure to a source you trust, cut it or find the real one.
- Sources and citations. If the output cites a law, a study, a policy, or a case, open it. Confirm it exists and says what the output claims. This is the check that is easiest to skip and most embarrassing to miss.
- What is missing. AI answers the question you asked. It will not necessarily mention the exception, the deadline, or the requirement you did not know to ask about. Ask yourself what someone who really knows this subject would add.
- Your own information. Names, account details, prices, and commitments should match your records — and nothing confidential from another document should have found its way into this one.
- Tone and promises. Read it as the person receiving it. Does it promise something you cannot deliver? Apologize for something you are not responsible for? Sound like your organization at all?
Checking habits that hold up
- Check against the source, not your memory. Open the policy, the record, or the contract.
- Compare numbers side by side. Swapped digits slip past silent reading.
- Give every specific a second look. Proper names, dollar amounts, dates, and deadlines.
- Ask the tool where it got something, then check that yourself. A source it names still has to be opened.
- Be honest about time. If the review of a two-page document took thirty seconds, it was not a review.
- Plan for tired reviewers. People get worn out. Put high-stakes checks earlier in the day and do not stack a long run of them back to back.
Who should review
The reviewer has to know the subject well enough to catch a wrong answer. That sounds obvious, but it is an easy gap to miss: the person using the tool is often the person who wanted help with the subject in the first place.
For high-stakes work, the reviewer should be someone other than the person who generated it, and they should know they are accountable for what goes out. Put their name on it. And when nobody on staff knows the subject well enough to check the output, that is a sign the task needs an expert, not a tool.
When the check keeps finding problems
If reviewers are rewriting most of what the tool produces, stop and look at that honestly. Sometimes the fix is better instructions: give the tool the source material, the audience, and the format you want, instead of a one-line request. Sometimes the fix is a smaller job, such as a first outline instead of a finished letter. And sometimes the right answer is that this task is not a good fit for AI in your organization right now. Taking a tool out of a step is a legitimate decision, not a failure. Review time is real work, and it belongs in the estimate for the job.
Keep a light record
For medium and high-stakes work, a short note is enough: what was generated, who checked it, and what they changed. It shows the review happened. Over time, the "what they changed" column tells you where the tool is weak for your kind of work. If you need more than that, see how to document AI use so it holds up later.
Train the check, not just the tool
Training that only covers how to get output from a tool leaves out half the job. Give people practice judging it.
One exercise works well: hand staff a piece of AI output with a few planted errors — a wrong date, a citation that does not exist, a promise your organization does not make — and have them find them. Catching real mistakes teaches people what to look for in a way a checklist on its own does not.
Then come back to it. As people get comfortable, review gets lighter. An occasional spot-check by a supervisor keeps the standard from quietly slipping, and it is a chance to tell someone they caught something that mattered.
If you are still deciding what to teach nontechnical staff first, start with the four things that prevent incidents — checking output is one of them.
Questions people ask
Do I need to check everything AI writes?
Check everything, but not at the same depth. A low-stakes internal draft needs a read-through. Anything about a person's job, money, health, or legal rights, or anything sent outside as a deliverable, needs a full check against sources and a second reviewer.
How do I fact-check AI output?
Trace every number, name, date, quote, and citation back to a source you trust, and open that source instead of relying on memory. If a figure or reference cannot be traced, cut it or replace it with one you have verified.
Can AI make up sources?
Yes. AI tools can produce citations, quotes, and references that look real and are not, or that exist but do not say what the output claims. Open every cited source before you rely on it.
Who should review AI-generated work?
Someone who knows the subject well enough to spot a wrong answer. For high-stakes work, that should be a different person from the one who generated it, and their name should go on the record of the review.
What does human in the loop mean?
It means a person checks and approves AI output before it is used. It only works when the reviewer knows what to check, knows the subject well enough to judge it, and has enough time to do it properly.