When Checking AI Work Takes Longer Than Doing It Yourself, Narrow the Job

Your team has had AI tools for a while now. Drafts, summaries, extracted fields and reconciliations come back in seconds.

Then they go to whoever knows the work best, because somebody has to check them. That person's afternoon goes into the checking, and nobody logs the minutes.

A few months in, the most experienced person in the room says what experienced people say about any tool that costs them time: it's quicker to do it myself.

An experiment that timed the check, and the courts that had to decide how much of a machine-sorted pile a person must read, point the same way.

The time a check takes is set first by how the job was cut. A better prompt or model, on its own, changes how often the reviewer finds a mistake, not what they have to redo to look for one. A reviewer trusts an AI output by redoing whatever part of the work could be wrong.

Ask the AI for a judgment (summarize this contract, prepare this entry, answer this question) and the reviewer makes the judgment again. Ask it for a field and make it say where the field came from, and the reviewer checks a match.

So narrow the job until every output points at its source. Send people only what fails the checks, plus the judgment calls the job still holds. Start once those checks have been tested on that kind of document, and keep sampling what passed.

Stop any workflow that costs more minutes, from the run to the receiver, than the job took by hand, unless it lets fewer errors through.

Two limits. The evidence on review effort comes from one timed experiment with lay reviewers, from litigation document review (which counted documents read, not minutes), and from what the people doing the checking report. Read it as direction, not as minutes timed in your industry.

And where a rule makes a person answer for the output, they answer for it whatever the design. What the design changes is how much of the source work they have to redo.

Nobody Counts the Checking, So the Saving Is a Feeling

A lawyer asked r/biglaw in April 2026 the question every reviewer gets to sooner or later: "Are you seeing meaningful productivity gains, or does the time spent reviewing and correcting AI output largely cancel them out?"

The same post answered it for its author: the gains "feel minimal in practice, because I spend significant time reviewing outputs and correcting mistakes".

One study looked past what people say to what the records show.

Anders Humlum and Emilie Vestergaard surveyed 25,000 workers at 7,000 Danish workplaces, in eleven occupations that include accountants, financial advisors, legal professionals and office clerks. Then they matched the answers to the national registers of the same people's hours and earnings (their 2025 working paper).

People said they saved a little time. The registers found no change in hours or pay: whatever effect there was, it was too small to tell apart from zero. And the tools made new work, part of it checking what the tools produce.

The 2.8% is self-reported, and the authors treat it as an upper bound. The registers can't show where the time went, only that the saving people reported didn't show up in hours or pay.

The checking sits in the work a company your size runs on. Among the AI-related new tasks people described, reviewing and correcting AI output was 24% for office clerks, 20% for financial advisors, 14% for accountants and 9% for legal professionals.

It also lands on people who never chose the tool. In one r/Lawyertalk thread about clients who bring their lawyers ChatGPT drafts, the replies were blunter than any study.

Those minutes belong in the count. A workflow that saves its user ten minutes and costs the person who receives the output fifteen has lost five, whatever the user tells you in the hallway.

When we build an agent, the checking counts in the hours it's measured by, the receiver's too.

The Check Costs Whatever the Reviewer Has to Redo

What the redo costs depends on what the AI handed back. A field copied from a document is checked by opening the document at that field, and a total by holding it against the ledger.

A citation is checked by opening the case and reading the passage. A memo's conclusion is checked by doing the research again, and by then the check costs about what the work did.

A summary is the row teams expect to be cheap. Checking one without the contract is like grading a book report on a book you haven't read, so the reviewer reads the book.

The Stanford researchers who tested the leading AI legal research tools in 2024 describe the last row from the inside. Their definition of a hallucination includes the misgrounded answer: a statement the cited source doesn't support.

Catching one, they write, "requires users to click through to cited references, read and understand the relevant sources, assess their authority, and compare them to the propositions the model seeks to support". Longer answers, they add, take "substantially more time to check, verify, and validate".

A 2026 experiment from CSIRO, Australia's national science agency, still a preprint, timed what happens when you change what the reviewer is handed (Irons and colleagues). In it, 162 people in the US judged answers an AI had pulled out of science news articles, about 30% of them made wrong on purpose.

One group checked each answer against the whole article, one against the three passages a search ranked most relevant, and one against an explanation written by another AI model.

The pointed passages were faster than the whole article and caught at least as many of the wrong answers. The AI-written explanation was the fastest of the three and caught under half of the wrong answers.

On answers that were a single fact, the three groups were about equally accurate. On answers that summarized or listed, the pointed passages lost their edge: reviewers holding them got 77% right, against 85% for those with the whole article.

The authors call that split exploratory, and they read it this way: a few short passages can leave out what a summary rests on. A single field sits in the line it came from, while a summary draws on the whole document. That's the case for narrowing the job before anyone checks it.

The job you hand the AI decides the job you hand the reviewer.

Here's what that does to a job that looks like a clean win.

Say your customs coordinator prepares an entry from a commercial invoice, a packing list and the purchase order, and the broker looks it over before filing: about 30 minutes by hand, both of them counted. (The minutes are an illustration; put your own in.)

An AI tool drafts the whole entry in a minute. The coordinator can't tell which line came from where, so they re-read all three documents against the draft: 20 minutes. Two lines need fixing: 4 minutes.

The broker's look before filing turns into a re-check of every classification, because last month's draft got one wrong: 8 minutes. That's 33 minutes against 30, and the team still believes it saved time, because the draft took a minute.

Narrow the Job Until Every Output Points at Its Source

Now cut the same entry differently. The AI reads one document family, commercial invoices, as each one lands in the entry queue, and returns fields instead of an entry: value, quantity and country of origin for each line, each with its source line on the invoice.

A script then checks what needs no judgment. The lines add up to the invoice total, the quantities match the packing list, the unit prices match the purchase order. A line that fails goes to the coordinator, who opens it at the pointer.

Classification, the part of the job that is a judgment, stays with the coordinator. Each line shows its part number beside the code your company used for it last time, so a repeat part is a match to confirm and only a new one takes real thought.

A script then fills the entry from the confirmed fields. The broker still looks over every entry before it's filed, now with the pointers and the confirmed codes in front of them.

And every week a sample of the filed entries goes back to a person. A match only proves a field equals its source; it can't show the field came from the right invoice, or that no line was skipped.

Count it again on the same assumptions, for an entry with no new part numbers. Each new one adds the minutes classification always took.

The narrow cut comes in at 15 minutes against 30. At an illustrative 400 entries a month, that's about 100 hours a month back, before the cost of building and keeping the checks.

A week of timing tells you whether the minutes hold. The error rate takes longer to show, and showing it is the sample's job.

A fair objection: this takes the AI out of the one part of the job that needed thinking. It does, on purpose.

Reading an invoice into fields, line by line, is the minute of work the model does well, and it's the part a person can check by looking. The judgment stays with someone whose work can be checked too.

Work that is judgment from end to end, a memo or a contract summary, can't be cut into fields. There, the check shrinks only once someone has read a subset in full and the tool held up on it. The stop rule decides the rest.

None of this earns a light check on day one. The checks need testing on documents they haven't seen, one document family at a time, before anyone stops reading the passed pile.

It's the cut we'd make before building any agent: one document family, outputs that point back to their source, and checks that run before anyone reads a line. If one of your AI workflows is the 33-minute version of a 30-minute job, that's the one to cut first.

Sample What the Checks Let Through

"Or are you saying you review what's given to you and then spot check a sample/given rate?" a lawyer asked in the same r/biglaw thread.

Litigators settled that question more than a decade ago, in document review, where the pile runs to hundreds of thousands of documents and nobody can read it all twice.

In the TREC 2009 Legal Track, review teams worked through 836,165 documents and five production requests. Maura Grossman and Gordon Cormack compared the teams that used technology-assisted review, where a system ranks the documents and people read what it puts forward, with the estimate for people reading everything (their 2011 paper).

People read 1.9% of the collection and, on average, found more of the relevant documents: 76.7% against an estimated 59.3%. The authors call that recall gap too uncertain for a firm conclusion. Precision more than doubled, and that difference is overwhelmingly significant.

The courts followed, and what they approved is the part to copy. In 2012, in what appears to be the first opinion approving computer-assisted review, the judge in Da Silva Moore v. Publicis Groupe wrote that it "is not perfect, the Federal Rules of Civil Procedure do not require perfection."

The protocol he approved still made the producing party look at the pile the system set aside: a random sample of 2,399 documents from what it had excluded as irrelevant, before the final production.

Three years later the same judge called it "black letter law" that courts will permit it, and held that it shouldn't be held to a higher standard than manual review.

In 2018 a federal court in Illinois wrote the sample down as a protocol. In re Broiler Chicken Antitrust Litigation validates a production with 3,000 documents reviewed blind by a subject matter expert.

Of those, 500 come from what was marked responsive, 500 from what people marked not responsive, and 2,000 from what the system set aside without anyone reading it. If the set-aside piles still hold a substantial number of responsive documents, the review goes on and the validation runs again.

The order applies a version of the same test to a review done entirely by people.

Put your own workflow in that frame. The lines your checks flag are the pile a person reads against the source. The lines that passed are the set-aside pile: nobody goes back to the documents for them, and that's where your errors hide, because nothing marked them.

The frame differs in one place, and not in your favor. A document review's misses are documents never produced, and the courts accept some; your passed pile gets filed or sent, so its errors ship.

A pilot that came back clean tells you the checks worked on the documents you tried. The sample of the passed pile tells you whether they still work on this month's. It never stops, because the documents change: a new supplier, a new invoice layout, a new kind of contract.

Banking regulators wrote the same idea into a rule outside litigation.

Since October 1, 2025, mortgage lenders that use automated valuation models to value a borrower's home for a mortgage decision must hold them to quality control standards that include random sample testing and reviews, with the controls left to each lender's size, complexity and risk (the 2024 interagency rule).

The opposite failure is a set of checks that flags nearly everything. An exception queue that sends most items to a person is a full review with an extra step in front of it, and a sign the job is still cut too wide.

You Stay Responsible for the Output Even When the Source Check Gets Shorter

A lawyer on r/Ask_Lawyers put the rule in two sentences: "Lawyers are responsible for the contents of pleadings they sign and submit to court. If you're not 100% convinced it's legit work, don't sign your name to it."

That's accountability, and no design removes it. Most of these rules let the firm decide how much of the source work that person redoes. A few name a check of their own.

The American Bar Association's ethics opinion on generative AI (advisory; each state's own rules are the ones that bind) says the right amount of verification depends on the tool and the task.

Its example: a lawyer who tested a tool's contract summaries on a smaller subset, read those contracts and found the summaries accurate, would not necessarily have to review the entire set of documents.

Then the sentence that settles who answers: "Regardless of the level of review the lawyer selects, the lawyer is fully responsible for the work on behalf of the client."

Responsibility stays with the lawyer. How far they go back into the source documents is their choice, and testing the tool can shorten it. Reading the outputs is another matter: the same opinion still has lawyers review them for accuracy.

A few checks are fixed by rule. A paper filed in a US federal civil case carries the attorney's certification that, after an inquiry reasonable under the circumstances, its legal contentions are warranted by existing law or by a nonfrivolous argument for changing it (Rule 11).

When filings in Mata v. Avianca cited opinions ChatGPT had made up, the court spelled out that "existing rules impose a gatekeeping role on attorneys to ensure the accuracy of their filings."

Two years later a federal court in Arizona went through a brief citation by citation. The brief cited authority 19 times, and only 5 to 7 of those citations pointed to cases that existed and supported what they were cited for.

The court found well over the majority fabricated, misleading or unsupported, with deficiencies consistent with AI hallucinations. Counsel never said AI was used.

The firm's own process had a supervising attorney who "spot-checks the legal authorities and citations." The court found that process wasn't followed: "Failures occurred at every level." It revoked the lawyer's permission to appear in the case, struck the brief, and gave the client 45 days to find a new lawyer or go on alone.

That's the receiver's rework at its most expensive: the client paid for a spot check that existed on paper and never ran.

Outside the courts the pattern holds. In US customs the importer of record must use reasonable care to enter, classify and value its goods (19 U.S.C. 1484).

Customs and Border Protection's own checklist asks whether a responsible and knowledgeable person in the company reviewed the documentation, and says the checklist has no legal or binding effect (its reasonable care guide). In our reading, that fixes who answers and leaves the depth of the review to the importer.

The valuation-model rule from the last section is the rare one that names the check itself.

So where a rule puts your name on the output, the design's job is to cut the source work you redo before signing: the pointers, the flagged items and the checks already run arrive with the output. Your name stays on it either way, and in a court filing every citation still gets checked.

Better Models Will Shorten the Check, Not Tell You Where to Look

The strongest objection to all of this is that it's a lot of design for a problem the next model will shrink. The r/biglaw post that opened this piece ends on that hope: "has experience shown you that the tool is reliable enough that you've stopped checking as carefully?"

The argument goes: models keep getting better, purpose-built tools already beat general chatbots, and a more accurate tool should mean a shorter check.

They will shorten it. They still won't tell you which answer is wrong. Count only the answers where each tool committed to a direct answer, and the share that hallucinated looks like this:

Part of the lower headline rate was the tools answering less. Ask Practical Law AI gave an incomplete answer 63% of the time; when it did answer directly, it hallucinated about as often as GPT-4, 46% against 47%.

In a job cut wide, a wrong answer that looks like a right one costs the same to find as before: you still read all of them.

The format that makes a check feel easiest also lowers the catch. In the CSIRO experiment, reviewers handed the AI-written explanation accepted 86.5% of the answers when 70% were correct.

Research on automation bias found the pattern long before chatbots: with imperfect decision aids, people make errors of omission and of commission, experts as well as novices, and neither training nor instructions prevents it (Parasuraman and Manzey, 2010).

And a rarer error needs a bigger clean sample to show it's rare. If 100 passed items show no errors, the true rate could still be close to 3%, at the usual 95% confidence.

Showing it's under 1% takes about 300 clean items, and under 0.1% about 3,000 (the statistics behind that). A better model doesn't retire the sample. Showing it's as good as it looks takes a bigger clean sample than the old one needed.

Time One Week of the Workflow, Then Narrow It or Stop It

When checking AI work takes longer than doing it yourself, the job was cut too wide. On its own, a better prompt or model shrinks the exceptions and leaves the redo where it was.

Pick the AI workflow your team runs most and give it one week.

1. 01

   Time all of it

   The run, the checking, the fixes, and the minutes of whoever receives the output, against the minutes the job took by hand, with the errors that got through counted on both sides.

2. 02

   Narrow what loses

   Cut it to one document family and outputs that point at their source, with checks that run before anyone reads a line.

3. 03

   Stop what still loses

   If the narrowed version still takes longer and catches no more errors than the work done by hand, stop it and give the hours back to the people who were checking.

If the week shows the AI side winning, keep the sample running. The documents will change, and so will the model under the tool.

If your busiest workflow comes out on the wrong side, time the next one the same way before you add any tool.