Human in the loop is a design decision, not a disclaimer
Adding 'reviewed by a human' to a footer changes nothing. Here is what it takes to keep a person genuinely in charge of an AI-assisted decision.
By the Maple Crew team · · 4 min read
Most AI products claim to keep a human in the loop. In many of them the loop is a button labelled Approve under a result the user has no practical way to question. That is a disclaimer. Real human oversight has to be built into the product, and it shows up in specific design choices.
Separate the recommendation from the decision
The system produces a recommendation. A person records a decision. These are two different objects, stored separately, and the second one carries a name and a reason. In Maple Insight the output of an assessment is always labelled as a recommendation for review. The decision screen is a different step, completed by an authorised person, with a mandatory justification.
Show the evidence, then the conclusion
A reviewer can only disagree with something they can inspect. Source data (the bank statements, the credit bureau report) and assessed conclusions (scores, labels) should be visibly different things on the screen, and each conclusion should link back to the data that supports it. If a reviewer has to take the result on faith, they are not reviewing.
Say when you do not know
Missing information is not a zero, and it is not a clean record. A system that quietly fills gaps produces confident results on thin data. The honest output is 'insufficient data', with the list of what is missing and the score withheld until it arrives.
Let people override, and remember that they did
Controls will sometimes block a legitimate case. An analyst should be able to switch a blocking control off, but only with a written reason, and the failed control should remain visible as an alert in the result. Overrides that leave a trace are a feature. Overrides that vanish are a risk.
Refuse the shortcuts
- No automatic approvals, however good the score looks.
- No figures the system cannot justify, such as a probability of default from an uncalibrated model.
- No protected characteristics among the inputs. Age and sex are never scored.
None of these choices is technically difficult. They are product decisions, and they need to be made at the start, because they shape the data model and every screen that follows.