All insightsResearch 01

Where a person should stay in the loop

Published
Reading time
8 minutes

A method for classifying automated decisions by cost of error and reversibility, with a worked example table showing where review belongs.

Every automation makes decisions, even the simple ones. It decides which rep gets a lead, whether a ticket is urgent, whether an invoice reminder goes out today. The question for any of them is how much a person should be involved.

"Keep a human in the loop" is easy to say and hard to use. Taken literally it means approving everything, which wears people out. Taken loosely it means nobody checks anything until a customer complains. This note gives a method for deciding where review belongs.

This is a framework note. It is not an empirical study, and the example table uses made-up classifications to show the method. Your own ratings will differ because your business, customers and tolerance for error differ.

The two questions

For each decision an automated system would make, ask two things.

1. What does it cost if the decision is wrong? Rate it Low, Medium or High, relative to your business.

Rating What it looks like
Low A person fixes it in a few minutes. No customer or outside party notices.
Medium Fixing it takes real time or a small amount of money. A customer might notice.
High Money leaves the business, a customer or regulator is affected, or trust is damaged.

Set the dollar lines yourself. A $200 mistake is trivial to one company and serious to another. Decide in advance what Low, Medium and High mean in dollars and hours for you, and write it down.

2. Can it be undone? Rate it Easy, Hard or None.

Rating What it looks like
Easy Change a field, reverse an entry. Nobody outside the company saw the wrong version.
Hard It can be corrected, but only with effort, an apology or a second transaction.
None It cannot be taken back: a message was sent, money was paid, a record was deleted with no backup.

The two questions are independent. A decision can be cheap and permanent (a text sent to the wrong person) or expensive and reversible (a wrong price on a draft quote nobody has seen).

Four modes of oversight

The ratings map onto four modes. Each describes how much a person is involved.

Mode Name What happens
1 Automatic The system acts and logs what it did. Nobody reviews individual actions.
2 Automatic with review The system acts. A person reviews a sample, or reviews everything after the fact in a daily summary.
3 Approve first The system prepares the action. A person approves, edits or rejects it before it happens.
4 Person decides The system gathers the facts and recommends. A person makes the decision and takes the action.

The grid

Combine the two ratings to find a starting mode.

Easy to undo Hard to undo Cannot undo
Low cost of error 1 Automatic 2 Automatic with review 2 Automatic with review
Medium cost of error 2 Automatic with review 3 Approve first 3 Approve first
High cost of error 3 Approve first 4 Person decides 4 Person decides

Read it as a starting point. The grid gives a default and the next section adjusts it.

Two rules that override the grid

Some categories get a minimum mode no matter what the grid says. We apply this one ourselves:

  • Money, payroll, sensitive information and messages to customers get at least Mode 3. An approval step comes before the system acts.

Money and customer contact are where a quiet error does the most damage. The grid cannot see that, so the override sits above it.

One refinement is worth considering. A reminder with fixed, pre-approved wording is a customer message, but approving every send wastes time. The approval can attach to the template: a person signs off the wording once, and sends are logged and sampled. Anything that changes the wording, such as a drafted reply to a specific question, still needs approval each time.

Adjustments

After the grid and the override, adjust for three things.

Quality of the input. If the data feeding a decision is messy (free-text notes, scanned documents, fields people fill in inconsistently), move one mode toward more review until you have evidence it behaves.

Volume. Approval queues have a failure mode of their own: people approve everything without looking. If 200 items a day need approval and each takes 30 seconds, that is 100 minutes a day, and attention will fall off. Split the queue. Let the system act alone below a threshold (an amount, a customer tier) and send only the larger or stranger cases to a person. Review where it matters.

Evidence. A new system starts with more oversight than it will end with. Earn a lower mode with measured results.

A worked example

The classifications below are made up for illustration. They are not from a client.

Decision Cost of error Reversibility Grid result Override Final mode
Tag the lead source in the CRM Low Easy 1 None 1 Automatic
Move a deal to the next stage when a proposal is sent Low Easy 1 None 1 Automatic
Sort a support ticket by topic and urgency Medium Easy 2 None 2 Automatic with review
Merge duplicate contact records (no backup) Medium None 3 None 3 Approve first
Send an appointment reminder from a fixed template Low None 2 Customer message 3 Approve first (template level)
Reply to a customer's question about an order Medium None 3 Customer message 3 Approve first
Send a quote drafted from job notes Medium Hard 3 Money, customer message 3 Approve first
Send an overdue invoice reminder Medium None 3 Money, customer message 3 Approve first
Issue a refund High Hard 4 Money 4 Person decides
Apply a late fee to an account High Hard 4 Money 4 Person decides
Prepare hours for payroll High Hard 4 Payroll 4 Person decides

Count the modes: two decisions are fully automatic, one is automatic with review, five need approval first, and three are made by a person. Mode 4 still saves time. The system pulls the account history, the policy and the amount, so the person decides in a minute instead of ten.

Notice where the grid and the override disagree. The appointment reminder scores a 2 on the grid and ends at a 3 because it is a customer message.

Measure the loop

A control you do not measure becomes a ritual. Track three numbers for every decision type in Mode 2 or 3:

Measure How to calculate What it tells you
Override rate Items a person changed or rejected, divided by items reviewed How often the system is wrong in ways a person catches
Review time Minutes per item, times items per day What the oversight costs
Escaped errors Mistakes found after the action, with their cost What review is missing

Here is a made-up illustration of the arithmetic. In a month, a reviewer checks 120 drafted replies and edits 6. The override rate is 6 / 120 = 5%. If each review takes 30 seconds and there are 40 a day, review time is 40 x 30 = 1,200 seconds, or 20 minutes a day. A 5% override rate with minor edits is a different thing from 5% where each edit fixes a wrong amount.

Log the reason as well as the count. "Changed tone" is a wording problem. "Wrong amount" is a rule problem that may justify moving the decision to a higher mode.

Graduating a decision to less oversight

Move a decision down a mode only on evidence, and set the rule before launch. A template for the rule:

  1. Measure for a set period, for example four weeks of live runs, chosen by the owner in advance.
  2. Set the threshold in advance. For example, an override rate below a figure the owner picks, and no escaped error above a stated cost.
  3. Move one step at a time. From 3 to 2, never from 3 to 1.
  4. Keep sampling after the move. If the sample shows the rate rising, move back up.
  5. Never graduate the overridden categories below Mode 3 without a deliberate decision by the owner, written down.

Moving up needs no ceremony. After an escaped error, move the decision up a mode first and investigate second.

Using it on your own workflows

List every decision the automation would make, including the small ones. Rate each, read the mode from the grid, apply the overrides and adjust. Then write the result next to the workflow with a named person responsible for each Mode 3 or 4 decision. A review that belongs to nobody does not happen. The output is a short table that anyone in the business can read and challenge.