What automation should never decide for you

Most software you buy promises to save time. A lot of it does. The trouble starts when a tool that was meant to save you a few minutes quietly starts making choices you would have wanted to make yourself. The time saved is real. The judgement lost is also real, and it is harder to notice until something goes wrong.

This article is about where that line sits. Not whether to automate at all, but which decisions you should keep a hand on even after the rest of the work is handed off. The short version: automate the task, keep the judgement. The rest of this explains what that means in practice.

The difference between a task and a judgement

A task is a step that goes the same way every time. Send the receipt. Tag the enquiry. Move the record to the next stage. Copy the figure from one place to another. These are the parts of your week that are tiring precisely because they are repetitive, and a computer does repetitive things faster and more consistently than any person.

A judgement is a step where the right answer depends on context the system does not fully hold. Whether to waive a late fee for a long-standing customer. Whether an unusual order is a mistake or a genuine large purchase. Whether a complaint needs an apology or a refund. These look like single decisions, but each one draws on things that rarely sit in your data: tone, history, intent, the mood of the relationship.

The useful rule is that automation should carry out judgements, not form them. It can prepare, sort, remind and draft. The moment of deciding, when the outcome affects a person or money in a way that is hard to reverse, is where you stay in the room.

We have written separately about how to tell whether a task should be automated, which is the question one step before this one. This article assumes you have decided something can be automated and asks what the machine should still not be allowed to conclude on its own.

What automation is genuinely good at

Be generous here, because the answer is "quite a lot". Automation is good at anything with a clear rule and a clear input. If a new lead comes in, send an acknowledgement within a minute. If an invoice is thirty days overdue, add it to a review list. If a form is missing a phone number, flag it. None of these decisions is really a decision. They are rules you would apply yourself every time, so letting a tool apply them is a gain with no cost.

It is also good at preparation. Drafting a reply you will read before it sends. Summarising a long thread so you start informed. Pulling the three figures that actually matter to a choice, which is a habit worth building on its own and one we covered in numbers that change a decision. In each case the tool does the gathering and a person does the concluding.

The decisions that need a person

Some decisions should never land on a customer or a colleague without a human having seen them first. A few recurring ones:

• Anything that ends a relationship. Closing an account, cancelling a service, rejecting an application.
• Anything that moves money the wrong way is hard to undo. Refunds above a threshold, price changes, credit decisions.
• Anything that reads intent. Deciding a customer is being abusive, or that a message is spam, or that an account is fraudulent.
• Anything legal or safety-related. Eligibility, compliance status, anything where being wrong is not just inconvenient but harmful.

The common thread is cost of error. Where a mistake is cheap and reversible, let the tool run. Where a mistake is expensive or cannot be taken back, keep a person between the system and the outcome.

Why trusting the tool too much is its own risk

There is a specific trap worth naming. When a system usually gets things right, people start to accept its output without checking, even when the evidence in front of them says otherwise. Researchers call this automation bias, defined as the tendency for humans to favour suggestions from automated systems and to ignore contradictory information, even when that information is correct [1].

This matters because the better your automation is, the stronger the pull to stop checking it. A review step that everyone rubber-stamps is not really a review. If you keep a person in the loop, give them a reason and the time to actually look, not just a button to click. The goal is a judgement, not a signature.

The line the law already draws

You are not the first to work out where this boundary sits. Data protection law in the UK and the EU has a rule aimed squarely at it. Under the GDPR, a person has the right not to be subject to a decision based solely on automated processing where that decision produces legal effects concerning them or similarly significantly affects them [2]. The accompanying guidance gives concrete examples of what counts, naming the automatic refusal of an online credit application and e-recruiting practices without any human intervention [3].

You do not need to be a lawyer to take the lesson. The decisions the law singles out are the same ones common sense flags: the ones that meaningfully affect a person. Where a choice changes someone's standing, their money, or their access to something, a human should be involved in making it, not merely informed afterwards. Building that in is cheaper than retrofitting it once a decision has gone wrong.

A test you can apply this week

For any step you are thinking of automating, ask three questions:

• If this goes wrong, how hard is it to undo?
• Does the right answer depend on something the system cannot see?
• Would I be comfortable explaining this decision to the person it affected, knowing no one looked at it?

If undoing it is hard, if context matters, or if you would not be comfortable, keep a person in the decision. Automate everything that leads up to it. Automate the follow-through. Leave the moment of deciding to a human.

How we think about it at 360REV

We build on the same principle. The system does the gathering, sorting, drafting and reminding, and it keeps the record of who decided what and when, so a person always sits at the point where a decision affects a customer. That division is the same one we argue for throughout the blog, including in what AI should and should not do in your business.

Automation is not the opposite of judgement. Used well, it clears away the repetitive work so you have more attention left for the decisions that actually need you. The skill is knowing which ones those are, and refusing to give them away just because a tool offered to take them.