When should a business not use AI?

Reject AI when it fails any of six decision gates

A business should not use artificial intelligence when rules suffice, representative evidence is unavailable, errors cannot be contained, verification erases the value, data rights are unresolved, or no workable fallback exists. Artificial intelligence, or AI, means software that infers an output from patterns in data instead of following only instructions written in advance.

This is a rejection test, not a scorecard. A strong answer to one gate does not compensate for a failed answer elsewhere. An accurate system can still be unsuitable if nobody can correct what it decides, and a useful one can turn unlawful the moment the business discovers it never had permission to use the underlying data.

The test prevents an attractive demonstration from becoming the default answer to an unexamined problem. Government procurement guidance makes the same distinction: select the technology that fits the need because generative AI, which produces new text, images, audio, or code from learned patterns, has limitations and can be expensive to operate. ¹

Run the six gates in order

Each gate asks for evidence that can be inspected during a buying or project approval meeting. A promise to investigate after deployment is a failed gate.

A rejection does not have to end the project. It changes the proposed mechanism. The answer might become deterministic automation, a narrower decision aid, a searchable knowledge system, or a process redesign with no AI component.

Gate one: use rules when the decision can be specified

A rules engine is software that applies explicit conditions such as thresholds, required fields, and permitted transitions. Prefer it when the correct output can be described in advance and exceptions can be enumerated.

Document routing, duplicate detection, format conversion, eligibility thresholds, and fixed approval chains often fit this pattern. The EU AI Act itself distinguishes narrow procedural tasks such as structuring data, classifying documents, and detecting duplicate applications from uses that warrant AI-specific risk treatment. ²

The good answer identifies a residual judgment that rules cannot express, then shows why learning from examples addresses it. “The inputs are unstructured” is insufficient. Unstructured inputs can often be converted into fields before ordinary rules handle the decision.

Take an illustrative case: an insurance operations lead is told a claims model is needed because incoming claim forms arrive as scanned PDFs with no fixed layout. Looking closer, the actual judgment, whether the policy covers the claimed loss, is a fixed lookup against the policy wording once the form is read. The unstructured problem is solved by a form parser, not by a model that learns to approve claims.

The alternative costs engineering time to define conditions and maintain exceptions. AI shifts that cost rather than removing it. The business must instead collect examples, evaluate model behavior, monitor drift, and investigate outputs that cannot be explained by a single rule. Generative AI, predictive AI, and automation solve different problems helps separate these mechanisms before a supplier is selected.

Reject AI at this gate when its only contribution is replacing visible instructions with learned behavior.

Gate two: require evidence from the intended population

Representative evidence means test cases reflect the people, documents, events, and conditions on which the system will operate. Volume alone does not establish representation.

Ask who appears in the evaluation set, who is missing, how recent the cases are, and whether the difficult exceptions occur at their real operating frequency. The EU AI Act requires training, validation, and testing data for high-risk systems to be relevant and sufficiently representative for the intended purpose. ²

The Gender Shades audit shows why this distinction changes a purchase decision. Two facial-analysis benchmarks contained 79.6 percent and 86.2 percent lighter-skinned subjects, and evaluated commercial systems showed substantial performance disparities across demographic groups. ³

The good answer supplies results divided by the groups and conditions that matter to the business. The bad answer supplies one average, a generic benchmark, or a vendor demonstration selected from convenient examples.

If representative cases are unavailable, use AI only to collect evidence without making consequential decisions. A shadow trial, where the system produces outputs that do not affect real cases, can reveal whether an evaluation set can be built. Reject deployment if the missing evidence concerns the people most exposed to error.

Gate three: reject errors whose consequences cannot be bounded

Containment means limiting how far an incorrect output can travel before detection and correction. The relevant unit is not the model response. It is the full chain of actions that response can start.

A mistaken product description can be edited. A mistaken fraud label can freeze access, generate an investigation, influence later reviewers, and enter records copied across systems. Accuracy does not reveal that difference.

For each output, name the reversible boundary. Identify who can stop the action, how the affected person can challenge it, which downstream records must be corrected, and whether the original state can be restored. What happens when the AI gets one wrong? provides the failure-path analysis this gate requires.

The Post Office Horizon scandal was not an AI deployment, but it is a direct warning about treating machine output as reliable evidence. A UK minister’s statement to Parliament described blameless people losing livelihoods, marriages, reputations, mental health, and in some cases their lives after flawed system output was relied upon against them. ⁴

The alternative is to keep the consequential decision outside the automated path. Let the system retrieve records, draft a recommendation, or flag a case, while an accountable process establishes the facts independently. Reject AI if an error becomes evidence for its own correctness or cannot be corrected across every downstream system.

Gate four: count verification as operating work

Verification is the work required to establish that an output is acceptable before anyone relies on it. Reading an answer is not verification. The reviewer needs access to the source material, a standard for checking it, enough time, and authority to reject it.

Human review also creates automation bias, the tendency to accept a machine recommendation too readily. The EU AI Act requires overseers of high-risk systems to be prepared for this specific failure mode. ²

Model the workload before approving the project. Assume, for illustration, 1,000 outputs each month, four minutes to check every output, and an additional twelve minutes to rework 10 percent of them. The modelled estimate is 5,200 minutes, or 86 hours and 40 minutes of monthly review and rework. This estimate has not been measured in production, and it excludes reviewer training, escalation, and quality audits.

Picture the person who actually carries that workload: a billing supervisor handed the same 1,000 cases to sign off before the books close. Reading each one against the invoice and the purchase order, at four minutes apiece, is not a formality squeezed into a spare hour. It is a recurring block of that supervisor's month, and skipping it does not remove the underlying risk. It only removes the record that anyone checked.

Compare that total with the work the system removes. If verification requires reconstructing the answer from the original evidence, generation has contributed little. How much of my team's time will checking the AI take? turns this gate into a workload calculation.

The good answer budgets named reviewers and tests whether they catch planted failures. The bad answer says a person will approve each output without explaining how approval establishes correctness.

Gate five: resolve data rights before uploading data

Data rights are the legal and contractual permissions governing collection, processing, disclosure, reuse, retention, and deletion. Having access to a file does not establish all of those permissions.

For personal data, the General Data Protection Regulation limits processing to specified and legitimate purposes. Data collected for one purpose cannot automatically be reused for an incompatible AI purpose. ⁵ Processing also needs an applicable lawful basis rather than a general belief that the use is beneficial. ⁵

Ask the supplier what data entered model training, what customer inputs are retained, whether those inputs train later models, where subprocessors receive them, and how deletion propagates. Providers of general-purpose AI models in scope of the EU AI Act must publish a sufficiently detailed summary of training content, giving buyers a concrete disclosure to request and inspect. ²

The good answer is a documented chain of authority for each data source. The bad answer relies on confidentiality language while leaving collection rights, training rights, or deletion obligations unanswered.

Consider an illustrative case: a marketing team wants to fine-tune a response generator on an archive of customer support transcripts. Those transcripts were collected under a privacy notice that names one purpose, resolving the ticket in front of the agent. Feeding them into a model-training pipeline is a different, incompatible purpose, and no engineering safeguard fixes a permission the business never had in the first place.

Reject the proposed use until those rights are resolved. Redaction, local processing, synthetic examples, or a narrower data set may produce a different proposal that passes.

Gate six: preserve a safe fallback

A fallback is the tested process used when the AI service, its inputs, its supplier, or its reviewers are unavailable. It must sustain the minimum acceptable operation without depending on the failed component.

Name the trigger for switching, who has authority to switch, which records remain accessible, and how work completed during the interruption returns to the main process. Then rehearse it before retiring the existing route.

The good answer describes degraded but safe operation. The bad answer points to a manual process whose staff, permissions, or source data will disappear after launch. Staff behavior matters here because a fallback that nobody understands or accepts exists only on paper. Will my staff actually use it? covers the adoption evidence needed before that dependency is accepted.

Reject AI when failure would stop a critical process or force staff to trust outputs they cannot independently reconstruct.

The procedure does not prove that humans are safer

These gates reject unsupported deployment claims. They do not establish that unaided human judgment is accurate, fair, or consistent.

A replication involving recidivism, meaning whether a person commits another offence, found statistical algorithms outperformed human predictions in three of the four data sets examined. ⁶ Replacing an algorithm with a person can therefore preserve or worsen the underlying error.

Compare every proposed AI process with the real fallback, including its delays, inconsistent decisions, missing records, and appeal path. The correct response to a failed gate may be a better rule, a constrained decision aid, or a redesigned human process. It is not automatically the process already in place.

The procedure also cannot decide whether a business objective is legitimate. A well-evidenced, reviewable, reversible system can still pursue manipulation or discrimination. Where law prohibits a use, operational safeguards do not make it acceptable. Where the law permits discretion, accountable people must still decide which decisions should never be delegated.