Most problems described to us as AI problems are search, classification or data quality problems. Here is how we tell the difference, and how we design for wrong answers.
The question we are asked is usually "can we add AI to this". The question worth answering is "what is the task, and what happens when the answer is wrong".
Unstructured input, structured output. Turning a supplier email, a scanned form or a free-text field into records. This used to require either a rules engine that broke on the first unexpected format, or a person. It is a genuinely good fit — the input varies infinitely, the output shape is fixed, and correctness is checkable.
Summarising long documents where the reader can verify. A summary of a case file, with links back to the source, is useful even when imperfect, because the reader can check it against the original.
Semantic search over your own content. "Find things about late deliveries" matching a document that says "consignment arrived after the agreed window". Keyword search cannot do this. Embeddings can, and the failure mode is a slightly worse result — not a wrong fact.
Anything where a wrong answer is silently authoritative. If the output goes straight into a decision, a record or a customer communication without a human or a rule in between, you have built a system that will confidently be wrong at a rate you have not measured.
That is not an argument against using the technology. It is an argument for designing the surrounding system so a wrong answer is recoverable.
Constrain the output shape. Ask for structured output and validate it against a schema. An answer that does not parse is a failure you can detect; an answer in prose is one you cannot.
const ExtractedInvoice = z.object({
supplierName: z.string().min(1),
invoiceNumber: z.string().min(1),
totalMinor: z.number().int().nonnegative(),
currency: z.enum(['GBP', 'EUR', 'USD']),
lineItems: z.array(z.object({
description: z.string(),
quantity: z.number().positive(),
unitPriceMinor: z.number().int().nonnegative(),
})).min(1),
});
const parsed = ExtractedInvoice.safeParse(JSON.parse(response));
if (!parsed.success) return queueForHumanReview(document, parsed.error);Check what can be checked. Do the line items sum to the total? Does the supplier exist in your database? Arithmetic and lookups are cheap, deterministic, and catch a large share of plausible-looking errors.
Route uncertainty to a person. A review queue is not an admission of failure. It is the mechanism that lets you deploy at all, and it gives you a measured error rate rather than an assumed one.
Keep the provider behind an interface. Model quality, pricing and availability all change. If your application calls one vendor's SDK from thirty places, you have made a decision you cannot revisit.
Before any of this: is the data you would feed it actually good enough? A model cannot recover information that was never captured. If half your records have an empty description field, the useful project is fixing capture, not adding inference on top of absence.
That is the most common finding when we look at these projects. It is also the least exciting, which is presumably why it gets skipped.
A permanent link and an email link, rather than a row of share buttons. A share widget would mean loading somebody else’s script — and their tracking — on every article you read here.