All posts
Photo: Florent Bertiaux — Pexels
AI & evaluation
Pavan Kumar TV7 min read

If the answer fits in a dropdown, it is not an agent problem

The default assumption on a freight AI roadmap is that smarter ops means a model on every decision.

Walk an architecture review and you will eventually find that assumption sitting on a question the database could already answer. Which of five exception types is this? Which charge family does this line belong to? Which desk owns a mail labelled "customs"? Should this invoice go to review because it is over a fixed threshold?

Those are dropdowns. Someone enumerated the options last year. Putting a language model in front of them does not add judgment. It adds latency, cost, and a new way to be randomly wrong about something that used to be reliably boring.

So where is judgment actually scarce, and what test stops a roadmap from spending it on enums?

The tax that used to be invisible

For a while, putting a model on everything felt free. The bill was small, the demo was green, and nobody asked what it cost to be approximately right on a charge code.

Three things are now impossible to ignore inside a live freight operation.

Replay. Deterministic code gives finance the same accrual from the same inputs next month. A model call regenerates. When someone asks why a detention line was approved in March, "the prompt usually works" is not an audit trail.

Latency under load. A registry lookup returns before the operator's eye finishes the row. A frontier call that is fine once becomes a queue when pre-alerts arrive in hundreds. Compound that across mailbox → extract → route → match and the desk feels the wait as product failure, not infrastructure trivia.

Variance where liability lives. Approximate tone in a draft email is acceptable. Approximate free-time calculation is not. Approximate HS digits without a citation are worse than a slow human, because they fail inside a system that looks automated.

None of that is an argument against models. It is an argument for putting them where approximate is allowed, and keeping clocks, codes, and thresholds where approximate is a finding.

The scarcity is elsewhere

The scarce thing in an ops team is attention under ambiguity.

A commodity description that is almost enough to classify and almost enough to get someone fined. A carrier invoice line whose code matches nothing in the annex vocabulary. A mailbox thread that is a rate request, a complaint, and a document chase in one forward chain. A detention claim where the free-time clock and the notice obligation disagree depending on which PDF you believe.

That is where a model earns its keep: language that was never designed for a form, evidence that has to be assembled, and a safe abstention when the pack is incomplete.

Charge codes, service levels, and approval bands are not scarce. They are inventories. Treating them as reasoning problems is how roadmaps get thick and operations get fragile.

Patterns that do not need a model

A useful walk of any logistics AI roadmap: check whether any of these describe what is actually being built.

Routing across a known intent list. Once ops has named the intents (rate request, customs query, pre-alert, complaint, POD chase), the valuable system is the map from intent to desk, queue, and SLA. Re-deriving that list with a model on every message is theatre. Propose a new intent when something novel arrives. Do not re-litigate the known set.

Threshold gates. "Over £X goes to review" is a comparison. So is "outside free time by N days." So is "margin below policy." If a moderately experienced ops lead can write the rule in a paragraph, write the rule.

Lookups against your own rate annex or shipment record. Asking a model what your contract said on the twelfth is slower and less honest than reading the versioned annex. The model does not remember your tenant. Your database does.

Chat over fields that were already keyed into the TMS. Structured data in a database was a solved problem before agents. A nicer query interface over completed work photographs well and does not touch the inbox where the hours still go.

Scheduled chasers rebranded as agents. If the cadence is fixed and the template is known, it is a job with a clock. Calling it autonomous does not change the failure mode when the carrier portal moved a button.

Calculations dressed as judgment. Billable weight, free-time end, duty from a known code and rate. If it can be computed, compute it. Models are not clocks and they are not calculators.

A predictable objection: but the agent reasons across tools. Orchestration is not the same as judgment. Most production "agents" are state machines with a model choosing the next transition. When the transitions are knowable, choose them in code. When they are not, the model is doing real work at the edge of a deterministic flow, not in place of it.

Patterns where a model earns its keep

Turning hostile documents into structure. Commercial invoices, packing lists, rate cards, carrier PDFs that change layout without notice. Prose into rows is the job.

Drafting from intent. A first-pass dispute email, a customer update, a clarification to an origin agent. The model is not intimidated by the blank page. The human knows what must not be promised.

Judgment on fuzzy criteria. Does this thread sound like a complaint hiding inside an ops update? Is this commodity description enough to classify, or must we abstain? Rubric yes, regex no.

Exception residue. The deterministic path covers the bulk. What remains is a mess of one-offs. That residue is often where a model is cheaper than another hire, if the abstention and the evidence pack are first-class.

The test underneath: is the alternative a faster deterministic version of the same thing, or is the alternative "a person does it, or nobody does"? Only the second case clears the bar.

A test we use on our own work

Before anything is allowed to call a model in production, we ask whether the output set is closed.

If it is closed, write the registry. Mailbox intent routing was the obvious example. Rate cards are the same shape: extraction can be probabilistic; the verifier that checks weight breaks, minima, and currency against the card grammar should not be. An LLM that "mostly" reads a grid is a demo. A round-trip check against structure is software.

Classification has the cleanest version of the rule. Where a citable authority exists, retrieve and attach the citation. Where it does not, abstain. We have already published the negative result on the other mistake: pulling more rulings into retrieval without regard to whether they were the right rulings made the system worse. Volume is not authority. The roadmap version of that mistake is sprinkling model calls onto every box because empty boxes look unfinished in a deck.

What goes wrong when the roadmap ignores the test

Finance discovers that last month's accruals cannot be replayed, only regenerated. A carrier dispute stalls because the evidence pack is a confidence score. Two tenants with opposite surcharge policies get the same "industry" recommendation. An incident review ends in prompt archaeology instead of a rule that can be fixed.

None of that is a frontier-model limitation. It is an architecture choice: spending judgment tokens on work that needed a versioned table.

Vendors will keep shipping chat over the TMS and agents over enums, because those screens photograph well. Buyers can push back with one question. Was this ever uncertain, or did we just not bother to encode it?

A heuristic for the next roadmap review

Before adding a model to a workflow, build the deterministic version first. If it covers the common case, ship it. Use the model only for the residual, and only if the residual is worth the replay, latency, and variance cost.

That inverts the default of the last two years. Instead of "where can we add AI?", ask "where does the registry fail, and is the failure worth a model call?" Most of the time the answer is no. Sometimes it is yes. Those are the Friday-afternoon pains that still hurt when the happy-path demos are over.

If the answer fits in a dropdown, put it in a dropdown. Save the model for the day the dropdown is wrong.