Code Chooses Tools, Not LLMs

The core question in AI system design isn't "should we use an LLM?" It's: "who decides which tools the system calls?"

In an agentic system, the LLM sees a set of available tools and decides which ones to call, in what order, with what arguments. In a deterministic system, code decides the tool sequence. The LLM never sees the tools.

Agentic: LLM sees tools → LLM decides which to call → execution follows
Deterministic: Code decides tool sequence → tools execute → no LLM in the decision loop

The agentic pitch is seductive for many workflows. Handles edge cases you didn't code. Reads messy natural language. Adapts to new situations. For triage, drafting, summarization—this is often right.

For refunds, we chose deterministic. Here's why and what we're learning.

Why refund issuance is different

A refund decision is binary and consequential: money leaves the business or it doesn't. What matters is delivery status, days since delivery, order amount, customer history. That's all structured data. No messy prose to interpret. And when it goes wrong, it's not "slightly worse UX." It's real money leaving the company:

• Chargebacks from customers disputing AI refunds they claim never to have requested
• Fraud vectors that a model might learn to exploit (issuing refunds to test patterns)
• Support cleanup when the LLM's "hallucinated policy exception" commits the company to something legally inconsistent
• Audit failures when regulators ask why a model made a financial decision and you can't explain it

So we're building it the boring way. Fixed sequence of data lookups. Deterministic eligibility check against policy. Hardcoded decision path. No model decides whether money moves. Ever.

What we rejected: LLM decides which tools to call

In the agentic pattern, the LLM sees your tools and decides which ones to call. For a refund system, this means:

• LLM sees: get_customer(), get_order(), get_delivery(), issue_refund(), send_email()
• LLM decides: which tools to call, in what order, with what arguments
• LLM decides: "should I call issue_refund() now, or should I gather more info first?"
• LLM decides: "this customer's tone suggests they deserve a refund"

The problem: on day 1, with one phrasing, the LLM calls tools in order A. On day 2, with slightly different context, it calls tools in order B. The decision is not reproducible. Not auditable. Not defensible to a regulator.

The agentic pattern: LLM sees available tools and decides which to call. Decision power is distributed across a system you can't fully audit.

What we kept: tool patterns, not tool autonomy

We borrowed patterns from agent infrastructure, but removed the autonomy. We built a scoped tool gateway that:

• Enforces token-based authentication on every tool call
• Logs every access in an immutable audit trail
• Restricts which tools can be called (no dynamic discovery)
• Scopes each tool to specific data (customer data only, order data only)

This is defensive infrastructure, not AI autonomy. The gateway doesn't decide when to call tools—the code does. The LLM never talks to the gateway. The code talks to the gateway.

We initially named it SecureMCPGateway (implying Model Context Protocol compliance). It wasn't. It was just a hardcoded dispatcher with three allowed function names and no dynamic discovery.

We renamed it ScopedToolGateway. Clearer. Honest about what it does: enforce boundaries around tool access, not emulate an LLM framework.

The lesson: Adopt the infrastructure patterns that matter (audit logs, scope boundaries, access control). Reject the framework patterns that give LLMs autonomy they shouldn't have.

Where we'd actually use a model

As this system grows, the natural place for an LLM is upstream:

• Parsing free-text support requests into structured categories (damaged, late, missing items, other)
• Drafting the decline email in a friendlier tone than a template allows
• Triaging which of several policies applies when the request is genuinely ambiguous

All of that is advisory. A human or a deterministic rule still gates the money-moving step. The model is a helper that makes a human decision faster, not a replacement for it.

The production pattern we're using

If you're building a money-critical workflow, here's the architecture that's working for us:

• Scoped data access: A gateway that allows only the lookups required (customer status, order history, delivery date). No fishing expeditions.
• Structured inputs: Parse free text into structured schemas before the decision logic touches it.
• Deterministic decision logic: A state machine or rule engine that is testable, auditable, and changes only through code review and deployment.
• Permanent audit trail: Every access, every decision, every who-what-when, stored to a database that can't be edited retroactively.
• Advisory LLM layer (optional): Upstream of the decision, or downstream in communication-never in the hot path.

Architecture in practice

The system we built (deterministic automation with human approval):

Deterministic workflow with human approval: eligible refunds wait for support team review before issuance. Every decision and every approval is logged and auditable.

Request flow: code controls all tool decisions

• Step 1 - Parse input: Application takes free-text request, extracts customer ID and order ID. LLM not involved in tool decisions yet.
• Step 2 - Call tools in fixed sequence: Code (not LLM) decides to call: get_customer() → get_order() → get_delivery(). Same sequence, every time.
• Step 3 - Scoped gateway validates: Each tool call is authenticated with a token. No other tools can be called. No dynamic discovery.
• Step 4 - Policy lookup: Code (not LLM) looks up policy rules. Returns eligible status and window. Advisory only—no decisions here.
• Step 5 - Deterministic decision: Fixed rule: IF (status in [damaged, lost] AND days_since_delivery ≤ 30) THEN eligible, ELSE decline. Binary logic, no model judgment.
• Step 6 - Create ticket: Eligible → ticket_status = pending_approval. Not eligible → ticket_status = declined (auto-resolved).
• Step 7 - Human approval: Support agent reads ticket, reviews eligibility, calls approve_refund(). Human makes the actual decision to move money.
• Step 8 - Refund issuance: Database constraint prevents duplicate refunds for the same order. Transaction logs everything.
• Audit trail: Every tool call, every decision, every state change written to PostgreSQL. Queryable, immutable, compliance-ready.

Two gates, not one: code decides eligibility, a human decides approval. Neither gate is an LLM.

Security & compliance built in:

• Token-based gateway prevents unauthorized data access
• Database-level uniqueness constraints (not application logic) enforce "one refund per order"
• Transactional retry logic handles concurrent requests without race conditions
• Permanent, immutable audit trail designed with regulatory review requirements in mind
• No model judgment in hot path = no unexplainable decisions in a compliance audit

Support operations dashboard: every ticket, approval, and audit event tracked and reviewable.

The business case: Why CTOs and CEOs should care

For CTOs: A deterministic refund system is operationally simpler. You audit it once, test it end-to-end, then ship it with confidence. An LLM-based system requires continuous monitoring, hallucination detection, and a support team trained to reverse bad decisions. In production, simpler beats smarter.

For CEOs: The cost math is brutal. A single erroneous AI-issued refund can cost thousands. A run of bad decisions can trigger chargebacks, payment processor review, and account suspension. A compliance violation discovered in an audit can cost millions. A deterministic system removes that tail risk entirely. You know exactly what the system will do, and regulators know you're not gambling with customer money.

Risk breakdown:

• Fraud surface: An LLM learns from patterns. If it sees "customer called 3 times" → "refund approved," it might learn "call lots, get refunds." A rule engine can't learn; it can only do what you told it to do.
• Audit liability: When a regulator asks "why did this refund happen?", a deterministic rule is defensible. "Delivery status was 'damaged' and within 30 days" is auditable. "The model thought the tone suggested the customer was upset" is not.
• Support load: An LLM requires a human-in-the-loop for edge cases. A deterministic system routes edge cases to a human ticket (transparent escalation). Same outcome, but the failures are predictable and logged.
• Chargeback exposure: Chargebacks happen when a customer disputes a refund they didn't authorize. An LLM refunding without customer confirmation is a chargeback vector. A system that creates a ticket and requires approval is not.

Request processing: deterministic logic means every decision is auditable and repeatable.

When you're tempted to use an agent

Before reaching for an agent framework or a language model for a financial workflow, ask:

• Is this decision actually non-deterministic? Or would a rule engine or state machine work?
• Can I test and audit every decision path?
• If the model hallucinates, what's the blast radius?
• Does it make my system easier to operate-or just more impressive?

If your answer to the last one is "just more impressive," you're adding an attack surface and a failure mode for no gain. Deterministic code that's well-tested and auditable beats a model with judgment you can't fully verify-especially anywhere real money, real PII, or real irreversible actions are involved.

We'd rather ship the boring version that's provably correct than the exciting version that's probably fine.