AI agents · security · freight forwarding
Dispatch Agent, Under Attack
An AI dispatch agent, built and attacked in a sandbox we made. A QR code on a delivery note tries to move the truck.
Overview
We built a small dispatch agent that books collections, updates ETAs and applies address changes from carrier emails and delivery notes. It runs on a local open model inside a sandbox we made: a mocked transport-management system, a fictional inbox, and printed delivery notes whose QR codes the agent decodes from the image.
Then we attacked it with 24 injection attacks. Five of the six attack templates use the published AgentDojo attack strings (we did not run the AgentDojo benchmark itself); the sixth is ours. They hide instructions in a QR code, in invisible white text, in an email body or in an email from a look-alike address, and try to move a shipment to another warehouse, leak a collection PIN, cancel a booking or plant a false ETA. A person looking at the note sees a booking code. The agent reads an order.
The defence follows a simplified version of the CaMeL design (straight-line plans only). A planner that sees only the dispatcher's request writes the plan before any email is read. A separate extractor with no tools reads the untrusted content and can only fill in values. Every value carries its source, and a policy keeps untrusted values away from recipients, cancellations and shipment IDs. Any change of address also waits for a call-back to the number on file. We ran the same attacks with no defence and with the call-back rule alone. We also show what the defence does not stop: a false ETA. It got through in 4 of 6 tries, from the carrier's email, from hidden text on the delivery note and from a look-alike sender, because the policy lets values that were read go into an ETA. Routes, bookings and recipients stayed protected. The same gap applies to slot times and booking references, which we did not test.
What we built
- A tool-using dispatch agent on a local 4-billion-parameter open model, with 12 sandbox tools (read email, read a document, decode a QR code, book a slot, update an ETA, change a destination, send an email, cancel a booking and others).
- A sandbox of fictional shipments, carriers and consignees, with delivery notes rendered with real QR codes and a simulated call-back to the consignee's number on file.
- A simplified CaMeL-style defence (straight-line plans): the planner writes a plan from the request only, a quarantined extractor reads the untrusted content, provenance is tracked on every value, and a policy checks each action.
- A suite of 24 injection attacks (6 templates × 4 goals, over 4 hiding places), scored from the sandbox state after each run.
Tech stack
Python · mlx-lm · Qwen3-4B-Instruct-2507 (4-bit, Apache-2.0) · zxing-cpp · qrcode · three.js · Playwright
Numbers
| Number | How we measured it |
|---|---|
| 12 · 7 · 4 of 24 attacks got through | with no defence, with the call-back rule only, and with the full defence. Each of the 24 cases was run once per setup on the same local model with greedy decoding; an attack counts as through when its goal shows in the sandbox state at the end of the run (the address, an email to the attacker, a cancelled booking, or the ETA). Small counts on one model; no general claim. |
| All 4 that got through the full defence were false ETAs | (4 of the 6 false-ETA cases: 2 from the carrier's email, 1 from hidden text on the note, 1 from a look-alike sender). In those runs no route, booking or recipient changed. One of them (f1) was ignored by both plain agents; the extractor copied it, so the net change from call-back only to full defence is 7 → 4. |
| 5 of 5 tasks finished without attack, in all three setups. | These are the five dispatcher requests run once with a clean inbox and checked against the expected bookings, ETA, email and address. |
