CBAM compliance agent

7 % → 81 %

accuracy on the hardest field · 50 documents

7 % → 81 %

Hardest field: 7 % → 81 % accuracy (95 % CI 66–91 %)

How it was measuredField accuracy against ground truth across evaluation stages; Wilson score intervals
On what50 synthetic supplier documents (37 with measured emissions)
Keep in mindSynthetic documents from known templates; accuracy on real supplier paperwork is unmeasured

Supervised by Mr. Fahmi Bedbabis · carried out at ninety. Startup House, Sfax

The problem

CBAM compliance agent · before

Paperwork chaos
EU importers must declare the carbon in their steel and aluminium; the inputs hide in supplier paperwork of every format.
Scripts break
Fixed extraction scripts break as soon as a field is missing.
Wrong but plausible
An LLM that invents an emissions figure produces a wrong number that looks exactly like a right one.

CBAM compliance agent · after

The AI reads
An AI agent does the reading and reconciling.
The engine computes
A deterministic, unit-tested engine computes every compliance number; the model never can.
A tight scope
Steel and aluminium, synthetic supplier documents, local open-weight models only.

CBAM compliance agent

Step 1 of 6 A declarant uploads a supplier document; Odoo hands it to the orchestrator.
Step 2 of 6 A local LLM (qwen2.5-14B on Ollama) picks the next tool each turn; after 8–10 turns without a result, a human takes over.
Step 3 of 6 Seven tools: reading a field tries a pattern match first and the LLM only as a fallback.
Step 4 of 6 Four pure functions compute emissions and certificate liability; 20 tests written first caught a sign-inversion bug.
Step 5 of 6 Safeguards: known facts override the model, and a measured value needs a verbatim quote plus plausibility bounds.
Step 6 of 6 At most one validated write per line; the full trace is saved for audit, and anything unresolved goes to a human.

Sources