Trust & test evidence
A compliance tool is only worth using if it is right. This page shows exactly how we test Stampbench, what the results are, and — just as importantly — what we have not yet proven. Every figure below comes from an artefact in our repository, not from an estimate.
1. Automated test suite
Every commit runs the full suite. The tests are not smoke tests — they assert specific rule outcomes (for example, that a one-cent discrepancy in the totals chain is rejected, and that a legitimate one-cent VAT rounding difference is accepted).
| Package | Tests | What they cover |
|---|---|---|
| @stampbench/core | 39 passing | Rule engine, UBL + CII parsing, generation, totals arithmetic, round-trips |
| stampbench (CLI) | 12 passing | Validate/generate commands, CI exit codes, JSON output, error paths |
| Web application | 6 passing | Plan/quota logic, API-key hashing, rate limiting |
One structural guarantee is enforced by test: generate → parse → validate round-trips cleanly. A document we produce, re-read from XML, must validate against the same rules — so the generator and the validator cannot silently drift apart.
2. The official XRechnung test suite
We run the public test corpus published by KoSIT (the German coordination office for IT standards) — itplr-kosit/xrechnung-testsuite@v2026-01-31, XRechnung 3.0.2 — through our engine on every parity run.
Both syntaxes are covered: 45 UBL and 41 CII (ZUGFeRD / Factur-X style) documents. Zero exceptions means no document in the official corpus crashed the parser — a baseline we hold to.
The three we flag — published openly
These are the reference documents our engine currently rejects. We publish them rather than hide them: they are open questions about whether our implementation of these two rules is stricter than the reference, and they are the first thing our parity run will settle.
| Document | Syntax | Rule we raise |
|---|---|---|
| instances/extension/05.01a-INVOICE_ubl.xml | UBL | BR-CO-16 |
| instances/technical-cases/cius/01.05_minimal_test_ubl.xml | UBL | BR-DE-16 |
| instances/technical-cases/cius/01.05_minimal_test_uncefact.xml | CII | BR-DE-16 |
3. KoSIT parity harness — what it is, and what it has not yet proven
Status: harness built, dual-run pending. We have not yet published a parity figure, because we have not yet run one. Any vendor claiming “100 % parity” without a reproducible report should be asked for the report.
The official KoSIT validator is the legal reference for XRechnung. Our harness pins exact versions of it (validator 1.6.2, XRechnung configuration 3.0.2) with byte-exact download verification, runs both validators over the same corpus, and emits a per-document delta report classified into two categories:
- False green — KoSIT rejects, we accept. This is the dangerous direction: it means we would have told you an invoice was fine when it was not. Our target for this number is zero, permanently.
- False alarm — KoSIT accepts, we reject. Noisy but safe; it costs you time, not a rejected invoice.
An honest limitation we will state on the report itself: the official corpus consists mostly of documents that should validate, so it tests the false-alarm direction well and the false-green direction only weakly. A mutation corpus is the next step.
4. Adversarial code review
Before launch the codebase was put through a multi-agent adversarial review across four dimensions — security, rule correctness, API/billing logic, and framework misuse — in which every claimed finding had to survive a second reviewer whose job was to refute it. Twenty-five findings survived that check and were triaged; all high-severity items were fixed before this page was written.
The most consequential fix is worth naming, because it is exactly the class of bug that makes compliance tools untrustworthy: our totals comparison originally used a ±0.011 tolerance on the EN 16931 summation rules, which meant a genuine one-cent-wrong invoice could pass. Those rules are exact equalities; the tolerance is now 0.005 (float-noise only), and the ±0.01 rounding allowance is applied solely to BR-CO-17, the one rule that involves multiplication. There is a regression test for both directions.
Other fixes in the same pass: invalid or revoked API keys now return 401 instead of silently falling back to the anonymous tier, duplicate subscriptions are blocked at checkout, and all API inputs are bounded and rejected if non-finite.
5. What we do not claim
We would rather lose a deal than win one on an overstatement.
- This is not legal advice, and passing Stampbench is not a legal guarantee that an authority or access point will accept your invoice. For certification-grade sign-off, run the official KoSIT validator too. Our job is to make that run pass.
- Rule coverage is a documented subset, not the complete rule set — exactly which rules are implemented is listed in the docs. It grows with every release, and the ruleset version is stamped into every response.
- No parity percentage yet, as stated above.
- No SOC 2 or ISO 27001 certification. We are early; we will say so plainly rather than imply otherwise.
- Generation is UBL today. Validation reads both UBL and CII; Factur-X PDF generation is on the roadmap, not shipped.
Verify any of this yourself
The rule engine is MIT-licensed and the parity harness is in the same repository. You do not have to take our word for any figure on this page — run it.
npm test # the full suite node tools/parity/download.mjs # fetch the pinned official validator + corpus node tools/parity/run.mjs # dual-run and emit the delta report
Version pinning is built into the product too: every validation response carries meta.rulesetVersion and meta.specVersions, so you can always answer “which spec release was this checked against?” in an audit.
Questions about any figure here? See our security practices or get in touch — we will show you the artefact.