← writing

Who audits the auditor?

June 9, 2026 · 4 min · security · static-analysis · agents · solidity · tooling

I built a small Solidity static-analysis tool — scaudit, a dependency-free scanner that reads a contract and flags reentrancy, tx.origin auth, unchecked low-level calls, unprotected selfdestruct, and a dozen other things, plus some gas hints. Writing the detectors is the easy, satisfying part. It’s also the part that means almost nothing, because a security tool’s entire worth is whether you can trust its verdict — and the way that trust dies isn’t a crash, it’s a quiet lie.

A security scanner fails in two directions, and both are silent:

  • False negative — a green light on vulnerable code. The worst outcome, because it manufactures confidence. You ran the tool, it said clean, you shipped the reentrancy.
  • False positive — a red flag on safe code. Less dangerous but corrosive: enough of them and people stop reading the output, which converts every real finding into a false negative by way of fatigue.

So the interesting phase of building this wasn’t the detectors. It was handing the finished detectors to a different, stronger model — one that hadn’t written them and had no stake in them looking good — with a single instruction: construct the contracts these miss, and the safe ones they wrongly flag.

The two ways a detector lies: false negatives and false positives contract is VULNERABLE contract is SAFE tool: FLAG tool: clean true positive ✓ caught it FALSE POSITIVE cries wolf → tx.origin in a log; uint x = 5 FALSE NEGATIVE manufactures confidence selfdestruct in fallback; returns(bool) true negative ✓ stayed quiet
The audit lived entirely in the two off-diagonal cells — the lies. Every finding it returned was a contract that landed in the wrong box.

What it found

The detectors were heuristics — regex and brace-counting over source, no compiler, no AST — and heuristics break on structure. The auditor found exactly where:

  • A function’s returns (bool) clause was being parsed into its modifier list. So a function that returned a value looked, to the access-control detector, like it had a guarding modifier — and its findings were suppressed. False negatives, by punctuation.
  • receive() and fallback() weren’t being segmented as functions at all, so a selfdestruct sitting in a fallback was simply invisible.
  • A nonReentrant modifier silenced unrelated checks, because the engine treated “has any modifier” as “is access-controlled.”
  • On the other side: tx.origin passed to an event got flagged as phishable auth (it’s a log), and uint x = 5 got flagged as a reentrancy state-write (it’s a declaration). Crying wolf.

Each one became a regression fixture — a tiny contract that must trip the detector, or must not. The suite went from 29 tests to 46. And then I did the part the fleet can’t do for me: re-ran them myself, and pointed the CLI at a vulnerable fixture (one HIGH finding, correct line) and a clean one (no findings) to watch recall and precision with my own eyes.

scaudit is a first-pass triage aid. It has no control-flow graph, no taint analysis, no inheritance resolution; it reads one file at a time; its access-control detector is high-false-positive by nature. Slither compiles the contract and reasons over an IR; Mythril runs symbolic execution; real coverage is fuzzing and formal verification. I put all of that in the README in plain words, because a tool that overstates its reach is just a better-dressed false negative.

But the structure is the point worth keeping. The same agent workflow that built scaudit is, itself, this idea: a builder model and a separate, sharper auditor model that exists only to disbelieve it. The most useful thing you can do with a second model isn’t more output — it’s adversarial doubt aimed at the first one. Who audits the auditor? A different auditor, who was told to assume it’s lying.