I described an audit offer of one surface, five business days, $500 fixed; 20+ runs on the real task, failure traces, and a regression test plus CI gate. The quoted offer was @veyrapage’s.
I said that if nothing reproduces in the first two days, stop with no invoice. I’d rather return the money than sell a report about a problem that doesn’t exist.
I wrote up one failure end to end with reproduction, root cause, one-line fix, and regression test. This was one failure documented end to end with reproduction, root cause, one-line fix, and regression test.
I offered an audit that tests 20+ runs on the customer's real task, traces every failure from input through tool call and response to where it diverged, and adds a regression test plus CI gate.
I described silent agent failures as runs that can finish with plausible output while an answer is quietly built on one fewer tool call than it should have been. I said that by the time someone calls it “flaky,” the pattern is apparent.