Talk to Paul about
How Paul thinks
Benchmarks aren't everything
Believes that a model’s performance on public leaderboards can be misleading; real‑world task benchmarks matter more.
For you: Consider testing your own tasks, not just public leaderboards.
Regression tests protect
Turns each real failure into a regression test to ensure improvements don’t break existing behavior.
For you: Add regression tests whenever you fix a bug.
Verification is the bottleneck
Spends more time reviewing agent pull requests than writing code, seeing verification as the main bottleneck.
For you: Invest in better verification workflows for your agents.
Ask this first
- How do you design a robust evaluation harness for AI agents?
- What’s the most surprising failure you’ve caught with regression tests?
- Can you share tips for improving verification speed in agent development?
Paul's brain
Recently updated
Every production AI agent must answer three essential questions – can it work on benchmarks, does it retain behavior after changes, and…
In early October 2026 I posted the third thread of my X series about Decode, focusing on a systematic error‑analysis and regression‑testing…
In early October 2026 I expanded my X thread on evolving the coding agent Decode from a manually‑babysitted tool into an autonomous…
Verification is the real bottleneck now. I spend more time reviewing agent PRs than writing code myself.
In September 2026 I posted about the early stages of building the coding AI agent **Decode** and the challenges of evaluating it reliably…
In early October 2026 I posted a series of LinkedIn updates describing practical AI‑engineering tactics such as cheap confidence‑based…
In early October 2026 I shared a series of LinkedIn posts about building agentic software factories. I explained how naïve tool‑calling…
I maintain an open‑source portfolio of AI/ML projects that span LLM‑based applications, MLOps pipelines, and deep‑reinforcement‑learning…
I am an AI Engineer, author, and educator based in Timișoara, Romania. Over more than a decade I have shipped 20 applications, written the…