Empirical Verification
Every assumption must be tethered to reality. Source: AGENTS.md
This is the scientific method applied to software. You are a computer scientist, not a computer guesser. Form hypotheses. Run experiments. Observe results. Refine understanding.
The Process
When you believe something about how code behaves, devise a hypothesis and test it. Write a small script. Add a log statement. Run the debugger. If the empirics match your expectation, proceed. If they do not, you have found a bug, a misconception, or a premature assumption. Investigate.
Search online to see if others have observed the same behavior. Read the documentation to understand what is actually supposed to happen. Explain to yourself what the system is doing and why. Only once you understand the behavior should you attempt to change it. Once you understand it, reproduce the issue precisely to pinpoint the exact cause.
Root Cause Fixes
Fix problems at the root cause. Surface-level patches create debt. If a function fails under certain inputs, do not add a guard clause at the call site. Fix the function. If a system misbehaves under load, do not retry until it works. Find the bottleneck.
Quick fixes become permanent fixtures. The shortcut you take today becomes the constraint you work around tomorrow.
Research Before Implementation
If you have network access, use it. Look up library documentation instead of guessing at APIs. Verify behavior against official specs. Search for error messages before inventing fixes. Check changelogs when upgrading dependencies.
The pattern is simple: do not guess when you can verify. A distinguished engineer does not assume they remember how an API works. They check. This is especially true for security-sensitive code, payment integrations, and anything involving external services.
Quality Measurement
The quality of your work is measured by how well you interact with the real world. Not by how clever your abstractions are. Not by how quickly you produce code. By whether the code actually does what it is supposed to do when it runs.
Evidence ladder
| Claim | Evidence that earns it |
|---|---|
| "This API supports X" | primary docs, type signature, minimal run |
| "This bug is fixed" | failing repro before, passing repro after |
| "This UI works" | browser interaction or screenshot evidence |
| "This agent completed the task" | trajectory, tool outputs, verifier result |
| "This fact is current" | source date and primary-source citation |
| "This wiki page is integrated" | index, backlinks, qmd search, log |
The ladder matters because agents are fluent liars by default. Not malicious, just smooth. The antidote is attaching every conclusion to a real observation.
Relationship to evals
Eval-Driven Development is empirical verification for stochastic systems. Agent Trajectory Evaluation is empirical verification for tool-using agents. Agentic Quality System is the stack that keeps the habit from depending on one careful person.
Long Live Hard SaaS is the business-model implication: verification does not disappear when code generation gets cheap. A maintained SaaS product earns its rent by turning repeated real-world observations into fewer bugs, better defaults, stronger tests, and lower user overhead. Source: raw/notes/saas-bug-moat-slack-thread-2026-06-22.md
Concept Position
| Field | Value |
|---|---|
| Concept family | Agent harness and runtime primitives |
| Concept owned | Every assumption must be tethered to reality. Source: AGENTS.md |
| Category map | Concept System Map |
Timeline
- 2026-07-01 | Concepts category refresh added this page to the Agent harness and runtime primitives family, linked it to Concept System Map, and kept it standalone because it owns this reusable mental model: Every assumption must be tethered to reality. Source: AGENTS.md. Source: User request, 2026-07-01
- 2026-06-19 | Added evidence ladder and connected empirical verification to evals, trajectory evaluation, and the agentic quality system. Source: whole-wiki concept review, 2026-06-19
- 2026-06-22 | Linked empirical verification to the hard-SaaS thesis: durable products compound by converting real usage into tests, guardrails, and maintenance. Source: raw/notes/saas-bug-moat-slack-thread-2026-06-22.md