Test Before Bulk Protocol
Before any 10+ item operation, process 3-5 real items first, inspect quality, then bulk only after the approach is correct.
Routing
Open this page before mass edits, batch ingests, skill migrations, media downloads, frontmatter rewrites, and broad quality passes. If the operation requires judgment, produce a small representative sample before touching the full corpus.
Bulk mistakes compound faster than good work. Sampling lets the agent fix the method while the blast radius is small. Source: conventions-test-before-bulk
Applies To
- batch wiki page creation
- raw source compilation batches
- mass skill creation or migration
- multi-file refactors
- bulk frontmatter or link rewrites
- automation changes across many files
- project enrollment across multiple repos
- generated surface rewrites or regeneration script changes
- media downloads and media processing
- any other operation where one bad rule could multiply
Protocol
- Define the intended invariant.
- Select 3-5 representative items.
- Process the sample end to end.
- Inspect output against the real quality gates.
- Show Kevin the sample when judgment or taste matters.
- Fix the procedure, not one item at a time.
- Bulk execute in small commits or checkpoints.
- Run validation after each batch.
Mechanical, low-risk work can skip the sample: document the scope, then verify after. private page
Failure Mode
The rule fails when an agent scales an uninspected transformation and then spends the rest of the session patching avoidable damage. private page
When Approval Is Required
Ask Kevin before bulk when the operation changes personal knowledge, deletes/merges pages, rewrites voice/style, or affects 10+ files with non-mechanical judgment. Mechanical index rebuilds and deterministic formatting do not need approval.
Done
Bulk work is done when the invariant holds across the batch, not when the command finished.
Timeline
- 2026-11-02 | Became the one owner of this rule; Test Before Bulk now points here. Source: wiki overhaul memo Phase 5, 2026-11-02
- 2026-07-01 | Added generated-surface rewrites to the bulk-sampling scope so pack, registry, redirect, and index changes are tested before broad application. private page
- 2026-06-18 | Expanded sampling protocol with applicability, phases, approval rule, and done definition. private page
- 2026-05-31 | Meta page formalized from test-before-bulk convention. Source: conventions-test-before-bulk.md