Source Compile Automation Setup

How to set up the source-compile automation in Codex and Cursor Automations.

What It Does

The source-compile automation closes the loop between raw data collection and knowledge compilation:

1. Syncs X bookmarks and AI session transcripts from local sources
2. Enriches X bookmarks with resolved links, linked READMEs/articles/transcripts, author self-threads, author artifact replies, and demo media
3. Researches promoted bookmark entities online before writing durable pages
4. Audits recent agent sessions for friction patterns and skill opportunities
5. Compiles findings into wiki page updates, skill improvements, and router/tool updates
6. Logs everything to wiki/log.md and state.json

The local runner can access Cursor/Codex transcript paths and local skill directories; the cloud runner can only process raw material already committed to the repo. Source: automations/source-compile.md; automations/source-compile.cursor-automation.md

Runner Decision

If the run discovers a repeated compile pattern, update or create the relevant skill. Source compile is not only ingestion; it is also skill maintenance.

Manual Trigger

In any agent session (Cursor, Codex, Claude Code):

Run the source-compile automation.

The agent reads automations/source-compile.md and follows the 5-phase pipeline.

Cursor Automations (Cloud Cron)

1. Go to cursor.com/automations
2. Create new automation
3. Name: Source Compile
4. Schedule: Daily-compatible for cloud, or on push to raw/
5. Repo: kevin-wiki
6. MCPs: GitHub
7. Instructions: Copy the text inside the code block from automations/source-compile.cursor-automation.md

Limitation: Cloud sandbox cannot access local paths (~/.cursor/, ~/.codex/). The sync scripts must be run locally first and committed to raw/. The automation only processes data already in the repo.

Codex Automation (Local Cron)

Codex automations run locally with full filesystem access - they can read transcripts and skills directly. This is the recommended runner for source-compile. The current heartbeat expects source compile to run every four hours locally because it shares the sub-daily capture loop with email and Slack sync.

Setup via sqlite (preferred)

sqlite3 ~/.codex/sqlite/codex-dev.db "INSERT INTO automations (id, name, status, cwds, rrule, model, reasoning_effort, prompt, next_run_at, created_at) VALUES (
  lower(hex(randomblob(16))),
  'source-compile',
  'ACTIVE',
  '[\"$HOME/Documents/GitHub/kevin-wiki\"]',
  'FREQ=HOURLY;INTERVAL=4',
  'o3',
  'medium',
  'Read automations/source-compile.md and execute the full 5-phase pipeline. You have local filesystem access. Run sync scripts first, including X bookmark sync, enrichment, and media download. Process raw/x-bookmarks/enriched/*.json using the x-bookmark-absorb skill: inspect resolved links, selfThread, artifactReplies, media, and online sources; add tweet directives for original posts and artifact cards for source-critical links/repos/docs/demos/videos; update hub pages, dedicated pages, skills/tools/routers, index, qmd, state, and log. Read transcripts directly from ~/.cursor/projects/*/agent-transcripts/ and ~/.codex/sessions/. Commit all changes with message: auto(source-compile): YYYY-MM-DD codex run',
  datetime('now'),
  datetime('now')
);"

Setup via Codex UI

1. Open Codex
2. Go to Automations
3. Create new:
   • Name: source-compile
   • Working directory: ~/Documents/GitHub/kevin-wiki
   • Schedule: Every 4 hours
   • Model: o3
   • Prompt: (see below)

Codex Prompt

Read automations/source-compile.md and execute the full 5-phase pipeline.

You have local filesystem access. Use it:
- Run ./scripts/sync-x-bookmarks.sh and ./scripts/export-ai-sessions.sh
- Run npx tsx scripts/enrich-x-bookmarks.ts before bookmark absorption
- Read raw/x-bookmarks/enriched/*.json and inspect resolvedLinks, selfThread, artifactReplies, and media
- Research promoted bookmark tools/projects/people online, preferring primary sources, GitHub, docs, and independent discussion
- Read Cursor transcripts from ~/.cursor/projects/*/agent-transcripts/*/*.jsonl
- Read Codex sessions from ~/.codex/sessions/YYYY/MM/DD/*.jsonl
- Read/write Cursor skills at skills/personal/ (loaded through ~/.cursor/skills) and Codex skills at ~/.codex/skills/

After completing:
1. Write output to outputs/<date>/source-compile/codex.md
2. Update wiki pages as specified in the automation
3. Update state.json (automations.source-compile and compile.last_bookmark_id)
4. Append to wiki/log.md
5. Commit: auto(source-compile): <date> codex run

Verifying It Works

After the first automated run, check:

# See if it ran
grep "source-compile" wiki/log.md | tail -3

# Check state
cat state.json | python3 -c "import json,sys; print(json.load(sys.stdin)['automations'].get('source-compile', 'never run'))"

# See output
ls outputs/*/source-compile/

Done Criteria

A source-compile run is complete only when:

• output file exists under outputs/<date>/source-compile/
• wiki pages are updated with citations
• skills or protocols are updated if transcript mining found repeated behavior
• tool/skill routing docs are updated if bookmark research changes Kevin's stack
• generated packs, capsules, skill registry output, redirects, and qmd are refreshed when source compile changes them
• state.json records the run
• wiki/log.md has a summary
• build-index and qmd have run after page edits

---

Timeline

• 2026-07-01 | Updated local Codex setup from daily to every four hours to match the current sub-daily source/email/Slack capture loop. Source: User request, 2026-07-01; Sub-Daily Cadence
• 2026-07-01 | Added generated-surface and artifact-audit closeout to source compile setup, matching the current X bookmark and pack/capsule workflow. Source: User request, 2026-07-01
• 2026-06-19 | Updated setup for daily X bookmark deep absorption: Codex local cron should use the canonical kevin-wiki path, run enrichment before absorption, inspect author artifact replies, research promoted entities online, and route stack changes into skills/tool routers. Source: User request, 2026-06-19
• 2026-06-18 | Added runner decision table and done criteria distinguishing local filesystem runs from Cursor cloud runs. Source: User request, 2026-06-18; automations/source-compile.md