X For You Algorithm (2026 Open Source)

The For You feed ranks posts with a Grok-1-derived transformer ("Phoenix") instead of hand-engineered features, and runs a separate Python daemon ("Grox") that uses Vision-Language Models as LLM-as-judge for content moderation. Open-sourced by xAI on 2026-05-15. Source: xai-org/x-algorithm README, github.com/xai-org/x-algorithm, 2026-05-15

Provenance

xai-org/x-algorithm is the official, current source of the "For You" feed (Rust + Python, ~26K stars, last pushed 2026-05-15). It supersedes the 2023 twitter/the-algorithm release. Source: gh repo view xai-org/x-algorithm, 2026-06-17

Three sources are tracked together here:

• xai-org/x-algorithm — For You ranking + moderation (this page). [Verified: repo README + source files.]
• twitter/communitynotes — Community Notes bridging scorer → X Community Notes Scoring (Bridging Algorithm). [Verified: still actively pushed, 2026-06-17.]
• codebreaker77/X-Algo-Breakdown — a third-party (~15-star) markdown walkthrough of the repo. Useful as a reading guide, not an authority; claims below are checked against the actual source. Source: shared analysis post (codebreaker77), via User, 2026-06-17

System architecture

The feed is an orchestration of five components. Source: xai-org/x-algorithm README + top-level tree, 2026-05-15

Request flow: For You request → Home Mixer → query hydration (engagement history + user features) → candidate sources (Thunder in-network + Phoenix retrieval out-of-network) → hydration → filtering → Phoenix scoring → weighted scoring → author diversity → feed. Source: xai-org/x-algorithm README architecture diagram, 2026-05-15

The death of heuristics

The README states it directly: "We have eliminated every single hand-engineered feature and most heuristics from the system. The Grok-based transformer does all the heavy lifting by understanding your engagement history." The transformer is "ported from the Grok-1 open source release by xAI, adapted for recommendation system use cases." Source: xai-org/x-algorithm README, 2026-05-15

This is the core architectural shift from 2023: no manual weighting of follower count, account age, or historical engagement rate. The model consumes a raw sequence of a user's interactions and predicts what they will do next. The packaged demo checkpoint is a "mini Phoenix" (256-dim embeddings, 4 attention heads, 2 transformer layers, ~3 GB via Git LFS); production uses a larger Mixture-of-Experts variant. Source: xai-org/x-algorithm README § Updates, 2026-05-15

Note on the breakdown's "200+ files, ripped out all heuristics" framing: the repo's own wording is "every hand-engineered feature and most heuristics" — filtering rules (dedup, age, blocked authors, muted keywords) still exist in Home Mixer. Source: xai-org/x-algorithm README § Filtering, 2026-05-15

Scoring — weighted sum over many predicted actions

Phoenix predicts a probability for each of a set of user actions, then ranks by a weighted sum. The README diagram enumerates 15 actions: P(favorite), P(reply), P(repost), P(quote), P(click), P(profile_click), P(video_view), P(photo_expand), P(share), P(dwell), P(follow_author), plus the negative signals P(not_interested), P(block_author), P(mute_author), P(report). (The breakdown post counts "19"; the exact set lives in the model config.) Source: xai-org/x-algorithm README § Scoring, 2026-05-15

Final Score = Σ (weight_i × P(action_i))

Positive actions (like, repost, share, dwell) carry positive weights; negative actions (block, mute, report, not-interested) carry negative weights and push content down. Author diversity then attenuates repeated authors. Source: xai-org/x-algorithm README § Scoring/Ranking, 2026-05-15

Takeaway for ranking: dwell time, replies, reposts, and shares are first-class objectives, and a post that provokes blocks/mutes/reports is actively demoted — not merely un-promoted.

Grox — VLM moderation as LLM-as-judge

Grox is a standalone async Python service that pulls from Kafka streams (grox/data_loaders/kafka_loader.py) and runs classifiers over content. Instead of keyword/rule filters it uses Vision-Language Models in an LLM-as-judge pattern. Verified classifiers under grox/classifiers/content/: spam.py, reply_ranking.py, banger_initial_screen.py, safety_ptos.py, post_safety_screen_deluxe.py. Source: xai-org/x-algorithm grox/ tree + README § Grox, 2026-05-15

Three implementation details, all confirmed in grox/classifiers/content/safety_ptos.py:

1. Structured output via <json> extraction. The classifier parses model output with result_pattern = re.compile(r"(.*)<json>(.*)</json>", re.DOTALL) and validates the captured group with a Pydantic model. The conversation is built to force the model to emit a <json> block directly, bypassing conversational filler. Source: safety_ptos.py, 2026-05-15
2. Near-deterministic temperature. vlm_config.temperature = 0.000001 for classification, making the judge effectively greedy. Source: safety_ptos.py, 2026-05-15
3. Conditional Chain-of-Thought ("Deluxe Mode"). SafetyPtosPolicyClassifier(deluxe: bool = False); when deluxe is set, build_convo calls _strip_thinking_restrictions(content), which rewrites the system prompt to permit a <think> block so the model can reason about ambiguous media (e.g. violent vs. educational footage) before emitting the final JSON. Simple cases skip thinking; ambiguous policy checks escalate to it. Routing lives in task_post_safety_screen_deluxe.py / plan_post_safety.py. Source: safety_ptos.py + post_safety_screen_deluxe.py, 2026-05-15

Policies are enumerated in a SUPPORTED_POLICY_CATEGORIES set ("PTOS" — Platform Terms of Service safety categories); the breakdown's "7 policies" count is plausible but not separately verified here. Source: safety_ptos.py, 2026-05-15

Slop and "banger" screening

grox/classifiers/content/banger_initial_screen.py and task_pub.py reference a slop signal: a VLM prompt evaluates text formatting and vocabulary and assigns a slop score for low-effort / AI-generated-looking content, while a "banger" screen scores high-quality posts. Downstream this modulates reach. The actionable consequence — post that reads like LLM output gets throttled — is captured as a reusable rule in Algorithmic Slop Throttling (Posting for the X Algorithm). Source: xai-org/x-algorithm grox/classifiers/content/banger_initial_screen.py, 2026-05-15

Why it matters

• LLM-as-ranker + LLM-as-judge in production. Both the ranking core (Phoenix) and the moderation layer (Grox) are transformer/VLM driven, orchestrated alongside Rust serving infra — a concrete reference for Language Orchestration / agent-orchestration work and LLM Inference Serving Architecture.
• Prompt-engineering patterns worth reusing: assistant-prefill / <json> forcing for reliable structured output, near-zero temperature for classification, and conditional CoT gated by difficulty. See Algorithmic Slop Throttling (Posting for the X Algorithm) for the posting-side implications.

Architecture Position

---

Timeline

• 2026-07-01 | Architecture category refresh added this page to the AI serving and public algorithms family, linked it to Architecture System Map, and kept it standalone because it owns this boundary: X For You ranking architecture and Phoenix model-based recommendation flow. Source: User request, 2026-07-01

• 2026-06-17 | Captured from Kevin (paste + "store … keep in mind when xposting"). Verified the xai-org/x-algorithm repo and the Grox source against the shared codebreaker77 breakdown: Grok-1-derived Phoenix ranker, weighted multi-action scoring, <json> prefill, temperature=0.000001, _strip_thinking_restrictions Deluxe Mode, slop/banger screens all confirmed in source. Flagged breakdown embellishments ("all heuristics" → "most"; "19 actions" → 15 enumerated). Source: User, 2026-06-17; xai-org/x-algorithm, 2026-05-15

• 2026-05-15 | xAI open-sourced the For You algorithm update: end-to-end Phoenix inference pipeline, pre-trained mini model, Grox content-understanding service, ads blending, expanded query/candidate hydrators. Source: xai-org/x-algorithm README § Updates, 2026-05-15