{"slug": "programmatic-memory-for-long-horizon-llm-agents", "title": "Programmatic memory for long-horizon LLM agents", "summary": "PRO-LONG, a minimal memory addition for LLM agents that appends every observation, action, and outcome to a structured log.txt and retrieves it programmatically, improves performance on the ARC-AGI-3 public game set by 18 percentage points on average over the same coding agents without the log, matches or exceeds specialized harnesses at 4.2–5.8x fewer billed tokens, and reaches 97.4% best@2 with Fable 5 at a total cost of $1,750, according to a paper released by the developers on arXiv.", "body_md": "PRO-LONG is a minimal memory addition for LLM agents on long-horizon tasks. The harness appends every observation, action, and outcome to a single structured log.txt, and the agent retrieves and reasons over it programmatically (grep, Python). There are no subagents or specialized retrieval mechanisms, and the system prompt is about 30 lines.\n\nOn the full [ARC-AGI-3](https://three.arcprize.org/) public game set, PRO-LONG improves over the same coding agents without the log by 18 percentage points on average, matches or exceeds specialized harnesses at 4.2–5.8x fewer billed tokens, and reaches **97.4% best@2 with Fable 5 at a total cost of $1,750.**\n\n**Paper:** [arxiv.org/abs/2607.20064](https://arxiv.org/abs/2607.20064)\n\nRequires Python (3.12 recommended) and Docker.\n\n```\ngit clone git@github.com:alexisfox7/PRO-LONG.git\ncd PRO-LONG\npython -m venv .venv\nsource .venv/bin/activate\npip install -e .\n\n# codex backend\ndocker build -t prolong-agent/codex-sandbox:latest docker/codex-sandbox\ndocker build -t prolong-openai-proxy docker/openai-proxy\n\n# claude-code backend\ndocker build -t prolong-agent/claude-sandbox:latest docker/claude-sandbox\ndocker build -t prolong-anthropic-proxy docker/anthropic-proxy\n```\n\nCreate a `.env`\n\nfile:\n\n```\nARC_API_KEY=...\nCODEX_API_KEY=...              # codex backend\nCLAUDE_CODE_OAUTH_TOKEN=...    # claude-code backend (default)\nANTHROPIC_API_KEY=...           # claude-code backend with --api-key\n```\n\nThe agent container only mounts the game workspace and, by default, has no network access except a proxy to the model API.\n\n```\nprolong-swarm --suite all -m gpt-5.5 --max-actions 500\nprolong-swarm --suite all --backend claude-code -m claude-opus-4-6\nprolong-swarm --game ls20,ft09 -m gpt-5.5\n```\n\nResults are written to `evaluation_results/`\n\n.\n\n| Flag | Default | Description |\n|---|---|---|\n`--backend` |\n`codex` |\n`codex` (OpenAI Codex CLI) or `claude-code` (Claude Code CLI) |\n`--suite` |\n— | Game suite (`all` only) |\n`--game` |\n— | Comma-separated individual game names or full IDs |\n`--max-actions` |\n500 | Max actions per game |\n`--model` , `-m` |\nBackend-specific | `gpt-5.5` for Codex; `claude-opus-4-6` for Claude Code |\n`--effort` |\n`high` |\nEffort level (claude-code backend) |\n`--reasoning-effort` |\n`none` |\nReasoning effort (codex backend) |\n`--operation-mode` |\n`online` |\n`online` / `offline` / `normal` |\n\nThe agent's access to game history is controlled by `--log-window`\n\nand `--workspace`\n\n. These are the ablation conditions from the paper:\n\n| Condition | Flags | History available |\n|---|---|---|\n| prolong | (default) | Full game log |\n| lw25 | `--log-window 25` |\nLast 25 action sections of the log |\n| no-log (in-prompt) | `--log-window -1` |\nNo log file; the current board is added to the prompt |\n| stateless | `--workspace stateless` |\nFull log, but the workspace is wiped each call |\n\n`scorecards/`\n\ncontains the official online scorecards, including all 25 Fable 5 runs from the paper (`fable_online_scorecards.txt`\n\n); each can be verified on arcprize.org. `release_logs/`\n\ncontains logs for the Fable 5 online runs: game logs, agent transcripts, and workspaces. Logs for the remaining ablations will be added.\n\n```\nprolong_agent/\n├── agent/\n│   ├── base.py               # base architecture\n│   ├── codex_agent.py        # Codex CLI backend\n│   ├── claude_code_agent.py  # Claude Code backend\n│   ├── swarm.py              # CLI entry point\n│   ├── action_queue.py       # action execution\n│   ├── game_state.py         # board/log formatting\n│   └── prompts.py            # prompts (~30 lines)\n├── environment/\n│   ├── arcagi3.py            # ARC-AGI-3 API wrapper\n│   ├── runner.py             # per-game loop\n│   └── config.py\n├── metrics/\n└── utils/\n```\n\nThis repo was formerly the Read-Grep-Bash (RGB) Agent, see our original [blog post](https://blog.alexisfox.dev/arcagi3) on the ARC-AGI-3 preview games.\n\n```\n@misc{fox2026prolong,\n  title={PRO-LONG: Programmatic Memory Enables Long-Horizon Reasoning},\n  author={Fox, Alexis and Wang, Junlin and Rosu, Paul and Dhingra, Bhuwan},\n  year={2026},\n  eprint={2607.20064},\n  archivePrefix={arXiv},\n  primaryClass={cs.AI},\n  url={https://arxiv.org/abs/2607.20064},\n}\n```\n\n", "url": "https://wpnews.pro/news/programmatic-memory-for-long-horizon-llm-agents", "canonical_source": "https://github.com/alexisfox7/PRO-LONG", "published_at": "2026-08-17 10:57:44+00:00", "updated_at": "2026-08-17 11:11:27.937751+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-agents", "ai-research"], "entities": ["PRO-LONG", "ARC-AGI-3", "Fable 5", "OpenAI Codex", "Claude Code", "arXiv"], "alternates": {"html": "https://wpnews.pro/news/programmatic-memory-for-long-horizon-llm-agents", "markdown": "https://wpnews.pro/news/programmatic-memory-for-long-horizon-llm-agents.md", "text": "https://wpnews.pro/news/programmatic-memory-for-long-horizon-llm-agents.txt", "jsonld": "https://wpnews.pro/news/programmatic-memory-for-long-horizon-llm-agents.jsonld"}}