# EVERYBODY IS EVERYONE — An Open Experiment in AI Memory and Continuity

> Source: <https://discuss.huggingface.co/t/everybody-is-everyone-an-open-experiment-in-ai-memory-and-continuity/192162#post_8>
> Published: 2026-11-22 04:06:36+00:00

**Recommendation:** Trace one accepted correction end to end — accepted state, committed IDs, retrieved IDs, delivered prompt and raw answer — and score crate, caution, courier and cited event IDs separately rather than as one success rate.
More in the playbook "Evals and testing for LLM apps": <https://wpnews.pro/playbooks/evals-testing>

For now, I ran a few small experiments:

The question I wanted to examine was your harder one: **if a decision is corrected, the system is restarted, and the old answer comes back, which record reached the model, and what did the model do with it?** I started from the Cedar/Lark example and the four resumption conditions in your [public harness packet](https://huggingface.co/datasets/Everyone-Belongs/dag-harness/tree/main), then built a small independent event-tracing harness. These are third-party, controlled experiments, **not runs of Aria’s own runtime**.

**A concrete old-answer case:** I deliberately left the SQLite event history containing **E1, E2, E3**, including the accepted change from **Cedar** to **Lark**, while making a separate search view expose **E1 only**. The newly launched reader gave the model E1; it answered **Cedar**. With the full, up-to-date view, it received E1/E2/E3 and answered **Lark**. The old answer was faithful to the limited evidence it saw, but wrong for the currently accepted decision. This was an **intentionally injected index-lag condition**, not a defect observed in SQLite or Aria.

That distinction seems directly useful for your question. A returned old answer can mean the correction was never accepted or committed; it was stored but did not survive retrieval or prompt assembly; or it reached the model and lost a conflict with older evidence. Those cases have different next checks. Keeping an older historical event is not itself a failure: a question about *what was originally decided* may correctly require Cedar.

**My suggested default is a trace for just one correction**, not a large new benchmark. Capture the accepted/current revision, the IDs actually committed, the IDs retrieved or represented in a checkpoint, the IDs and text delivered in the final model input, and the raw response. Your existing results sheet already asks for the crucial last two observations. If the system lacks a distinct index, skip that field; if a checkpoint replaces raw records, include its source IDs.

```
Accepted state:       E2, Lark
Canonical history:    E1, E2, E3
Searchable view:      E1             [intentionally stale]
Selected/delivered:   E1
Model answer:         Cedar
Answer citations:     none

Control: canonical E1,E2,E3; delivered E1,E2,E3; answer Lark
```

The two controlled runs should not be merged into one success rate:

| Experiment | Audited observation | 
|---|---|
| Four published resumption views, each repeated with four crate-name pairs, 16 fresh prompts | Correct crate for the **supplied view: 16/16** ; caution 13/16; pending task 15/16; all three fields 12/16 | 
| SQLite canonical events, separately reopened reader, derived FTS5 view, delivered prompt, model answer, 11 conditions | Answer matched its **delivered view: 11/11** ; crate support cited 10/11; complete field-level evidence cited 9/11 | 

Both runs used **Qwen2.5-3B-Instruct on a Colab L4**, fixed settings, and one attempt per condition. In the event trace, each reader interpreter restarted, **not the model process**. Different prompts and grading prevent treating 12/16 versus 11/11 as an improvement comparison. In the first run, an initial automatic 9/16 wrongly rejected citation-suffixed crate names; the corrected raw-answer audit is **12/16**.

The simplest decision path is: if E2 was not durably committed, inspect the write; if it is committed but absent from selection, inspect the projection/search; if selected but absent from the actual model input, inspect context assembly; if delivered and the answer is still Cedar, compare **E2 alone** with **E1 plus E2**. Finally, score the crate, the latch caution, the unassigned courier, and the supporting event IDs separately. A correct label need not be a complete operational handoff.

After an accepted correction, **one sanitized trace** of commit, selection, delivery, and reply can say more than an aggregate success percentage. If E2 never reached the model, investigate why; if it was delivered but Cedar won, test reader-level conflict. If a historical answer is wrong for today, keep both evaluation labels; if the crate is right but the warning or next task disappeared, score those fields separately.
