JITI kernel, OpenAI generations, and recovery ASCII diagrams

A developer built a kernel that keeps a running Lisp world alive while an external controller queries an OpenAI model for the next action to try. The model never edits memory directly; it proposes a single validated action that an SBCL worker checks against generation tags and invariants before accepting or restoring a checkpoint, rejecting stale proposals as out-of-date.

The project keeps a running Lisp world alive while an outside controller asks an OpenAI model what to try next. "What should we try?" | v +------------------+ +------------+ +------------------+ | Running Lisp | ---- | Controller | ---- | OpenAI Responses | | world + kernel | view | | prompt| API | +--------+---------+ +------+-----+ +--------+---------+ ^ | | | | action | | v | | +------+-----+ | +-----------------+ SBCL worker|<---------------+ result / state | | JSON proposal +-------------+ The model never directly edits memory. It proposes one small, validated action; the worker decides whether that action is safe and current. +----------------------------------------------------------------+ | Lisp world | | | | data table function definitions application state | | +-----------+ +-------------------+ +----------------+ | | | :x = 0 | | defun add ... | | counters, ... | | | +-----------+ +-------------------+ +----------------+ | | | | The world adapter knows how to: | | observe it checkpoint it restore it | | export it import it record accepted forms | +----------------------------------------------------------------+ The kernel manages only what the world adapter can describe. External files, arbitrary threads, and other side effects need their own adapter support. php +--------+ +-----------+ +------------+ +----------+ | Observe| - | Propose | - | Validate | - | Execute | | world | | one action| | generation | | in worker| +--------+ +-----------+ +------------+ +----+-----+ | v +--------+--------+ | Check invariants| +--------+--------+ | +-------------------+-------------------+ | | v v +----+-----+ +-----+----+ | Accept | | Restore | | revision | | checkpoint| +----------+ +----------+ php Time ------------------------------------------------------------- Kernel: view at generation 7 ------- state changes ------- generation 8 | ^ | | OpenAI: receives view 7 -------- returns action tagged 7 ---+ | Worker: sees current generation 8 | | | +-------------------- reject as stale --------------+ A generation is like a CS50 problem-set version number. The answer must match the version of the question that produced it. Worker stack still alive +------------------------------+ | worker-main | | +------------------------+ | | | evaluate application | | | | error | |<---- condition is signaled | +-----------+------------+ | | | | | v | | +------------------------+ | | | condition-loop | | | | restart menu: | | | | 0/0 use-value | | | | 0/1 retry | | | +-----------+------------+ | +--------------|---------------+ | v worker pauses and waits for controller A resume action selects a restart ID and supplies a Lisp list of arguments. The restart exists only while this worker is paused; it is not saved across a process restart. checkpoint C | v +--------+--------+ | outer evaluation| | | | repair 1 | | repair 2 | | resume call | +--------+---------+ | +---------+---------+ | | v v all checks pass any failure | | v v keep changes restore C The outer call and repairs share one provisional checkpoint. A failed repair rolls back the entire attempt, preventing half-applied changes. candidate world | +---------+----------+ | | v v Safety invariants Goal predicates "must never break" "what we want eventually" | | v v failure = reject failure = keep working | v all goals pass = success This allows useful intermediate revisions: a candidate can be incomplete while still being safe. Process A Disk Process B --------- ---- --------- accepted world ---- export ---- revision-123/ manifest.sexp exported world | +-- CURRENT = revision-123 crash | v recover-session | +--------------+--------------+ | | v v read CURRENT read diagnostics | | v v import revision keep recent history | | +--------------+--------------+ | v fresh worker, fresh stack Recovery restores accepted managed code and data. It does not replay unfinished Lisp forms or pretend that an old call stack survived the crash. store/ ├── CURRENT points to the accepted revision ├── revision-123/ │ ├── manifest.sexp identity, parent, SBCL version, events │ └── exported world managed data and reconstructible code └── events.sexp diagnostic attempts and conditions CURRENT is authoritative. A torn final diagnostic record is archived during recovery and cannot replace the accepted revision. php observe - propose - generation-check - checkpoint - execute - pause/repair if needed - safety-check - accept or restore - publish accepted revision - observe again Suppose the running application begins with one rule: price = 100.00 rate = 5% tax = 5.00 total = 105.00 The user asks: Create a sales-tax calculator using a 5% tax rate. progn defun sales-tax price price 0.05 defun total-price price + price sales-tax price view generation 0 | v checkpoint world | v install definitions | v safety checks pass | v accept revision 1, generation 1 js sales-tax 100.00 = 5.00 total-price 100.00 = 105.00 The user asks: Support different tax rates. progn defun sales-tax price rate price rate defun total-price price rate + price sales-tax price rate php revision 1 | | checkpoint v try new definitions | +-- checks pass -- accept revision 2 | +-- checks fail -- restore revision 1 js total-price 100.00 0.05 = 105.00 total-price 100.00 0.08 = 108.00 The user asks: Use different rates for food and non-food items. progn defun sales-tax price rate price rate defun total-price price item-type let rate if eq item-type :food 0.02 0.08 + price sales-tax price rate +------------------+ | total-price | | price, item-type | +--------+---------+ | v item-type = :food? / \ yes no | | rate 0.02 rate 0.08 \ / v v price + price rate js total-price 100.00 :food = 102.00 total-price 100.00 :clothing = 108.00 candidate calculator | +---------+----------+ | | v v goal predicates safety invariants desired behavior must-never-break rules | | v v food total is 102.00 tax is never negative | | v v incomplete goal failure rejects attempt means keep working For example: ;; Goal: desired behavior lambda and = total-price 100.00 :food 102.00 = total-price 100.00 :clothing 108.00 ;; Safety invariant: never accept a negative tax lambda and = sales-tax 100.00 0.02 0 = sales-tax 100.00 0.08 0 If a proposal accidentally makes food tax negative: defun total-price price item-type + price if eq item-type :food price -0.02 price 0.08 revision 2 | v checkpoint C | v run bad proposal | v safety invariant fails | v restore C | v revision 2 is still live The next user request starts from the last accepted calculator, not from the half-applied bad change. php +------------+ +------------+ +------------+ | Revision 1 | ---- | Revision 2 | ---- | Revision 3 | | fixed 5% | | caller rate| | food rules | +------------+ +------------+ +------------+ | | | v v v total 100 total 100,.08 total 100,:food 105 108 102 accepted revision 3 | v process crashes | v read CURRENT | v load revision 3 | v start fresh worker | v food and non-food behavior returns The accepted calculator and managed data return from disk. The old call stack and live restart objects do not; those exist only inside the original worker.