I am experimenting with a small syntax/workflow idea for multi-step LLM work.
The problem I keep running into is that the useful part of an LLM workflow is not just the prompt. It is the trail around it:
Chat is good for exploration, but it is a weak record. Scripts are reproducible, but often too rigid for work that starts exploratory.
The shape I am testing is a local-first notebook where each step is an “intent cell”. I am calling the syntax ICC DSL, for Intent-Cell Coding.
A cell keeps readable task text together with execution details:
c1 Collect
> auto
@file -markdown context.md
c2 Review
> fast
%from c1
@file -json review.json
c3 Final
> best
%from c1
%from c2
@file -markdown final.md
The part I am thinking hardest about is file/context identity. A rerun is not really comparable unless the system records exact input versions, hashes, snapshots, retrieval results, or upstream artifact refs.
I am curious how people here think about reproducibility for LLM demos and workflows.
Do you usually keep this state in notebooks, app logs, model/dataset cards, traces, config files, or generated run manifests?