Revision Prompting: improves industrial LLM processes

Revision Prompting, a technique introduced by an unnamed author, improves industrial LLM processes by operating on input and output revisions instead of full re-runs, reducing token costs and ensuring consistency. The method constructs a prompt that includes the original input, output, and a diff of the updated input, prompting the LLM to generate a patch that updates the output. An example shows translating an e-bike product page from English to German, where a battery upgrade from 80 km to 100 km range is handled with a two-line patch instead of a full re-translation.

The Problem that Revision Prompting solves We prompt LLMs in two ways: - Ad-hoc prompting - Prompts LLMs manually, with a custom instruction per call. Examples - Asking a coding agent to implement a new feature. - Asking a chatbot to draft an email. - Industrial prompting - Prompts LLMs as part of an automated process, with the same instruction across calls. Examples - Extracting structured information from invoices as part of an accounting pipeline. - Translating documentation pages into other languages as part of a release process. Industrial prompting typically processes some Input data with an Instruction to produce some Output . Whenever the Input gets updated, industrial prompting naively re-runs the Instruction on the UpdatedInput to produce the UpdatedOutput . This approach has two downsides: - Lack of consistency - LLMs are non-deterministic, so the UpdatedOutput differs from the original Output beyond what the UpdatedInput necessitates. - Full processing time and token costs - Although only parts of the input have changed, we produce the UpdatedOutput in full. This is as expensive as the production of the original Output . Revision prompting resolves both downsides of naive re-runs by operating on the input and output revisions instead of the full input and output. How Revision Prompting works Assume you have processed some Input with an Instruction to produce some Output by prompting an LLM with Instruction : Input Now, Input has been updated, and you also want to process the UpdatedInput . Revision Prompting processes the UpdatedInput by constructing the RevisionPrompt as Instruction : Input produced Output . The input got updated as follows: diff . Input , UpdatedInput Please produce a patch to update the output. The LLM responds to the RevisionPrompt with the OutputPatch that we apply to the Output to obtain the UpdatedOutput . Example You translate the product page of an e-bike to German with the prompt The Vela 3 e-bike has a range of 80 km. Its battery recharges in three hours. The frame is made from recycled aluminium. Every Vela 3 includes a two-year warranty. and the LLM produces Das E-Bike Vela 3 hat eine Reichweite von 80 km. Sein Akku lädt in drei Stunden auf. Der Rahmen besteht aus recyceltem Aluminium. Jedes Vela 3 hat zwei Jahre Garantie. Later, a battery upgrade increases the range from 80 km to 100 km. Instead of re-translating the whole page, you prompt The Vela 3 e-bike has a range of 80 km. Its battery recharges in three hours. The frame is made from recycled aluminium. Every Vela 3 includes a two-year warranty. produced Output Das E-Bike Vela 3 hat eine Reichweite von 80 km. Sein Akku lädt in drei Stunden auf. Der Rahmen besteht aus recyceltem Aluminium. Jedes Vela 3 hat zwei Jahre Garantie. The input got updated as follows: diff Input, UpdatedInput - The Vela 3 e-bike has a range of 80 km.+ The Vela 3 e-bike has a range of 100 km. Please produce a patch to update the output. The LLM responds with Applying the OutputPatch to the original Output produces the updated translation. The OutputPatch contains only two lines of text instead of a full re-translation. Unchanged content stays consistent with the original translation. Why Revision Prompting works - Consistency - By supplying the LLM with the input revision diff , we ensure that the Input , UpdatedInput OutputPatch is limited to what the input changes necessitate. Everything not touched by the OutputPatch remains identical to the original Output . Therefore, the UpdatedOutput is consistent with the original Output . - Time & cost savings - Revision Prompting feeds the original Output back in as part of the prompt, so the LLM only generates the short OutputPatch . Most tokens thereby move from the output to the input. Since processing time scales roughly with the length of the output, this eliminates most of the processing time. It also converts most of the output token cost into much cheaper input token cost. If the re-run happens within a couple of minutes of the original run, prompt caching reduces part of the input token cost as well. Revision Prompting in practice - Revision Formats - The ideal formats for encoding diff and the Input , UpdatedInput OutputPatch depend on the Instruction . The POSIX https://pubs.opengroup.org/onlinepubs/9799919799/utilities/diff.html is a useful generic format. For JSON outputs, the diff utility JSON Patch https://jsonpatch.com/ format works well. - Expected Savings - The time & cost reduction scales with the size of the input changes and the sensitivity of the output to changes in the input. In our own industrial prompts, Revision Prompting reduces time by ~80%, and costs by ~65%.