# Context Compression: Making AI Agents Forget Without Losing the Plot

> Source: <https://dev.to/rijultp/context-compression-making-ai-agents-forget-without-losing-the-plot-5g7a>
> Published: 2026-07-24 20:09:52+00:00

*Hello, I'm Rijul. I'm building git-lrc, a micro AI code reviewer that runs on every commit. It's free and source-available on GitHub. Star git-lrc to help more developers discover the project. Do give it a try and share your feedback*

Suppose you are working with an AI agent to fix an API.

You give it a simple instruction:

Fix the API 500 error.

The agent might go through a workflow like this:

```
Agent:
→ Reads the logs
→ Searches the codebase
→ Checks the database
→ Checks recent commits
→ Runs tests
→ Tries a fix
```

After 30 tool calls, the agent's context can become huge.

But most of that information may no longer be useful.

The agent does not need to keep every detail of the investigation forever.

It does not need:

What it really needs is the **current state of the investigation**.

Imagine the agent has accumulated **40,000 tokens of context** during the investigation.

That context takes up valuable space.

As the agent continues working, the context window gradually fills up.

Eventually, the agent may have less room for new information, which can lead to:

So instead of carrying the entire history forward, we can compress it.

The original 40,000 tokens might become:

```
Goal:
Fix the API 500 error.

Found:
The error started after deployment v1.4.
The database is healthy.
The payment service is missing PAYMENT_API_KEY.

Tried:
Restarting the service. No effect.

Next:
Fix the environment configuration and retest.
```

This is **context compression**.

Context compression is not simply about making the context shorter.

The goal is to:

Remove the information that is no longer useful while preserving what the agent needs to continue working.

The agent does not need to remember every step it took.

It needs to remember the important conclusions from those steps.

The simplest technique is **pruning**.

You remove information that is no longer useful.

For example:

```
Agent:
→ Searched for config.yaml
→ Found nothing

Agent:
→ Searched for settings.yaml
→ Found nothing

Agent:
→ Searched environment variables
→ Found PAYMENT_API_KEY is missing
```

Once the agent has found the actual cause, the failed searches may no longer be useful.

They can be removed from the active context.

The important information is:

```
PAYMENT_API_KEY is missing.
```

Pruning is essentially:

Remove what the agent no longer needs.

Instead of keeping the entire conversation, we can convert it into a structured summary.

For example:

```
Goal:
Fix the API 500 error.

Facts:
- The database is healthy.
- The error started after deployment v1.4.
- PAYMENT_API_KEY is missing.

Decisions:
- Do not modify the database.
- Fix the environment configuration.

Completed:
- Checked the logs.
- Verified database connectivity.
- Inspected environment variables.

Next Action:
- Add PAYMENT_API_KEY.
- Restart the service.
- Retest the API.
```

The original investigation may have taken thousands of tokens.

But the distilled state contains the information the agent needs to continue.

A useful structure might be:

```
Goal
Facts
Decisions
Completed Work
Next Action
```

Distillation is essentially:

Turn a long history into a structured state.

Sometimes an investigation contains knowledge that can be reused in future situations.

For example:

```
Specific experience:

Missing API key caused the payment API to fail.
              ↓
Reusable knowledge:

Check environment variables when an API integration fails.
```

The agent is no longer just remembering what happened in one specific incident.

It is extracting a general rule from that experience.

This can be useful for:

Generalisation is essentially:

Turn a specific experience into reusable knowledge.

An AI agent does not need to carry its entire history forever.

It needs to preserve the parts of that history that are still useful.

A long investigation might look like this:

```
40,000 tokens of raw history
              ↓
       Context compression
              ↓
      Goal + Facts + Decisions
              ↓
          Next Action
```

The agent can then continue working with a much smaller context while retaining the information that actually matters.

**Context compression is not about forgetting everything.**

It is about forgetting the right things.

AI agents write code fast. They also silently remove logic, change behavior, and introduce bugs -- without telling you. You often find out in production.

[git-lrc](https://github.com/HexmosTech/git-lrc) fixes this. It hooks into git commit and reviews every diff before it lands. 60-second setup. Completely free.

Any feedback or contributors are welcome! It's online, source-available, and ready for anyone to use.

Give it a ⭐ [star on Github](https://github.com/HexmosTech/git-lrc)