# Colab free tier killed my 7B fine-tune — here's the autopsy

> Source: <https://promptcube3.com/en/threads/7074/>
> Published: 2026-08-20 17:01:03+00:00

# Colab free tier killed my 7B fine-tune — here's the autopsy

## The setup

- Base model: Mistral-7B-v0.1 (4-bit quantized via bitsandbytes)
- Dataset: 12k Alpaca-format examples (~400MB tokenized)
- Method: LoRA rank 16, alpha 32, targeting q_proj/v_proj
- Colab: Free tier, T4 GPU (16GB VRAM), 12GB RAM, 2-hour disconnect

``` python
# The config that seemed reasonable on paper
from transformers import TrainingArguments

training_args = TrainingArguments(
    output_dir="./mistral-7b-lora",
    per_device_train_batch_size=1,
    gradient_accumulation_steps=16,
    num_train_epochs=3,
    learning_rate=2e-4,
    fp16=True,
    optim="paged_adamw_8bit",
    logging_steps=10,
    save_steps=500,
    max_steps=1500,  # ~2 hours at this rate
)
```

## What broke

**First run:** OOM at step 47. The gradient accumulation buffer + optimizer states + model weights pushed past 16GB. T4 has 16GB but Colab's overhead eats ~2GB before you start.

**Second run:** Switched to `gradient_checkpointing=True`

, dropped batch to 1, accumulation to 32. Made it to step 200 before the runtime disconnected. Colab free kills at exactly 2 hours — no warning, no checkpoint recovery.

**Third run:** Added `save_steps=100`

and `save_total_limit=3`

. Got to step 800. Then the 12GB system RAM filled up (dataset caching + tokenizer + intermediate tensors) and the kernel died with `MemoryError: Unable to allocate 1.2 GiB`

.

## The actual limits I hit

| Resource | Free tier | What I needed |

|----------|-----------|---------------|

| VRAM | 16GB (T4) | ~14GB usable for 7B 4-bit + LoRA |

| System RAM | 12GB | 16GB+ for dataset + overhead |

| Max session | 2 hours | 4-6 hours for 3 epochs |

| Disk | ~70GB | Fine, not the bottleneck |

## What *does* work on free Colab

**1.5B-3B models** full fine-tune (Phi-2, TinyLlama, Gemma-2B)**7B LoRA** with aggressive quantization (4-bit +`max_memory={0: "13GiB"}`

) + dataset streaming**Inference only**— 7B-13B 4-bit runs fine for generation

``` python
# Streaming dataset avoids RAM explosion
from datasets import load_dataset

dataset = load_dataset("json", data_files="train.jsonl", split="train", streaming=True)
dataset = dataset.shuffle(buffer_size=1000, seed=42)
```

## My workaround (not pretty)

Ended up renting a RunPod A100 40GB for $1.10/hr. Same code, finished 3 epochs in 47 minutes. Cost: ~$0.85 total.

Colab Pro ($10/mo) gets you T4/V100 priority and 24hr sessions — might work for 7B LoRA if you're patient. But free tier? Save yourself the grief.

**TL;DR:** Free Colab cannot reliably fine-tune 7B models. The 2-hour hard limit + 12GB RAM ceiling + 16GB VRAM (shared) is a triple constraint. Use it for inference, small models, or prototyping — not production fine-tunes.

[Next Debugging the timeout cascade that killed our UPI integration →](/en/threads/6976/)