In this video I continue the fine-tuning series on my channel. This time I go through the whole pipeline, not just the training part: renting GPUs on Runpod, fine-tuning a model with Unsloth Studio, deploying it as an inference endpoint using Runpod's Serverless, and then actually using that endpoint inside a Pydantic AI Agent.
This is meant to cover the full path: rent the GPU, train the model, deploy it, and get it hooked up to something that can actually call it.
I go in-depth on creating a Pod (a dedicated GPU instance container), picking a GPU, setting up storage, and getting SSH access working. The whole setup also works through the accompanying Jupyter Notebook, but I show the terminal option too.
One thing I didn't call out clearly enough in the video: run apt update && apt upgrade -y right after you SSH in, before installing anything else. A fresh Pod's package index is often out of date, so skipping this can mean installs failing or pulling older versions of tools than you'd expect.
Before you get into training, it's worth understanding how storage works on a Pod, because it's easy to get caught out. You've got 3 options:
Volume disk is usually what's mounted on your /workspace directory.
If you install Unsloth Studio without setting UNSLOTH_STUDIO_HOME first, like this:
export UNSLOTH_STUDIO_HOME=/workspace
Unsloth Studio will quietly ignore /workspace and fall back to /root/.unsloth/studio instead, which is outside the volume disk and sitting on the container disk. You won't notice anything's wrong until you stop the pod to save some money, come back the next day, and your training setup is gone. Ask me how I know!
I didn't go in-depth on optimizing Runpod usage in the video itself, but a couple of things are worth knowing:
Once the model was trained and deployed, the last piece was actually using it. I set up a Pydantic AI agent that calls the fine-tuned model through the endpoint. I went with Pydantic AI's OpenAI provider for this, since vLLM (the inference framework running behind the endpoint) supports the OpenAI response format.
Fine-tuning is best when you're trying to modify the behavior of a model, like response style, tone etc. You can also teach it new knowledge, but even after fine-tuning, models can still hallucinate or fall back on old data. For anything that needs to stay accurate and up to date, pairing your fine-tuned model with Retrieval Augmented Generation (RAG) is still the safer route, since RAG pulls in fresh, sourced info at request time instead of relying on whatever got baked in during training.
If you enjoyed this, feel free to subscribe to the channel.
Thanks and happy coding!