This is a short article that demonstrates how to fine-tune a small language model without writing a single line of Python code. The fine-tuning job is dispatched on the Modal Cloud platform and uses a T4 GPU. The fine-tuned adapters are then pushed to Hugging Face, and metrics are pushed to W&B. We use a library called ** TRLoom** that provides us the ability to dispatch fine-tuning jobs with a single yaml configuration file.
Dataset used: b-mc2/sql-create-context available on huggingface. For simplicity we consider only the first 2000 items of the train split of the dataset. This can be extended based on your usage.
Base Model: HuggingFaceTB/SmolLM2–135M-Instruct. We use a 135M Instruction-tuned model.
Step 1:
Install the dependency. “all” installs wandb and modal dependencies.
pip install "trloom[all]"
Step 2:
Setup modal and authenticate via browser
python -m modal setup
Step 3:
Add secrets to Modal Cloud
python -m modal secret create wandb WANDB_API_KEY=your_key_herepython -m modal secret create huggingface HF_TOKEN=your_hf_token_here
Step 4:
So we are all set. Our environments are prepared to take up the fine-tuning job. Create a configuration file for your fine-tuning configurations. Refer to the yaml below.
Step 4.1 (Optional)
Validate the yaml:
trloom validate sft_config.yaml
You should get the response like below:
{ "ok": true, "method": "sft", "trainer": "SFTTrainer", "config_class": "SFTConfig", "experimental": false, "output_dir": <whatever the output directory is>, "wandb_enabled": true, "modal_enabled": true}
Step 5:
Run the fine tuning job:
trloom run sft_config.yaml
Your fine tuning job will be live on Modal and You will get the link of W&B runs in the Modal container logs.
After the training has been completed the Model will be pushed to hub.
The W&B runs will be visible in the project page of W&B
Thanks for reading. TRLoom is now open for contributions. A star to the repo ⭐️ would be much appreciated.
For documentation: https://saqlain2204.github.io/trloom
Github Repo: https://github.com/saqlain2204/trloom
Pypi: https://pypi.org/project/trloom/
How to Fine-Tune a Small Language Model Without Writing a Single Line of Python Code on Modal Cloud was originally published in Towards AI on Medium, where people are continuing the conversation by highlighting and responding to this story.