{"slug": "why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai", "title": "Why Fine-Tuning Is No Longer Your First Choice for Custom AI?", "summary": "Harvey, a legal AI company, found in 2025 that seven general-purpose Frontier AI models surpassed its custom fine-tuned model on its own legal benchmark, despite no custom legal fine-tuning. Bloomberg's BloombergGPT, trained from scratch, was outperformed by GPT-4 and ChatGPT on many financial benchmarks. These results suggest that fine-tuning is no longer the first choice for custom AI, as general models have caught up and techniques like RAG offer alternatives.", "body_md": "Is fine-tuning large language models still needed today?\n\nLet’s take a well-known legal AI company called Harvey. Back in 2023, they fine-tuned their own model — their own custom AI — in partnership with OpenAI. In blind tests, attorneys preferred this fine-tuned model over the Frontier model at the time, which was GPT-4. They preferred it **97%** of the time. A win for fine-tuning: they built a custom AI that lawyers actually preferred over the off-the-shelf leading model.\n\nSo the lesson is, if a general-purpose model isn’t quite right for some specific use case like legal work, then fine-tune it. Right?\n\nFirst, let’s define what fine-tuning actually is.\n\nWe start with a base model, a base LLM. This is what comes off the shelf — either working with a Frontier lab directly or picking an open-source model. The base model is trained on a massive amount of data; effectively, we scrape information from the internet and use it to train the base model, so there’s a lot of general knowledge baked into the model's weights.\n\nFine-tuning takes that base model and customizes it by continuing its training, but now on a much more focused dataset — some specific documents really focused in one particular area. It might be legal contracts, or an organization’s internal support tickets: the stuff that isn’t sitting around on the internet waiting to be scraped by base models.\n\nThe result is a new model, a fine-tuned model, and that fine-tuned model incorporates the information from the focused dataset plus all of the weights from the base model, now adjusted with that additional data. This resulting model should get better at certain narrow tasks. That is the wonder of fine-tuning.\n\nBut in practice, how well does it work?\n\nLet’s go back to that legal AI company Harvey. In 2025, they created their own legal benchmark to measure how effective their models were at performing particular tasks, and they tested their fine-tuned system against the latest crop of Frontier AI models — the custom model against a bunch of general-purpose Frontier AI models available at the time.\n\nThe result? **Seven of the general-purpose models had now surpassed the company’s custom model on the benchmark** — models that had never received any custom legal fine-tuning, yet were still better.\n\nBloomberg saw something similar. They famously trained BloombergGPT from scratch, and later evaluations found GPT-4 and ChatGPT outperforming BloombergGPT on many financial benchmarks.\n\nSo where does that leave fine-tuning today? Is all that custom training worth doing when big frontier general models keep getting smarter on their own?\n\nTo answer that, it’s worth considering how general models have, in many cases, caught up to custom-trained ones. There are a few reasons.\n\nGeneral models have gotten better — but if we’re not adjusting weights, how do we make a general model behave like a specialist, like a legal scholar, for example? It turns out there’s a whole stack of customization techniques that don’t touch the model weights at all.\n\n**The first is RAG, retrieval-augmented generation.** Instead of training the documents into the model, the application retrieves — that’s the R in RAG — the documents at query time and then feeds them into the prompt.\n\n```\n# Minimal RAG: retrieve relevant chunks at query time, then feed them into the promptfrom openai import OpenAIclient = OpenAI()def answer_with_rag(question: str, vector_store, k: int = 5) -> str:    # R — retrieve the most relevant documents for this specific query    docs = vector_store.similarity_search(question, k=k)    context = \"\\n\\n\".join(d.page_content for d in docs)    # A + G — augment the prompt with that context, then generate    prompt = (        \"Answer the question using only the context below.\\n\\n\"        f\"Context:\\n{context}\\n\\n\"        f\"Question: {question}\"    )    resp = client.chat.completions.create(        model=\"gpt-5\",        messages=[{\"role\": \"user\", \"content\": prompt}],    )    return resp.choices[0].message.content\n```\n\nIn practice, this means the model never “learns” your documents — it simply reads the most relevant ones fresh on every query, so your knowledge base can change without ever retraining a thing.\n\n**There’s also the consideration of context, specifically context engineering.** The idea is that a good prompt is a carefully assembled bundle of context: the system prompt, the relevant data, maybe some format guidelines and the like, all packaged together.\n\n```\n# Context engineering: assemble the prompt as a deliberate bundle of contextdef build_context(system_prompt: str, retrieved_data: str, format_rules: str, user_query: str):    return [        {\"role\": \"system\", \"content\": system_prompt},          # who the model should be        {\"role\": \"system\", \"content\": f\"Relevant data:\\n{retrieved_data}\"},  # grounding        {\"role\": \"system\", \"content\": f\"Output format:\\n{format_rules}\"},    # guardrails        {\"role\": \"user\",   \"content\": user_query},              # the actual ask    ]\n```\n\nNotice there’s no training here at all — the “specialisation” comes entirely from how deliberately the prompt is assembled, not from the weights.\n\n**The third thing to consider is agent skills** — those MD files you can create. Skills are folders of files that package up procedural knowledge: basically how to do something, and the tools to use to do it. The model loads them on demand when it sees a task that calls for them. So instead of fine-tuning a model to know how to write SQL queries against a very specific schema, a SQL agent skill can tell the model exactly what to do — and any general-purpose model can use that skill.\n\nHere’s what a minimal SQL agent skill might look like — a simple Markdown file that hands the model the schema, the rules, and the tool it needs:\n\n```\n---name: sql-reporting-agentdescription: Write and run SQL against the analytics warehouse schema.---# SQL Reporting Skill## Schema- orders(id, customer_id, amount, status, created_at)- customers(id, name, region, signup_date)## Rules- Always filter out status = 'cancelled' for revenue queries.- Use explicit JOINs, never comma joins.- Return at most 1000 rows unless asked otherwise.## Tools- run_sql(query: str) -> table   # executes read-only SQL and returns rows## Procedure1. Restate the question as a metric + dimensions + time range.2. Draft the SQL using the schema above.3. Validate column names against the schema before running.4. Call run_sql, then summarise the result for the user.\n```\n\nSo essentially, fine-tuning isn’t the only path to customization. There’s a whole stack of options that work without ever touching model weights.\n\nHaving all these no-weight options is a good thing, because fine-tuning is not free. In addition to the training run itself, there’s a cost in collecting the examples, evaluating results, and avoiding regressions — plus a cost of maintaining the custom models as the frontier models move on.\n\nAll of this begs the question: does anyone still need to fine-tune at all?\n\nYes — but for a much narrower set of reasons than back in 2023.\n\nThere’s a modern technique called **LoRA**, low-rank adaptation, that lets a team fine-tune by training a small adapter that sits on top of an existing base model. We’ve got the base model with its weights, and then this adapter that sits on top of it. Most of the original weights stay locked. In fact, a lot of what gets labeled as fine-tuning in production today is some flavor of LoRA or a related parameter-efficient method.\n\n```\n# LoRA fine-tuning: train a small adapter, keep the base weights frozenfrom peft import LoraConfig, get_peft_modelfrom transformers import AutoModelForCausalLMbase = AutoModelForCausalLM.from_pretrained(\"base-llm-7b\")lora_config = LoraConfig(    r=8,                       # rank of the adapter (small = few extra params)    lora_alpha=16,    target_modules=[\"q_proj\", \"v_proj\"],    lora_dropout=0.05,    task_type=\"CAUSAL_LM\",)model = get_peft_model(base, lora_config)model.print_trainable_parameters()# Only the adapter is trainable; the original weights stay locked.\n```\n\nNotice that only the small adapter is trainable while the base model stays frozen — which is exactly why LoRA is so much cheaper and faster than full fine-tuning.\n\nFine-tuning does still make sense in certain situations:\n\n```\n# Reinforcement fine-tuning (RFT): sample answers, grade them, reward the good onesdef rft_step(model, prompt: str, expected_ans: str, grader) -> None:    candidates = model.sample(prompt, n=4)  # sample candidate answers    # Programmatically grade each candidate    scored = [(ans, grader(expected_ans, ans)) for ans in candidates]    # Pseudocode: update the model so higher-scoring answers become more likely    update_policy(model, prompt, scored)def exact_match_grader(expected: str, answer: str) -> float:    # Works only when there is a definitive right answer    return 1.0 if answer.strip() == expected.strip() else 0.0\n```\n\nThis is why RFT only works when correctness can be measured programmatically — no grader, no reward signal, no training.\n\nSo fine-tuning isn’t dead. From a practical decision framework today, I think of the order going something like this:\n\nBut what do you think? Does fine-tuning still have its place? Let me know in the comments.\n\nIf you found this helpful, consider clapping👏 so others can find it too and follow me for more amazing technical AI content!\n\n[Why Fine-Tuning Is No Longer Your First Choice for Custom AI?](https://pub.towardsai.net/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai-ddd69c23298f) was originally published in [Towards AI](https://pub.towardsai.net) on Medium, where people are continuing the conversation by highlighting and responding to this story.", "url": "https://wpnews.pro/news/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai", "canonical_source": "https://pub.towardsai.net/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai-ddd69c23298f?source=rss----98111c9905da---4", "published_at": "2026-08-03 12:01:03+00:00", "updated_at": "2026-08-03 12:22:15.819571+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-research", "ai-products"], "entities": ["Harvey", "OpenAI", "GPT-4", "Bloomberg", "BloombergGPT", "ChatGPT"], "alternates": {"html": "https://wpnews.pro/news/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai", "markdown": "https://wpnews.pro/news/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai.md", "text": "https://wpnews.pro/news/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai.txt", "jsonld": "https://wpnews.pro/news/why-fine-tuning-is-no-longer-your-first-choice-for-custom-ai.jsonld"}}