LLM Lifecycle

A developer explains the LLM lifecycle by comparing it to OS deployment, covering pretraining, fine-tuning, prompt engineering, evaluation, and deployment. The post highlights the analogy between fine-tuning methods like T-Few and hotfixes, and discusses GPU cluster resource management.

This LLM life cycle is an easy reading for an Infra admin comparing to Deployment life cycle of an OS. 1. Pretraining — building the golden image This is where the base model learns general language understanding from massive datasets — expensive, done once by the model provider Cohere, Meta, OpenAI . Think of this like building a golden Windows Server image : a huge, centralized effort to bake in a broad, general-purpose baseline that many teams will later build on top of. You don't do this yourself — you consume it, the same way you don't build your own base OS image from source, you pull the vendor-provided baseline. 2. Fine-Tuning — customizing the image for your environment This is where you adapt that general-purpose base model to perform well on a specific task using your own labeled data. Directly analogous to customizing a golden image for a specific business unit — installing department-specific software, applying tailored GPOs, baking in your org's configuration standards. The deck's distinction between Vanilla retrain most of the model's weights — slow, expensive vs. T-Few only touch a tiny ~0.01% slice of additional layers — fast, cheap maps almost exactly to full reimage vs. hotfix/patch: Vanilla is like rebuilding the entire image from scratch every time you need a change; T-Few is like applying a targeted config/registry patch on top of the existing image without touching the base. 3. Prompt Engineering — runtime configuration, no rebuild needed This is adjusting how you ask the model to behave — instructions, examples, formatting — without touching the model's underlying weights at all. This is like adjusting a config file, environment variable, or GPO setting at the client/session level rather than rebuilding or repatching the image. Fast, reversible, zero deployment risk — which is exactly why the course's own guidance "start with a simple prompt" before reaching for RAG or fine-tuning mirrors the sysadmin instinct of always trying the cheap, reversible fix before a heavier one. 4. Evaluation and Iteration — UAT and health checks before wider rollout This is where you test the model's output — using metrics like accuracy how many predictions were right and loss how wrong the wrong ones were, so it's more like error severity than a pass/fail count — against a validation dataset, and iterate: adjust hyperparameters, retrain if needed. Think UAT/pilot testing before a full production rollout, plus ongoing benchmarking — similar to how you'd validate a patched image in a staging environment before pushing it fleet-wide, using defined KPIs rather than just "it seems to work." 5. Deployment and Ongoing Maintenance — go-live and lifecycle management Deploy the model or fine-tuned version for real inference traffic, then monitor performance and collect feedback for future fine-tuning cycles. This is your production rollout + patch/maintenance cycle. Dedicated AI Clusters Hosting vs. Fine-tuning — think resource pools/clusters in vCenter: a fine-tuning cluster needs more GPU "horsepower" 2 units than a hosting cluster 1 unit , the same way a build/compile server needs more compute than a server just serving traffic. One base model endpoint + multiple custom model endpoints sharing the same GPU cluster — this is exactly like multiple VMs sharing a hypervisor host's resources , or several App Pools sharing an IIS server — efficient because they share a common "base" and only the deltas differ. GPU memory reload overhead when switching between unrelated models — this is your vMotion / cold-migration overhead intuition: moving between things that share the same base is cheap; switching to something fundamentally different means a full reload, which costs time and resources. The one-line summary for your notes: Pretraining = build the golden image someone else does this → Fine-tuning = customize the image for your org → Prompt Engineering = runtime config tweaks → Evaluation = UAT/benchmarking → Deployment = go-live + ongoing patch/monitor cycle. Same lifecycle discipline you already run for infrastructure, just applied to a model instead of a server fleet.