Fine-Tuning

A base language model like GPT-4 or Claude knows a lot about the world but nothing about your specific business. Fine-tuning teaches the model your domain by training it on examples of the inputs and outputs you care about.

The process works like this: you prepare a dataset of example inputs and desired outputs, then run a training process that adjusts the model's weights. After fine-tuning, the model responds in ways that align with your examples. It might learn your company's terminology, follow a specific format, or handle domain-specific tasks more accurately.

Modern fine-tuning usually means parameter-efficient fine-tuning (PEFT) rather than updating all of a model's weights. LoRA (Low-Rank Adaptation) and QLoRA train a small number of additional parameters (typically 0.1-1% of the base model) while keeping the base weights frozen. This cuts training cost and memory by 10-100x compared to full fine-tuning, with competitive quality for most tasks. For reasoning-style tasks, RLHF and DPO (direct preference optimization) add another layer where the model is trained to prefer better outputs over worse ones using human or AI preference pairs.

Fine-tuning is most useful when you need the model to adopt a specific style, consistently follow a complex format, or develop deep expertise in a narrow domain. Medical report generation, legal document drafting, code generation for a specific framework, and structured extraction tasks are common fits.

The main alternative is RAG. RAG gives the model relevant context at runtime without changing the model itself. For most enterprise use cases, RAG is the better starting point. It's cheaper, faster to set up, and easier to update when your data changes.

Fine-tuning makes sense when RAG isn't enough. If the model needs to behave differently (not just know different things), fine-tuning is the right tool. Many production systems use both: a fine-tuned model for style and format, with RAG for current knowledge. When not to fine-tune: when your training set is under 500 examples (prompt engineering will do better), when the task changes frequently (you'll re-train constantly), or when the base model already performs well with a good prompt.

A medical coding vendor wants an LLM that reads physician notes and outputs ICD-10 and CPT codes. The base Claude Haiku with a good prompt hits 78% code accuracy on a labeled eval set of 500 notes, which isn't enough for billing-grade output. The team has 12,000 expert-labeled note-to-code pairs from the past two years of production.

They fine-tune Claude Haiku via AWS Bedrock using 10,000 pairs for training and 2,000 held out for eval. The training job runs for about 4 hours and costs around $180 in compute. The fine-tuned model hits 94% accuracy on the held-out set, with the biggest gains on specialty-specific codes (cardiology, orthopedics) where the base model had been weakest. Per-call inference cost on the fine-tuned model is about 8% higher than base Haiku on Bedrock.

In production, the fine-tuned model handles first-pass coding with a confidence threshold: codes above 0.9 confidence go straight to billing, codes between 0.7 and 0.9 get a human coder review, and codes below 0.7 get a full human workflow. Human coders now review about 30% of notes instead of 100%, so each coder handles 3x more volume. Payback on the fine-tuning project was roughly 6 weeks. The team re-trains the model every quarter as new coding guidelines come out.