Prompt Engineering

The way you ask an AI model to do something changes what you get back. Prompt engineering is the discipline of figuring out what works. It's the cheapest and fastest way to improve AI output quality before reaching for more complex techniques like fine-tuning or RAG.

Basic techniques include: giving the model a role ("You are a senior compliance analyst"), providing examples of desired output (few-shot prompting, typically 2-5 examples), specifying the exact format you want (JSON with a schema, bullet points, a specific template), and asking the model to think step by step before answering (chain-of-thought prompting).

More advanced techniques include breaking a complex task into smaller sub-tasks handled by separate prompts, using structured prompts with clearly labeled sections (XML tags work particularly well with Claude), and adding constraints that prevent common failure modes. Telling the model "if you're not sure, say so instead of guessing" measurably reduces hallucination rates in production RAG systems. Self-consistency prompting, where the same question is run multiple times and the majority answer is returned, improves reliability on reasoning tasks at 3-5x the cost.

In production systems, prompts are treated as code. They get version-controlled, tested, and reviewed. A small change in wording can significantly affect output quality, so teams maintain prompt libraries, use prompt-management tools like PromptLayer, LangSmith, or Humanloop, and run offline evaluations before deploying prompt changes. Treating prompts as untested strings embedded in application code is how teams end up with regressions nobody noticed.

Prompt engineering has limits. When you need the model to have specific knowledge it wasn't trained on, you need RAG. When you need it to behave consistently in a way that prompting alone can't achieve, you need fine-tuning. But for most use cases, a well-engineered prompt gets you 80% of the way there.

The other limit: prompt engineering is model-specific. A prompt tuned for GPT-4o may behave differently on Claude Sonnet, and both will differ from Gemini 2.0. Switching base models usually requires re-evaluating prompts against your test set. DSPy and automated prompt optimization tools (like OPRO or APE) try to make this less painful, but the tax is real. Budget for it in migration plans.

An e-commerce retailer uses an LLM to extract structured product attributes (color, size, material, brand) from free-text supplier descriptions. The first prompt is a simple "Extract color, size, material, and brand from this description and return JSON." Claude Haiku gets 78% field-level accuracy on a labeled test set of 400 descriptions.

The team iterates. Version 2 adds a role ("You are a product data specialist for a fashion retailer") and raises accuracy to 81%. Version 3 adds 4 few-shot examples covering tricky cases (multi-color garments, size ranges, rare materials). Accuracy jumps to 87%. Version 4 wraps the input in XML tags and explicitly instructs the model to return null for missing fields instead of guessing, which cuts the false-positive rate on "material" from 14% to 3%. Accuracy now 91%.

Version 5 moves from free-text JSON to structured outputs using Anthropic's tool-use pattern with a typed schema. This eliminates format errors entirely and pushes parseable output to 100%, with accuracy stable at 91%. The team stops iterating and deploys version 5 to production, saving each prompt version plus its eval scores in their prompt repo. When they later test Claude Sonnet for a quality lift, the same evaluation pipeline measures it against version 5 on the same test set. Sonnet hits 94% accuracy. The team evaluates whether the quality bump justifies the 4x cost increase for that workload.