Golden Test Sets

What makes a test set golden

A "golden test set" is a curated collection of (input, expected_output) pairs where the expected outputs have been verified by humans to represent correct, high-quality responses. "Golden" means: (1) human-verified — not generated by the same LLM you're evaluating; (2) stable — the expected answers don't change unless you deliberately update them; (3) representative — covers the full range of inputs your system will encounter in production, including edge cases; (4) versioned — tracked in Git so you know exactly what changed and when.

Building your first golden set

Start small and iterate:

1. Collect 50–100 real or simulated inputs: For a pre-launch system, manually craft inputs covering your intended use cases. For a live system, sample from recent production traffic.
2. Generate candidate outputs: Run your current best system on each input to get a starting point.
3. Human review and correction: Have domain experts review each candidate output, marking it pass/fail and rewriting failed outputs to the ideal response. This is the most expensive step — budget 5–15 minutes per example.
4. Categorise examples: Tag each example (e.g. "edge_case", "typical", "adversarial") so you can analyse performance by category.
5. Set acceptance criteria: Define what score on the golden set means "ready to ship" (e.g. 90% pass rate on typical, 75% on edge cases).

Example structure and storage

import json
from pathlib import Path
from dataclasses import dataclass, asdict
from typing import Optional

@dataclass
class GoldenExample:
    id: str
    category: str          # "typical", "edge_case", "adversarial", "regression"
    input: str | dict      # the user input or full conversation
    expected_output: str   # human-verified ideal response
    expected_facts: list[str]  # key facts that must appear in any acceptable answer
    forbidden_content: list[str]  # things that must NOT appear
    notes: str = ""        # why this example is in the set
    added_date: str = ""
    added_by: str = ""

# Store as JSONL — one example per line, easy to diff in git
def save_golden_set(examples: list[GoldenExample], path: str):
    with open(path, "w") as f:
        for ex in examples:
            f.write(json.dumps(asdict(ex)) + "\n")

def load_golden_set(path: str) -> list[GoldenExample]:
    examples = []
    with open(path) as f:
        for line in f:
            examples.append(GoldenExample(**json.loads(line)))
    return examples

# Store in git alongside your code:
# evals/golden/customer_support.jsonl
# evals/golden/product_descriptions.jsonl
# evals/golden/sql_generation.jsonl

Running golden set evaluations

import asyncio
from typing import Callable

async def run_golden_eval(
    examples: list[GoldenExample],
    system_under_test: Callable,
    judge_llm: Callable,
) -> dict:
    results = []
    for ex in examples:
        output = await system_under_test(ex.input)

        # Check hard assertions first (fast, cheap)
        hard_pass = all(fact.lower() in output.lower() for fact in ex.expected_facts)
        hard_fail = any(forbidden in output.lower() for forbidden in ex.forbidden_content)

        # LLM judge for soft quality (slower, more expensive)
        judge_score = await judge_llm(
            f"Rate this response 1-10 compared to the ideal.\n"
            f"Question: {ex.input}\nIdeal: {ex.expected_output}\nActual: {output}"
        )
        results.append({
            "id": ex.id,
            "category": ex.category,
            "hard_pass": hard_pass and not hard_fail,
            "judge_score": judge_score,
            "output": output,
        })

    by_category = {}
    for r in results:
        cat = r["category"]
        if cat not in by_category:
            by_category[cat] = []
        by_category[cat].append(r)

    return {
        "overall_pass_rate": sum(r["hard_pass"] for r in results) / len(results),
        "mean_judge_score": sum(r["judge_score"] for r in results) / len(results),
        "by_category": {
            cat: {
                "pass_rate": sum(r["hard_pass"] for r in items) / len(items),
                "mean_score": sum(r["judge_score"] for r in items) / len(items),
            }
            for cat, items in by_category.items()
        },
    }

Maintaining and growing the set

A golden set is a living document:

• Add regressions immediately: Every time a bug reaches production, add it as a golden example. This is your regression test suite.
• Monthly review: Prune examples that no longer represent real use cases; update expected outputs if your quality bar has risen.
• Systematic growth: Each sprint, add 5–10 new examples sampled from production edge cases. Target the categories where your pass rate is lowest.
• Never delete without replacement: If an example no longer applies, replace it with a similar current one. Shrinking golden sets means losing coverage.

Golden sets vs A/B testing

Golden sets and A/B testing are complementary. Golden sets catch regressions — changes that break cases that previously worked. A/B testing measures improvements — whether a new prompt does better on the full distribution of real traffic. Use golden sets to gate releases (don't ship if pass rate drops), and A/B testing to measure whether a change actually improves the metrics you care about in production.

Gotchas

• Don't use LLM to create golden labels: If you use GPT-4 to generate both your training data and your golden set, you're measuring consistency with GPT-4, not actual quality. Golden labels must come from humans.
• Expected output is not the only acceptable output: For generative tasks, many outputs are equally valid. Your evaluation should check for correctness and quality, not exact match to the expected string. Use fact-based assertions and LLM judges rather than exact match.
• Example contamination: If golden examples leak into the training data for your fine-tuned models, you're measuring memorisation. Keep golden sets strictly held out.