Testing LLM Applications: Evaluation, Mocks, and Regression Checks
A practical testing strategy for LLM applications covering unit tests, retrieval checks, tool calls, evaluation datasets, safety cases, and CI regression gates.
On this page
Testing an LLM application is still software testing, even when the model output is variable. The system has prompts, retrieval, tools, model calls, parsing, policies, and user-facing behavior. Each layer needs an appropriate test.
#Use a testing pyramid
Unit tests: validate parsing, chunking, filters, authorization, and prompt construction.
Contract tests: validate provider responses, tool schemas, and structured output.
Integration tests: exercise retrieval, databases, queues, and model adapters.
Evaluation tests: score representative inputs for quality, grounding, and safety.
End-to-end tests: verify the user workflow and recovery.
#Test deterministic boundaries
Input limits, secret redaction, permission filters, citation mapping, JSON parsing, retries, timeouts, and tool authorization should have exact assertions without involving a model. A model judgment is not a security test.
#Mocks and evaluation data
Use realistic fixtures for success, malformed output, refusal, timeout, rate limit, and provider outage. Build an anonymized evaluation set containing normal, ambiguous, empty, long, adversarial, and out-of-scope requests. Version the set and add real incidents after their causes are understood.
#Metrics that matter
Depending on the workflow, measure extraction accuracy, classification agreement, groundedness, citation correctness, retrieval recall, schema validity, refusal quality, latency, and token usage. LLM-as-judge scoring can help with open-ended text, but calibrate it against human review.
#RAG, tools, and agents
For RAG, verify that the correct source appears in the candidate set before scoring the answer. Test deleted documents, stale indexes, tenant filters, and duplicate chunks. For agents, test invalid arguments, unauthorized resources, timeouts, excessive steps, and approval gates.
#CI and observability
Run deterministic tests on every change. Run a small evaluation smoke set on pull requests and a broader set on release or scheduled runs. Record prompt and model versions, retrieval IDs, tool outcomes, latency, and token usage while redacting sensitive content.
#Common mistakes
Testing only final wording.
Using one tiny evaluation set for every workflow.
Accepting a judge score without calibration.
Letting tests call real destructive tools.
Logging full prompts without a privacy review.
#Frequently asked questions
#Can LLM outputs have exact assertions?
Yes, for schemas, safety decisions, citations, and required properties. Open-ended text needs combined signals.
#Should evaluation run on every commit?
Fast tests should. Expensive model evaluations can use a smoke subset and broader scheduled runs.
#Conclusion
Reliable LLM testing combines ordinary software tests with representative evaluation. Validate deterministic boundaries exactly, measure retrieval and generation separately, test permissions and failure paths, and turn incidents into regression cases.