LangChain LangGraph Testing: An End-to-End Guide for QA

Most QA teams still treat LLM applications like ordinary microservices. They write a unit test, mock the HTTP call, and ship it. That approach falls apart the first time the model returns a different answer on every run. LangChain LangGraph testing is a different discipline: you test prompts, chains, agents, and checkpointed state, and you accept that some assertions can never be fully deterministic. This guide covers the full stack I use to test LangChain and LangGraph apps end to end, from fake models to state replay, with code you can run this week.

Table of Contents

• Why LangChain LangGraph Testing Is Different
• What You Are Actually Testing: Four Layers
• The LangChain LangGraph Testing Stack in 2026
• Unit Testing LangChain Chains with Fake Models
• Testing RAG Pipelines and Retrievers
• Testing LangGraph Agents: State, Checkpoints, and Replay
• Evaluating Outputs: Faithfulness, Relevance, and LLM-as-a-Judge
• Common Failures Teams Hit
• India Context: What Hiring Managers Want
• Key Takeaways
• FAQ

Why LangChain LangGraph Testing Is Different

LangChain is the most downloaded agent framework in Python right now. It pulled 270 million PyPI downloads last month, and LangGraph, the graph engine that powers its agents, pulled another 71 million (PyPI Stats). On GitHub, LangChain sits at 144,427 stars and LangGraph at 39,889 as of this writing (LangChain, LangGraph). A large share of the QA engineers I talk to are now being asked to test these apps without ever having tested an LLM before.

The core problem is simple. A regular function is deterministic; a model call is not. If you write assert result == expected against a live model, the test fails on the next run because the model phrased the same answer differently. The fix is to split the test surface into deterministic parts (prompt shape, tool selection, state transitions, retrieval) and non-deterministic parts (the exact wording of the output), then test each with a different tool.

The cost of getting this wrong is not theoretical. I watched one team burn two sprints chasing a “flaky” suite that was really just live-model assertions failing on rephrasing. Every red build trained developers to ignore the pipeline, which is worse than having no pipeline at all. Flaky tests in an LLM app do not slow you down; they teach the whole team that test results are meaningless.

Deterministic code, non-deterministic output

I split every LLM test into two buckets. Bucket one holds things that must be exact: the tool name, the JSON schema, the state update, the number of documents returned. Bucket two holds things that can vary: tone, phrasing, exact sentence structure. You assert hard on bucket one and evaluate bucket two with an LLM judge or a metric. Teams that assert hard on bucket two end up with a flaky suite and a lot of anger.

A test surface that keeps growing

An old web app had one surface: the response. An LLM app has at least four, and each needs its own strategy. A prompt template change can break every downstream test. A tool description edit can make the model pick the wrong tool. A graph edge added in the wrong place can make an agent loop forever. State that is not checkpointed correctly cannot be replayed, which blocks debugging entirely.

What You Are Actually Testing: Four Layers

Before you write a single test, map the layers. Most test plans I review are missing two of them.

Prompts and templates

System prompts, few-shot examples, and message templates. Test that variables render, that the system prompt contains the required guardrails, and that the rendered prompt fits the model context window. These are cheap, fast, deterministic tests that catch the most expensive production bugs, because a malformed system prompt is usually the reason an agent misbehaves.

A quick pattern I use everywhere is a one-line assertion on the rendered prompt length: build the template with the largest real inputs you support, render it, and assert the token count stays under your model limit with headroom. Tokenizers are approximate, so a rough character-based ceiling works fine as a smoke check. When someone adds a bigger few-shot example or a longer tool schema later, that test fails before the prompt silently truncates in production.

Chains and tools

A chain is a sequence of steps: prompt, model, output parser, maybe a tool call. Test that the chain calls the right tool, parses the right schema, and routes to the right branch. Use fake models so the test never touches a live API.

Agents and graphs

LangGraph agents are state machines. Nodes transform state, edges define control flow, and checkpoints persist state after each step. Test the transitions, the loop termination, and the ability to resume from a checkpoint. This is where most agent bugs hide, and where LangGraph 1.2.11 adds new controls I cover below.

Retrieval and context

RAG apps live or die by what the retriever returns. Test that the right chunks come back for a query, that irrelevant chunks are excluded, and that the grounded answer actually cites the retrieved text instead of hallucinating. Evaluation frameworks exist precisely for this layer.

The LangChain LangGraph Testing Stack in 2026

You do not need ten tools. I use three groups, and they cover about 90 percent of what ships. Worth noting: langchain-core 1.5.6 shipped on August 17, 2026, so pin your versions and re-run the suite on upgrades.

Fake and mock models

LangChain ships fake chat models in langchain_core for exactly this. GenericFakeChatModel returns messages you feed it in sequence, which makes it perfect for testing chains and graph nodes deterministically. FakeListChatModel cycles through a list of responses. Use these for unit tests and control-flow integration tests, and reserve live models for the evaluation layer.

Tracing and observability

You cannot debug an agent by reading logs. You need traces. LangSmith is LangChain’s tracing platform and captures every step, token, tool call, and state update in one view (LangSmith docs). I covered production LLM observability in depth in AI Observability for QA, but the short version is this: if a flaky agent failure is not visible in a trace, it effectively did not happen, because no one can reproduce it.

Evaluation frameworks

For judging output quality you want a framework, not hand-rolled assertions. DeepEval, Promptfoo, and Ragas are the three I recommend, and I compared them line by line in DeepEval vs Promptfoo vs Ragas. DeepEval gives you G-Eval, faithfulness, and answer relevancy. Promptfoo gives you assertions, red-teaming, and multi-provider testing. Ragas focuses on RAG metrics like context precision and recall. You pick one based on whether you test agents, prompts, or retrieval, which I also walk through in AI Eval Pipelines.

Here is the order I set up a new project in:

1. Add fake models and write chain unit tests.
2. Add retriever tests with presence and absence assertions.
3. Compile the graph with a checkpointer and add state replay tests.
4. Turn on tracing so failures are reproducible.
5. Add an evaluation stage with thresholds, gated to merge requests.

Unit Testing LangChain Chains with Fake Models

Here is the pattern I use for chains. The fake model returns a scripted message, so the test is deterministic and runs in milliseconds with no API key.

from langchain_core.language_models.fake_chat_models import GenericFakeChatModel
from langchain_core.prompts import ChatPromptTemplate

def test_extracts_action_from_ticket():
    # Scripted output: the model always returns this exact message
    llm = GenericFakeChatModel(messages=iter(["Action: create_bug_report"]))
    prompt = ChatPromptTemplate.from_messages([
        ("system", "Extract the action from the ticket. Reply 'Action: <tool>' only."),
        ("human", "{ticket}"),
    ])
    chain = prompt | llm

    result = chain.invoke({"ticket": "Login page crashes on Safari"})

    assert result.content == "Action: create_bug_report"

This catches prompt shape, message ordering, and variable rendering. It does not validate that a real model would actually pick create_bug_report, and that is the point. You push the judgment call to the evaluation layer instead of baking it into a flaky unit test.

For a chain with an output parser, test the parser as a separate unit. Feed it the raw string and assert on the parsed object. Never test a live model and a parser in the same test, because then a model phrasing change fails a parser test that has nothing wrong with it.

Testing RAG Pipelines and Retrievers

Retrieval is where most RAG apps break, and it is the easiest layer to test deterministically. Your retriever is plain code over an index, so there is no non-determinism to fight.

def test_retriever_returns_relevant_chunks(vectorstore):
    retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
    docs = retriever.invoke("How do I reset my password?")

    assert len(docs) == 3
    joined = " ".join(d.page_content.lower() for d in docs)
    assert "password" in joined
    assert "reset" in joined


def test_retriever_excludes_unrelated_chunks(vectorstore):
    docs = vectorstore.as_retriever(search_kwargs={"k": 3}).invoke("How do I reset my password?")
    joined = " ".join(d.page_content.lower() for d in docs)
    assert "pricing" not in joined

The first test asserts the right chunks come back. The second asserts irrelevant chunks stay out, which is the test everyone forgets. A retriever that returns the password doc but also three pricing docs still produces a bad answer, because the model gets distracted by the extra context.

For the grounded answer itself, do not assert exact wording. Assert that the answer cites the retrieved text. That is what faithfulness metrics measure, and I cover them in the evaluation section below.

Testing LangGraph Agents: State, Checkpoints, and Replay

LangGraph models an agent as a graph. Nodes change state, edges route between nodes, and a checkpointer persists state after every step. The current release, LangGraph 1.2.11 (published August 11, 2026), exposed a trace_policy argument on add_node and shipped the checkpoint libraries at 4.2.0, checkpoint-postgres 3.1.2, and checkpoint-sqlite 3.1.1 (LangGraph 1.2.11 release notes). For QA, the checkpoint work matters more than anything, because it is what makes state replay and retry testable.

Here is a minimal graph and the tests that matter for it.

from typing import Annotated, TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
from langgraph.checkpoint.memory import MemorySaver

class State(TypedDict):
    messages: Annotated[list, add_messages]

def assistant(state):
    return {"messages": [("assistant", "Hello, how can I help?")]}

builder = StateGraph(State)
builder.add_node("assistant", assistant)
builder.add_edge(START, "assistant")
builder.add_edge("assistant", END)
graph = builder.compile(checkpointer=MemorySaver())

Test the state transition

def test_assistant_appends_message():
    config = {"configurable": {"thread_id": "t-1"}}
    result = graph.invoke({"messages": [("user", "hi")]}, config)

    assert result["messages"][-1].content == "Hello, how can I help?"
    assert result["messages"][-1].type == "ai"

Test checkpointed state and replay

def test_state_is_checkpointed_and_replayable():
    config = {"configurable": {"thread_id": "t-2"}}
    graph.invoke({"messages": [("user", "hi")]}, config)

    # Read the persisted state back, as if in a new process
    snapshot = graph.get_state(config)
    assert snapshot.values["messages"][-1].content == "Hello, how can I help?"

    # Rewind and replay from the checkpoint
    graph.update_state(config, {"messages": [("user", "tell me more")]})
    result = graph.invoke(None, config)
    assert len(result["messages"]) >= 2

This is the test that catches the bug I see most often in agent work: a team ships an agent, it fails once in production, and no one can reproduce the failure because the state was not persisted. If you can get the state back and replay it, a production failure becomes a unit test in under an hour. I wrote a deeper piece on this in LangGraph State Persistence Testing, including retry semantics and how to test them.

One gotcha: MemorySaver keeps state in memory, which is fine for tests but not for production. When you test against Postgres or SQLite checkpoints, run the same suite against the real checkpointer once in CI, because serialization bugs only show up when state crosses a process boundary.

Test retries and the new trace_policy

LangGraph 1.2.11 also lets you attach a trace_policy to add_node, which gives you finer control over what gets traced per node. For QA that matters because it means you can trace only the nodes you care about during a test run instead of drowning in every token. Retry semantics are the other piece worth testing: when a step fails, your graph should resume from the last checkpoint rather than restarting from zero, and that resume path deserves its own test, not a manual click-through. You drive it by calling invoke again with the same thread_id after injecting a failure into the failing node.

Evaluating Outputs: Faithfulness, Relevance, and LLM-as-a-Judge

Unit tests stop where judgment starts. Once the graph, retriever, and state are verified, you still have to answer one question: is the output actually good? That is where metrics come in.

The three metrics I use most:

• Faithfulness: does every claim in the answer trace back to the retrieved context? This catches hallucination.
• Answer relevancy: does the answer actually address the question, or did it drift?
• Context precision and recall: did the retriever pull the right chunks, and did it miss any?

Here is a DeepEval example wired into a pytest suite, because that is the fastest way to get these metrics running.

from deepeval import assert_test
from deepeval.test_case import LLMTestCase
from deepeval.metrics import FaithfulnessMetric, AnswerRelevancyMetric

def test_answer_is_faithful_and_relevant():
    test_case = LLMTestCase(
        input="How do I reset my password?",
        actual_output="Go to Settings, then Security, and tap Reset password.",
        retrieval_context=[
            "To reset your password, open Settings and tap Security.",
            "Choose Reset password and follow the on-screen steps.",
        ],
    )
    assert_test(test_case, [FaithfulnessMetric(), AnswerRelevancyMetric()])

These metrics are non-deterministic, so they cost money and run slowly compared to fake-model tests. Keep them in a separate CI stage that runs on merge requests, not on every commit, and set thresholds instead of exact passes. A faithfulness score below 0.8 fails the build; a score of 0.81 versus 0.83 does not.

I also cache the judge results in CI. LLM-as-a-judge is itself an LLM call, so it costs tokens every time it runs. Caching means a merge request that did not touch the prompts or retrieval skips the expensive stage entirely, and you only re-evaluate when the thing under test actually changed. It is the difference between an evaluation bill of a few dollars a month and one that surprises your finance team.

Common Failures Teams Hit

After auditing a lot of these test suites, here are the recurring mistakes, in order of how often I see them.

• Asserting exact wording against a live model. The single biggest source of flaky CI in LLM testing. Assert on structure, not phrasing.
• Testing the prompt and the model in the same test. You cannot tell which one broke. Isolate them.
• No checkpoint tests. If you cannot replay state, you cannot reproduce or retry failures, and you cannot write regression tests from production incidents.
• Retrieval tests that only check presence, never absence. Irrelevant context breaks answers just as badly as missing context.
• Running LLM-judge metrics on every commit. They are slow and costly. Gate them to merge requests and set thresholds.
• Ignoring the context window. A rendered prompt that exceeds the model limit fails silently or truncates the instructions. Add a cheap length assertion.

One more from my own team: pin your model versions. A minor model upgrade can change tool-calling behavior enough to flip a whole suite, and you want to know that happened in CI, not at 3 AM from a customer.

India Context: What Hiring Managers Want

In India, LLM testing has moved from a nice-to-have to a line item on SDET job descriptions. Product companies in Bengaluru and Hyderabad now list LangChain, LangGraph, and at least one evaluation framework alongside Selenium and Playwright. The AI-focused SDET roles I see posted pay ₹25 to 40 LPA, and the ones that require you to actually build agent test harnesses, not just prompt ChatGPT, sit at the top of that range.

What separates the hires is not knowing the libraries. It is being able to answer two questions in the interview: how do you make an LLM test deterministic, and how do you reproduce a production agent failure. If you can talk through fake models and checkpoint replay, you are ahead of most candidates. The market wants testers who treat AI apps as systems, not magic.

If you are a manual tester or a Selenium-era automation engineer, do not wait for a job to force you into this. A weekend is enough to build one LangGraph agent, compile it with a checkpointer, and write the five tests in this guide. That single project answers both interview questions better than any certification, and it is exactly the kind of artifact that gets you the top of that ₹25 to 40 LPA band.

Key Takeaways

LangChain LangGraph testing comes down to separating deterministic structure from non-deterministic output, then testing each with the right tool.

• Use fake models like GenericFakeChatModel for chain and graph unit tests so they stay fast and deterministic.
• Split the test surface into prompts, chains, agents, and retrieval, and test each layer on its own.
• Test retrieval for both presence and absence of chunks.
• Compile your graph with a checkpointer and test state replay, because that is how you turn a production failure into a regression test.
• Push output quality to LLM-judge metrics like faithfulness and answer relevancy, run them on merge requests only, and set thresholds.

The stack is not the hard part. The hard part is the discipline of not asserting on things you cannot control. Once you get that, testing LangChain and LangGraph is just testing a stateful system with an unpredictable function in the middle, and you already know how to do that.

FAQ

Can I test LangChain apps without paying for LLM calls?

Yes. Fake models like GenericFakeChatModel and FakeListChatModel return scripted responses, so unit and control-flow tests cost nothing. Reserve live model calls for the evaluation stage.

How do I make an LLM test deterministic?

Mock the model, assert on structure (tool name, schema, state, document count), and move wording quality to metrics. Never assert exact output text against a live model.

What is the difference between unit testing and evaluating an LLM app?

Unit tests check that the system does the right thing: the right tool, the right state, the right chunks. Evaluation judges the quality of the output: faithfulness, relevance, precision. You need both, and they belong in different CI stages.

Which evaluation framework should I start with?

For agents and general output quality, DeepEval. For prompt-level assertions and red-teaming, Promptfoo. For RAG-specific metrics, Ragas. I broke the trade-offs down in this comparison.

Do I need LangSmith to test LangGraph?

No. LangSmith is for tracing and debugging, and it makes production failures far easier to reproduce, but your unit tests and checkpoint replay tests run fine without it.