Claude Code E2E Testing: The Full Loop with e2e TesterArmy
Claude Code can walk checkout, freeze it in e2e TesterArmy, and debug a Pay rename from stored CI runs.
Claude Code can open your app in a real browser, tap through a flow, and write the test from the controls it just used.
The catch is CI. A locator that looked right in chat still fails at 2 a.m., Claude relaxes the assertion to go green, and nobody can tell if guest checkout failed last night too.
This guide walks the full loop for Claude Code E2E testing: wire e2e by TesterArmy, freeze guest checkout, replay from cache, then read the red job from trace.md and from stored runs.
Use this split before you read the rest. Every row is the same checkout.
| Step | Who owns it | Checkout example |
|---|---|---|
| Walk | Claude Code, through e2e mcp | Tap guest checkout on the live page |
| Freeze | You, in tests/checkout.e2e.ts | agent.act('pay as a guest'), then expect() |
| Replay | The cache, after a later check passes | Same taps, no model call |
| Check | A locator expect() you keep | Heading "Order confirmed", total $42.00 |
| Diagnose | trace.md, then one merged run | Button "Pay" is gone, "Place order" is on screen |
What Claude Code E2E testing actually is
Claude Code E2E testing is an agent writing, running, and debugging an end-to-end test against a live app, then leaving a spec that CI can rerun without the chat. The model is a coworker in the editor. The runner still owns pass, fail, waits, and artifacts.
The agent walks a real app, writes or repairs a spec, and reads a failure. In e2e TesterArmy the live walk is e2e mcp. The outcome still has to land in expect().
Three jobs get mixed together. Authoring is Claude walking checkout and writing tests/checkout.e2e.ts. The run is npx e2e run in a real Chromium session. Diagnosis is reading why Pay missed Place order, not guessing from a stack trace.
Playwright E2E testing already covers the run. You install a browser, click, assert, and fail the job when Order confirmed never shows.
Claude Code does not replace that contract. It fills the gap before and after: it should not invent a locator, and it should not "heal" a money check. Ordinary Playwright test automation still owns the green job.
Playwright also ships Playwright Test Agents. npx playwright init-agents --loop=claude adds a planner, a generator, and a healer as Claude Code subagents. They write ordinary @playwright/test files. The CI job still runs npx playwright test.
e2e by TesterArmy is the other path. Web tests still drive Chromium, Firefox, and WebKit through Playwright. The npm package is e2e. You can mix a goal (agent.act) with a locator expect(). A verified goal can replay from cache with no model call. Tests with no agent steps need no model at all.
Keep guest checkout as the spine for the rest of this page:
import { test, expect } from 'e2e';
test('guest checkout shows an order confirmation', async ({ app, agent, screen }) => {
await app.open('/checkout');
await agent.act('pay as a guest with buyer@example.com');
await expect(screen.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
await expect(screen.getByTestId('order-total')).toHaveText('$42.00');
});
The agent may tap Pay. Code still owns Order confirmed and $42.00. If Claude deletes those two lines to make the run pass, you no longer have a regression test.
Agentic testing tools that only draft a file in chat stop at authoring. Claude Code E2E testing is only useful if the spec still fails when /api/pay returns 500.
Three MCP servers, three jobs
MCP is how Claude Code calls a tool outside the chat. For Claude Code E2E testing you can attach three servers. They do not do the same job.
| Server | Install in Claude Code | What Claude sees | What it must not become |
|---|---|---|---|
| Playwright MCP | claude mcp add playwright npx @playwright/mcp@latest | Accessibility snapshots of a live page | Your CI runner |
| e2e mcp | claude mcp add --scope project e2e -- npx e2e mcp | A live e2e session, locator checks, test lines | History from last night's shards |
| A results MCP | Wired after the first green checkout | Stored runs, flakes, artifacts | A substitute for expect() |
Playwright MCP is a local browser server. Claude reads roles and names, not pixels. The Claude Code install is one CLI line, and it is live in the next session. The same add command is how you install Playwright MCP on Claude Code.
Use Playwright MCP when the suite is still @playwright/test. Ask Claude to walk checkout, then write tests/checkout.spec.ts with getByRole and web-first Playwright assertions. Run npx playwright test. Do not keep the MCP server in CI. It is for authoring and debug, not the nightly job.
e2e mcp is the TesterArmy server. It needs no model of its own. A test it runs needs a model only for an agent step with no recording. Each action prints the test line that repeats it, from a locator that matched one node:
Tapped #n31.
Test line: await screen.getByRole("button", "Sign in").tap();
If Claude guesses getByLabel("Email") and zero nodes match, the session fails there. You never wait out a 30 second CI timeout to learn the label is "Email address". That is the whole point of checking a locator before writing it, the same habit you want from Playwright locators.
A session does not write a JUnit file. Claude still runs the suite with npx e2e run. After a failure, open_session {test: "tests/checkout.e2e.ts:3"} opens the screen where the test stopped. Claude should not walk checkout from / again.
Who should skip a second MCP server: a repo that already has a green Playwright suite and one happy-path checkout. Add e2e mcp when you are actually writing *.e2e.ts files. Two browsers in one chat just double the noise.
Claude still has to pick a server per job. Live page, live e2e session, and stored runs are not interchangeable.

No public dataset supports a chart here for how much faster MCP authoring is. Time saved is your own checkout, timed with npx e2e run tests/checkout.e2e.ts.
Set up Claude Code with e2e TesterArmy
Start Claude Code E2E testing from a checkout that already fails in a boring way. If Order confirmed is not green on Chromium with locators you typed, a model will generate noise on top of a broken pay endpoint.
You need Node.js 24.8 or newer, or 22.22.3 or newer on Node 22. On Windows, run e2e inside WSL.
Paste an init prompt into Claude Code, or do it by hand:
npx e2e init
Pick Web (Playwright). Pick a model provider, a subscription, or None. None is the right first choice. The generated example makes no model calls, so it needs no API key.
npx e2e init writes e2e.config.ts, an example test, the e2e skill, and MCP config. Existing files stay intact. The skill lands in .agents/skills/e2e/ and, for Claude Code, .claude/skills/e2e/. Add this line to project instructions if the agent does not pick it up:
End-to-end tests use e2e; read .agents/skills/e2e/SKILL.md before writing or running one.
Register the live session:
claude mcp add --scope project e2e -- npx e2e mcp
claude mcp list
You should see e2e connected. Inside a session, /mcp shows the same thing. That live session is what Claude Code E2E testing uses to check a locator before it writes the spec.

The optional plugin installs extra skills, not the npm package:
claude plugin marketplace add tester-army/e2e
claude plugin install e2e@e2e
Point the target at the app. Keep the example model line only if you chose a gateway.
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { gateway } from 'ai';
export default {
targets: [
{
name: 'web',
engine: web({ browser: 'chromium' }),
app: { url: process.env.APP_URL ?? 'http://localhost:3000' },
},
],
agents: { default: { model: gateway('openai/gpt-6-luna-fast') } },
} satisfies E2EConfig;
That model string is a typical gateway example. It reads AI_GATEWAY_API_KEY. OpenRouter, a direct provider key, or a local server also work if the provider supports tool calls.
Subscriptions are the other route. npx e2e login openai uses ChatGPT Plus or Pro. Copilot, OpenCode Console, and SuperGrok have their own logins. Claude Max or Team is different. There is no e2e login claude. Agent steps on Anthropic go through an API key billed from the plan's monthly API credits, not from Claude Code usage limits.
If you want Claude Code itself to execute agent.act, that is @e2e-dev/acp with claudeCode(). The built-in agent and the Claude Code executor are two configs. Do not mix them in your head.
Run the example once:
npx e2e run
e2e downloads a browser, opens the app, and prints the result. Then ask Claude: "Write a test for guest checkout while you walk it in the app with e2e mcp. Freeze locators you checked. Keep Order confirmed and the total in expect()."
Default timeouts are wider than Playwright because an agent step can take more than a click. Playwright uses a 30 second test timeout, a 5 second expect timeout, and no per-action cap. e2e uses a 120 second test timeout, a 30 second action timeout, and a 5 second assertion timeout. Claude Code E2E testing hits those budgets while it iterates on checkout.

While Claude iterates, keep the loop tight:
npx e2e run tests/checkout.e2e.ts:3 --action-timeout 5000 --retries 0
--action-timeout 5000 makes a wrong locator fail in 5 seconds, not 30. --retries 0 matters because a retry never replays from cache. It runs every goal live. Keep those flags on while Claude Code E2E testing is still chasing a locator.
When the test passes, run it once with --no-cache. Every goal hits the live agent. That is the check that a recording is not what keeps checkout green by itself.
Write a checkout test Claude can keep
Claude Code E2E testing should walk the flow in e2e mcp, copy the test lines that matched one node, then put a locator expect() after each atomic agent.act. One idea per act. Then a check you named.
A good hybrid looks like this:
import { test, expect } from 'e2e';
test('guest checkout shows an order confirmation', async ({ app, agent, screen }) => {
await app.open('/checkout');
await screen.getByLabel('Email').fill('buyer@example.com');
await agent.act('pay as a guest');
await expect(screen.getByRole('heading', { name: 'Order confirmed' })).toBeVisible();
await expect(screen.getByTestId('order-total')).toHaveText('$42.00');
});
Known fields stay as locators. The messy chrome (which button, which guest form) can be a goal. The money line never is.
Tip: Give each agent.act one outcome, then follow it with an expect on a known control. That check fails when the agent takes a wrong turn. It is also what makes the step eligible for the replay cache.
String names in e2e are exact and case-sensitive. Playwright matches a case-insensitive substring. A query that hits several elements throws LOCATOR_AMBIGUOUS instead of clicking the first match. That strictness is why Claude must run locate before it writes getByRole.
The replay cache is how Claude Code E2E testing stays cheap in CI. A recording is saved only when a check after agent.act passes. Locator matchers such as toHaveText, URL assertions, agent.assert, and agent.waitFor count. A plain value expect, expect.poll, agent.extract, or the test simply passing do not.
Only agent.act replays. agent.assert, agent.waitFor, and agent.extract call the model on every run. A suite full of judgments will spend tokens even with a warm cache.
Entries are JSON under .e2e/cache/. They hold actions, target descriptions, typed text, and end checks. They do not hold prompts, screenshots, or secret values. npx e2e init gitignores that directory, so CI starts with no recordings and every agent step runs live. To share replays, remove the ignore line, commit the directory, and review entries like test data.
On the next run the runner checks the starting route, finds each control by role, name, test id, and context, repeats the action, and checks the end state. The summary reports replayed, handed off, or missed.
When Pay now becomes Place order, replay waits up to 15 seconds, then hands the step to the live agent from the current screen. A stale recording is a hand-off, never a tap on the closest control. The test file is never edited. The recording updates only after a later check passes again.

A scripted lookup does not get that hand-off. But I ran that case on e2e 0.22.0 against a local checkout page. The test still asked for button "Pay". The screen listed button "Place order". Chromium 156.0.8078.4 waited 30.0 seconds and failed:
LOCATOR_NOT_FOUND
locator matched no nodes within getByRole("button", name: "Pay")
Asked for: button "Pay"
Waited: 30.0s
The trace page named the heading "Checkout" and the button "Place order". It did not tap Place order. That is the failure you want on a price or a permission. It is also why Claude Code E2E testing still needs a locator expect(), not a fully scripted Pay click you will rename next sprint.

Both runners can live in one project, which is the practical Playwright vs e2e split. e2e only reads tests/**/*.e2e.ts by default. Leave stable specs on Playwright until you have read a hand-off.
Do not let Claude "fix" a failing total by widening $42.00 to any text. The same LLM agent in Playwright rule still applies: a test that already passed once is not a reason to rewrite the assertion. If the heading is gone, fix the app.
Recommendation, and who should skip it: commit the replay cache only if someone reviews the JSON in the pull request. If review is "looks good" on generated steps, skip agent.act on checkout. Keep locators you typed. A recording you never read is the test you cannot defend.
What breaks in CI and how Claude debugs it
When Claude Code E2E testing hits CI, the red job is almost never "Claude is bad." It is checkout on Chromium, heading not visible, cache stale, or /api/pay returning 500. Fix the layer trace.md names before you rewrite the goal.
CI mode turns on when CI is set to anything other than empty, 0, or false. These defaults change:
| Setting | Local | CI |
|---|---|---|
| retries | 0 | 1 |
| workers | Half your cores | 1 |
| cache, when unset | read-write | read-only |
| test.only | Allowed | Rejected with ONLY_IN_CI |
Sharding works the way Playwright sharding users expect: --shard 1/3 on each job. The hole is at the end. e2e cannot merge shard reports yet. Each job uploads its own report.json and junit.xml. Playwright in GitHub Actions has the same shard-blindness until something merges the run.
jobs:
e2e:
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3]
env:
AI_GATEWAY_API_KEY: ${{ secrets.AI_GATEWAY_API_KEY }}
steps:
- uses: actions/checkout@v4
- run: npm ci
- run: npx @e2e-dev/web install chromium --with-deps
- run: npx e2e run --shard ${{ matrix.shard }}/3 --reporter list,junit || npx e2e run --last-failed
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: e2e-shard-${{ matrix.shard }}
path: .e2e
--last-failed has to run in the same step, because it reads the previous report.json. It writes new attempts under rerun-1/ inside each test folder. A red job on shard 2 does not tell you whether checkout on shard 1 failed last night.
Exit codes carry meaning. 0 means every selected test passed. 1 is a test failure. 2 is a config or policy error, including REPLAY_STALE. 3 is an infrastructure failure such as an engine or model provider. 130 means the run was interrupted.
Without --strict-cache, a stale recording quietly hands the step to the agent. Every CI run then pays until someone records it again. With the flag, the step fails with REPLAY_STALE and makes no model call. That exit is 2, and it is not retried.
CI=1 npx e2e run --strict-cache
Use it when a silent hand-off is more expensive than a red job.
Note: Changing the model does not invalidate a recording. Renaming the test, the describe group, or the target, moving the file, or changing the instruction, params, engine, or app identity does. Retries never replay, so pass --retries 0 while you iterate.
Playwright retries still classify a fail-then-pass as flaky inside one run. e2e does the same on the CI retry. Neither tells you this checkout failed on most nights last week.
Claude Code E2E testing should debug in this order:
- Open .e2e/results//trace.md. The terminal prints ❯ trace .e2e/results/.../trace.md under the failed test.
- Read the failing line, the cache decision, the app log, the agent's last turns, and the screen at failure.
- Fix every line the evidence covers. The failing line is rarely the whole bug. A test stops at its first failed step.
- To see a screen the page does not show, open it in e2e mcp by URL. Do not walk the flow from / again.
- Do not open raw --ai-trace output in the chat. It repeats prompts and can be several megabytes.
trace.md plays the role the Playwright trace viewer plays for Playwright suites. It is Markdown. Preview it. The Pay rename run listed six nodes, including button "Place order", and still failed the Pay lookup. That is enough to stop Claude from inventing a CSS selector.
If /api/pay is 500, stop. Do not hand the step to the agent and call it healed. The app failed. So if Pay now became Place order and Order confirmed still appears, a hand-off is fair. Read the new recording before you commit it.
A JUnit file per shard is enough to make a job go red. It is not enough to say whether that failure is new, flaky, or the third time this week. The runner writes report.json, screenshots, and trace.md. It does not keep history across nights. It does not merge shards. It does not score a test that passes on retry on Monday and fails clean on Thursday.
That history is the missing half of Claude Code E2E testing. Claude can write the spec. Claude can read tonight's trace.md. Claude cannot see last Tuesday unless you store the run.
Where TestDino MCP closes the e2e loop
TestDino is a companion for this runner, not a second framework. The @testdino/e2e reporter uploads each finished run with its cases, retries, steps, errors, console output, screenshots, and videos. After Claude Code E2E testing finishes a job, that upload is how last night stays in the editor.
npm install -D @testdino/e2e
import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testdino } from '@testdino/e2e';
export default {
targets: [{ name: 'web', engine: web(), app: { url: 'http://localhost:3000' } }],
reporters: ['list', testdino()],
} satisfies E2EConfig;
export TESTDINO_TOKEN="your-api-key"
npx e2e run
The reporter needs e2e 0.16.0 or newer and below 1.0, Node.js 22.12 or newer for the reporter itself, an API key, and a Git repository so commit and branch attach to the run. Without a token, nothing uploads and the run continues.

Here are the holes in the e2e loop, and the practical fix:
| Hole in e2e TesterArmy | What you feel in CI | Practical fix |
|---|---|---|
| Shard reports cannot be merged | Three JUnit files, three stories | Set the same TESTDINO_CI_RUN_ID on every shard. On GitHub Actions use ${{ github.run_id }}-${{ github.run_attempt }} |
| Jobs that run different files never become one run | checkout and billing look unrelated | Split mode: TESTDINO_SPLIT plus TESTDINO_SPLIT_ID |
| Flaky is only fail-then-pass inside one job | Monday retry-pass, Thursday clean fail | Cross-run flake scores across nights |
| No HTML report, and trace.md stays on the runner | Claude cannot see last night | Dashboard keeps steps, errors, screenshots, videos. Keep trace.md local for agent turns |
| A JUnit file cannot say if checkout failed on this branch before | Every red job feels new | Run history tied to commit and branch |
| Results do not stream | You wait until npx e2e run exits | That limit stays. The e2e reporter uploads when the run finishes |
| Quarantine does not change the build result | A known flake still fails the job | Do not expect Playwright Test Controls behavior on e2e runs |
| Re-run commands on the dashboard are written for Playwright | Copy-paste npx playwright test is wrong | Rerun with npx e2e run tests/checkout.e2e.ts |
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
TESTDINO_CI_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}
A shard started without that id uploads as its own run. An agent that only saw shard 2 will misread checkout if checkout lived on shard 4. Cross-run flaky tests detection is what Monday's retry-pass and Thursday's clean fail look like together.
When jobs run different files, sharding is the wrong tool:
TESTDINO_SPLIT=1/2 TESTDINO_SPLIT_ID="$SPLIT_ID" npx e2e run tests/checkout.e2e.ts
TESTDINO_SPLIT=2/2 TESTDINO_SPLIT_ID="$SPLIT_ID" npx e2e run tests/billing.e2e.ts
Each target appears as its own project, so web and iOS stay separate. Artifacts up to 50 MB upload in PNG, JPEG, WebM, ZIP, JSON, TXT, and Markdown. MP4 device recordings are not uploaded. The e2e trace (trace.md) is not uploaded. Files still uploading 40 seconds after the run finishes are dropped. Visual comparison is not available, because this framework produces no expected and diff images.
Now connect the editor. TestDino MCP is how Claude Code E2E testing queries those stored runs without you pasting a dashboard URL. Add the local MCP server once:
claude mcp add testdino --env TESTDINO_PAT=your-token -- npx -y @testdino/mcp@latest
claude mcp list
Use a Personal Access Token (TESTDINO_PAT), not the project TESTDINO_TOKEN you put in CI. Ask Claude to run health and list projectId values. Most query tools need that id.
Then the useful prompts are small:
- Did guest checkout fail the same way on every retry in the last run?
- When did guest checkout shows an order confirmation last pass on this branch?
- Is this LOCATOR_NOT_FOUND new, or did Pay miss Place order last week too?
The MCP overview lists those shapes. Debug Playwright failures with AI is the hand-off you can paste into Claude.
A fail-then-pass is flaky, so do not rewrite the assertion. A test that failed every attempt is deterministic, so a longer timeout is the wrong first patch. After you push, ask whether the fix held. A test that passes only on retry is not fixed.
You now have three MCP servers in one Claude Code session. Playwright MCP or e2e mcp sees the live page. The TestDino MCP server sees history. Use it after the run, not instead of expect().
Skip this upload if the suite never leaves a laptop and nobody asks "did this fail yesterday." Keep it if Claude is about to patch checkout from one red job. One job is a snapshot. History is the loop.
Conclusion
The loop is simple. A prompt without a spec is a demo. Walk checkout in e2e mcp. Freeze locators that matched one node. Keep Order confirmed and $42.00 in expect(). Replay from cache in CI. Read trace.md when Pay misses. Query stored runs when you need last Tuesday.
e2e by TesterArmy can let a model tap. It cannot merge shards, keep nightly history, or tell a new LOCATOR_NOT_FOUND from a week of the same miss. Upload the finished run. Point Claude at that run. Then decide whether Place order is a fair hand-off or a broken pay endpoint.
Open tests/checkout.e2e.ts tonight. Run it once with --no-cache, then again with the cache on. If Order confirmed is gone, fix the app. If Pay was renamed, read the hand-off, keep the total, and ask Claude whether that failure already happened on this branch.IP
FAQs

Krupa Gandhi
QA Tester