How to Handle e2e Shard Reports: Merge 3 Shards Into 1 Run Free With TestDino

Split a test run across 3 machines and it finishes sooner, but you get 3 separate e2e shard reports back instead of 1. Every CI pipeline makes that trade today, and the new e2e framework from TesterArmy made it a one-flag option.

The pain lands after the run. The official e2e docs say the framework can't merge shard reports yet, so a 3-shard matrix leaves 3 report files, 3 exit codes, and no single answer to "did the suite pass", a gap that Playwright sharding closed years ago with a merge command.

This guide shows what each shard writes, how teams work around the gap, and how the free TestDino reporter merges e2e shard reports into 1 run with 1 environment variable. We built a 9-test demo and ran it 3 ways to show the difference.

What are e2e shard reports and why does e2e split them?

A quick naming note first. In this post "e2e" is the open-source framework from TesterArmy, published on npm as e2e under the Apache-2.0 license, not the general practice of end-to-end testing. It drives browsers through Playwright and adds AI agent steps, which is why Playwright vs e2e is the comparison most teams start with.

The framework launched publicly on October 1, 2026, and the GitHub API counted 12,972 stars for the tester-army/e2e repository on November 4. It is still pre-1.0, so some of what follows will change, and we note the version behind every claim.

How e2e sharding cuts a suite

The CLI reference describes --shard <index/total> as one contiguous slice of the selected tests, cut after every other filter has run. Slices differ by at most one test, a serial group is never split, and each shard loads only the setup tests its own tests need.

The order is the report order, so every machine cuts the suite the same way from the same command. A shard with nothing to run fails with NO_TESTS unless you pass --pass-with-no-tests, which matters when a matrix has more shards than tests.

npx e2e run --shard 1/3 --reporter list,junit
npx e2e run --shard 2/3 --reporter list,junit
npx e2e run --shard 3/3 --reporter list,junit

What one shard leaves behind

We ran a 9-test demo shop as 3 shards with e2e 0.22.0 on November 4, 2026. Each shard printed its own summary, such as Tests 3 passed (3), and wrote a full report.json that lists all 9 discovered tests.

Only 3 rows in each file are marked selected. The other 6 carry "cause": "filtered", "reason": "outside the shard", and the summary.selected count covers the slice alone. Nothing in the file says "I am shard 2 of 3", so the index has to come from the artifact name.

Add the junit.xml from --reporter list,junit and the results/ folder with a trace.md page per test, and each machine ends with 4 complete outputs for one third of the run. They are correct and useful on their own. The trouble starts when the slice is all you ever get.

The problem: 3 shards, 3 reports, and no single answer

The e2e CI docs are direct about it: "Upload each shard's report.json and junit.xml under a name with the shard index. e2e can't merge shard reports yet." That sentence has been there since sharding arrived.

Pull request 389, which introduced --shard and merged on September 22, 2026, lists "a note that merging shard reports is not implemented yet" in its own description. Version 0.22.0 shipped on October 30 with the note still in place.

What the gap costs a team

The missing merge shows up in 5 places, each traceable to the official docs rather than to opinion.

1. No single pass or fail. Each shard job has its own exit code, so branch protection needs every shard job as a required check, and changing the shard count breaks the rule set.
2. Totals split N ways. Passed, failed, flaky, and skipped exist only per shard, and so do the model token and cost figures in usage. Someone adds them up by hand.
3. One JUnit file per shard. Most CI test tabs accept a glob, but you still upload N files with index-suffixed names.
4. A pull request comment per shard. The @e2e-dev/github reporter warns that matrix replicas of one job share a comment, and the last one to finish wins. Its fix is a key per entry, which gives you 3 comments for 3 shards.
5. Evidence in N artifacts. A failure's trace.md lives in the upload of the shard that ran it, so the first debugging step is guessing which artifact to open.

Our demo made the arithmetic concrete. With one deliberate failure in the orders filter, the shards reported 3 passed, 3 passed, and 2 passed with 1 failed, and a human had to add those into 8 passed and 1 failed.

The same split hits the money figures that make e2e different from a scripted runner. Each shard's usage block carries its own model tokens and estimated cost, so "what did this run cost" needs a calculator too.

Teams have found a few ways to live with that, and none of them is free of maintenance.

How teams handle e2e shard reports today

Playwright users rarely think about this problem. The official Playwright sharding docs pair the blob reporter with npx playwright merge-reports, and the same pattern is spelled out for CI matrices in the Playwright CI reports guide.

The open-source TestDino Playwright skill packages that matrix-plus-merge-job pattern as a guide an AI coding agent can follow. e2e users have no equivalent command yet, so the workarounds look like this.

The do-it-yourself script

The fourth row is what most teams build first. Our demo repo includes a 60-line script that reads every report.json, prints one row per shard, and adds up the summaries, which is the kind of thing a Playwright custom reporter would do in that ecosystem.

It works, and it also shows the limits. The script has no idea whether a shard is missing, cannot link to the trace page of a failure, and knows nothing about the previous 50 runs. Every one of those questions comes up within a week of sharding a real suite.

Why the gap is widening

The number of teams about to hit this is not small. The npm downloads API counted 623,284 downloads of the e2e package in the week of October 27 to November 2, 2026, and the weekly figure grew about 15 times in 5 weeks.

Many of those installs will end up sharded, because agent steps spend seconds on model calls that a scripted click never needs. That is the audience TestDino built its reporter for.

TestDino's shard merger: 1 run ID, 1 test run, $0

TestDino published @testdino/e2e 1.0.1 on October 27, 2026. It is a reporter object for e2e's reporter API, which means you add it to the reporters array and it uploads the finished run with its test cases, retries, steps, errors, console output, screenshots, and videos.

The merge is one line in the @testdino/e2e docs: "Set the same TESTDINO_CI_RUN_ID on every shard. TestDino merges the shards into 1 test run." A shard started without that variable uploads as its own test run.

Why a server-side merge fits sharding

The shards never talk to each other, and with this design they never have to. Each job uploads independently when its run finishes, and the dashboard groups the uploads by run ID as they arrive. There is no merge job, no artifact download step, and no extra CLI to pin.

That also makes it CI-neutral. The same two variables work on GitHub Actions, GitLab, Azure DevOps, or a laptop, which is more than the GitHub-only pull request comment can say. A shard that fails still uploads, so the merged run turns red instead of going missing.

Because TestDino tracks each test case across runs, a test keeps its history whichever shard ran it this time. The flaky test detection tools comparison explains why a cross-run view is the point: a flaky verdict inside one shard tells you about today, and a reliability score tells you about the month.

What free means here

The TestDino pricing page lists the Free plan as free forever with 5,000 test executions a month, 1 user, and 1 month of analytics retention. Flaky detection, the dashboard, and Test Controls are included.

One uploaded test case is one execution, so our 9-test demo uses 9 per run, which puts about 555 sharded runs a month inside the free allowance. Keep uploading the .e2e folder as a CI artifact too, because the e2e trace page never leaves the runner.

There is also a split mode for jobs that run different files on purpose, driven by TESTDINO_SPLIT and TESTDINO_SPLIT_ID. For a plain --shard matrix, the run ID alone does the job, and setting it up takes 5 steps.

How to merge e2e shard reports with TestDino in 5 steps

The prerequisites from the docs: e2e 0.16 or newer and below 1.0, a Node.js version your e2e release supports, a TestDino API key, and a Git-initialized repository, because the reporter reads the commit and branch. Our demo ran e2e 0.22.0 on Node 24.22.2.

1. Install the reporter with npm install -D @testdino/e2e.
2. Add testdino() to the reporters array in e2e.config.ts, next to list.
3. Create a project API key in TestDino under Project Settings, then API Keys, then Generate New Key, and store it as TESTDINO_TOKEN.
4. Set the same TESTDINO_CI_RUN_ID on every shard, taken from your pipeline, not generated per job.
5. Run each shard as usual with npx e2e run --shard i/N and open the link printed on the TestDino line.

npm install -D @testdino/e2e

import type { E2EConfig } from 'e2e';
import { web } from '@e2e-dev/web';
import { testdino } from '@testdino/e2e';
export default {
  targets: [
    {
      name: 'web',
      engine: web({ browser: 'chromium' }),
      app: { url: 'http://127.0.0.1:0', command: { executable: 'node', args: ['server.mjs', '{port}'] } },
    },
  ],
  reporters: ['list', testdino()],
} satisfies E2EConfig;

Running it from a laptop first

You do not need CI to see the merge. Set the token and a run ID in one shell, then run the shards one after another, which is exactly what the generate API keys guide expects for local uploads.

export TESTDINO_TOKEN="your-project-api-key"
export TESTDINO_CI_RUN_ID="local-$(date +%Y%m%d-%H%M%S)"
npx e2e run --shard 1/3 --reporter list,junit --output .e2e-shards/1
npx e2e run --shard 2/3 --reporter list,junit --output .e2e-shards/2
npx e2e run --shard 3/3 --reporter list,junit --output .e2e-shards/3

The --output flag keeps each shard's files in a separate folder so you can open the 3 reports side by side. Each run ends with a TestDino line that either links to the run or tells you why nothing was uploaded.

Our own capture shows all 3 shards printing one run link: shard 1 all green, shard 2 with a flaky inventory test that passed on retry, and shard 3 with the failed orders filter. Each shard still wrote its own report.json and junit.xml on disk, so the per-shard files are there if you ever need them.

Two details from the e2e reporters reference are worth knowing before you wire this into CI. The --reporter flag replaces only the built-in reporters, so the testdino() object keeps running, and every reporter gets a one-minute deadline once the run finishes.

A laptop proves the merge works. A matrix is where it pays off.

Running the merge on a GitHub Actions matrix

The workflow below is the e2e docs' 3-shard matrix with two environment variables added at the job level, so every shard inherits them. The docs suggest ${{ github.run_id }}-${{ github.run_attempt }} as the run ID, and that choice is deliberate.

name: e2e shards
on: [push, pull_request, workflow_dispatch]
permissions:
  contents: read
jobs:
  e2e:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shard: [1, 2, 3]
    env:
      TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
      TESTDINO_CI_RUN_ID: ${{ github.run_id }}-${{ github.run_attempt }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 24
          cache: npm
      - run: npm ci
      - run: npx e2e-web install chromium --with-deps
      - run: npx e2e run --shard ${{ matrix.shard }}/3 --reporter list,junit
      - uses: actions/upload-artifact@v4
        if: ${{ !cancelled() }}
        with:
          name: e2e-shard-${{ matrix.shard }}
          path: |
            .e2e/report.json
            .e2e/junit.xml

Why run_id plus run_attempt

The run ID is shared by all 3 jobs of one workflow run, which is what merges them. The attempt number changes when someone re-runs the workflow, so a retry becomes a new test run instead of mixing into the first attempt's totals, the same separation the rerun only failed tests pattern depends on.

Keep fail-fast: false. Without it, one failing shard cancels the others, and the merged run would hold only the shards that finished instead of a complete red result. The artifact upload stays too, because the e2e trace.md page is not uploaded to TestDino and is the fastest way to read an agent failure.

Tip: Never generate the run ID inside the job. A timestamp or a random string evaluated per shard produces 3 different IDs and 3 separate runs. Use a value the pipeline already shares, such as the GitHub run ID, the GitLab pipeline ID, or the Azure build ID.

Retries, reruns, and other CI systems

e2e switches to 1 retry and 1 worker when CI is set, so a flaky test gets its second attempt inside the same shard and arrives in TestDino as one flaky result. We kept that default rather than the --last-failed rerun chain, because a rerun finishes a second run and we did not test how two uploads with one ID are reconciled.

The pattern ports directly. On GitLab a parallel: 3 job exposes CI_NODE_INDEX and CI_NODE_TOTAL for the shard flag and CI_PIPELINE_ID for the run ID, as the Playwright in GitLab CI setup does for Playwright. Picking the shard count is a separate question, and the Sharding Calculator in TestDino's free tools gives a starting number from your suite duration.

Once the matrix is green, the interesting part is what the dashboard does with 3 uploads.

What the merged run looks like in TestDino

Each shard job ends with the same TestDino line linking to one run, which is the first visible difference. Without the run ID, each shard printed its own link followed by the reporter's warning, shards uploaded as separate runs: set TESTDINO_CI_RUN_ID to merge them.

The run list: 3 rows become 1

Those 3 uploads became runs 24, 25 and 26 in the Test Runs list, 3 tests each, all green, and all meaningless on their own. The push with the run ID set arrived as run 29, 1 row with a Sharded badge, 9 tests, 7 passed, 1 failed and 1 flaky, and a 53 second duration.

The 2 rows in between are worth a look. Run 27 is an earlier attempt where shard 1's upload failed on the laptop's connection, and the list shows it as a sharded run with 6 tests rather than a green suite. Run 28 is a single shard pushed on its own.

That 6-test row is what a lost upload looks like. TestDino keeps the shards that reported and counts those, 2 expected and 2 reported in the API record, so the dashboard does not flag the gap. A sharded run with fewer rows than your matrix has jobs is the thing to check.

The run page

The Summary tab opens with the Shards panel: a row per shard with its status, duration, a pass and fail breakdown, and test counts. The panel also marks the slowest shard, prints an imbalance ratio, 5.7 times here with shard 3 at 10.9 seconds against shard 1 at 1.9 seconds, and says where the failures concentrated, which in our case read "100% of failures, shard 3".

Below it sit the run totals, 1 failed and 1 flaky, and a Detailed Analysis list you can narrow by shard. Clicking a shard row adds or removes it from that analysis, which is the quickest way to ask what shard 3 broke.

That imbalance figure is the first thing to check before you try to reduce Playwright CI runtime in a sharded pipeline, because a gap of 5.7 times means the fastest runner would sit idle for most of the slowest shard's time. The shard count is the second, since the panel lists only the shards that reported, and 2 rows under a 3-job matrix means an upload was lost.

Across runs, not just across shards

Open the failed test and its page carries the shard that ran it, both attempts with the same assertion error, and a reliability score that treats a result from shard 3 today and shard 1 tomorrow as the same test. The Playwright test history view works the same way for e2e uploads, which is what makes resharding safe.

On paid plans the Why Failing tile adds a category, Unstable in this case, next to the exact locator and the expected versus observed text. That is the raw material for test failure analysis: a failure that is new on this commit, one that has flipped for a week, and one that has failed 5 runs in a row look different in the history even when they look identical in a JUnit file.

Those limits are listed in the reporter docs, and the live streaming one matters most for long runs, since the Playwright realtime reporting experience fills the dashboard test by test. For a sharded e2e run, you see the whole picture once the slowest shard ends.

Each run is also tied to its commit and branch, which the reporter reads from the Git repository it requires. That link is what lets the dashboard answer whether this test failed on this code before, a question no per-shard file can answer on its own.

A merged run is easy to get and almost as easy to break. The failures are predictable.

6 mistakes that split a merged run again

Every one of these produces a clear line on the terminal, and most of them are called out in the troubleshooting section of the reporter docs.

1. One shard without the run ID. The terminal says shards uploaded as separate runs: set TESTDINO_CI_RUN_ID to merge them. Set the variable in the job-level env, not inside one step.
2. A different ID per shard. A value computed per job, such as a timestamp, gives N runs. Use the pipeline's shared ID.
3. A missing token on one job. That shard prints skipped (TESTDINO_TOKEN is not set) and never arrives, so the Shards panel lists one row fewer than the matrix has jobs.
4. The execution limit. When the organization is over its monthly executions the tests still run, but the terminal says not recorded and nothing is saved until the plan is upgraded or executions are added.
5. More shards than tests. The empty shard fails with NO_TESTS and its job goes red. Add --pass-with-no-tests if the matrix can outgrow the suite.
6. Re-running only the failed jobs. A GitHub re-run increments run_attempt, so the re-run shards upload under a new ID as a run of their own. Re-run the whole workflow, or accept that the second run holds only the shards you repeated.

Tip: Before you debug e2e shard reports in the dashboard, read the TestDino line at the bottom of each shard's terminal output. It names the exact cause: skipped, not uploaded, not recorded, or shards uploaded as separate runs.

Two more are worth a sentence each. Mixing json with list in --reporter is an INVALID_CONFIG error, so keep list,junit, and artifacts still uploading 40 seconds after the run ends are dropped, which is one more reason to keep the CI artifact upload for large screenshot sets.

When the run count in the dashboard matches the pipeline count in CI, the merge is doing its job. The remaining question is whether sharding was worth it at all, and the CI Budget Calculator in TestDino's free tools models what a smaller matrix or a flake fix saves per month.

Conclusion

Sharding an e2e suite gives you speed and takes away the single report, and the framework's own docs say the merge is not there yet. The cost is 3 files, 3 exit codes, and a pull request comment per shard until someone writes a script.

TestDino's reporter removes that work with one shared TESTDINO_CI_RUN_ID. Every shard uploads itself, the dashboard merges the e2e shard reports into 1 run with per-shard rows, an imbalance ratio and failure concentration, and the Free plan covers 5,000 executions a month.

The pieces that matter after the merge are the same ones Playwright test reporting teams already rely on: history per test, a flaky score, and a failure classification. A test intelligence platform that reads e2e runs now gives TesterArmy users the same view, without waiting for the merge command to land upstream.

FAQs