Franklin Reduced Playwright CI Costs by 40% with TestDino
Franklin reduced Playwright CI costs by 40% using TestDino while improving flaky test visibility and cutting debugging time by up to 20%.
Franklin builds a financial platform for e-commerce businesses, including payment cards with high spend limits, expense management, and cashback on card expenses.
Because the product operates in the financial domain, release confidence and test stability directly affect day to day engineering work.
The team runs Playwright tests in CI and previously relied on Currents for reporting. While this worked initially, the setup became harder to justify as the test suite grew.
Flaky tests increased across environments. Investigating failures required repeated manual effort, and the reporting layer became expensive relative to the value it provided.
We were using Currents earlier, but as our test suite grew, it felt expensive and still lacked the visibility we needed.
Franklin adopted TestDino to gain better insight into flaky tests, simplify debugging, and reduce reporting costs. Over time, the team reported noticeable reductions in debugging effort and meaningful cost savings.
This case study explains what changed and how Franklin uses TestDino today.
Company Overview
Franklin provides a financial platform for e commerce businesses. The product includes:
- Payment cards with high spend limits
-
Expense management tooling
-
0.5 percent cashback on card expenses
Given the financial nature of the product, reliable CI feedback and stable test execution are essential for maintaining release confidence.
Johan Frølich Overview
Johan Frølich is the CTO and Co Founder at Franklin, where he leads engineering for financial products built for e commerce businesses.
He has a background in software engineering and platform development, with experience building and operating reliable systems at scale. At Franklin, Johan focuses on maintaining stable CI pipelines, improving test reliability, and ensuring developers have clear, trustworthy feedback from automated tests.
Franklin’s Playwright Testing Setup
-
Framework: Playwright
-
CI: GitHub
-
Previous reporting: Currents
Before TestDino, Franklin relied on Currents for Playwright reporting. Over time, the team found that it offered fewer features while becoming increasingly expensive as usage grew.
Key Challenges Faced by Franklin
As the Playwright suite and CI usage scaled, several issues became clear.
1. Limited visibility into failures
When test runs failed, the team lacked historical context to understand whether failures were recurring, flaky, or environment specific. Every failure required manual investigation.
2. Flaky tests with no effective tracking
Flaky tests increased as the test suite expanded across environments. The team struggled to understand test stability and identify which failures could be safely deprioritized.
This directly affected confidence in CI results.
3. Lack of intelligent insights and automation
The previous reporting layer did not provide intelligent failure insights. Debugging relied heavily on manual inspection rather than structured analysis.
4. High maintenance effort and price
The reporting setup required ongoing effort to maintain while also being costly relative to the value delivered.
How TestDino Helped in Franklin’s Workflow
With TestDino, it became much easier to understand failures and quickly tell whether something was flaky or a real issue.
What Changed in Practice
1. One Centralised Dashboard for All Playwright Runs
Instead of jumping between CI artifacts and tools, Franklin now starts investigations from a single dashboard.
The interface is intuitive and easy to use, which really helps when investigating failed tests day to day.
2. Tracks Flaky Tests with History and Context
Flaky tests became visible and actionable instead of speculative, enabling focused stabilization work.
3. Branch and Environment Mapping per Execution
Branch and environment context reduced manual comparison across runs and helped the team understand where instability originated.
Seeing test behavior clearly across branches and environments made a big difference for us. It became much easier to understand where instability was coming from.
4. GitHub and Linear Integration
Failures stayed close to the development workflow, reducing context switching and speeding up response times.
Where TestDino Helped
1. Flaky Test Dashboard
The flaky test dashboard stood out as the most impactful improvement.
TestDino made it much easier to find and fix flaky tests compared to our previous setup.
2. Ease of Use in Day to Day Debugging
Ease of use directly affected productivity during daily debugging.
Results After Adopting TestDino
Franklin saw clear operational improvements after adopting TestDino.
1. Debugging Time Reduced
TestDino reduced the effort required to investigate failures, even though exact time savings were not measured.
2. Easier Flaky Test Stabilization
Flaky tests became easier to identify and fix, reducing noise and unnecessary reruns.
3. Time and Cost Savings
Franklin estimates meaningful efficiency gains after switching to TestDino.
We estimate around 10 to 20% time saved on test and error related work, and roughly 30 to 40% cost savings compared to our previous reporting setup.
4. AI Native Approach by TestDino
AI driven insights helped teams focus on meaningful failures rather than noise.
Measurable Improvements After Adopting TestDino
| Area | Before TestDino | After TestDino |
|---|---|---|
| Failure investigation | Manual inspection with limited context | Centralized dashboard with history |
| Flaky tests | Hard to identify and track | Easier to find and fix |
| Environment visibility | Difficult to compare stability | Clear branch and environment mapping |
| Debugging effort | Repetitive and time consuming | Reduced by 10–20% |
| Reporting costs | Higher cost tool | 30–40% cost savings |
| Developer experience | Fragmented and noisy | Clear and intuitive |
What Would Regress if TestDino Were Removed
Without TestDino, Franklin would lose key visibility into test behavior and flaky patterns.
TestDino has become part of how we investigate Playwright test failures, especially for visibility into flaky tests.
Who Benefits Most at Franklin
-
Developers: Primary users for debugging, flaky test analysis, and CI investigation
Final Takeaways
Franklin adopted TestDino to solve scaling issues with Playwright reporting.
The outcome:
-
Better insight into flaky tests
-
Faster and simpler debugging
-
10–20% time savings
-
30–40% cost savings
-
Strong support and customer involvement
The product already has everything we want, but what really stands out is the communication, support, and how involved we are as customers.
Table of content
Flaky tests killing your velocity?
TestDino auto-detects flakiness, categorizes root causes, tracks patterns over time.
Follow Us
