AI Hallucination Detection: 3 Checks That Fail a Made-Up Answer
Struggling with a chat that invents your refund policy? Use 3 checks to catch that AI hallucination in CI.
Support chats are answering policy questions in full sentences, and some of those sentences add a number the policy never had. AI hallucination detection is the check that flags that gap before a customer relies on it.
The UI test still goes green. The bubble rendered, the button worked, and the pull request merged. Then support spends the afternoon walking back a 90-day refund the page does not offer.
This guide stays on one refund chat. You will split the answer into claims, fail the spec when a claim is invented, and keep that answer attached to the CI run.
What AI hallucination detection checks on a chat answer
AI hallucination detection asks one question of the text on screen: does this sentence follow from the source you gave the model, or did the model add something else?
AI hallucination detection checks whether a chat answer follows the source you gave the model. On this refund bot, that source is the policy file. A claim that adds 90 days or a phone number fails the check.
On a RAG chat, the source is the chunk the retriever returned, not the model's memory of the internet. A faithful answer is one where every claim follows from that chunk. The score is how many claims the source supports, divided by how many claims the answer made.
Wrong in the outside world and unfaithful to the page are not the same thing. If your source says water has a smell, then "water is odorless" fails this check, even when everyday knowledge says otherwise.
There are two shapes. One contradicts the source. The other adds something the source never mentions, so you cannot prove it from that page.
Here is the policy file this post uses, tests/fixtures/refund-policy.txt:
Refunds are available within 30 days of delivery.
Shipping is not refunded.
This policy does not cover exchanges.
The customer asks: "How long do I have to return a delivered order, and do you refund shipping?"
A bad answer looks like this: "You can return a delivered order within 90 days, and we refund shipping. Call 1-800-555-0199.

"90 days" contradicts "30 days." That is intrinsic. "We refund shipping" contradicts "Shipping is not refunded." That is intrinsic too. The phone number is extrinsic. The policy never mentions a phone, so the model filled a gap with a prop.
A second question belongs in the same spec: "Can I exchange this?" A bad answer invents "Exchanges are free within 14 days." A grounded answer says the policy does not cover exchanges. If your test only hunts for contradictions, that 14-day line can slip through, because nothing in the file says "14" is false. It is simply absent.
You also want the answer to address the question. "We ship worldwide" can be fine in some other doc and still ignore the refund question. One score should not hide the other.
When the policy file is the source you trust, compare the answer to that file. When the bot retrieved chunks at runtime, compare the answer to those chunks. If you score a live retrieval against the wrong page, you will blame the writer for a search miss.
Why the UI test stays green
You already have a Playwright E2E testing spec for this widget. It opens /support, types the question, clicks Send, and waits until the assistant bubble is visible.
That test is doing its job. It proves the page rendered a reply. It does not prove the reply is allowed to say what it said.
The cost shows up after the merge. A customer quotes the 90-day line back to support. The policy team has to issue an exception or admit the bot was wrong. The CI log looks clean because nobody asserted the number.
Playwright assertions split into two kinds, and the refund bug hides in that split. A web-first assertion retries until the bubble is visible. A generic assertion runs once on a string you already hold. toBeVisible() belongs on the bubble. toContain('30 days')belongs on the text after you have read it.
A screenshot will not save you here. Visual testing compares pixels. A wrong sentence in the same font, same bubble, same layout can sit inside your diff threshold. The lie is in the words.
So the spec has to read the words.

import { test, expect } from '@playwright/test';
test('refund bubble renders', async ({ page }) => {
await page.goto('/support');
await page.getByLabel('Ask about an order').fill(
'How long do I have to return a delivered order, and do you refund shipping?',
);
await page.getByRole('button', { name: 'Send' }).click();
await expect(page.getByTestId('assistant-message').last()).toBeVisible();
});
That spec passes on the 90-day answer. The failure you want is still one assertion away.
Split the answer into claims before you score it
A whole-answer score hides the sentence that hurts you. Say the source is "born in Germany on 14 March 1879" and the answer says "born in Germany on 20 March 1879." The country is supported. The date is not. Half the claims pass, and the date is still wrong.
Do the same math on the refund answer.
"You can return a delivered order within 90 days." Unsupported.
"We refund shipping." Unsupported.
"Call 1-800-555-0199." Unsupported.
Three claims, zero supported, faithfulness 0. If the model gets the window right and invents only the phone number, you are at 2/3, about 0.67. A gate set at 0.6 would pass. The customer still gets a fake phone number.
That is the bug in averaging. One invented claim is enough to fail a policy answer. The ratio is a report. The gate is "any claim unsupported."
Tip: Fail the spec when any one claim is unsupported. A faithfulness average of 0.67 can still hide the fake phone number in the third sentence.
You can split sentences in the test process without a model. This helper is deliberately dumb. It breaks on ., ?, and !.
export function splitClaims(answer: string): string[] {
return answer
.split(/(?<=[.!?])\s+/)
.map((claim) => claim.trim())
.filter((claim) => claim.length > 0);
}
It catches "within 90 days. We refund shipping." It misses "within 90 days and we refund shipping" because that is one sentence. When your bot writes long "and" sentences, do not trust this splitter alone. Break on "and" only when you have read a few real answers and seen that it does not slice a real clause in half. Or hand the same two strings to a judge that lists the claims for you.
Keep the ratio next to the list, not instead of the list:
export function faithfulnessScore(supported: boolean[]): number {
if (supported.length === 0) return 0;
const hits = supported.filter(Boolean).length;
return hits / supported.length;
}
supported is the part you must not fake. A boolean array that you typed by hand is a unit test of the formula. In CI, those booleans come from a rule or from a judge call that quotes the policy. If you hard-code true, you have tested nothing.
Three checks that catch the refund-policy lie
Use three checks, in this order. Stop at the first hard failure. The later checks explain the miss. They should not wash it out.
| Check | Input | Fails the refund example when |
|---|---|---|
| Required fact | A phrase the policy states | The answer never says 30 days |
| Forbidden claim | A phrase the policy rules out, or a pattern you never publish | The answer says 90 days, refunds shipping, or gives a phone number |
| Faithfulness | Supported claims divided by all claims | Any claim is unsupported, even if the average looks high |
The required fact is the cheap pass. For this bot, the answer must contain "30 days" on the return-window question. That string is in the policy, so a paraphrase like "one month" will fail this check even when a human would accept it.
Skip the required-fact string if your product is allowed to paraphrase. A closed policy with a fixed sentence can keep it. A bot that rewrites "30 days" as "a month from delivery" should not be gated on that exact phrase. Those teams should skip the keyword and judge the claim instead.
The forbidden list is how you catch the obvious lie the same day. "90 days", "refund shipping", and a phone-number pattern do not need a model. A word match is good at the blatant miss. It is weak on a reworded invention.
So keep the forbidden list for numbers, phone numbers, and promises your policy explicitly denies. Do not expect it to catch a reworded invention.

The third check is faithfulness against the file you trust. This refund spec can use the policy file, because you already know the right page. If you later point the same spec at live retrieval, switch the comparison text to the chunks the request returned. Otherwise you will punish the writer for a search miss.
A word-overlap check catches 90 versus 30. It misses "you have a full quarter to send it back." A judge can look at meaning, but one score is still a lead. Calibrate it on your own labeled answers before you copy a threshold from somewhere else.
Who should skip the judge in CI: a team whose allowed answers are a short list of sentences. Exact required facts are enough, and a judge adds a network call that can flake. Who should skip sampling-based checks: a UI test that sends one message. Comparing several generations of the same question needs those extra calls. One Playwright click does not produce them. Skip that until you store every sample.
The run is only useful if the answer text is on it. The next section puts that text there on purpose.
Wire the three checks into one Playwright spec
The spec lives in the suite you already run. This is still Playwright test automation: one config, one npx playwright test, Chromium unless you add a project. You do not need a second runner for the words.
Playwright architecture is why the assert can see the sentence. The test process tells the browser to click Send, then reads the bubble back as a string. The judgment happens in Node, on that string, next to the policy file.
Load the policy once with a Playwright fixtures file if several specs share it. For one spec, readFileSync is enough and the path is obvious in the failure.
If the chat page grows past this form, move the locators into a page object model and keep the claim checks in the spec. The page object should not know what "90 days" means. The spec should.
Playwright locators for this widget: the label on the textbox, the Send button's role, and a test id on the assistant bubble. last() matters because a thread keeps older bubbles.
Commands here use testInfo.attach and trace: 'retain-on-failure-and-retries'. Those options are in current Playwright. Retries stay on CI only, and forbidOnly fails the build if a focused test was left in.
import { defineConfig } from '@playwright/test';
export default defineConfig({
forbidOnly: !!process.env.CI,
retries: process.env.CI ? 2 : 0,
reporter: 'html',
use: {
baseURL: 'http://localhost:3000',
trace: 'retain-on-failure-and-retries',
},
});
trace: 'retain-on-failure-and-retries' records every attempt and keeps the trace when an attempt failed or when it was a retry. A judge that fails and then passes still leaves you the first answer. on-first-retry would skip the original failure's trace. You want that file.
import { readFileSync } from 'node:fs';
import { test, expect } from '@playwright/test';
import { splitClaims } from './support/claims';
const policy = readFileSync('tests/fixtures/refund-policy.txt', 'utf8');
async function ask(page: import('@playwright/test').Page, question: string) {
await page.goto('/support');
await page.getByLabel('Ask about an order').fill(question);
await page.getByRole('button', { name: 'Send' }).click();
const bubble = page.getByTestId('assistant-message').last();
await expect(bubble).toBeVisible();
return (await bubble.innerText()).trim();
}
test('return window does not invent a policy', async ({ page }, testInfo) => {
const answer = await ask(
page,
'How long do I have to return a delivered order, and do you refund shipping?',
);
await testInfo.attach('assistant-answer', {
body: answer,
contentType: 'text/plain',
});
await testInfo.attach('refund-policy', {
body: policy,
contentType: 'text/plain',
});
const folded = answer.toLowerCase();
expect(folded, 'missing the real window').toContain('30 days');
const forbidden = ['90 days', 'refund shipping', '1-800-555-0199'];
for (const claim of forbidden) {
expect(folded, `invented claim: ${claim}`).not.toContain(claim);
}
const claims = splitClaims(answer);
const supported = claims.map((claim) => {
const line = claim.toLowerCase();
const contradicts = forbidden.some((bad) => line.includes(bad));
return !contradicts;
});
const score = supported.filter(Boolean).length / supported.length;
expect(supported.every(Boolean), `unsupported claim in: ${claims.join(' | ')}`).toBe(true);
await testInfo.attach('faithfulness', {
body: String(score),
contentType: 'text/plain',
});
});
The assertion message is the method. invented claim: 90 days shows up in the Playwright report without a rerun. The two attachments are the answer and the policy, so the next person does not have to reproduce the chat to see the lie.
Open that attachment before you add another metric. The 90-day line is either in the file or it is not.
This faithfulness number is a stand-in. It marks a claim unsupported only when a forbidden phrase appears. A word list is a blunt filter. It will not catch "you have a full quarter to send it back." Add a judge only for claims that survive the list, and store a short quote from the policy next to the claim. A bare 0.82 does not show the lie.
There is a second test to add in the same file for the exchange question. The required behavior is abstention: the answer should say exchanges are not covered. Fail if it invents a day count. Do not score that abstention as a miss. A model that always answers looks perfect on the return-window question and still fabricates on the exchange question.
npx playwright test tests/refund-chat.spec.ts
Then open the report:
npx playwright show-report

If you have not installed the runner yet, the setup in learn Playwright is the same one this spec expects: npm init playwright@latest, then the spec above.
When the assertion fails, open the trace from that run before you change the threshold. The threshold is the temptation. The attachment is the evidence.
Keep the bad answer in CI instead of retrying it away
Locally, leave retries at 0. You want the invented claim on the first run, while you can still read the bubble. Retries belong on CI only, and that split is the right one here: retries: process.env.CI ? 2 : 0.
Note: Keep retries at 0 on your machine. On CI, a fail-then-pass still means the first answer was wrong, so keep that trace and read both sentences.
On CI the retry is a safety net for a crashed browser, not a way to shop for a nicer answer. A refund bot that says "90 days" once and "30 days" on the retry is not fixed. Playwright will mark that test flaky if the retry passes. Treat that flake as a product bug.
retain-on-failure-and-retries is what makes the comparison possible. You get a trace for the failing attempt and a trace for the retry. Open both in the Playwright Trace Viewer. If the DOM is the same and only the sentence changed, the model moved. If the bubble was empty in the first trace, your test read too early, and toBeVisible() was not enough. Wait for a non-empty bubble, or for the loading flag to detach, then read innerText().

Playwright in GitHub Actions should upload the HTML report and the test-results folder on every conclusion of the job, including failure. If the upload step is skipped when the test fails, the attachment you just added never leaves the runner.
A judge call needs one more rule. Pin the judge model name in the attachment. A score from last week and a score from today are not the same check if the judge changed. Fail the spec when the judge returns no quote. A score without a quote is how a fluent lie gets a 0.9.
Comparing several samples of the same question is a separate job, not this UI spec. If you add it, store each sample as its own attachment. Do not average them into the UI test and retry until the average clears.
This is also where an LLM evaluation framework is wider than hallucination detection. Relevance, tone, and latency can live in that framework. The refund spec should stay narrow: unsupported claims fail the build. Mixing them into one green score is how the phone number survives.
TestDino's @testdino/playwright reporter can sit beside the HTML reporter and stream the run, including sharded jobs, so the failed attempt and the retry land on one timeline. Fail-then-pass is marked flaky, and the stability percentage shows whether this chat spec keeps flipping. Then come back to the lesson: open the two traces and read the two answers.
If the same spec flips on three builds and the policy file did not change, stop raising the threshold. Pull the attached answers. The drift is in the model or in retrieval, and the attachments are the diff.
Conclusion
You do not need a new runner to catch this. Read the bubble, split the answer, and fail on the first claim the refund policy does not allow.
Keep the answer and the policy on the failed run. If the report names the invented claim, the spec is doing the job. If a fair paraphrase of "30 days" fails the string check, drop that required phrase and judge the claim instead.
Run npx playwright test tests/refund-chat.spec.ts and open the HTML report. Which sentence in your last failed answer is missing from the source?
