JUnit Test Report Analyzer
Analyze JUnit XML from CI runs in your browser with pass-rate gates, failure triage, retry clues, skipped cases, and slow-test flags.| {{ header }} | Copy |
|---|---|
|
{{ emptyResultTitle(tab.key) }}
{{ emptyResultNote(tab.key) }}
|
|
| {{ cell.value }} {{ cell.value }} |
Continuous integration logs are written for machines first and people second. A failed job can contain compiler output, dependency noise, container setup lines, and hundreds of passing test names before the one result that matters. JUnit XML gives that same run a smaller, structured record: which suites ran, which test cases completed, how long they took, and which cases reported a failure, error, or skip.
The phrase "JUnit XML" is used across many test ecosystems, not only Java or JUnit itself. Pytest, Maven Surefire, Gradle, Jest adapters, GitLab CI, Jenkins, and other tools can emit or consume JUnit-style reports, but they do not all write exactly the same optional fields. That is why a report review should focus on stable evidence first: suite names, testcase names, outcome children, elapsed time, and the aggregate counts written by the producer.
| Term | Meaning in a report | Review question |
|---|---|---|
| Test suite | A named group of test cases, often matching a job, class, file, namespace, or runner output. | Did every expected suite write a report? |
| Test case | One recorded test result with a name, optional class name, duration, and optional outcome child. | Which exact case needs debugging or rerun evidence? |
| Failure | A test ran and an assertion-style expectation did not hold. | What product behavior or expectation changed? |
| Error | The test run hit setup, fixture, environment, teardown, or uncaught-exception trouble. | Did infrastructure or test harness setup break before the assertion? |
| Skipped | A case was reported but did not complete as a normal pass or failure. | Is the skip intentional, temporary, or hiding coverage? |
A release decision usually needs more than one count. Pass rate shows the broad health of the run, but one failed payment test, one setup error in authentication, or one skipped migration test can matter more than a large number of ordinary passes. Slow tests add another signal because they can make feedback loops expensive even when the job is green.
JUnit XML should be treated as the artifact that was successfully written, not as proof that every planned test ran. A crash before report generation, a misconfigured artifact path, expired report files, duplicate test names, or a runner that writes partial XML can make the report look cleaner or smaller than the real job. The safest review checks both the parsed report and the CI configuration that produced it.
How to Use This Tool:
Start with the JUnit-style XML artifact from the test job you want to review. Use one focused report or one related group of suite reports rather than a mixed archive from several unrelated jobs.
- Enter a short CI job or suite name so Suite Health, JSON, and downloaded reports identify the run.
- Set Pass-rate target to the percentage your release gate requires. Values are limited to the 0% to 100% range.
- Paste the report into JUnit XML report, use Browse XML/TXT, or drop an XML or TXT file onto the source field. Files larger than 2 MiB are rejected before analysis.
- Use Normalize if the XML is hard to scan after pasting. Use Load sample when you want to compare the expected result shape before using your own artifact.
- Open Advanced when timing or flake clues matter. Slow test cutoff controls which passing cases are called out as slow, and Flaky signal terms adds text matching across suite, test, message, and retry text.
- If a warning says JUnit XML parse failed or no suite/testcase entries were found, repair or re-export the artifact before trusting the gate readout.
- Read Suite Health first, then inspect Failure Triage for the cases that need action. Use Suite Outcome Map when several suites need a visual comparison.
Before sharing the result, confirm the total test count and suite names match the CI job you meant to review.
Interpreting Results:
The large percentage is the overall pass rate. Read it with the gate passes or gate fails badge, because the gate requires both a pass rate at or above the target and zero failed or errored test cases.
Suite Health is the release summary. A row with Failures or Errors above zero needs debugging before approval. A row with a good Pass rate and a nonzero Retry signals count may pass the gate but still needs stabilization work.
Failure Triage is intentionally broader than failed tests. It can include failures, errors, skipped cases, retry-marked passes, and passing cases at or above the slow-test cutoff. Start there when the job needs a rerun decision, owner handoff, or performance follow-up.
A clean gate does not prove the CI job was complete. Compare the suite count and job name with the pipeline configuration, especially after runner crashes, sharded jobs, artifact expiration, or a sudden drop in total tests.
Technical Details:
JUnit-style reports are XML documents built around <testsuite> and <testcase> elements. Suite attributes such as tests, failures, errors, skipped, and time summarize a group. Case attributes such as classname, name, and time identify one recorded test and its duration in seconds.
Case status comes from direct child elements. A <failure> child marks an assertion-style failure, an <error> child marks an execution error, and a <skipped> child marks a skipped case. When none of those children is present, the case counts as passed. Retry evidence is separate: <flakyFailure>, <flakyError>, <rerunFailure>, and <rerunError> can mark a case that needed more than one attempt.
Formula Core:
Pass rate uses counted test cases after failures, errors, and skipped cases are removed from the passed count. The gate then applies a second rule that blocks any hard failure, even when the pass-rate percentage meets the target.
A report with 120 tests, 1 failure, 0 errors, and 2 skipped cases has 117 passed cases and a 97.5% pass rate. With a 95% target, the percentage is high enough, but the gate still fails because failures + errors is greater than zero. Displayed percentages are rounded for reading, while the decision uses the numeric value behind the display.
| Evidence | Use | Boundary |
|---|---|---|
<testsuite name> |
Labels suite rows and chart categories. | A missing suite name falls back to a generated suite label. |
Direct <testcase> children |
Drive per-case status, duration, messages, retry markers, and triage rows. | When direct cases exist, they take priority over suite aggregate counts for that suite. |
| Suite attributes | Provide tests, failures, errors, skipped, and time when no direct case rows are available. | Nonnumeric or missing values are treated as zero where a number is required. |
| Outcome children | Set each case to failure, error, skipped, or pass. | Only direct outcome children on the testcase decide that case status. |
| Retry children and flaky terms | Raise retry counts or add a flake signal in triage. | No retry marker does not prove a test is stable across earlier runs. |
| Rule | Boundary | Result meaning |
|---|---|---|
| Gate pass | passRate >= target and failures + errors = 0 |
The run meets the configured percentage target with no hard failures. |
| Below target | passRate < target with no failure or error cases |
The job needs review because skipped or missing successful cases pulled the percentage below policy. |
| Hard fail | failures + errors >= 1 |
Any failed or errored testcase blocks the gate before release approval. |
| Slow pass | testcase time x 1000 >= Slow test cutoff |
The case passed but should be reviewed for feedback-time cost. |
| Flaky signal | Retry child present, or a configured flaky term appears in suite, case, message, or retry text. | The case may need stabilization even if the final outcome is pass. |
Advanced Tips:
- Set Pass-rate target to the release policy for the job, not to the number you expect from the current run. A lower target can hide skipped cases that should be reviewed.
- Use a repository-specific Slow test cutoff. A 500 ms cutoff is useful for a small unit suite, while browser, integration, or end-to-end suites may need a higher value to avoid noisy slow-pass rows.
- Add only high-signal words to Flaky signal terms, such as
rerun,retry,timeout, or a known flaky marker from your runner. Broad terms can make ordinary assertion messages look unstable. - When several suites are present, compare the total tests and suite names against the CI artifact list. A missing shard or expired artifact can make the pass rate look better than the real pipeline.
- Use Copy CSV or row copy from Failure Triage for owner handoff. Copying the whole JSON report is better for audit notes, but it can include class names and messages that should stay inside the team.
- Open Suite Outcome Map after reading the tables when you need a quick split of clean passes, retry signals, failures, errors, and skips across the largest suites.
Limitations and Privacy:
The analyzer reads the XML or text file in the browser and does not need to send the report to a server for parsing. It does not run tests, fetch CI artifacts, compare against repository history, or know which suites were supposed to exist. File input is limited to 2 MiB, and malformed XML must be fixed before the pass rate or gate badges are meaningful.
JUnit reports can contain class names, test names, assertion messages, stack traces, and environment clues. Even when parsing stays local, avoid sharing copied rows, JSON, or downloaded reports outside the team unless those details are safe to disclose.
Worked Examples:
Release job with one assertion failure
A backend report records 120 tests, 117 passed cases, 1 failure, 0 errors, and 2 skipped cases with a Pass-rate target of 95%. Suite Health shows a 97.5% Pass rate, but Gate note still says to fix failed or errored tests because the single failure is a hard block. Failure Triage should point to the failing testcase and its assertion message.
Below-target run without hard failures
A smoke suite has 40 tests, 36 passes, 0 failures, 0 errors, and 4 skipped cases. With a 95% target, Suite Health reports a 90.0% Pass rate and a below-target gate. The next check is the skip reason, because the job did not produce enough completed passes to satisfy the policy even though nothing failed.
Green run with retry and timing clues
A UI suite eventually passes every case but includes a <rerunFailure> marker on checkout retries payment. If another passing testcase has time="0.80" and Slow test cutoff is 500 ms, Failure Triage can list one flaky pass and one slow pass. The gate may pass, but the cases still deserve follow-up.
Truncated XML from a failed artifact upload
A copied report that ends before the closing </testsuite> tag returns a JUnit XML parse failed warning instead of partial confidence. Re-export the artifact, paste the full XML, and confirm Suite Health shows the expected suites before reading the pass rate.
FAQ:
Does a high pass rate always mean the gate passes?
No. The gate also requires zero failures and zero errors. A 99.9% pass rate still fails when any testcase has a <failure> or <error> child.
Can it read pytest, Maven, Gradle, Jest, Jenkins, or GitLab reports?
Yes, when the artifact is JUnit-style XML with <testsuite> or <testcase> elements. Producer-specific extras are ignored unless they match the outcome or retry markers used by the analyzer.
Why are skipped tests not counted as passed?
Skipped cases did not complete as ordinary pass/fail assertions. They reduce the passed count, remain visible in Suite Health, and can appear in Failure Triage when they need review.
Why did the source parse with no useful rows?
The pasted text may be a console log, an HTML report, or XML without <testsuite> or <testcase> entries. Use the actual JUnit XML artifact from the CI job.
Does a missing retry signal mean the test is stable?
No. Retry markers depend on the test runner and CI adapter. Treat retry counts and flaky-term matches as useful clues when present, then confirm stability with recent run history.
Glossary:
- JUnit XML
- A common XML report shape used by many test frameworks and CI systems to exchange unit-test results.
- Test suite
- A named group of test cases, usually represented by a
<testsuite>element. - Test case
- An individual recorded test result, usually represented by a
<testcase>element. - Failure
- An assertion-style result where the test ran and the expected condition did not hold.
- Error
- An execution problem such as setup, fixture, environment, teardown, or uncaught-exception trouble.
- Retry signal
- A rerun or flaky marker in the XML, or configured text that suggests the case needed more than one attempt.
- Slow test
- A passing testcase whose reported duration meets or exceeds the selected slow-test cutoff.
References:
- Surefire XML Report Schema, Apache Maven Project, 2026-05-28.
- JUnit plugin, Jenkins.
- JUnit Plugin Pipeline steps, Jenkins.
- Unit test reports, GitLab Docs.
- Creating JUnitXML format files, pytest documentation.
- Maven Surefire Plugin, Apache Maven Project.