OpenAPI Operation Coverage Checker

A test suite can grow while newly documented API behavior remains untouched. Operation coverage asks a narrower and more useful question: which HTTP method-and-path operations in the current OpenAPI contract have a corresponding piece of test or request evidence?

The denominator matters as much as the percentage. GET /orders and POST /orders are separate operations even though they share a path. Deprecated operations, OpenAPI 3.1 webhooks, and intentionally ignored methods can be included or excluded, so two coverage numbers are comparable only when they use the same scope.

Operation

One supported HTTP method on one OpenAPI path or in-scope webhook.

operationId

A case-sensitive identifier that should be unique across the API and can provide a stable evidence key.

Evidence row

A test reference, method/path line, request log, cURL line, CSV row, or identifier that can be matched to an operation.

Coverage gap

An in-scope operation with no matching evidence, or evidence that cannot be attached cleanly to the contract.

A match proves only that a reference exists. It does not prove the request succeeded, assertions passed, authorization was tested, error responses were covered, or business workflows are complete. Access logs can show traffic without showing an assertion, while a test name can remain after the underlying test was skipped or deleted.

Stable identifiers improve review quality, but method and templated path matching remains useful for request traces. A concrete path such as /orders/ord_123 can match /orders/{orderId} when the method also agrees. Duplicate operation identifiers make ID-based evidence ambiguous and should be corrected before they become release keys.

Use operation coverage as a release inventory. It helps owners find missing or stale evidence quickly, while the test runner, CI system, and service telemetry remain responsible for proving behavior.

How to Use This Tool:

Choose a contract, evidence set, and denominator that describe the same release.

1. Paste or load the OpenAPI source. Use automatic dialect detection for ordinary OpenAPI 3.0 or 3.1 JSON or YAML, and select loose method/path lines only for an inventory without a structured document.
2. Paste the Test evidence and choose its format when automatic detection cannot distinguish identifiers, request lines, or CSV data.
3. Select the Matching rule. Use operation ID first when identifiers are stable, or method plus templated path when the evidence comes from requests and logs.
4. Set the operation scope, ignored HTTP methods, gate profile, and integer coverage target from 0 through 100.
5. Review missing operations, unmatched evidence, unparsed rows, and duplicate IDs. Correct those records before treating the percentage as release evidence.

Interpreting Results:

Coverage percent tells you how much of the selected operation set has at least one matching evidence row. The operation ledger shows the numerator directly. The gap queue deserves equal attention because stale, ambiguous, or unparsed evidence can make the percentage look more reassuring than the inputs justify.

• A release gate passes when coverage meets or exceeds the target.
• A strict gate also requires zero unmatched rows, zero unparsed rows, and zero duplicate in-scope operationId values.
• An advisory gate always passes locally and is intended for review rather than enforcement.
• A matched request or ID is not proof of a passing assertion. Confirm status, assertions, and CI outcome in the real test system.

Technical Details:

An OpenAPI Paths Object supplies path items, and each supported HTTP method under a path becomes a separate operation. OpenAPI 3.1 can also describe incoming webhooks with path-item semantics. The selected scope decides whether deprecated operations and webhooks enter the denominator before matching begins.

Transformation Core

The comparison follows a deterministic sequence so that scope and matching decisions remain visible.

1. Parse structured JSON or YAML, or read loose METHOD /path lines.
2. Collect GET, POST, PUT, PATCH, DELETE, OPTIONS, HEAD, and TRACE operations after applying deprecated, webhook, and ignored-method rules.
3. Parse each evidence row as an explicit operationId, method/path request, HTTP log or cURL line, or CSV/Newman-style row.
4. Match by the selected rule. Identifier comparison is exact and case-sensitive. Method/path comparison requires the same method and allows one concrete non-slash segment for each OpenAPI path parameter.
5. Mark every matched operation covered, then derive missing operations, unmatched evidence, unparsed evidence, duplicate identifiers, and per-method totals.

Query strings and fragments are removed from request paths before method/path comparison. A trailing slash is optional for that comparison, but base-path or version-prefix differences are not rewritten automatically.

Formula Core

Every in-scope operation contributes once to the denominator, regardless of how many evidence rows match it.

The percentage is compared with the selected integer target using an inclusive boundary: target met when coverage ≥ target.

Rule Core

Each source is limited to 1,000,000 characters and the parsed scope to 10,000 operations. If automatic structured parsing fails but method/path lines are present, automatic mode can resolve to the loose-line dialect and retain a parser warning.

Privacy and Accuracy Notes:

The OpenAPI source and test evidence are processed in the browser and are deliberately kept out of the page URL. Files are read locally.

• Remove credentials, tokens, private hostnames, and customer data from logs before pasting or exporting evidence.
• Callbacks, links, response codes, assertions, test pass status, and business-path completeness are outside the coverage denominator.
• Referenced multi-file OpenAPI documents are not fetched. Supply a resolved document when external references affect the operation set.
• Repeat comparisons with the same dialect, scope, ignored methods, matching rule, and target.

References:

• OpenAPI Specification 3.1.1, OpenAPI Initiative.
• OpenAPI Specification 3.0.3, OpenAPI Initiative.