OpenAPI Operation Coverage Checker

API teams often know that a contract changed before they know which tests still exercise it. OpenAPI operation coverage turns that uncertainty into a countable review: each documented HTTP method and path becomes an operation that can be checked against evidence from contract tests, smoke tests, access logs, cURL commands, CI exports, or operation identifier manifests.

The important unit is the operation, not the route file or controller name. In OpenAPI, GET /orders and POST /orders are separate operations because they promise different behavior, authorization rules, response codes, and failure cases. A single path can therefore be half covered when read requests are tested but create, update, or delete requests are missing.

Coverage reviews are most useful when the denominator is stable. Teams need to agree which operations are in scope before judging the percentage, especially when a contract includes deprecated endpoints, webhooks, server base paths, generated operation identifiers, or methods that are intentionally outside the release gate. Changing that scope after the fact can make one coverage number look better without adding any new tests.

Operation set

The methods and paths that should have evidence, usually from an OpenAPI or Swagger document.

Evidence inventory

The test, log, command, CSV, or identifier lines that claim an operation was exercised.

Match rule

The way evidence is attached to an operation, such as operationId matching or method plus path matching.

Review queue

The missing operations, unmatched evidence, duplicate identifiers, and parser warnings that need owner review.

Operation coverage catches several problems that ordinary test-count reports miss. A test suite can contain many assertions while still ignoring a newly added PATCH route. A stale smoke test can keep calling /v1/orders/search after the contract moved the operation. A duplicated operationId can make identifier-based evidence look cleaner than it is. The percentage is therefore a starting point for release review, not proof that the API is fully tested.

A useful coverage run separates two questions. First, did every in-scope operation appear in the evidence? Second, is the evidence trustworthy enough for the decision being made? The first question can be measured mechanically. The second still needs human review of assertions, authorization, schemas, negative cases, and production behavior.

How to Use This Tool:

Use one API contract and one related evidence inventory per run so the percentage, queue, and ledger all describe the same release scope.

1. Paste or browse an OpenAPI source. OpenAPI JSON, YAML, Swagger 2.0, and loose METHOD /path lines can build the operation set. If the summary reports zero operations, switch the OpenAPI dialect or normalize the source before reviewing results.
2. Add the Test inventory. Operation ID lists, method-path rows, access-log snippets, cURL commands, CSV rows, and Newman-style exports can become evidence. The inventory badge shows how many rows were parsed and how many stayed unmatched.
3. Set API name, Coverage target, and Coverage gate profile. The release gate compares coverage with the target, while the strict write-operation profile adds extra pressure to uncovered POST, PUT, PATCH, and DELETE operations.
4. Choose a Coverage match mode. Use operation ID or method plus path for mixed evidence, operation ID only for stable manifests, method plus templated path for logs, and server-prefix stripping when evidence includes a known base path.
5. Check Advanced before trusting the denominator. Deprecated operations, webhook path items, and ignored HTTP methods change which operations count toward coverage.
6. Read Coverage Gate Metrics first, then work through the Operation Coverage Ledger, Coverage Gap Queue, Inventory Match Audit, and Method Coverage Matrix. Parser warnings, duplicate identifiers, and unmatched inventory rows need review even when the target is met.

Use Normalize when pasted evidence is noisy. If useful rows disappear, change the inventory format and rerun the comparison before deciding that a gap is real.

Interpreting Results:

The summary percentage answers only the first coverage question: how many in-scope operations had matching evidence. The release risk often sits in the supporting rows. A target met badge can still appear beside uncovered operations, duplicate operation identifiers, parser warnings, or unmatched inventory that points to contract drift.

Do not treat 100.0% as proof of test depth. It means every counted operation had at least one matching evidence entry. Review the ledger, queue, and inventory audit to confirm assertions, authorization checks, schema coverage, and live-service behavior are still represented elsewhere in the test process.

Technical Details:

OpenAPI describes operations as HTTP methods attached to path templates. Path parameters such as {orderId} describe one segment of a concrete request path, so evidence from GET /orders/ord_123 can match GET /orders/{orderId} when templated path matching is allowed. Query strings are removed before path comparison because coverage belongs to the operation, not to a particular filter value.

Operation identifiers add a second matching path. They are useful for generated clients and contract-test manifests, but they must stay unique. When two operations share the same identifier, one evidence line may map to more than one operation, which turns the result into review evidence instead of clean coverage proof.

Formula Core

Coverage is calculated after the operation denominator is built from the selected source, deprecated-operation setting, webhook setting, and ignored-method list.

For example, 27 covered operations out of 30 in-scope operations produces 90.0% in the summary. The detailed JSON keeps the percentage to two decimal places, so the same run is recorded as 90.00 there.

Rule Core

Boundary Conditions

Advanced Tips:

• Keep the Coverage target stable before comparing release runs. Changing the target after the evidence is parsed can hide a real denominator or testing gap.
• Start with Auto-detect OpenAPI or Swagger, then pin the OpenAPI dialect when parser warnings say the selected format does not match the source.
• Use Operation ID only only when operation identifiers are unique and trusted. If duplicate identifiers appear, switch to a method-path match mode or fix the contract before using ID evidence as the main proof.
• Review Deprecated operations, Webhook path items, and Ignored HTTP methods before the run becomes release evidence. These settings change the denominator that drives the coverage percent.
• Use CI contract gate when parser warnings, duplicate IDs, or unmatched inventory should block CI evidence. Use Exploratory audit for early planning where the queue matters more than pass or review status.
• Export the artifact that matches the handoff. The ledger supports audit review, the gap queue supports owner assignment, the inventory audit catches stale evidence, and the method matrix shows whether missing coverage is concentrated in one HTTP method.

Privacy and Accuracy Notes:

The contract and inventory text are analyzed in the browser. File browse and drop actions read local text files into the input fields, and files larger than 1 MB are refused by the file loader. Remove tokens, internal hostnames, customer identifiers, unreleased route names, and sensitive log details before sharing exported evidence outside the intended team.

Coverage evidence proves only that an operation was referenced under the selected rules. It does not prove assertion quality, negative-case testing, authentication coverage, authorization behavior, response schema validation, rate-limit handling, or whether the deployed service matches the contract.

Worked Examples:

Release target met with a remaining queue

An orders API has 48 in-scope operations and the inventory matches 45 of them. Operation coverage reports 93.8% against a 90% target, so the release gate can pass under the release profile. The Coverage Gap Queue still lists the three uncovered operations, which should be assigned or accepted before the release notes claim complete coverage.

Strict profile blocks a missing write route

A smaller admin API has 10 operations, 9 covered, and one uncovered DELETE /users/{userId}. The percentage is exactly 90.0% against a 90% target, but Strict write-operation gate reports a blocker because a state-changing operation lacks evidence. The next useful action is to add a delete test or exclude the route only if it is outside the active release scope.

Unmatched evidence after a base-path change

A contract contains GET /orders/{orderId}, while access logs show GET /v1/orders/ord_123. The Inventory Match Audit may show the log line as unmatched until the match mode strips the known server prefix or the contract source includes the matching server base path. Once the path basis is aligned, the ledger can mark the operation covered.

FAQ:

Glossary:

Operation

An HTTP method on an OpenAPI path, such as GET /orders or PATCH /orders/{orderId}.

Operation identifier

A stable name for an operation, commonly used by generated clients, manifests, and reports.

Evidence inventory

The list of tests, logs, commands, CSV rows, or identifiers used to prove that operations were exercised.

Gate profile

The review policy that decides how target gaps, warnings, unmatched evidence, and write-operation gaps are summarized.

Path template

An OpenAPI path with placeholders in braces, such as /orders/{orderId}.

Deprecated operation

An operation marked as retired or discouraged but still present in the contract.

Method Coverage Matrix

The chart view that compares covered and missing operations by HTTP method.

References:

• OpenAPI Specification v3.2.0, OpenAPI Initiative, September 19, 2025.
• OpenAPI Paths and Operations, OpenAPI Initiative.
• Providing Documentation for Webhooks, OpenAPI Initiative.
• API Servers, OpenAPI Initiative.