| Metric | Value | Copy |
|---|---|---|
| {{ row.label }} | {{ row.value }} | |
| No metrics available. | ||
| Field | Pattern | Example | Type | Copy |
|---|---|---|---|---|
|
{{ field.renameError }}
|
{{ field.pattern }} | {{ field.sample || '—' }} | {{ field.type }} | |
| No captured fields detected. | ||||
| Sample | Status | Captured | Coverage | Copy |
|---|---|---|---|---|
| #{{ row.index }} · {{ row.snippet }} |
{{ row.status === 'matched' ? 'Matched' : row.status === 'failed' ? 'No match' : row.status === 'error' ? 'Error' : 'Blank' }}
{{ row.message }}
Fields
{{ field.name }}={{ field.value }},
, ...
|
{{ row.capturedCount }} | {{ (row.coverage || 0).toFixed(1) }}% | |
| No samples analysed yet. | ||||
Filebeat and Elasticsearch become much easier to work with when repeated parts of a raw log line land in stable fields instead of staying buried inside one long message string. A person can spot a status code, URL path, or host name by eye, but filters, alerts, and dashboards work better when those values are captured under names such as http.response.status_code, url.path, or host.name. This generator helps with that first parsing draft by turning one representative log line into a Grok expression you can inspect before you wire it into an ingest pipeline.
You can start from nine built-in presets for common formats or stay on the custom path for application logs. The page returns more than one output: a Grok pattern preview, editable field mapping, pattern metrics, a sample validation view, a token-type chart, a JSON summary, and, when enabled, an ingest pipeline snippet. That makes it useful both for quick copy-and-paste work and for slower review where another engineer needs to see how the parser was assembled.
The tool works best when one line really does represent the format you need to parse. It is less dependable when a source changes shape across lines, hides important values behind inconsistent quoting, or needs multiline context such as stack traces. The page also ignores everything after the first non-empty pasted line, so a clean result should be treated as a draft to test against more real samples rather than proof that the full log stream will parse cleanly.
Pattern drafting, field review, validation, and export all happen in the browser. The tool does not define a server-side parsing endpoint for your log text, which makes it suitable for preparing patterns without sending sample lines to a separate helper service.
Elastic's grok processor extracts structured fields from a single text field by applying named pattern fragments. The generated pipeline snippet in this tool follows that model closely. It builds one grok processor against the source field you choose, keeps ignore_missing enabled, leaves trace_match off, and can append a simple event.module setter when you supply a module tag. If you enter a pipeline ID, that ID is added to the exported JSON as well.
The generator has two drafting paths. The preset path uses fixed patterns and known field targets for Apache combined and common access logs, Apache error logs, Nginx ingress and error logs, HAProxy HTTP logs, IIS W3C logs, Postgres CSV logs, and RFC 3164-style syslog. The custom path is heuristic. It looks for one timestamp based on the selected hint, then scans for standalone log levels, key=value fragments, key: value fragments, IP addresses, host names, HTTP methods, paths, durations, status codes, and a trailing message segment. Between each detected capture it escapes the literal separators from your sample line so the finished pattern stays anchored to the original punctuation.
sample line -> preset template or heuristic markers -> Grok captures -> field names -> match check
field review -> optional field rename -> updated pattern -> optional ingest pipeline JSON
copy, download, or export tables, JSON, and chart output for handoff
ECS naming rules matter here because consistent field sets make search and dashboards more reusable. Elastic recommends populating @timestamp, mapping as many fields as possible to ECS, keeping names lowercase, and using dotted field sets such as host.* or http.*. The generator follows that direction when Normalize field names is on. Known keys like status, path, method, and pid are mapped to recognizable ECS-style targets, while unknown keys are normalized under custom.*. A field prefix can then be applied on top of most captured names when you want the draft to live inside a dataset-specific namespace.
Type inference is intentionally narrow. Numeric values can become :int or :float suffixes, and epoch timestamps are emitted as a numeric long. True and false values are recognized as boolean-like in the field typing output, but the generated Grok fragment still uses a word token rather than a dedicated boolean cast. That is helpful to know if you expect the exported pattern to do every downstream conversion by itself.
| Stage | What the tool does | What you still need to verify |
|---|---|---|
| Preset path | Loads a ready-made sample, pattern grammar, and field map for nine common log families. | Confirm your real line truly matches that format and does not contain extra columns or missing segments. |
| Custom path | Detects timestamps, log levels, structured pairs, loose tokens, and a trailing message to assemble one anchored pattern. | Check that punctuation, quoting, and token order were modeled correctly rather than merely guessed well enough to look plausible. |
| Field naming | Maps familiar keys to ECS-style names and normalizes unknown keys under custom.* when requested. |
Decide whether the names fit your data model or need manual renaming before handoff. |
| Sample validation | Compiles the generated Grok into a regular expression and checks it against the accepted sample line. | Retest with additional real lines one at a time, because the page validates only the first non-empty pasted line. |
| Pipeline export | Builds a minimal ingest pipeline JSON document around the generated pattern. | Add any production-specific failure handling, enrichment, and broader pipeline policy outside this draft. |
The page also keeps the draft inspectable. You can rename fields directly in the mapping table, and the underlying Grok pattern is rewritten to match the new names. That is a practical advantage over opaque pattern generators because you can refine the parser without starting from scratch.
Start with a preset whenever your source clearly looks like Apache, Nginx, HAProxy, IIS, Postgres CSV, or classic syslog. That route is faster because the field names, token types, and pattern order are already tuned for a known structure. Choose the custom path when your line mixes fixed text with smaller structured fragments such as level=INFO, status=200, request_id=abc123, or duration=35.
The quality of the result depends heavily on the quality of the sample you paste. Use a line that contains every important field you want to preserve, not the shortest or cleanest line in the file. If one rare field appears only on certain lines, the parser you draft from a simpler example may never capture it.
Timestamp hint when the date shape is ambiguous. A good hint prevents the first token from being treated as ordinary text.Normalize field names on when you want ECS-style names. Turn it off only if you need source keys preserved as-is.Field prefix when the captures belong inside a dataset or app-specific namespace and you want to avoid collisions with existing fields.Detect key=value pairs on for app and ingress logs that mix narrative message text with structured fragments. Turn it off for rigid positional formats where equals signs are mostly noise.Infer numeric & boolean types for cleaner number handling, but remember that booleans are labeled in the output rather than emitted as a special Grok cast.A strong fit for this tool is a repeatable single-line format with predictable separators. A weak fit is multiline data, a source with several incompatible layouts, or a line where the real structure lives inside nested quoted payloads. In those cases, the draft can still be useful as a starting point, but it should not be mistaken for a finished parser.
The fastest stop signal is disagreement between the outputs. If the field mapping looks sensible but the sample still shows No match, the line probably depends on punctuation or quoting the generator modeled incorrectly. Fix that before you trust the pattern anywhere else.
Sample log line. If you paste more than one, the tool keeps only the first non-empty line.Custom and let the heuristic path draft the parser.Timestamp hint, Normalize field names, Detect key=value pairs, Infer numeric & boolean types, and an optional Field prefix so the draft reflects the log style you actually want to ship.Grok Pattern tab and then check Pattern Metrics. A clean-looking pattern is not enough if validation errors or a failed match appear.Field Mapping and rename any awkward captures. The tool updates the underlying pattern for you, so this is the right place to fix naming before export.Sample Analysis, Pattern Types, and JSON if you need a fuller audit trail for handoff or ticket notes.Include ingest pipeline JSON only when you want a pipeline draft. Then set the source field, optional pipeline ID, and optional module tag.The most useful outputs are not all measuring the same thing. Confidence score reflects how strong the generator thinks its draft is. Known field coverage measures how many captures map to recognized field concepts. Matches and Failures tell you whether the compiled pattern actually matched the accepted sample line. Those numbers should be read together, not in isolation.
| Signal | What it actually tells you | What it does not prove |
|---|---|---|
Confidence score |
How strong the preset or heuristic draft looks based on the tokens it recognized. | That the parser will hold up on real production variation. |
Known field coverage |
How much of the capture set maps to recognizable field names rather than generic or custom names. | That the line matched successfully or that the field model is complete. |
Matches / Failures |
Whether the generated pattern matched the accepted sample line after compilation. | That sibling lines from the same source will also match. |
Literal characters escaped |
How much exact punctuation from the sample had to be preserved in the final pattern. | That a high count is automatically bad. Some formats genuinely depend on exact separators. |
Pattern Types |
The mix of token families in the draft, shown as a pie chart with image and CSV export. | That each token choice is semantically perfect for every future line. |
Good results usually line up in a simple way: the pattern matches, the main fields look familiar, and the metrics do not hide a surprising amount of generic text. Weak results tend to split apart. For example, you may see high naming coverage and still get a failed sample match because one bracket, quote, or separator was modeled too literally.
The export surfaces help with review. Pattern, pipeline JSON, metrics, field tables, sample analysis, chart output, and the structured JSON tab can all be copied or downloaded, which makes it easier to hand the draft to someone else without re-explaining the result from memory.
An Apache combined line is the cleanest preset example because the order is stable and the fields are familiar. The generated pattern captures the client address, identity fields, timestamp, request method, path, HTTP version, status code, byte count, referer, and user agent. In a good run, the summary badges and the field table agree with each other, which makes the result a strong starting point for an access-log ingest pipeline.
Suppose your log looks like level=INFO message="Started request" duration=35 path=/api/items status=200 after a timestamp and host prefix. The custom path can detect the timestamp, severity, message, numeric duration, request path, and status code, then map the recognizable keys toward ECS-style names. If the resulting field map looks right but the sample fails to match, the issue is usually not the field choice. It is usually the exact punctuation around the values.
If your logs arrive under event.original rather than message, turn on pipeline export and change the match field before copying the JSON. You can also add a pipeline ID and module tag for a more reviewable handoff. The result is still intentionally minimal: one grok processor, one optional set processor for event.module, and no extra error branch or enrichment logic.
No parser service is defined for this page. Pattern drafting, validation, and export are handled in the browser.
The generator only accepts the first non-empty line as the drafting sample. That keeps the parser focused on one canonical format, but it also means you need to test additional lines one at a time.
custom.*?That happens when the key was not recognized as one of the tool's built-in canonical mappings and name normalization was enabled. It is the tool's way of keeping the field lowercase and namespaced without pretending it knows the exact ECS target.
Not in the exported pattern. Numeric values can gain :int or :float suffixes, but true and false values are only labeled as boolean-like in the field typing output.
Yes. The field table lets you rename captures, and the Grok pattern updates to match. Empty names, whitespace, duplicate names, and colons are rejected so the result stays usable.