Fields and shapes

You will learn how Cordo discovers fields and ranks shapes from ingested JSON.

Fields

As events arrive, Cordo observes JSON paths and value types and evolves a per-stream field catalog. You filter and aggregate on those learned paths — you do not hand-write JSON extracts for everyday queries.

Typical catalog facts for a field:

Path (for example level, http.method, items[].id)
Learned type (string, number, boolean, object, array, …)
Samples (redacted when sensitive)
First / last seen

Explore the catalog on Fields. Windowed cardinality and top values (from the analytics store) are available there for operational fields and may lag slightly behind raw ingest.

Common semantics on Fields groups compatible names (Severity, Message, Service, Environment, request/trace id, Timestamp) so a level filter can also match severity or log.level. Query chips show those extra paths as also ….

Envelope originals such as log._raw are hidden in Events and field displays; Copy still includes the stored payload.

Field and shape catalogs have soft/hard caps. When a cap is active, Fields and Insights show a banner; Streams has the full Schema learning panel. Events still ingest; new paths or fingerprints are skipped. This is not a type conflict.

Type conflicts

When the same path is observed with incompatible concrete types (for example string then object, or number then bool), Cordo keeps the first learned type and records the other in the Conflict column on Fields. Conflicting values do not corrupt analytics columns; incompatible inserts become null for numeric/bool columns.

Conflict does not clear itself when later events match the original type. Nested children of an object can still appear in the catalog when object-shaped events arrive (for example log.msg while log itself shows a conflict).

Prevention: emit one consistent JSON shape per path. A frequent DigitalOcean case is plain-text stdout in log mixed with nested JSON — see OpenSearch-compatible ingest. To clear a stuck conflict after producers are fixed, use Resolve on Fields (Clear a field type conflict).

Shapes

A shape is a recurring structural fingerprint: which shallow keys appear together on an event.

Depth includes top-level keys and one nested level (parent.child, parent[].child).
Key presence matters: a field whose JSON value is null still counts as present (distinct from an absent key).
Each distinct key set gets a fingerprint, event count, share of stream events, and optional redacted sample.

Shapes answer “what layouts do we actually see?” Fields answer “what can we query?”

Where shapes show up

Surface	Role
Insights	Ranked shapes table, suggested views, preview/create
Saved views	Optional structural cohort matcher (`exact` or `contains`)
Query / agents	Same matchers when a view or query includes `shape`

Near-identical shapes that differ only by sparse optional fields may be grouped into families on Insights suggestions. The shapes table itself stays exact so you can still debug schema drift fingerprint by fingerprint.

User-agent enrichment

If the stream has Parse user agents enabled, ingest may add derived {field}_parsed objects beside detected user-agent fields. Those paths appear in the field catalog like any other learned fields. See Streams and ingest.