Detect and alert

You will learn how metrics, monitors, alerts, and notification channels turn a saved query into something Cordo watches for you.

Query and views answer what is happening right now. Detect answers tell me when something changes — without you re-running a search. Detection is built on the same learned fields and aggregates you already use; it records those signals and evaluates rules against them on a schedule.

The four nouns

Noun	What it is
Metric	A recorded aggregate — a bounded count/sum/percentile query snapshot that Cordo runs on a schedule and stores as a time series.
Monitor	A detection rule over a saved view, a recorded metric, or anomaly findings. Decides when a signal is in breach.
Alert	A fired monitor instance with a lifecycle (firing → resolved) and immutable evidence of why it fired.
Notification channel	A reusable destination — webhook, email, Slack-compatible, or PagerDuty — that alerts are delivered to.

A metric feeds monitors and keeps history cheaply; a monitor watches a metric or view; an alert is what a monitor produces when its condition holds; a channel is where that alert is sent.

Anomaly detection adds two more nouns that feed the same alerts — patterns and findings. See Patterns and findings.

Metrics

A metric is an aggregate query you have chosen to record. Instead of only running on demand, Cordo evaluates it on a fixed recording interval and stores each result as a point in a series, with optional grouping.

Query-time aggregates stay available without recording — a metric is for signals you want to keep and watch.
Recorded-metric retention is independent of raw event retention, and is typically longer and cheaper.
Series are bounded by cardinality and count caps. When a cap prevents a new series from recording, the reason is visible to the stream owner.

Monitors

A monitor evaluates one source — a saved view or a recorded metric — and decides whether it is in breach. Cordo supports four condition types:

Condition	Breaches when
Threshold	The value is above / below / equal to a target within the window. A zero-valued aggregate is present data, not absence.
Absence	An expected signal stops arriving (no data / no rows) in the window.
Change	The value moves too far versus the equivalent prior period (delta or percent).
Baseline	The value deviates from a learned seasonal baseline. Baseline detection stays explainable — you see the baseline and the deviation, not just a score.

A Findings monitor is the exception: its source is the stream's findings, filtered by kind, minimum severity, service, and pattern. It has no condition or window to set — after each anomaly scan it raises one alert per matching finding (default) or one alert per co-firing incident, and resolves when the finding or every incident member closes. See Alert on findings.

Window vs frequency

Two intervals govern every monitor:

Window — how much time each evaluation looks at (for example, the last 5 minutes of data).
Frequency — how often the evaluation runs (for example, every 1 minute).

Eval offset shifts that window behind wall clock so recording and materialization can catch up (default 5m). The form explains the offset as looking N minutes behind wall clock.

View-sourced monitors also wait automatically while events from the window are still being written to analytics (materialize_outbox). The timeline shows that skip as waiting for analytics (analytics_lag); the same window evaluates once the backlog clears. Metric sources already wait on recorded buckets. Eval offset remains an extra shift behind wall clock when you want more settling time.

For metric sources these use the same interval tokens as recording. See Relative lookbacks.

Grouping

When the source is grouped (for example, by status or route), a monitor can evaluate per group so one noisy series does not mask another. Each group carries its own breach/OK streak, and groups are never merged into a single value.

Flap suppression and cooldown

Monitors are deliberately slow to cry wolf. Confirmation is configurable per monitor (trigger_periods / recovery_periods, default 2, range 1–10):

A first breach is pending, not an alert — it takes N consecutive breaches to open an alert.
It takes N consecutive clears to resolve.
After a resolve, a cooldown (one frequency) passes before the monitor can re-open, so a flapping signal does not spam you.

Optional recovery value adds value hysteresis on top of that period count. While an alert is open, a period counts toward recovery only when the observed value (or deviation, for change/baseline) clears past the recovery threshold — below the fire threshold for gt / gte, above it for lt / lte. Values between the two thresholds keep the alert open and do not advance the OK streak. With no recovery value set, clear uses the same threshold as fire.

A monitor with no attached channel still evaluates and shows alerts in-app — it just has nowhere to deliver them until you attach one.

Alerts

An alert is what a monitor produces when its condition holds through flap suppression. Alerts have a clear lifecycle — firing then resolved — and one open alert per monitor + group at a time.

Every alert is explainable. It links back to:

the originating monitor,
the evaluated window,
the metric or view values that triggered it (including compare / baseline / deviation for change and baseline conditions), and
a bounded sample of contributing events.

Fire-time and resolve-time evidence are captured immutably, so the exact observation that opened an alert survives later evaluations.

Monitor history

Absence of an alert does not prove health — a monitor may be disabled, skipped, starved of data, in cooldown, or failing to evaluate. Monitor history is an immutable, seven-day timeline that answers what has this monitor actually been doing?

It records state and observed-value samples, lifecycle edges (fired / resolved / enabled / disabled), skipped or not-ready windows, and bounded evaluation errors.
Healthy, pending, firing, cooldown, no-data/hold, disabled, and error periods are shown distinctly.
Gaps in coverage are surfaced as unknown, never inferred as healthy.
You can inspect ranges of 1h, 6h, 24h, 7d, or a custom range; older detail is purged after seven days, and the UI tells you when a request reaches that boundary.

Notification channels

A channel is a reusable delivery destination you attach to one or more monitors:

Type	Delivers to
Webhook	An HTTPS endpoint (optionally signed)
Email	One or more recipients over SMTP
Slack	A Slack-compatible incoming webhook URL
PagerDuty	PagerDuty Events API (trigger / resolve)

Destinations and channel secrets are write-only after save. Cordo never returns the complete webhook or Slack URL, email address, signing secret, or PagerDuty routing key. People with the Settings stream role see only a non-reusable destination description and can replace a destination by entering it again; people with Query access see the channel name, type, enabled state, monitor attachment, and sanitized delivery health.

Creating, changing, deleting, testing, and attaching channels requires the Settings stream role or higher. You can test-fire a channel before relying on it, and delivery retries with backoff. Health errors preserve useful status such as an HTTP response code, but do not echo third-party URLs, headers, or response bodies.

What detect is not

Not an external-metrics backend — Cordo records metrics from your logs, it does not scrape Prometheus or ingest OTLP metrics.
Not a general APM rules engine — detection is scoped to log-derived signals.
Not a replacement for Query — you still explore interactively; detect is for the signals worth watching.
Not only rules you write — anomaly detection watches every busy pattern and recorded metric against its own history, and a Findings monitor turns that into alerts.