Skip to content

Alert on findings

You will learn how to send findings to Slack, email, PagerDuty, or a webhook with a Findings monitor, and how to choose which findings are worth an alert.

Findings stay in the app until a Findings monitor picks them up. The monitor uses the same notification channels as every other monitor.

Before you start

  1. Select a stream and turn on anomaly detection.
  2. Have the Settings stream role or higher.
  3. Create a channel if you don't have one: on Streams, click New notification channel and choose Slack, Email, PagerDuty, or Webhook. See Detect and alert.

1. Start a Findings monitor

Start from scratch:

  1. Open Monitors and click New monitor.
  2. Under Source, choose Findings.

Or start from what you are looking at, with the filters filled in:

Where Click Prefills
A finding Alert on findings like this Its kind, service, and pattern
A finding's Alerts card, when nothing alerted Add a Findings monitor Its kind, service, and pattern
A pattern on Patterns Alert on findings The pattern

If anomaly detection is off for the stream, the form warns Anomaly detection is off with a Turn on anomaly detection link — the monitor won't alert until it is on.

2. Choose which findings alert

Field Choices Default
Kinds New pattern, Spike, Drop, Error rate, Volume, Metric All
Minimum severity Info, Warning, Critical Warning
Only these services Up to 50 services Any service
Only these patterns Up to 50 pattern numbers Any pattern
Alert grouping Per finding (default) or Per incident Per finding
  • With the default Warning, Info findings — new patterns without errors or warnings — don't alert. Choose Info to include them.
  • Only these services matches findings tied to a service, which is Error rate findings. Leave it empty for the others.
  • Per incident sends one alert for co-firing findings from the same outage (same scan, and later findings that share a service or pattern with that group). Prefer this when one failure raises many findings.
  • Name the monitor. Prefilled monitors suggest a name such as Spike findings · pattern #7 · billing.

There is no window, threshold, or schedule to set: the monitor checks after each anomaly scan, so every 15 minutes or every hour depending on the stream.

3. Attach channels

  1. Under Notify, check each channel that should receive the alerts.
  2. Leave Also notify when resolved checked if you want a message when the finding resolves. PagerDuty always gets the resolve, so it can close the incident.
  3. Click Create.

How alerts behave

Per finding (default):

  • One alert per finding subject — a pattern, a service, or a metric series. A finding that keeps being detected stays one alert.
  • Escalation re-notifies. If the finding gets more severe, for example from Warning to Critical, the channels hear about it once more.
  • Resolves with the finding. The alert resolves when the finding resolves, is dismissed, is muted, or stops matching the monitor's filters.

Per incident:

  • One alert per co-firing group. Findings first detected in the same scan (and later findings that share a service or pattern with that group) share one alert that lists the members.
  • Joins update without a new page. When another finding joins, the alert is updated; channels hear about it as an update (PagerDuty stays the same incident). Severity escalation still re-notifies.

  • Resolves when every matching member closes. Dismissing or muting a finding removes it from the incident.

  • Learning findings never alert. Neither do findings detected before the monitor was created.

Each alert carries the finding's headline, severity, and kind (or the incident lead plus member list), and its alert page shows why it was flagged, the baseline chart, the contributing events, and an Open finding link. The finding's own page lists the alerts raised for it.

Monitors created before Metric findings

A Findings monitor stores the kinds you picked. Monitors created before Metric findings existed list the five event kinds, so they don't alert on metric findings until you edit them and check Metric. New monitors include every kind.

Check what it has done

Open the monitor on Monitors. Its condition reads like Spikes · warning+ · pattern #7 · checked after every anomaly scan, and its history shows when finding alerts were open and which findings caused them.

Next