Skip to content

OpenSearch-compatible ingest

You will learn how to forward logs with OpenSearch-compatible _bulk (including DigitalOcean App Platform).

Cordo exposes a minimal OpenSearch-compatible HTTP surface on the API host root (not under /v1) so platforms that speak OpenSearch can ship without a sidecar.

Auth

  • HTTP Basic
  • Username is ignored (for example doadmin)
  • Password = stream API key (lax_…)
  • The stream comes from the API key — the index name is cosmetic

Endpoints

Path Purpose
POST /_bulk, PUT /_bulk Bulk ingest
POST /{index}/_bulk, PUT /{index}/_bulk Bulk ingest with default index

Also: cluster health and index acknowledge stubs for forwarder probes. Max body size 10 MiB. Body: NDJSON bulk (index / create / update).

If a document has log as a string containing a JSON object, Cordo extracts it on every ingest path (not only OpenSearch) so nested fields can be learned. The original string is kept at log._raw. Plain-text stdout (for example shell echo) stays a single string in log and is not extracted.

Prefer JSON-shaped stdout from every component you forward. Mixing plain strings with nested JSON objects can leave a type conflict on path log on Fields — see Fields and shapes. After producers emit JSON consistently, clear it with Resolve on Fields (Clear a field type conflict).

DigitalOcean App Platform

  1. Create a stream API key — Manage stream API keys.
  2. In App Platform → Settings → Log Forwarding → OpenSearch:
  3. Endpoint: https://<your-cordo-host> (HTTPS)
  4. User: any non-empty value (for example doadmin)
  5. Password: the stream API key
  6. Index name: for example logs
  7. Select the components whose runtime logs should forward.
  8. Ensure job Run Command lines do not prefix the real command with debug shell that prints plain text (for example ls … &&). Plain lines become an unparsed string in log.

curl smoke test

curl -sS -u "doadmin:$KEY" \
  -H "Content-Type: application/x-ndjson" \
  --data-binary $'{"index":{"_index":"logs"}}\n{"@timestamp":"2024-10-25T17:34:15.309Z","log":"{\\"level\\":\\"info\\",\\"msg\\":\\"hi\\"}","do_app_name":"demo"}\n' \
  "$CORDO_API/logs/_bulk"

Elasticsearch _bulk clients that use the same NDJSON shape usually work against this shim.

Next