ServicesLiveness

Uptime and Analytics

Four read-only endpoints, all derived from the same two per-entity series - heartbeat (one sample per beat) and latency (one sample per beat that carried client_timestamp_ms). Pick by the question you are asking:

  • uptime - what fraction of a window was the entity reporting?
  • latency - how far away is it, and is that trending?
  • history - the raw samples, for charting or export.
  • aggregations - an arbitrary bucketed rollup of either series.

Uptime is gap analysis, not a beat count

Uptime does not count heartbeats. It walks the beats inside the window and charges downtime for every gap wider than the 90-second alive threshold (heartbeat_interval_seconds + grace_period_seconds):

  • Interior gap wider than 90s: gap - 90s of downtime, plus one outage.
  • Trailing gap (last beat to now) wider than 90s: same rule.
  • Leading gap (window start to first beat) wider than 90s: the entire gap is charged, with no 90-second allowance, plus one outage.
  • No beats at all in the window: 0%, downtime equal to the whole window, and outages: 0.

The consequence is that a window longer than the entity's history reads as near-zero, because the stretch before the first beat is counted as downtime. One entity, one heartbeat a few seconds old, three windows, captured live:

snug --output json liveness uptime -i device-001 -w 60s,5m,1h
{
  "entity_id": "docs4-up",
  "uptime": {
    "60s": { "percentage": 100.0, "downtime_seconds": 0,    "outages": 0 },
    "5m":  { "percentage": 0.01,  "downtime_seconds": 299,  "outages": 1 },
    "1h":  { "percentage": 0.0,   "downtime_seconds": 3599, "outages": 1 }
  }
}

The 60s figure is 100% only because the leading gap fits inside the 90-second allowance. So compare uptime windows against entities of comparable age, and do not read a low 24h number on a device provisioned an hour ago as an incident. The zero-sample case is the other trap: a window with no beats reports 0% uptime with outages: 0, so alert on percentage, not on outages.

Windows take s, m, h, or d suffixes, and a bare number is seconds (-w 30s,2m,1d,600 returned all four). The default is 1h,24h. A spec that parses to nothing is 400 BAD_REQUEST, No valid windows specified.

Latency

snug --output json liveness latency -i device-001 -r 3600 -b 600
{
  "entity_id": "docs4-sensor-1",
  "current_ms": 6.0,
  "stats": { "avg": 5.57, "min": 4.0, "max": 8.0, "std_dev": 1.18 },
  "trend": [ { "timestamp_ms": 1788007200000, "avg": 5.571428571428571 } ]
}

--range is the lookback in seconds (default 86400) and --bucket the trend bucket size (default 3600). stats is population standard deviation. An entity that never sent a client timestamp has no latency series at all, and answers with an empty object rather than nulls or a 404:

{ "entity_id": "docs4-nolat", "stats": {}, "trend": [] }

Note that trend values are unrounded while stats are rounded to two decimals, so a single-bucket trend point will not string-match its avg.

Raw history

snug --output json liveness history -i device-001 --limit 100
snug --output json liveness history -i device-001 --from 1788007500 --to 1788007700

--from and --to are Unix seconds (not the milliseconds the response uses), and default to the last hour. Each sample carries the heartbeat timestamp plus that beat's latency, when one exists:

{
  "entity_id": "docs4-sensor-1",
  "samples": [
    { "timestamp_ms": 1788007596165, "latency_ms": 8.0 },
    { "timestamp_ms": 1788007617634, "latency_ms": 5.0 },
    ...
  ],
  "total_in_range": 9,
  "has_more": true
}

total_in_range is the true count in the window, independent of the page: that response was --limit 3 against nine samples. There is no cursor, so walk a long window by moving --from and --to rather than paging. The server caps a page at 1000 samples whatever you ask for.

Custom aggregations

snug --output json liveness aggregations -i device-001 -r 300 -b 60 -a count -s heartbeat
{
  "entity_id": "docs4-sensor-1",
  "series": "heartbeat",
  "aggregation": "count",
  "bucket_ms": 60000,
  "data": [
    { "timestamp_ms": 1788007560000, "value": 2.0 },
    { "timestamp_ms": 1788007620000, "value": 5.0 }
  ]
}

--series is heartbeat or latency (default latency); --aggregation is one of avg, sum, min, max, count, first, last, range, std.p, std.s, var.p, var.s (default avg). count over the heartbeat series is the useful one for beat-rate charts, since heartbeat samples all carry the value 1.0 and every other function over them is therefore constant.

An unknown function or series is rejected with 400, but the payload carries a misleading code and message - the request is refused for the aggregation name, not the entity id:

{
  "status": 400,
  "msg": "Invalid entity ID format: unsupported aggregation: bogus",
  "error": "invalid_entity_id"
}

Branch on the 400 and read msg; do not treat invalid_entity_id here as a statement about the entity id.

A dashboard pattern

Fleet health in one pass - list the silent entities, then pull each one's recent uptime against a window that matches your heartbeat interval:

snug --output json liveness query --alive false \
  | jq -r '.entities[].entity_id' \
  | while read -r id; do
      snug --output json liveness uptime -i "$id" -w 1h \
        | jq -c '{entity_id, uptime_1h: .uptime["1h"].percentage}'
    done

Exact parameters and response schemas for all four endpoints are in the Liveness API reference.

On this page