Set up webhook notifications

Send instance events to an HTTPS endpoint you control. CosmicAC posts a payload when a job fails, degrades, or recovers, and when a served model's health changes. One webhook covers the whole instance.

Prerequisites

You need the following before you start:

A running CosmicAC deployment. See Installation.
An HTTPS endpoint that accepts POST requests. For Slack, create an incoming webhook and copy its URL.

Steps

Open the Notifications page

In the left navigation, click Settings. On the Settings page, under Instance, click Notifications.

Save the webhook URL

In Webhook URL, enter the HTTPS URL that receives the events, then click Save. CosmicAC sends nothing until you save a URL.

The first save also generates the signing secret and shows it once. Copy it now if you plan to verify signatures.

Choose the payload format

Under Format, select one of the following.

Generic JSON — a flat event object carrying id, type, schema, timestamp, instance, and the fields for that event type.
Slack-compatible — a Slack incoming webhook body carrying text and a color-coded attachments block. Red marks a failure, green a recovery, and orange a degraded state.

Select Slack-compatible when the URL points at Slack. CosmicAC posts the body straight to the incoming webhook URL, so it needs no Slack workspace credentials, bot token, or OAuth authorization.

Select the events to send

Under Events, turn on each event you want delivered. Delivery is opt-in, so CosmicAC never sends an event you leave off.

Event	Fires when
`job.failed`	A job transitions to Failed. Includes the failure reason.
`job.degraded`	Healthy replicas drop below desired. The endpoint stays live.
`job.recovered`	A job returns to Active from Degraded or Failed.
`job.restart_storm`	Any replica restarts three times within 10 minutes.
`model.health.down`	A served model's health enters Down.
`model.health.recovered`	A served model's health leaves Down.

Click Save to apply the selection.

Send a test event

Click Send test event. CosmicAC posts a test.ping payload to the saved URL. Repeated tests in quick succession are rate limited.

Confirm the delivery

Check Recent Deliveries. Each row lists the event type, the target URL, the response status, and how long ago CosmicAC sent it. A successful delivery shows the response code, such as 200 OK.

A delivery counts as failed when the receiver answers with a status outside the 200 range or does not answer in time.

Verify the signature on your receiver

This step applies when your receiver is a service you run. Skip it when the webhook URL points at Slack, because a Slack incoming webhook cannot run verification code and ignores the signature header.

Your receiver needs the signing secret to confirm that a payload came from CosmicAC. CosmicAC shows the secret once, when you first save the webhook URL. If you did not copy it then, click Regenerate to issue a new one.

Every request carries these headers.

X-Cosmic-Event — the event type, such as job.failed.
X-Cosmic-Delivery — the unique event identifier. Use it to discard duplicates.
X-Cosmic-Timestamp — the send time in milliseconds.
X-Cosmic-Signature — sha256= followed by the HMAC-SHA256 digest of the raw request body, keyed with the signing secret.

Recompute the digest over the raw body and compare it before you trust a payload.

const crypto = require('crypto')

const expected = 'sha256=' + crypto
  .createHmac('sha256', process.env.COSMICAC_WEBHOOK_SECRET)
  .update(rawBody)
  .digest('hex')

const signature = req.headers['x-cosmic-signature']
const valid = expected.length === signature.length &&
  crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(signature))