Alerts

Documentation Index

Fetch the complete documentation index at: https://docs.agentmark.co/llms.txt

Use this file to discover all available pages before exploring further.

​

Overview

• Monitor important metrics like cost, latency, and error rates
• Set thresholds for acceptable values
• Define time windows for measurement
• Receive notifications via Slack or webhooks
• Track alert history to analyze patterns

​

Available metrics

​

Creating alerts

1. Navigate to your app’s Alerts tab in the Dashboard
2. Click Create alert
3. Configure the alert:
   • Name — descriptive name for the alert
   • Metric — what to monitor
   • Threshold — value that triggers the alert
   • Time window — period over which the metric is measured
   • Evaluation name (for score alerts) — specific score to monitor
   • Aggregation type (for score alerts) — average or individual scores
   • Threshold direction (for score alerts) — alert above or below threshold

​

Notification options

​

Slack integration

1. Enable Send to Slack when creating an alert
2. Click Connect Slack if not already connected
3. Select the channel for notifications

​

Webhooks

1. Enable Use webhook when creating an alert
2. Configure your webhook endpoint in the Developers section
3. Receive alerts as HTTP POST requests with alert details

​

Alert status

• Triggered — the monitored metric has crossed the threshold.
• Resolved — the metric has returned to the acceptable side of the threshold.

​

Alert history

1. Navigate to the Alerts tab
2. Click an alert to see when it triggered, the value that triggered it, when it resolved, and how frequently it fires

​

Use cases

• Cost management — get notified when daily spending exceeds budget
• Performance monitoring — alert when latency degrades beyond acceptable levels
• Quality assurance — track when evaluation scores drop below quality thresholds
• Error detection — catch error rate spikes before they impact users

​

Best practices

• Set realistic thresholds based on your app’s normal behavior, not arbitrary values.
• Choose appropriate windows to match your usage patterns and avoid alert fatigue.
• Configure multiple channels — Slack plus a webhook for critical alerts.
• Review regularly and adjust thresholds as your usage patterns evolve.
• Start conservative — begin with higher thresholds and tighten them once you understand your baseline.

​

Next steps