Metrics

Metrics provide a high-level overview of how your application uses AI. Monitor costs, tokens, requests, latency, and more across models, users, and time periods.

​

Overview

AgentMark automatically tracks key metrics from your prompt executions, giving you insights into:

• Cost - Total spending on LLM providers
• Latency - Response times for AI requests
• Tokens - Input, output, and total token usage
• Models - Breakdown of model usage across your application
• Requests - Total volume and request rates

​

Key Metrics

​

Cost Tracking

Monitor your LLM spending:

• Total cost over time
• Cost breakdown by model
• Cost trends and projections

​

Latency Analysis

Track response times:

• Average latency across all requests
• Latency distribution and percentiles
• Slowest requests and bottlenecks
• Latency trends over time

​

Token Usage

Understand token consumption:

• Input tokens (prompts)
• Output tokens (completions)
• Total tokens used
• Token usage by model

​

Model Distribution

See which models are being used:

• Request count per model
• Cost per model
• Performance by model
• Model usage trends

​

Filtering and Analysis

Metrics can be filtered and segmented by: Time Period - View metrics for specific date ranges User - Analyze activity for individual users Model - Compare performance across different models Status - Filter by success, error, or specific outcomes Session - View metrics for specific sessions Custom Metadata - Filter by any custom attributes

​

Use Cases

Budget Management - Track spending and forecast costs Performance Optimization - Identify slow requests and bottlenecks Capacity Planning - Understand usage patterns for scaling decisions Model Comparison - Evaluate different models based on cost and performance

​

Best Practices

Regular Monitoring - Check metrics frequently to catch trends early Set Baselines - Understand your normal usage patterns Compare Periods - Analyze week-over-week or month-over-month changes Segment Analysis - Break down metrics by user, model, or custom attributes Correlate with Traces - Use metrics to identify areas for deeper investigation with traces Export Data - Download metrics for custom analysis or reporting

​

Next Steps