AgentMark builds on top of OpenTelemetry for collecting telemetry data from your prompts. This helps you monitor, debug, and optimize your LLM applications in production. Traces

​
What We Track

AgentMark automatically collects:
  • Inference Spans: Full lifecycle of prompt execution
    • Token usage
    • Response times
    • Model information
    • Completion status
    • Cost
    • Response status
  • Tool Calls: When your prompts use tools
    • Tool name and parameters
    • Execution duration
    • Success/failure status
    • Return values
  • Streaming Metrics: For streaming responses
    • Time to first token
    • Tokens per second
    • Total streaming duration
  • Sessions: Group related traces together
    • Organize traces by user interaction
    • Track multi-step workflows
    • Monitor batch processing jobs
    • Analyze performance across related operations
  • Alerts: Monitor critical metrics and get notified
    • Cost thresholds
    • Latency monitoring
    • Error rate tracking
    • Notifications via Slack or webhooks
    • Alert history for pattern analysis

​
Basic Usage

Enable telemetry in your AgentMark client: 
const myPrompt = agentmark.loadTextPrompt('my-prompt.prompt.mdx');
const props = { num: 1 };
const result = myPrompt.format({
  props,
  telemetry: {
    isEnabled: true,
    metadata: {
      userId: '1234567890',
      sessionId: 'session-123',
      sessionName: 'Example Session'
    }
  }
});

​
Learn More

For detailed information about spans, metrics, and custom configuration, see:

​
Have Questions?

We’re here to help! Choose the best way to reach us: