Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.agentmark.co/llms.txt

Use this file to discover all available pages before exploring further.

Instrument with the SDK to capture traces automatically. Explore them in your terminal (local) or in the Dashboard with filtering, search, graph view, dashboards, and alerts. Built on OpenTelemetry, AgentMark automatically collects telemetry data from your prompts and provides actionable insights.

Core concepts

Traces

A trace represents a complete request or workflow in your application. Each trace is identified by a unique trace ID and contains one or more spans. Traces carry top-level attributes such as metadata, tags, user ID, and session ID.

Spans

Spans are individual operations within a trace, forming a tree structure. AgentMark records three span types:
  • ai.inference — the full lifecycle of an LLM call, including model, tokens, cost, and response
  • ai.toolCall — a single tool execution, including name, arguments, and result
  • ai.stream — streaming response metrics such as time to first token and tokens per second

Span kinds

Every span has a semantic kind that categorizes the operation. Span kinds determine how spans can be filtered and how analytics are grouped on dashboards.
KindDescription
functionGeneric computation step (default)
llmA call to a language model
toolAn external tool or API call
agentAn orchestration loop that decides what to do next
retrievalA vector database query or document search
embeddingA call to an embedding model
guardrailA content safety or validation check
Set span kinds using observe() or ctx.span().

Sessions

Sessions group related traces together by session ID. Track multi-turn conversations, agent workflows, and batch processing runs. Each session aggregates cost, tokens, and latency across its traces. Learn more about Sessions →

Scores

Numeric evaluations attached to spans or traces. Set scores programmatically via the SDK using sdk.score(), or manually through annotations in the Dashboard.

Metadata and tags

Metadata — Custom key-value pairs attached to traces for context (environment, feature flags, customer tier). Automatically discovered as filter fields. Tags — String labels for categorization (environment, team, feature, release). Metadata → · Tags →

What gets tracked

Inference spans — Full prompt execution lifecycle: token usage, costs, response times, model information, completion status. Tool calls — Tool name, parameters, execution duration, success/failure status, return values. Streaming metrics — Time to first token, tokens per second, total streaming duration. Sessions — Group related traces by user interaction, multi-step workflow, or batch run. Alerts — Monitor cost thresholds, latency spikes, error rates, and evaluation scores.

Quick start

Enable telemetry when formatting your prompts:
import { client } from './agentmark.client';
import { generateText } from 'ai';

const prompt = await client.loadTextPrompt('greeting.prompt.mdx');
const input = await prompt.format({
  props: { name: 'Alice' },
  telemetry: {
    isEnabled: true,
    functionId: 'greeting-handler',
    // Session/user keys in metadata must be snake_case to group traces (see Sessions)
    metadata: {
      user_id: 'user-123',
      session_id: 'session-abc',
      session_name: 'Customer Support Chat'
    }
  }
});

const result = await generateText(input);
For full tracing setup including AgentMarkSDK, child spans, observe(), and span kinds, see Tracing Setup.

How data flows

Your application sends telemetry via the AgentMark SDK, which exports OpenTelemetry spans to the AgentMark gateway. The gateway processes and stores the data, powering the traces, metrics, and analytics views.
Spans are exported to the AgentMark Cloud gateway and stored in ClickHouse. View traces, dashboards, alerts, and analytics in the Dashboard.

Programmatic access

You can query traces, spans, sessions, scores, metrics, datasets, experiments, prompts, and runs programmatically through the REST API, or from an IDE agent via the agentmark-mcp MCP server (which exposes one MCP tool per gateway operation). Use either to build custom integrations, pull data into external tools, or automate monitoring workflows. Most endpoints are available on both the local dev server and the AgentMark Cloud gateway. The local server returns 501 not_available_locally for features that require ClickHouse aggregations (/v1/metrics and score analytics). Trace export (/v1/traces/export) is Cloud-only too, but has no local route — it returns 404. Use the capabilities endpoint to check which features a server supports.
# Query traces from the local dev server
curl -fsS "http://localhost:9418/v1/traces?limit=10"

# Get a specific trace with all its spans
curl -fsS "http://localhost:9418/v1/traces/<traceId>"

# Same calls against Cloud — set AGENTMARK_API_KEY + AGENTMARK_APP_ID
curl -fsS "https://api.agentmark.co/v1/traces?limit=10" \
  -H "Authorization: Bearer $AGENTMARK_API_KEY" \
  -H "X-Agentmark-App-Id: $AGENTMARK_APP_ID"

# Check which features are available on a given server
curl -fsS "http://localhost:9418/v1/capabilities"
The same operations are available as MCP tools (list_traces, get_trace, get_capabilities, …) when you run the agentmark-mcp server alongside your IDE.

Next steps

Tracing setup

Instrument your app with the SDK

Traces and logs

View execution timelines in the Dashboard

Sessions

Group related traces together

Alerts

Get notified of critical issues

Dashboards

Analyze usage, performance, and scores

API reference

Query traces, scores, and metrics via REST API

Have Questions?

We’re here to help! Choose the best way to reach us: