# AgentMark Docs

> AgentMark helps teams build reliable AI agents. Manage prompts, trace executions, run evaluations, and deploy agents with confidence.

## Docs

- [Add items to queue](https://docs.agentmark.co/api-reference/annotation-queue-items/add-items-to-queue.md): Adds one or more traces/spans/sessions to the queue for review. Duplicate `(queue_id, resource_id)` pairs are ignored silently.
- [Delete queue item](https://docs.agentmark.co/api-reference/annotation-queue-items/delete-queue-item.md): Removes an item from a queue. Cascades to reviewer records.
- [Get queue item](https://docs.agentmark.co/api-reference/annotation-queue-items/get-queue-item.md): Returns a single queue item by ID.
- [List queue items](https://docs.agentmark.co/api-reference/annotation-queue-items/list-queue-items.md): Returns every item enqueued for review, in the order they were added.
- [Submit review](https://docs.agentmark.co/api-reference/annotation-queue-items/submit-review.md): Submit a review (`completed` or `skipped`) on behalf of the authenticated user. When the queue's `reviewers_required` threshold is met, the item auto-advances to `completed`.
- [Update queue item](https://docs.agentmark.co/api-reference/annotation-queue-items/update-queue-item.md): Updates item status or assigned reviewer. Setting status to `completed` auto-records `completed_by` / `completed_at`.
- [Create annotation queue](https://docs.agentmark.co/api-reference/annotation-queues/create-annotation-queue.md): Creates a new queue for collecting human review on traces, spans, or sessions.
- [Delete annotation queue](https://docs.agentmark.co/api-reference/annotation-queues/delete-annotation-queue.md): Deletes a queue. Cascades to its items and reviewer records.
- [Get annotation queue](https://docs.agentmark.co/api-reference/annotation-queues/get-annotation-queue.md): Returns metadata for a single queue by ID.
- [List annotation queues](https://docs.agentmark.co/api-reference/annotation-queues/list-annotation-queues.md): Returns every annotation queue for the authenticated application, with per-queue progress counters (pending / in_progress / completed / skipped / total).
- [Update annotation queue](https://docs.agentmark.co/api-reference/annotation-queues/update-annotation-queue.md): Updates mutable queue metadata (`name`, `description`, `status`, `instructions`, `reviewers_required`, `score_config_names`). Fields not provided are left unchanged.
- [Authentication](https://docs.agentmark.co/api-reference/authentication.md): How to authenticate with the AgentMark Gateway API.
- [Get capabilities](https://docs.agentmark.co/api-reference/capabilities/get-capabilities.md): Returns a map of available API endpoints for the current target (cloud or local). Use this to discover which features are supported before calling other endpoints.
- [Get config](https://docs.agentmark.co/api-reference/config/get-config.md): Returns the effective project configuration synced from `agentmark.json` for the authenticated application, plus the current synced commit SHA when available.
- [Append dataset row](https://docs.agentmark.co/api-reference/datasets/append-dataset-row.md): Appends a single row to the specified dataset. The `datasetName` parameter is the dataset path without the `.jsonl` extension, URL-encoded.
- [Import dataset rows from spans](https://docs.agentmark.co/api-reference/datasets/import-dataset-rows-from-spans.md): Transforms one or more spans into canonical dataset rows and appends them to the specified dataset.
- [Import dataset rows from traces](https://docs.agentmark.co/api-reference/datasets/import-dataset-rows-from-traces.md): Transforms one or more traces into canonical dataset rows and appends them to the specified dataset.
- [List datasets](https://docs.agentmark.co/api-reference/datasets/list-datasets.md): Returns a list of dataset file paths for your application. Datasets are JSONL files containing test cases used in experiments.
- [Get experiment](https://docs.agentmark.co/api-reference/experiments/get-experiment.md): Retrieve a specific experiment by ID, including its per-item details (trace IDs, inputs/outputs, per-item cost/latency/tokens, and any scores attached to each trace).
- [List experiments](https://docs.agentmark.co/api-reference/experiments/list-experiments.md): Retrieve a paginated list of experiments (dataset runs). Each experiment is a group of traces that share a `DatasetRunId`, typically produced when a prompt is evaluated against every row in a dataset.
- [Files health](https://docs.agentmark.co/api-reference/health/files-health.md): Check the health of the files service and its dependencies.
- [Ingestion health](https://docs.agentmark.co/api-reference/health/ingestion-health.md): Check the health of the trace ingestion pipeline and its dependencies.
- [Service health](https://docs.agentmark.co/api-reference/health/service-health.md): Check if the gateway service is running. Returns healthy if all required environment variables are configured.
- [Get metrics](https://docs.agentmark.co/api-reference/metrics/get-metrics.md): Retrieve aggregated analytics metrics for your application. Returns a summary and an hourly time series for trace volume, latency, cost, token usage, and error rates.
- [API reference](https://docs.agentmark.co/api-reference/overview.md): Programmatic access to the AgentMark Gateway API.
- [Get LLM pricing](https://docs.agentmark.co/api-reference/pricing/get-llm-pricing.md): Returns per-model pricing for cost calculation. Response is a dynamic map keyed by model ID (e.g. `gpt-4o`, `claude-opus-4-6`). Prices are per 1,000 tokens.
- [List prompt logs](https://docs.agentmark.co/api-reference/prompts/list-prompt-logs.md): Retrieve a paginated list of prompt execution logs. Each log represents a prompt that was run, with its input, output, and metadata.
- [Create score](https://docs.agentmark.co/api-reference/scoring/create-score.md): Create a score record for a span or trace. Scores are used to track quality metrics, evaluation results, and human feedback.
- [Create scores (batch)](https://docs.agentmark.co/api-reference/scoring/create-scores-batch.md): Create up to 1000 scores in a single request. Each item is validated independently and the response always contains a per-item results array.
- [Delete score](https://docs.agentmark.co/api-reference/scoring/delete-score.md): Delete a score record by ID.
- [Get score](https://docs.agentmark.co/api-reference/scoring/get-score.md): Retrieve a single score record by ID. Returns the full score object including its value, label, reason, and source.
- [Get score aggregations](https://docs.agentmark.co/api-reference/scoring/get-score-aggregations.md): Returns aggregated statistics for scores grouped by name. Useful for understanding score distributions across your application.
- [Get score names](https://docs.agentmark.co/api-reference/scoring/get-score-names.md): Returns a list of distinct score names used in your application. Useful for building filter dropdowns and discovering available score types.
- [List scores](https://docs.agentmark.co/api-reference/scoring/list-scores.md): Returns a paginated list of scores for the authenticated application. Supports filtering by resource, name, source, and date range.
- [List sessions](https://docs.agentmark.co/api-reference/sessions/list-sessions.md): Retrieve a paginated list of sessions. Sessions group related traces together for multi-turn conversations, workflows, and batch processing.
- [Get span I/O detail](https://docs.agentmark.co/api-reference/spans/get-span-io-detail.md): Returns the full input/output payload for a specific span, plus parsed output objects and tool calls when present. Useful for rendering a single span in isolation without loading the entire trace.
- [List spans](https://docs.agentmark.co/api-reference/spans/list-spans.md): Query spans across all traces. Supports filtering by type, status, model, name, and duration range.
- [List spans for a trace](https://docs.agentmark.co/api-reference/spans/list-spans-for-a-trace.md): Returns every span belonging to the given trace, ordered by start time. Not paginated — traces are bounded by span volume (capped at ingest), so the full list is returned in a single response.
- [Get template](https://docs.agentmark.co/api-reference/templates/get-template.md): Retrieve a prompt template by its file path. Templates must have a `.mdx` or `.jsonl` extension.
- [Get trace](https://docs.agentmark.co/api-reference/traces/get-trace.md): Retrieve a specific trace by ID, including all its spans.
- [Ingest traces](https://docs.agentmark.co/api-reference/traces/ingest-traces.md): Ingest trace data in [OTLP (OpenTelemetry Protocol)](https://opentelemetry.io/docs/specs/otlp/) format. Traces are buffered in a queue and processed asynchronously.
- [List traces](https://docs.agentmark.co/api-reference/traces/list-traces.md): Retrieve a paginated list of traces. Supports filtering by status, user, model, and date range.
- [API versioning & stability](https://docs.agentmark.co/api-reference/versioning.md): How the AgentMark Gateway API versions itself, what counts as a breaking change, and how deprecations are communicated.
- [WebSocket Connect](https://docs.agentmark.co/api-reference/websocket-connect.md): Persistent WebSocket endpoint for connecting a deployed worker to the AgentMark gateway.
- [Components](https://docs.agentmark.co/build/components.md): Create reusable components to share prompting patterns across your prompts
- [Create a prompt](https://docs.agentmark.co/build/creating-prompts.md): Create prompts in the Dashboard visual editor or as local .prompt.mdx files
- [File attachments](https://docs.agentmark.co/build/file-attachments.md): Attach images and files to prompts using the ImageAttachment and FileAttachment tags.
- [Generating images](https://docs.agentmark.co/build/generation-types/image.md): Generate images from prompts using AgentMark with DALL-E, Stable Diffusion, and other image models.
- [Generating objects](https://docs.agentmark.co/build/generation-types/object.md): Generate structured JSON objects from prompts with schema validation using AgentMark.
- [Generation types overview](https://docs.agentmark.co/build/generation-types/overview.md): Understand the different types of content you can generate with AgentMark prompts
- [Generating speech](https://docs.agentmark.co/build/generation-types/speech.md): Generate speech audio from prompts using AgentMark with text-to-speech models.
- [Generating text](https://docs.agentmark.co/build/generation-types/text.md): Generate text completions from prompts using AgentMark with any LLM provider.
- [MCP integration](https://docs.agentmark.co/build/mcp.md): Use Model Context Protocol (MCP) tools from AgentMark prompts via adapters.
- [Build](https://docs.agentmark.co/build/overview.md): Create, run, and version prompts — in code or in the Dashboard
- [Playground](https://docs.agentmark.co/build/playground.md): Compare prompts and models side-by-side to find the best configuration before publishing
- [Running prompts](https://docs.agentmark.co/build/running-prompts.md): Run prompts from the Dashboard, CLI, or SDK — with streaming, tracing, and caching
- [Schema references](https://docs.agentmark.co/build/schema-references.md): Reuse JSON schema definitions across AgentMark prompts with $ref
- [Prompt syntax](https://docs.agentmark.co/build/syntax.md): Learn the AgentMark template syntax for creating dynamic prompts
- [Tools and agents](https://docs.agentmark.co/build/tools-and-agents.md): Extend prompts with function calling and build multi-step agent workflows
- [Version control](https://docs.agentmark.co/build/version-control.md): Track, compare, and manage prompt versions in the AgentMark Dashboard
- [Client config](https://docs.agentmark.co/configure/client-config.md): Configure your AgentMark client for models, tools, scores, and prompt loading
- [Environment variables](https://docs.agentmark.co/configure/environment-variables.md): Complete reference for all environment variables used by AgentMark
- [Model schemas](https://docs.agentmark.co/configure/model-schemas.md): Add built-in and custom models in AgentMark
- [Project config](https://docs.agentmark.co/configure/project-config.md): Learn how to configure AgentMark for your application needs
- [API keys](https://docs.agentmark.co/deploy/api-keys.md): Create, scope, and manage API keys for the AgentMark Gateway from the Dashboard.
- [Billing and usage](https://docs.agentmark.co/deploy/billing-and-usage.md): Pricing tiers, usage limits, entitlements, and rate limits
- [Deployment](https://docs.agentmark.co/deploy/deployment.md): Deploy your prompts and handler code to AgentMark Cloud using the deployment pipeline
- [Organizations and teams](https://docs.agentmark.co/deploy/organizations-and-teams.md): Organizations, apps, and how they fit together
- [Security](https://docs.agentmark.co/deploy/security.md): How AgentMark protects your data — SSO, encryption, PII masking, and data residency
- [Users and access control](https://docs.agentmark.co/deploy/users-and-access-control.md): Roles, permissions, custom roles, app-level access, and team management
- [Webhooks](https://docs.agentmark.co/deploy/webhooks.md): Receive alert notifications from AgentMark Cloud via webhooks
- [Human annotation](https://docs.agentmark.co/evaluate/annotations.md): Score traces with human reviewers — individually or in structured batch queues
- [Datasets](https://docs.agentmark.co/evaluate/datasets.md): Create test datasets for your prompts
- [Evaluate](https://docs.agentmark.co/evaluate/overview.md): Test and improve your prompts with datasets, evaluators, experiments, and annotations
- [Running experiments](https://docs.agentmark.co/evaluate/running-experiments.md): Test prompts with datasets using the CLI or SDK
- [Evaluations](https://docs.agentmark.co/evaluate/writing-evals.md): Write evaluation functions to score prompt outputs
- [Quickstart](https://docs.agentmark.co/getting-started/quickstart.md): Create your first AgentMark project, run a prompt, and run an experiment — in under 5 minutes.
- [Custom integration](https://docs.agentmark.co/integrations/custom.md): Learn how to create your own custom integration with AgentMark.
- [Default integration](https://docs.agentmark.co/integrations/fallback.md): Use AgentMark with the fallback adapter for raw prompt configuration output.
- [Overview](https://docs.agentmark.co/integrations/overview.md): Connect AgentMark to your AI SDK through adapters
- [Pydantic AI](https://docs.agentmark.co/integrations/python/pydantic-ai.md): Use AgentMark prompts with Pydantic AI in Python
- [AI SDK](https://docs.agentmark.co/integrations/typescript/ai-sdk.md): Use AgentMark prompts with the Vercel AI SDK
- [Claude Agent SDK](https://docs.agentmark.co/integrations/typescript/claude-agent-sdk.md): Use AgentMark prompts with Anthropic's Claude Agent SDK
- [Mastra](https://docs.agentmark.co/integrations/typescript/mastra.md): Use AgentMark prompts with Mastra agents
- [Core concepts](https://docs.agentmark.co/introduction/core-concepts.md): The main concepts to help you get started with AgentMark
- [Deployment modes](https://docs.agentmark.co/introduction/deployment-modes.md): The two ways to run AgentMark: local development and AgentMark Cloud.
- [What is AgentMark?](https://docs.agentmark.co/introduction/overview.md): AgentMark helps teams build reliable AI agents. Manage prompts, trace executions, run evaluations, and deploy with confidence — locally or in the cloud.
- [Alerts](https://docs.agentmark.co/observe/alerts.md): Monitor your application with customizable alerts
- [Cost and token tracking](https://docs.agentmark.co/observe/cost-and-token-tracking.md): Monitor LLM spending and token usage across your application
- [Dashboards](https://docs.agentmark.co/observe/dashboards.md): Monitor operational metrics, evaluation scores, and custom analytics in one place
- [Filtering and search](https://docs.agentmark.co/observe/filtering-and-search.md): Find specific traces and sessions using filters, date ranges, sorting, saved views, and shareable URLs
- [Metadata](https://docs.agentmark.co/observe/metadata.md): Attach custom key-value pairs to traces for filtering, debugging, and context
- [Observe](https://docs.agentmark.co/observe/overview.md): Monitor, debug, and optimize your LLM applications with tracing, dashboards, and alerts
- [PII masking](https://docs.agentmark.co/observe/pii-masking.md): Redact sensitive data from traces before they leave your application
- [Sessions](https://docs.agentmark.co/observe/sessions.md): Group related traces together for multi-turn conversations, workflows, and batch processing
- [Tags](https://docs.agentmark.co/observe/tags.md): Attach string labels to traces for categorization and filtering
- [Traces and logs](https://docs.agentmark.co/observe/traces-and-logs.md): Monitor and debug your prompts with distributed tracing
- [Tracing setup](https://docs.agentmark.co/observe/tracing-setup.md): Instrument your application with the AgentMark SDK to capture traces, spans, and metrics
- [CLI reference](https://docs.agentmark.co/sdk-reference/cli/commands.md): Complete reference for all AgentMark CLI commands
- [Python client setup](https://docs.agentmark.co/sdk-reference/python/client-setup.md): Install and configure the AgentMark Python SDK with Pydantic AI
- [Python dev server](https://docs.agentmark.co/sdk-reference/python/dev-server.md): Running the AgentMark development server with Python
- [AgentMark MCP Server](https://docs.agentmark.co/sdk-reference/tools/agentmark-mcp.md): Using AgentMark within your AI-powered code editors
- [MCP Trace Server](https://docs.agentmark.co/sdk-reference/tools/mcp-trace-server.md): Expose AgentMark traces to AI editors for debugging
- [Troubleshooting](https://docs.agentmark.co/sdk-reference/troubleshooting.md): Solutions to common errors and issues when using AgentMark
- [TypeScript client setup](https://docs.agentmark.co/sdk-reference/typescript/client-setup.md): Install and configure the AgentMark TypeScript SDK with your preferred adapter
- [Loaders](https://docs.agentmark.co/sdk-reference/typescript/loaders.md): Load prompts from different sources using ApiLoader and FileLoader
- [Type safety](https://docs.agentmark.co/sdk-reference/typescript/type-safety.md): Generate TypeScript types from your prompt schemas for compile-time validation and autocomplete.
- [Components](https://docs.agentmark.co/templatedx/components.md): Create reusable template fragments in TemplateDX for shared prompt logic and structure.
- [Expressions](https://docs.agentmark.co/templatedx/expressions.md): Use JavaScript-like expressions in TemplateDX templates for dynamic content and conditional logic.
- [FAQ](https://docs.agentmark.co/templatedx/faq.md): Frequently asked questions about TemplateDX syntax, setup, and usage.
- [Filters](https://docs.agentmark.co/templatedx/filters.md): Transform and format template values with built-in TemplateDX filter functions.
- [Overview](https://docs.agentmark.co/templatedx/introduction.md): TemplateDX is an extensible templating engine for AI prompts, built on Markdown and JSX.
- [Quickstart](https://docs.agentmark.co/templatedx/quickstart.md): Install TemplateDX and render your first prompt template.
- [Syntax Overview](https://docs.agentmark.co/templatedx/syntax.md): Complete syntax reference for TemplateDX
- [Editor integration](https://docs.agentmark.co/templatedx/syntax-highlighting.md): Set up syntax highlighting, schema validation, and MCP integration for TemplateDX files in VS Code, Cursor, Zed, and Claude Code.
- [Tags](https://docs.agentmark.co/templatedx/tags.md): Use built-in and custom tags in TemplateDX to control template flow and transform data.
- [Type safety](https://docs.agentmark.co/templatedx/type-safety.md): Editor-level type hints for TemplateDX templates. For runtime type safety, use the AgentMark type generator.
- [Variables](https://docs.agentmark.co/templatedx/variables.md): Pass dynamic data into TemplateDX templates via the props object.

## OpenAPI Specs

- [dashboard-openapi](https://docs.agentmark.co/dashboard-openapi.yaml)
- [openapi](https://docs.agentmark.co/openapi.yaml)

## Optional

- [GitHub](https://github.com/agentmark-ai/agentmark)
- [linkedin](https://www.linkedin.com/company/agentmark)