Project config - AgentMark Docs

You configure AgentMark projects through two files: agentmark.json for project-level settings, and agentmark.client.ts (or agentmark_client.py) for the prompt loader and your eval registry.

`agentmark.json`

The agentmark.json file lives at your project root and configures your AgentMark application. It’s read by both the CLI and AgentMark Cloud.

Basic example

A freshly-scaffolded agentmark.json (from npm create agentmark@latest) contains the four base fields plus one seeded model, so the Dashboard prompt editor’s model dropdown isn’t empty on first run:

agentmark.json

{
  "$schema": "https://raw.githubusercontent.com/agentmark-ai/agentmark/refs/heads/main/packages/cli/agentmark.schema.json",
  "version": "2.0.0",
  "mdxVersion": "1.0",
  "agentmarkPath": ".",
  "builtInModels": ["openai/gpt-5.5"]
}

Add the other optional properties documented below (scores, mcpServers, and others) as your project needs them.

Configuration properties

`$schema` (optional)

Points to the JSON Schema for editor autocompletion and validation.

"$schema": "https://raw.githubusercontent.com/agentmark-ai/agentmark/refs/heads/main/packages/cli/agentmark.schema.json"

agentmarkPath (required)

The base directory (relative to your project root) where AgentMark looks for the agentmark/ folder containing prompts, components, and datasets. Projects scaffolded with npm create agentmark@latest use ".", which puts the agentmark/ directory at the project root.

"agentmarkPath": "."

In a monorepo, set this to the relative path of the package containing your AgentMark files (for example, "packages/ai").

Use "." (or a relative path like "packages/ai"), not "/". A leading slash resolves to the filesystem root and breaks agentmark build.

`version` (required)

The AgentMark configuration version. Use "2.0.0" for new projects. AgentMark Cloud uses this to choose the storage folder for deployed prompts: versions >= "2.0.0" use the agentmark/ folder.

mdxVersion (optional)

The prompt format version. Use "1.0".

`builtInModels` (optional)

An array of model IDs allowed in prompts. When set and non-empty, prompt-core rejects any prompt whose model_name isn’t in the list. IDs use the provider/model format (for example, openai/gpt-5), which carries through to text_config.model_name (or object_config.model_name) on the neutral render so your call site or executor can map it to a provider model. Pricing and settings for these models come from the bundled AgentMark model registry.

"builtInModels": ["openai/gpt-5", "openai/gpt-5-mini", "anthropic/claude-sonnet-4-20250514"]

Populate it with agentmark pull-models, which emits the correct provider/model format. Run it interactively, or non-interactively with agentmark pull-models --provider <name> --models <csv> (use --provider <name> --list to print the available IDs as JSON first). See Model schemas for pulling models and defining custom schemas.

`scores` (optional)

Define score schemas for evaluation and human annotation. Each entry declares a score name and its type (boolean, numeric, or categorical).

"scores": {
  "accuracy": {
    "type": "boolean",
    "description": "Was the response factually correct?"
  },
  "helpfulness": {
    "type": "numeric",
    "min": 1,
    "max": 5,
    "description": "Rate helpfulness on a 1-5 scale"
  },
  "tone": {
    "type": "categorical",
    "description": "Response tone",
    "categories": [
      { "label": "professional", "value": 1 },
      { "label": "casual", "value": 0.5 },
      { "label": "inappropriate", "value": 0 }
    ]
  }
}

To add automated eval functions for these scores, define them in your client config using the evals option. See Evaluations for details.

modelSchemas (optional)

Define custom model configurations with settings, pricing, and UI controls. Use this for models not covered by builtInModels, or to customize settings for existing models.

"modelSchemas": {
  "my-custom-model": {
    "label": "My Custom Model",
    "cost": { "inputCost": 0.01, "outputCost": 0.03, "unitScale": 1000000 },
    "settings": { "temperature": { "type": "slider", "default": 0.7, "minimum": 0, "maximum": 2 } }
  }
}

See Model schemas for the full cost and settings reference.

mcpServers (optional)

Configure Model Context Protocol (MCP) servers that your prompts can reference as tools. Servers listed here are available in the AgentMark Dashboard prompt editor, and AgentMark surfaces them on the neutral render for prompts that reference them as mcp://<server-name>/<tool> in the tools: frontmatter. Connecting them at runtime is your call site’s or executor’s responsibility.

URL / SSE
Stdio

For remote MCP servers accessible via HTTP:

"mcpServers": {
  "docs": {
    "url": "https://example.com/mcp",
    "headers": {
      "Authorization": "Bearer your-token"
    }
  }
}

For local MCP servers that run as a subprocess:

"mcpServers": {
  "local-tools": {
    "command": "node",
    "args": ["./mcp-server.js"],
    "cwd": "/path/to/server",
    "env": {
      "API_KEY": "secret"
    }
  }
}

See MCP integration for usage in prompts.

Full example

An illustrative config showing every top-level field. The scaffolder doesn’t write all of them; see each field’s section above for when it applies.

agentmark.json

{
  "$schema": "https://raw.githubusercontent.com/agentmark-ai/agentmark/refs/heads/main/packages/cli/agentmark.schema.json",
  "version": "2.0.0",
  "mdxVersion": "1.0",
  "agentmarkPath": ".",
  "builtInModels": ["openai/gpt-5", "openai/gpt-5-mini", "anthropic/claude-sonnet-4-20250514"],
  "scores": {
    "correctness": {
      "type": "boolean",
      "description": "Was the response correct?"
    },
    "hallucination": {
      "type": "boolean",
      "description": "Did the response contain hallucinated content?"
    },
    "helpfulness": {
      "type": "numeric",
      "min": 1,
      "max": 5,
      "description": "Rate helpfulness on a 1-5 scale"
    }
  },
  "mcpServers": {
    "docs": {
      "url": "https://example.com/mcp"
    }
  },
  "modelSchemas": {
    "my-fine-tuned-model": {
      "label": "My Fine-tuned Model",
      "cost": {
        "inputCost": 0.005,
        "outputCost": 0.015,
        "unitScale": 1000000
      },
      "settings": {
        "temperature": {
          "label": "Temperature",
          "order": 1,
          "default": 0.7,
          "minimum": 0,
          "maximum": 2,
          "multipleOf": 0.1,
          "type": "slider"
        }
      }
    }
  }
}

Client configuration

Alongside agentmark.json, your project has a client file (agentmark.client.ts or agentmark_client.py) that loads prompts at runtime and renders them to the neutral shape ({ messages, text_config }); your own code or an executor makes the model call. The only required option is the loader; evals register via the evals option. npm create agentmark@latest scaffolds agentmark.json and the agentmark/ directory but not this file. Create it by following Client setup, which walks through the loader, evals, tracing, and type safety end to end.

Environment variables

Variable	Required	Description
`AGENTMARK_API_KEY`	Cloud mode	API key from AgentMark Dashboard settings
`AGENTMARK_APP_ID`	Cloud mode	App ID from AgentMark Dashboard settings
`AGENTMARK_BASE_URL`	No	Override the AgentMark Cloud endpoint (managed deployments inject it). Never point `ApiLoader.local` at it; the local dev server URL is plain `http://localhost:9418`
`OPENAI_API_KEY`	When calling OpenAI	OpenAI API key, read by your SDK or executor at the model call site
`ANTHROPIC_API_KEY`	When calling Anthropic	Anthropic API key, read by your SDK or executor at the model call site

See Environment variables for the complete list.

Have questions?

Reach out any time:

Email the team at hello@agentmark.co for support
Schedule an Enterprise Demo to learn about AgentMark’s business solutions

​agentmark.json

​Basic example

​Configuration properties

​$schema (optional)

​agentmarkPath (required)

​version (required)

​mdxVersion (optional)

​builtInModels (optional)

​scores (optional)

​modelSchemas (optional)

​mcpServers (optional)

​Full example

​Client configuration

​Environment variables

​Have questions?

`agentmark.json`

Basic example

Configuration properties

`$schema` (optional)

agentmarkPath (required)

`version` (required)

mdxVersion (optional)

`builtInModels` (optional)

`scores` (optional)

modelSchemas (optional)

mcpServers (optional)

Full example

Client configuration

Environment variables

Have questions?