Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.agentmark.co/llms.txt

Use this file to discover all available pages before exploring further.

​
Quickstart

npm create agentmark does the absolute minimum bootstrap — writes agentmark.json, creates an empty agentmark/ directory, installs the AgentMark agent skill into your IDE, and hands off to your AI tool. The AI tool reads your project, asks the docs MCP for the right integration pattern, and wires the SDK into your existing code. No template menu, no opinionated scaffolding.

​
Prerequisites

  • Node.js 18+
  • An AI-tool-aware editor: Claude Code, Cursor, VS Code (Copilot Chat), or Zed
  • An LLM provider API key (OpenAI, Anthropic, etc.) for the model you want to run

​
Step 1: Bootstrap

Run from inside your project directory (or pass a folder name to scaffold a fresh one): 
npm create agentmark@latest
The CLI asks two short questions, scaffolds, and exits: 
? Where would you like to set up AgentMark?  .
? Wire AgentMark MCP into which IDE clients?
  Space to toggle. Enter to submit. Skip all = empty selection.
  ◉ Claude Code
  ◉ Cursor
  ◉ VS Code
  ◉ Zed

✅ agentmark.json
✅ agentmark/ (empty, ready for your .prompt.mdx files)
✅ MCP wired (Claude Code): .mcp.json
✅ MCP wired (Cursor): .cursor/mcp.json
✅ MCP wired (VS Code): .vscode/mcp.json
✅ MCP wired (Zed): .zed/settings.json

📚 Installing AgentMark agent skill...
✅ Agent skill installed at ./.agents/skills/agentmark/

✨ AgentMark is wired up.

   Next: open this project in Claude Code, Cursor, VS Code, or Zed and say:

       "Set up AgentMark in this project."
Non-interactive (CI / scripting):
npm create agentmark@latest my-app -- --client all --overwrite
Flags: --path <dir> • --client <id|all> (repeatable or comma-separated) • --overwrite (replace existing agentmark.json) • positional folder name.

​
Step 2: Tell your AI tool to integrate

Open your project in Claude Code, Cursor, VS Code, or Zed and send the agent this message:
Set up AgentMark in this project.
The AgentMark skill takes over. It:
  1. Detects your project’s framework (Next.js, FastAPI, Hono, plain Node, etc.)
  2. Queries the docs MCP for the right integration recipe
  3. Proposes a concrete plan back to you — packages to install, where the client file goes, what your first prompt looks like
  4. After you confirm, installs the SDK, writes the client (agentmark.client.ts / agentmark_client.py), scaffolds a first prompt, and smoke-tests it
It will not touch your existing LLM-SDK call sites during setup. Migrating those is a separate confirmation — ask the agent when you’re ready.

​
Step 3: Add your provider key

The agent will tell you which env var to set for the model it picked. For OpenAI’s gpt-4o-mini (the common default) that’s: 
echo "OPENAI_API_KEY=sk-..." >> .env

​
Step 4: Run your first prompt

Start the dev server (keep it running in a separate terminal):
npx agentmark dev
Then run the prompt the agent scaffolded (the agent will tell you the path; chat.prompt.mdx is the conventional default):
npx agentmark run-prompt agentmark/chat.prompt.mdx --props '{"message":"hello"}'
The CLI prints the model output, token counts, cost estimate, and a 📊 View trace URL you can open in the browser for the full span tree.
The dev server listens on ports 9418 (API), 9417 (webhook), and 3000 (UI app). Override with --api-port / --webhook-port / --app-port if you need different ports.

​
Step 5: Run an experiment

An experiment runs a prompt against a dataset and scores each row. Add a test_settings block to your prompt’s frontmatter pointing at a .jsonl dataset (see Datasets for the row shape), then: 
npx agentmark run-experiment agentmark/chat.prompt.mdx --threshold 80
The CLI runs every row, applies your evaluators, prints a results table, and exits non-zero if pass rate is below --threshold — wire that into CI for prompt regression gating.
Need worked examples? See Example prompts — four copy-paste recipes covering all four generation types (object, text+tools, image, speech).

​
What’s in your project after bootstrap

FileSourcePurpose
agentmark.jsonCLIProject config — agentmarkPath, version, models, scores
agentmark/.gitkeepCLIEmpty prompts directory (drop .prompt.mdx files here)
.mcp.json (and per-IDE configs)CLIMCP wiring — docs MCP, agentmark-mcp (Cloud), agentmark-local (dev)
.agents/skills/agentmark/CLI (via npx skills add)Agent skill that knows AgentMark — teaches Claude Code / Cursor / etc.
agentmark.client.ts (or _client.py)SkillConfigured SDK client — added when you ask the AI tool to integrate
Your first .prompt.mdxSkillScaffolded by the AI tool, named for your use case
.envYouProvider API key(s); AGENTMARK_API_KEY / AGENTMARK_APP_ID for Cloud
The CLI ships only the unopinionated bits. Everything stack-specific comes from the AI tool reading your project + the docs MCP — so the integration matches whatever framework you’re already on.

​
Next steps

Build Prompts

Author .prompt.mdx files: text, object, image, speech

Example prompts

Copy-paste starters for all four generation types

Evaluate

Test prompts with datasets + evaluators; gate CI on regressions

Observe

Traces, sessions, cost-and-token tracking

Integrations

Vercel AI SDK, Mastra, Claude Agent SDK, Pydantic AI

Deploy

Git-based deploys to AgentMark Cloud

​
Have Questions?

We’re here to help! Choose the best way to reach us: