Skip to main content
This guide walks you through creating a new AgentMark project connected to the cloud platform. By the end, you’ll have a working project with example prompts, a local dev server, and cloud sync for prompt management, testing, and observability.

​
Prerequisites

  • Node.js 18+
  • An LLM provider API key (OpenAI or Anthropic, depending on your adapter choice)
1

Create Your AgentMark App

Run the interactive setup:
npm create agentmark@latest
The CLI will guide you through the following prompts:
PromptDescription
Project folderWhere to create your project (default: my-agentmark-app)
LanguageTypeScript or Python
AdapterYour preferred AI framework (AI SDK, Claude Agent SDK, Mastra, or Pydantic AI)
API keyYour OpenAI or Anthropic API key (can be skipped and added later)
Deployment modeChoose AgentMark Cloud to sync with the platform
IDEOptionally configure MCP servers for your editor
  • AI SDK (Vercel) — Recommended for most TypeScript projects
  • Claude Agent SDK — For Anthropic-native agent workflows
  • Mastra — For Mastra framework users
Select AgentMark Cloud as the deployment mode. This connects your project to the AgentMark platform for prompt management, datasets, tracing, experiments, and alerts.
Running in an existing project? The CLI detects your project setup and merges AgentMark files in without overwriting your existing code.
2

Set Up Environment Variables

After setup, your .env file will contain:
OPENAI_API_KEY=your_openai_key

# AgentMark Cloud credentials (required for production)
AGENTMARK_API_KEY=your_agentmark_api_key
AGENTMARK_APP_ID=your_agentmark_app_id
To get your AgentMark Cloud credentials:
  1. Sign in at app.agentmark.co
  2. Create a new organization and app (or select an existing one)
  3. Navigate to Settings to find your API key and App ID
3

Start the Development Server

cd my-agentmark-app
npm run dev
This starts three local services:
  • API server (port 9418) — serves prompts and collects traces
  • Webhook server (port 9417) — executes prompts via your adapter
  • Dashboard (port 3000) — view traces, sessions, and requests in your browser
4

Run Your First Prompt

In a separate terminal, run one of the example prompts:
npm run prompt agentmark/customer-support-agent.prompt.mdx
Run an experiment against a test dataset:
npm run experiment agentmark/party-planner.prompt.mdx
Or build and run the demo application:
agentmark build
npm run demo
5

Connect to the Platform

To sync your prompts with the AgentMark platform:
  1. Commit and push your project to a Git repository
  2. In the AgentMark platform, navigate to your app
  3. Connect your repository
Sync RepositoryOnce synced, you can edit prompts in the platform’s visual editor, and changes automatically deploy to your application via the cloud loader.

​
What’s in Your Project

File / DirectoryPurpose
agentmark/Prompt templates (.prompt.mdx) and test datasets (.jsonl)
agentmark.client.tsClient configuration — models, tools, and loader setup
agentmark.jsonProject configuration (models, evals, schema)
agentmark.types.tsAuto-generated TypeScript types for your prompts
dev-entry.tsDevelopment server entry point (customizable)
index.tsExample application entry point
.envEnvironment variables (API keys, credentials)

​
Available Scripts

ScriptCommandDescription
devnpm run devStart the local development server with dashboard
promptnpm run prompt <file>Run a single prompt with test props
experimentnpm run experiment <file>Run a prompt against its test dataset
buildagentmark buildCompile prompts for standalone use
demonpm run demoRun the example application (requires build first)

​
IDE Integration

If you selected an IDE during setup, your project includes MCP server configuration that gives your AI assistant access to AgentMark documentation and trace debugging. Supported editors: Claude Code, Cursor, VS Code, Zed

​
Next Steps

Core Concepts

Understand organizations, apps, and branches

Writing Prompts

Learn how to create and configure prompts

Testing & Evals

Test prompts with datasets and evaluations

Observability

Monitor traces, costs, and performance

​
Have Questions?

We’re here to help! Choose the best way to reach us: