Skip to main content
This guide walks you through creating a new AgentMark project connected to the cloud platform. By the end, you’ll have a working project with example prompts, a local dev server, and cloud sync for prompt management, testing, and observability.

​
Prerequisites

  • Node.js 18+
  • An LLM provider API key (OpenAI or Anthropic, depending on your adapter choice)
1

Create Your AgentMark App

Run the interactive setup:
npm create agentmark@latest -- --cloud
The CLI will guide you through the following prompts:
PromptDescription
Project folderWhere to create your project (default: my-agentmark-app)
LanguageTypeScript or Python
AdapterYour preferred AI framework (AI SDK, Claude Agent SDK, Mastra, or Pydantic AI)
API keyYour OpenAI or Anthropic API key (can be skipped and added later)
Deployment modeChoose AgentMark Cloud to sync with the platform
IDEOptionally configure MCP servers for your editor
2

Connect to the Platform

To sync your files with the AgentMark platform:
  1. Commit and push your project to a Git repository
  2. In the AgentMark platform, navigate to your app
  3. Connect your repository
Sync RepositoryOnce synced, you can edit prompts in the platform’s visual editor, and changes automatically deploy to your application via the cloud loader.
3

Start the Development Server

cd my-agentmark-app
npm run agentmark dev -- --remote
This starts three local services:
  • API server (port 9418) — serves prompts and collects traces
  • Webhook server (port 9417) — executes prompts via your adapter
  • Dashboard (port 3000) — view traces, sessions, and requests in your browser
It also creates a secure tunnel so you can test prompts, datasets, and experiments directly from the platform, and sets up a dev API key so you can send your first traces.
4

Send Your First Trace

npm run prompt agentmark/customer-support-agent.prompt.mdx
Run your first experiment:
npm run experiment agentmark/party-planner.prompt.mdx
5

Run AgentMark in the Platform

  1. Add your Tunnel Credentials from Step 3 to your App Settings
  2. Open a Prompt or Experiment, and click Run
Run Prompt

​
What’s in Your Project

File / DirectoryPurpose
agentmark/Prompt templates (.prompt.mdx) and test datasets (.jsonl)
agentmark.client.tsClient configuration — models, tools, and loader setup
agentmark.jsonProject configuration (models, evals, schema)
agentmark.types.tsAuto-generated TypeScript types for your prompts
dev-entry.tsDevelopment server entry point (customizable)
index.tsExample application entry point
.envEnvironment variables (API keys, credentials)

​
Available Scripts

ScriptCommandDescription
devnpm run agentmark devStart the local development server with dashboard
promptnpm run agentmark prompt <file>Run a single prompt with test props
experimentnpm run agentmark experiment <file>Run a prompt against its test dataset
buildnpm run agentmark buildCompile prompts for standalone use
demonpm run demoRun the example application (requires build first)

​
IDE Integration

If you selected an IDE during setup, your project includes MCP server configuration that gives your AI assistant access to AgentMark documentation and trace debugging. Supported editors: Claude Code, Cursor, VS Code, Zed

​
Next Steps

Core Concepts

Understand organizations, apps, and branches

Writing Prompts

Learn how to create and configure prompts

Testing & Evals

Test prompts with datasets and evaluations

Observability

Monitor traces, costs, and performance

​
Have Questions?

We’re here to help! Choose the best way to reach us: