Troubleshooting

This page covers common errors you may encounter when using AgentMark and how to resolve them.

Connection & Authentication Errors

AGENTMARK_API_KEY not set

Error: Error: AGENTMARK_API_KEY environment variable is not set Cause: You’re using ApiLoader.cloud() but haven’t configured your API key. Solution:

Get your API key from AgentMark Dashboard

Add it to your .env file:

AGENTMARK_API_KEY=your-api-key-here
AGENTMARK_APP_ID=your-app-id-here

Restart your development server

AGENTMARK_APP_ID not set

Error: Error: AGENTMARK_APP_ID environment variable is not set Cause: Cloud loader requires both API key and App ID. Solution:

Find your App ID in AgentMark Dashboard
Add to .env:
```
AGENTMARK_APP_ID=your-app-id-here
```

Loader connection failed

Error: Error: connect ECONNREFUSED 127.0.0.1:9418 Cause: The local development server isn’t running. Solution:

Start the dev server in a separate terminal:
```
agentmark dev
```
Verify it’s running at http://localhost:9418

If using a custom port, update your loader:

ApiLoader.local({ baseUrl: "http://localhost:YOUR_PORT" })

Invalid API key

Error: Error: 401 Unauthorized - Invalid API key Cause: The API key is incorrect or has been revoked. Solution:

Verify your API key in the dashboard
Generate a new key if needed
Update your .env file with the new key

Model Errors

Model not found / not registered

Error: Error: Model "gpt-4o" is not registered in the model registry Cause: The model name in your prompt frontmatter isn’t registered in your client. Solution:

Check the model name in your prompt:

text_config:
  model_name: gpt-4o  # This must be registered

const modelRegistry = new VercelAIModelRegistry()
  .registerModels(["gpt-4o"], (name) => openai(name));

Provider API key missing

Error: Error: OpenAI API key is missing or ANTHROPIC_API_KEY not set Cause: The AI provider’s API key isn’t configured. Solution: Add the provider’s API key to your .env:

# OpenAI
OPENAI_API_KEY=sk-...

# Anthropic
ANTHROPIC_API_KEY=sk-ant-...

# Google
GOOGLE_GENERATIVE_AI_API_KEY=...

Rate limit exceeded

Error: Error: 429 Too Many Requests - Rate limit exceeded Cause: You’ve exceeded the provider’s rate limits. Solution:

Wait and retry (most SDKs have built-in retry logic)
Reduce request frequency
Consider upgrading your provider plan
Use multiple API keys with rotation

Model not available in region

Error: Error: Model is not available in your region Cause: Some models have regional restrictions. Solution:

Check provider documentation for model availability
Use an alternative model
Consider using a VPN or different deployment region

Tool Errors

Tool not available

Error: Error: Tool "search_knowledgebase" is not registered Cause: The tool referenced in your prompt isn’t registered in the tool registry. Solution:

Check the tool name in your prompt:

text_config:
  tools:
    - search_knowledgebase  # This must be registered

const toolRegistry = new VercelAIToolRegistry()
  .register("search_knowledgebase", async ({ query }) => {
    return { results: await search(query) };
  });

Tool execution failed

Error: Error: Tool execution failed: Cannot read property 'x' of undefined Cause: The tool function threw an error, often due to missing or malformed input. Solution:

Add input validation to your tool:

.register("my_tool", async ({ param }) => {
  if (!param) {
    throw new Error("param is required");
  }
  // ... tool logic
});

Check the tool’s parameter schema matches the prompt definition

MCP Server Errors

MCP server not connecting

Error: Error: Failed to connect to MCP server "docs" Cause: The MCP server configuration is incorrect or the server isn’t running. Solution:

For stdio servers, verify the command works:

npx -y @modelcontextprotocol/server-filesystem ./data

For URL servers, verify the endpoint is accessible:
```
curl https://your-mcp-server.com/mcp
```

Check environment variables are set:

mcpServers: {
  docs: {
    url: "env(MCP_SERVER_URL)",  // Is MCP_SERVER_URL set?
  }
}

MCP tool not found

Error: Error: Tool "web-search" not found on MCP server "docs" Cause: The tool name doesn’t exist on the MCP server. Solution:

List available tools from the server (check server documentation)

Verify the tool name in your prompt:

tools:
  search: mcp://docs/web-search  # Is "web-search" correct?

Use wildcard to import all tools:
```
tools:
  all: mcp://docs/*
```

Prompt Errors

Prompt not found

Error: Error: Prompt not found: greeting.prompt.mdx Cause: The prompt file doesn’t exist or the path is incorrect. Solution:

Check the file exists at the expected path

Verify the path in loadTextPrompt():

// Relative to agentmarkPath in agentmark.json
await client.loadTextPrompt("greeting.prompt.mdx");

// Or with subdirectory
await client.loadTextPrompt("helpers/greeting.prompt.mdx");

Run agentmark build to verify all prompts are valid

Invalid frontmatter

Error: Error: Invalid frontmatter in prompt file Cause: YAML syntax error in the prompt’s frontmatter. Solution:

Validate YAML syntax (check for proper indentation, colons, quotes)

Common issues:

# Wrong - missing space after colon
model_name:gpt-4o

# Correct
model_name: gpt-4o

# Wrong - improper nesting
text_config:
model_name: gpt-4o

# Correct
text_config:
  model_name: gpt-4o

Template syntax error

Error: Error: Unexpected token in template Cause: Invalid TemplateDX syntax in the prompt body. Solution:

Check for unclosed tags:

{# Wrong - unclosed if #}
{% if condition %}
  Content here

{# Correct #}
{% if condition %}
  Content here
{% endif %}

Verify variable syntax: {props.name} not {{props.name}}
See TemplateDX Syntax for reference

Experiment & Dataset Errors

Dataset file not found

Error: Error: Dataset file not found: ./datasets/test.jsonl Cause: The dataset path in test_settings doesn’t exist. Solution:

Check the path in your prompt:

test_settings:
  dataset: ./datasets/test.jsonl  # Is this path correct?

Paths are relative to the prompt file location
Create the dataset file if it doesn’t exist

Invalid dataset format

Error: Error: Invalid JSON on line 3 of dataset Cause: The JSONL file has malformed JSON. Solution:

Each line must be valid JSON:

{"input": {"name": "Alice"}, "expected": "Hello Alice"}
{"input": {"name": "Bob"}, "expected": "Hello Bob"}

Validate with: cat dataset.jsonl | jq -c '.'
Check for trailing commas or missing quotes

Eval function not registered

Error: Error: Eval "exact_match" is not registered Cause: The evaluation referenced in test_settings isn’t registered. Solution:

const evalRegistry = new EvalRegistry()
  .register("exact_match", ({ output, expectedOutput }) => ({
    score: output === expectedOutput ? 1 : 0,
    passed: output === expectedOutput,
  }));

Pass evalRegistry to createAgentMarkClient()

Type Errors

Type mismatch after schema change

Error: TypeScript errors after modifying prompt schemas Cause: Generated types are out of date. Solution:

Regenerate types:

agentmark generate-types --root-dir ./agentmark > agentmark.types.ts

Or use the build command:
```
agentmark build
```

Props type error

Error: Argument of type '{ name: string }' is not assignable to parameter Cause: Props don’t match the prompt’s input_schema. Solution:

Check the prompt’s input schema:

input_schema:
  type: object
  properties:
    name: { type: string }
    age: { type: number }  # Required but missing
  required: [name, age]

Provide all required props:

await prompt.format({
  props: { name: "Alice", age: 30 }
});

CLI Errors

agentmark command not found

Error: command not found: agentmark Cause: CLI not installed globally or not in PATH. Solution:

Install globally:
```
npm install -g @agentmark-ai/cli
```
Or use npx:
```
npx @agentmark-ai/cli dev
```

Or use package.json scripts:

{
  "scripts": {
    "dev": "agentmark dev"
  }
}

Port already in use

Error: Error: listen EADDRINUSE: address already in use :::9418 Cause: Another process is using the port. Solution:

Use a different port:
```
agentmark dev --api-port 9500
```

Or kill the existing process:

# Find the process
lsof -i :9418

# Kill it
kill -9 <PID>

Still Having Issues?

If you’re still experiencing problems:

Check the logs - Run with DEBUG=agentmark:* for verbose output:
```
DEBUG=agentmark:* agentmark dev
```

Update packages - Ensure you’re on the latest versions:

npm update @agentmark-ai/cli @agentmark-ai/ai-sdk-v5-adapter

Ask the community - Join our Discord for help
Report a bug - Open an issue on GitHub

Have Questions?

We’re here to help! Choose the best way to reach us:

Join our Discord community for quick answers and discussions
Email us at [email protected] for support
Schedule an Enterprise Demo to learn about our business solutions

Getting Started

Prompts and Agents

Testing

Observability

Integrations

Python

Further Reference

​Troubleshooting

​Connection & Authentication Errors

​AGENTMARK_API_KEY not set

​AGENTMARK_APP_ID not set

​Loader connection failed

​Invalid API key

​Model Errors

​Model not found / not registered

​Provider API key missing

​Rate limit exceeded

​Model not available in region

​Tool Errors

​Tool not available

​Tool execution failed

​MCP Server Errors

​MCP server not connecting

​MCP tool not found

​Prompt Errors

​Prompt not found

​Invalid frontmatter

​Template syntax error

​Experiment & Dataset Errors

​Dataset file not found

​Invalid dataset format

​Eval function not registered

​Type Errors

​Type mismatch after schema change

​Props type error

​CLI Errors

​agentmark command not found

​Port already in use

​Still Having Issues?

​Have Questions?