Each AgentMark adapter ships a webhook handler class via its /runner export. These handlers implement a standard interface that processes prompt and dataset events from the AgentMark platform, so you don’t need to write low-level generation logic yourself.
Available Handlers
| Adapter | Handler Class | Import |
|---|
| AI SDK v5 | VercelAdapterWebhookHandler | @agentmark-ai/ai-sdk-v5-adapter/runner |
| AI SDK v4 | VercelAdapterWebhookHandler | @agentmark-ai/ai-sdk-v4-adapter/runner |
| Claude Agent SDK | ClaudeAgentWebhookHandler | @agentmark-ai/claude-agent-sdk-adapter/runner |
| Mastra | MastraAdapterWebhookHandler | @agentmark-ai/mastra-v0-adapter/runner |
Handler Interface
All webhook handlers implement the same interface:
interface WebhookHandler {
runPrompt(
promptAst: Ast,
options?: {
shouldStream?: boolean;
customProps?: Record<string, any>;
telemetry?: {
isEnabled: boolean;
metadata?: Record<string, any>;
};
}
): Promise<WebhookPromptResponse>;
runExperiment(
promptAst: Ast,
datasetRunName: string,
datasetPath?: string
): Promise<WebhookDatasetResponse>;
}
Creating a Handler
Each handler takes the AgentMark client (created with createAgentMarkClient()) as its constructor argument:
AI SDK v5 (Vercel)
AI SDK v4 (Vercel)
Claude Agent SDK
Mastra
import { VercelAdapterWebhookHandler } from "@agentmark-ai/ai-sdk-v5-adapter/runner";
const handler = new VercelAdapterWebhookHandler(client);
import { VercelAdapterWebhookHandler } from "@agentmark-ai/ai-sdk-v4-adapter/runner";
const handler = new VercelAdapterWebhookHandler(client);
import { ClaudeAgentWebhookHandler } from "@agentmark-ai/claude-agent-sdk-adapter/runner";
const handler = new ClaudeAgentWebhookHandler(client);
import { MastraAdapterWebhookHandler } from "@agentmark-ai/mastra-v0-adapter/runner";
const handler = new MastraAdapterWebhookHandler(client);
The runPrompt Method
Executes a single prompt and returns the result. The handler inspects the prompt’s configuration (text_config, object_config, image_config, speech_config) and calls the appropriate generation function.
Parameters
| Parameter | Type | Description |
|---|
promptAst | Ast | The prompt AST object from the webhook event payload |
options.shouldStream | boolean | Whether to stream the response. Defaults to true for AI SDK adapters, false for Claude Agent SDK and Mastra |
options.customProps | Record<string, any> | Custom input properties to pass to the prompt |
options.telemetry | object | Telemetry configuration with isEnabled flag and optional metadata |
Return Value
Returns a Promise<WebhookPromptResponse> — the response type depends on the generation type and whether streaming is enabled.
Streaming response (response.type === "stream"):
{
type: "stream";
stream: ReadableStream;
streamHeader: { "AgentMark-Streaming": "true" };
traceId?: string;
}
Response:
if (response.type === "stream") {
return new Response(response.stream, {
headers: { ...response.streamHeader },
});
}
// Text
{ type: "text", result: string, usage: TokenUsage, finishReason: string, toolCalls?: any[], toolResults?: any[] }
// Object
{ type: "object", result: unknown, usage: TokenUsage, finishReason: string }
// Image
{ type: "image", result: Array<{ mimeType: string, base64: string }> }
// Speech
{ type: "speech", result: { mimeType: string, base64: string, format: string } }
Example
const response = await handler.runPrompt(event.data.ast, {
shouldStream: true,
customProps: event.data.customProps,
});
if (response.type === "stream") {
return new Response(response.stream, {
headers: { ...response.streamHeader },
});
}
return NextResponse.json(response);
The runExperiment Method
Runs a prompt against all items in a dataset and streams the results. Each item is executed sequentially, with optional evaluation scoring.
Parameters
| Parameter | Type | Description |
|---|
promptAst | Ast | The prompt AST object |
datasetRunName | string | A name for this experiment run |
datasetPath | string (optional) | Path to the dataset file |
Return Value
Returns a Promise<WebhookDatasetResponse>:
{
stream: ReadableStream;
streamHeaders: { "AgentMark-Streaming": "true" };
}
Response:
const response = await handler.runExperiment(
event.data.ast,
event.data.experimentId,
event.data.datasetPath
);
return new Response(response.stream, {
headers: { ...response.streamHeaders },
});
{
"type": "dataset",
"result": {
"input": "What is 2+2?",
"expectedOutput": "4",
"actualOutput": "The answer is 4",
"tokens": 15,
"evals": [
{
"name": "accuracy",
"score": 1.0,
"label": "correct",
"reason": "Output matches expected result",
"passed": true
}
]
},
"traceId": "abc-123",
"runId": "run-456",
"runName": "test-dataset-run"
}
EvalRegistry
Register custom evaluation functions that automatically score dataset run outputs. The EvalRegistry is re-exported from every adapter package for convenience.
import { EvalRegistry } from "@agentmark-ai/prompt-core";
// or from any adapter:
// import { EvalRegistry } from "@agentmark-ai/ai-sdk-v5-adapter";
const evalRegistry = new EvalRegistry();
evalRegistry.register("correctness", ({ input, output, expectedOutput }) => {
const isCorrect = output === expectedOutput;
return {
score: isCorrect ? 1.0 : 0.0,
passed: isCorrect,
label: isCorrect ? "correct" : "incorrect",
reason: isCorrect
? "Output matches expected result"
: "Output does not match expected result",
};
});
EvalParams
interface EvalParams {
input: string | Record<string, unknown> | Array<Record<string, unknown> | string>;
output: string | Record<string, unknown> | Array<Record<string, unknown> | string>;
expectedOutput?: string;
}
EvalResult
interface EvalResult {
score?: number; // Numeric score (0-1 recommended)
passed?: boolean; // Pass/fail indicator
label?: string; // Classification label
reason?: string; // Explanation
}
Signature Verification
All incoming webhook requests must be verified using the x-agentmark-signature-256 header. Use the verifySignature function from @agentmark-ai/shared-utils:
import { verifySignature } from "@agentmark-ai/shared-utils";
const isValid = await verifySignature(
process.env.AGENTMARK_WEBHOOK_SECRET!,
request.headers.get("x-agentmark-signature-256")!,
JSON.stringify(payload)
);
if (!isValid) {
return NextResponse.json({ message: "Invalid signature" }, { status: 401 });
}
sha256={hex}.
Adapter Feature Support
| Feature | AI SDK v4 | AI SDK v5 | Claude Agent SDK | Mastra |
|---|
| Text generation | Yes | Yes | Yes | Yes |
| Object generation | Yes | Yes | Yes | Yes |
| Image generation | Yes | Yes | No | No |
| Speech generation | Yes | Yes | No | No |
| Streaming (default) | Yes (on) | Yes (on) | Yes (off) | Yes (off) |
| Tool calls | Yes | Yes | Yes | Yes |
| Telemetry | Yes | Yes | Yes | Yes |
Manual Implementation
If you need full control over generation logic, you can handle events manually without the webhook handler. See the event-specific documentation:
Have Questions?
We’re here to help! Choose the best way to reach us:
