Features & Improvements
- AgentMark init
- Updated examples in CLI init
- Rebrand Puzzlet -> AgentMark
- CLI: “run-prompt” for dataset + single props
- Webhook Helpers
- Alerts enhancements
- Google/GitHub auth
- Dataset runs directly via prompts
- Vercel v4 webhook helper
- Streaming to the platform
- Commit History + Rollbacks
Bug Fixes
- Ollama fix on init
JSONL Datasets

Evals & Scoring

Other
- Consolidating prompts, evals, and datasets into single “files”
- Officially rolled out alerts
- Some CLI improvements
- Minor bug fixes
Sessions

Alerts

Traces UI Improvements

Onboarding Improvements
We’ve improved our onboarding. Now, you can see your dashboard without having to sync your repo first. We also support modular onboarding, so you can skip steps you don’t need.2025-02-18
Add Trace Examples to Datasets, Load Trace in Prompt, Re-indexing, App UI Improvments, bug fixes
Adding Examples to Datasets

Adding Examples to Prompts

Re-indexing

App UI Improvements
You can now view your easily app’s repo configuration, including repo names, branch, and more.Type Safety
AgentMark aims to provide developers with the best developer experience possible. As part of this, we’ve just added type safety to our platform.- Types can now be generated via our CLI
- Fetching prompts from our CDN or AgentMark are now type-safe
- Prompts now support run/compile/deserialize functions

Datasets
Datasets now allow you to test your prompts in bulk against a large set of data.- Run your datasets in bulk against your prompts
- View previous runs, with inputs/outputs
- View traces associated with each run
- View high-level metrics for each run

Trace Grouping
Traces can now be grouped based on the trace function, and the component function. Trace groups together at the root level, while component allows for sub-groups.- New function added:
trace
- New function added:
component

CLI Improvements
Our CLI has been improved to provide a better developer experience.- AgentMark
init
can optionally create an example app - Added
pull-models
to walk through adding new models to your platform
Bug Fixes
- Fixed a bug which could cause an app’s templates to be deleted when a new app was created
- Fixed a bug which could cause some branches not to show up in the UI
- Fixed a bug which could prevent newly created local prompts from being synced to the platform
Other
- Improved UI for prompts input/output
- Paginate traces
- Improved UI theme for prompts
Overview
Features
- Prompt Management
- Observability
- Datasets
- CLI
- Platform Management
- Evals
Prompt Management

Observability

Datasets
Create datasets to test easily test your prompts in bulk against a large set of data.Read DocsCLI
We provide a CLI for initializing your AgentMark app, customizing it, and deploying it to the cloud. Add new models to your platform with just a single command. You can also develop w/ AgentMark locally using our serve command.bash
Platform Management
AgentMark offers an intuitive platform for creating new git-synced apps, adding team members with roles, and setting up API keys for users.AgentMark SDK

- Initial release of AgentMark
- Support for OpenAI, Anthropic, and other LLM providers
- MDX-based prompt templating
- Type-safe prompt development
- Tools and agents support
- Added comprehensive documentation
- Included examples and guides
- API reference documentation