Features & Improvements
- Terms of Service agreement — users are now prompted to accept terms before using the platform
- Non-blocking terms update policy — users are notified of terms changes without being blocked
- Streamlined registration — skip company setup during onboarding for faster account creation
- Admin Control Panel — platform admins can manage users, roles, and organization settings
- Profile email updates — update your profile email directly from account settings
- Email sync — profile email now stays in sync when your authentication email changes
- PostHog analytics integration for improved product insights
- BetterStack health monitoring with uptime tracking
- MCP Server API endpoints — programmatic access to the platform via MCP
- Platform admin changelog notifications — send changelog updates to users via email
- Annotations improvements and reliability fixes
- Organization loading performance improvements
Bug Fixes
- Fixed OAuth registration flow
- Fixed image prompt timeout for large image generation
- Strengthened password validation with improved error messaging
- Fixed request filters not working correctly
- Fixed dashboard display issues
- Fixed git sync and user invite flows
- Fixed dataset run execution
- Fixed GitLab commit history display
- Fixed template frontmatter handling
Features & Improvements
- Multi-tenant membership — users can now belong to multiple organizations and switch between them
- GitLab integration — connect GitLab repositories alongside GitHub for prompt management Read Docs
- Tracing revamp — improved observability UI with better data presentation Read Docs
Bug Fixes
- Fixed trace span selection unexpectedly jumping to top span
- Fixed edit prompt processing for ‘done’ message type
- Fixed annotations not appearing after being added
Features & Improvements
- Annotations — add notes and labels to trace scores for better evaluation tracking
- Interactive trace graph — visualize trace execution flow with resizable panels
- MCP integration documentation — guides and examples for using AgentMark as an MCP server
Bug Fixes
- Fixed aspect ratio issue that could break the app layout
- Fixed GitHub username not being saved on first OAuth login
- Fixed webhook delivery reliability
Features & Improvements
- Evaluation support — evaluate prompts and dataset outputs with scoring
- Eval-based alerts — get notified when evaluations detect issues
- Dataset V2 — improved dataset management with file loading support
- Enhanced dataset metrics and evaluation charts for better insights
- Automated pricing updates — model pricing stays current automatically
- Build failure notifications — project owners are notified when builds fail
- Improved permission checks for server actions
Bug Fixes
- Fixed OAuth registration login and redirect issues
- Fixed incorrect trace count when using filters
- Improved database error messages with readable descriptions
Features & Improvements
- AgentMark init
- Updated examples in CLI init
- Rebrand Puzzlet -> AgentMark
- CLI: “run-prompt” for dataset + single props
- Webhook Helpers
- Alerts enhancements
- Google/GitHub auth
- Dataset runs directly via prompts
- Vercel v4 webhook helper
- Streaming to the platform
- Commit History + Rollbacks
Bug Fixes
- Ollama fix on init
JSONL Datasets
Evals & Scoring
Other
- Consolidating prompts, evals, and datasets into single “files”
- Officially rolled out alerts
- Some CLI improvements
- Minor bug fixes
Sessions
Alerts
Traces UI Improvements
Onboarding Improvements
We’ve improved our onboarding. Now, you can see your dashboard without having to sync your repo first. We also support modular onboarding, so you can skip steps you don’t need.2025-02-18
Add Trace Examples to Datasets, Load Trace in Prompt, Re-indexing, App UI Improvments, bug fixes
Adding Examples to Datasets
Adding Examples to Prompts
Re-indexing
App UI Improvements
You can now view your easily app’s repo configuration, including repo names, branch, and more.Type Safety
AgentMark aims to provide developers with the best developer experience possible. As part of this, we’ve just added type safety to our platform.- Types can now be generated via our CLI
- Fetching prompts from our CDN or AgentMark are now type-safe
- Prompts now support run/compile/deserialize functions
Datasets
Datasets now allow you to test your prompts in bulk against a large set of data.- Run your datasets in bulk against your prompts
- View previous runs, with inputs/outputs
- View traces associated with each run
- View high-level metrics for each run
Trace Grouping
Traces can now be grouped based on the trace function, and the component function. Trace groups together at the root level, while component allows for sub-groups.- New function added:
trace - New function added:
component
CLI Improvements
Our CLI has been improved to provide a better developer experience.- AgentMark
initcan optionally create an example app - Added
pull-modelsto walk through adding new models to your platform
Bug Fixes
- Fixed a bug which could cause an app’s templates to be deleted when a new app was created
- Fixed a bug which could cause some branches not to show up in the UI
- Fixed a bug which could prevent newly created local prompts from being synced to the platform
Other
- Improved UI for prompts input/output
- Paginate traces
- Improved UI theme for prompts
Overview
Features
- Prompt Management
- Observability
- Datasets
- CLI
- Platform Management
- Evals
Prompt Management
Observability
Datasets
Create datasets to test easily test your prompts in bulk against a large set of data.Read DocsCLI
We provide a CLI for initializing your AgentMark app, customizing it, and deploying it to the cloud. Add new models to your platform with just a single command. You can also develop w/ AgentMark locally using our serve command.bash
Platform Management
AgentMark offers an intuitive platform for creating new git-synced apps, adding team members with roles, and setting up API keys for users.AgentMark SDK
Features
- Initial release of AgentMark
- Support for OpenAI, Anthropic, and other LLM providers
- MDX-based prompt templating
- Type-safe prompt development
- Tools and agents support
- Added comprehensive documentation
- Included examples and guides
- API reference documentation