Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.agentmark.co/llms.txt

Use this file to discover all available pages before exploring further.

When you connect a repository to AgentMark — GitHub or GitLab — every push gets a status check on the commit: AgentMark / Build. It’s the same pattern Vercel uses: if the deploy would fail, the PR/MR shows red and (once you make it required) blocks merge.

​
What gets checked

On both providers, the check runs every prompt and component file in your push through the same compiler the runtime uses (getTemplateDXInstance(...).parse(...)). If a file would fail to load at request time, the check fails — with that file annotated on the PR (GitHub) or summarized in the status description (GitLab). What surfaces as a failure today:
  • Template syntax errors — malformed MDX/JSX in .prompt.mdx or .mdx files (unclosed tags, stray HTML comments, unexpected characters).
  • Frontmatter errors — missing text_config/object_config/image_config/speech_config; invalid model names; missing required fields.
  • Schema reference errors — $ref paths that don’t resolve in the push.
What the check does not cover:
  • Your handler’s TypeScript or Python build — that runs on the code-deploy step, and surfaces in the AgentMark dashboard under the deployment’s build logs.
  • Files outside the agentmark directory configured by agentmark.json — only files AgentMark Cloud imports are validated.

​
Conclusion states

The check reports one of three conclusions per push:
ConclusionGitHub appearanceGitLab appearance
successgreen checksuccess status, “All prompt and component files compiled.”
failurered ✕, with per-file annotations on the PR’s Files Changed tabfailed status, description shows count by category (e.g. 3 prompt compile errors (1 frontmatter, 2 template syntax))
neutralgray, “No prompts to compile”success status, “No AgentMark files in this push.” — GitLab has no native neutral state, so we collapse to success

​
GitHub

​
Making the check required

GitHub’s “required status check” setting is what actually blocks merge — it’s a repo-level setting we deliberately don’t manage on your behalf, because doing so needs administration scope on the GitHub App and that’s a security surface most teams don’t want us holding. To enforce the check:
  1. In your GitHub repo, go to Settings → Branches → Branch protection rules.
  2. Add a rule for the branch you deploy from (usually main).
  3. Enable Require status checks to pass before merging.
  4. In the search box, type AgentMark and select AgentMark / Build.
    The check name only appears in GitHub’s picker after the first push that triggers it. If you don’t see it, push any commit to the repo first, then come back to this screen.
  5. Save.
From now on, any PR targeting that branch can’t merge until the check passes.

​
What a failure looks like

When a prompt fails to compile, the PR’s Files changed tab gets an inline annotation on the offending file. The annotation title categorizes the failure so you can spot the class of problem from the sidebar without opening each file:
  • Frontmatter error — fix the --- block at the top of the prompt.
  • Template syntax error — MDX/JSX in the body didn’t parse.
  • Schema reference error — a $ref couldn’t be resolved.
  • Prompt compile error — generic fallback.
Annotations include the parser’s line and column when the underlying templatedx error carries them. Frontmatter errors without a line number fall back to line 1 (the start of the frontmatter block).

​
GitLab

​
What the check looks like

GitLab’s commit status API is intentionally thinner than GitHub’s Checks API — it carries {state, name, description, target_url} and nothing else. So the GitLab experience differs in two ways:
  • No line-level annotations on the MR diff. Failure details ride on the status description as a categorized count: 4 prompt compile errors (1 frontmatter, 2 template syntax, 1 schema reference).
  • The target_url points to the AgentMark dashboard — click through for the per-file failure list with full error messages.
A planned follow-up will use GitLab’s MR Discussions API to post line-positioned comments alongside the status, giving GitLab users back the line specificity GitHub gets via annotations.

​
Making the check required

GitLab’s options depend on your tier:
GitLab tierMechanismLimitation
Free”Pipelines must succeed” in Settings → Merge requestsOnly blocks merge if the status comes from a real GitLab CI job. The Cloud-managed status posted by AgentMark Cloud is informational. To block merge on free tier, use the self-hosted CI job pattern in Hardcoded merge blocking via CI below — it runs agentmark build as a real pipeline job, which “Pipelines must succeed” can block on.
Premium / UltimateExternal Status Checks in Settings → General → Merge request approvalsAdd a rule pointing at AgentMark; the Cloud-managed status counts as a required external approval.

​
Hardcoded merge blocking via CI

The Cloud-managed status checks above are convenient — zero config beyond installing the App or wiring a webhook — but they depend on AgentMark Cloud being reachable, and on the provider tier supporting “required external checks” (GitLab Free doesn’t). For a fully self-hosted alternative that blocks merge on any provider regardless of tier, run agentmark build as a CI job in your own repository. The CLI compiles every prompt with the same compiler the runtime uses and exits non-zero on failure — failing the pipeline naturally and triggering the provider’s built-in “pipelines must succeed” merge guard. This pattern has three advantages over (or alongside) the Cloud-managed check:
  • Works on GitLab Free. “Pipelines must succeed” only blocks merge on pipeline failures, not external status checks — but agentmark build failing IS a pipeline failure.
  • Doesn’t depend on AgentMark Cloud uptime. The validation runs entirely inside your CI. If our webhook handler is degraded, your merge guard still works.
  • Faster feedback for big repos. No webhook round-trip — the CI job runs in seconds against the pushed commit directly.

​
GitLab CI

.gitlab-ci.yml
agentmark-build:
  image: node:22
  stage: test
  script:
    - npx --yes @agentmark-ai/cli build
  rules:
    - changes:
        - "agentmark/**/*"
        - "agentmark.json"
        - "**/*.prompt.mdx"
        - "**/*.mdx"
Then in Settings → Merge requests → Merge checks, enable Pipelines must succeed. From now on, an MR can’t merge until the agentmark-build job is green.

​
GitHub Actions

.github/workflows/agentmark-build.yml
name: AgentMark / Build
on:
  pull_request:
    paths:
      - "agentmark/**"
      - "agentmark.json"
      - "**/*.prompt.mdx"
      - "**/*.mdx"

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npx --yes @agentmark-ai/cli build
Then in Settings → Branches → Branch protection rules, add a rule requiring the AgentMark / Build check name. Same picker, same UX as the App-based check — your CI job’s status takes the place of (or runs alongside) the App’s.
You can run both the Cloud-managed check and the self-hosted CI check in parallel. They use the same name (AgentMark / Build) but appear as separate entries in the required-status picker on GitHub. On GitLab they appear as separate statuses on the MR pipeline. Belt-and-suspenders: if either signals failure, the merge blocks.

​
Re-running the check

There’s no “Re-run” button on either platform — the check is keyed to the commit SHA, and pushing a new commit (even an empty one) is the way to retry: 
git commit --allow-empty -m "Retry AgentMark check"
git push

​
Limitations

  • PRs from forks don’t get checks (GitHub) today. The check is posted from the push webhook, which forks don’t send to the upstream App. PRs from branches in the same repo do get checks. GitLab MRs from forked projects have the analogous limitation.
  • No retries on transient outages. If the status API itself errors when we post the result, we log it and move on; the deploy still runs. The check will sit at in progress (GitHub) / running (GitLab) until the provider’s timeout. You can re-push to recover.
  • One check per push, not per file. The aggregate conclusion comes from every prompt file in the change set.
  • GitLab line-level annotations are deferred to a follow-up that uses the MR Discussions API to post comments at failing line positions. Today, GitLab users get a categorized count in the status description; the full per-file list lives in the AgentMark dashboard.

​
Have Questions?

We’re here to help! Choose the best way to reach us: