Skip to main content
When you connect a repository to AgentMark, on either GitHub or GitLab, every push gets a status check on the commit: AgentMark / Build. If the deploy would fail, the PR/MR shows red. On GitHub you can make the check required so it blocks merge; on GitLab the status is informational, and merge blocking runs through a CI job.

​
What gets checked

On both providers, the check runs every prompt and component file in your push through the same TemplateDX compiler the runtime uses. If a file would fail to load at request time, the check fails, with that file annotated on the PR (GitHub) or summarized in the status description (GitLab). What surfaces as a failure:
  • Template syntax errors. Malformed MDX/JSX in .prompt.mdx or .mdx files (unclosed tags, stray HTML comments, unexpected characters).
  • Frontmatter errors. Missing text_config/object_config/image_config/speech_config, invalid model names, or missing required fields.
  • Schema reference errors. Malformed $refs that fail at parse time.
What the check does not cover:
  • Your handler’s TypeScript or Python build. That runs on the code-deploy step, and surfaces in the AgentMark Dashboard under the deployment’s build logs.
  • Files outside the directory set by agentmarkPath in agentmark.json. Only files AgentMark Cloud imports are validated.
  • $refs and imports that point at files outside the push. The check can’t validate those standalone, so it skips them rather than failing, which means a typo’d schema path passes the check and surfaces later at load time.

​
Conclusion states

The check reports one of three conclusions per push:
ConclusionGitHub appearanceGitLab appearance
successgreen checksuccess status, “All prompt and component files compiled.”
failurered ✕, with per-file annotations on the PR’s Files Changed tabfailed status, description shows count by category (e.g. 3 prompt compile errors (1 frontmatter, 2 template syntax))
neutralgray, “No prompts to compile”success status, “No AgentMark files in this push.” GitLab has no native neutral state, so we collapse to success

​
GitHub

​
Making the check required

GitHub’s required status check setting is what blocks merge. To enforce the check:
  1. In your GitHub repo, go to Settings → Branches → Branch protection rules.
  2. Add a rule for the branch you deploy from (usually main).
  3. Enable Require status checks to pass before merging.
  4. In the search box, type AgentMark and select AgentMark / Build.
    The check name only appears in GitHub’s picker after the first push that triggers it. If you don’t see it, push any commit to the repo first, then come back to this screen.
  5. Save.
From now on, any PR targeting that branch can’t merge until the check passes.

​
What a failure looks like

When a prompt fails to compile, the PR’s Files changed tab gets an inline annotation on the offending file. The annotation title categorizes the failure so you can spot the class of problem from the sidebar without opening each file:
  • Frontmatter error. Fix the --- block at the top of the prompt.
  • Template syntax error. MDX/JSX in the body didn’t parse.
  • Schema reference error. A malformed $ref failed at parse time.
  • Prompt compile error. Generic fallback.
Annotations include the parser’s line and column when the underlying templatedx error carries them. Frontmatter errors without a line number fall back to line 1 (the start of the frontmatter block).

​
GitLab

​
What the check looks like

GitLab’s commit status API is intentionally thinner than GitHub’s Checks API: it carries {state, name, description, target_url} and nothing else. So the GitLab experience differs in two ways:
  • No line-level annotations on the MR diff. Failure details ride on the status description as a categorized count: 4 prompt compile errors (1 frontmatter, 2 template syntax, 1 schema reference).
  • The target_url points to the AgentMark Dashboard. Click through for the per-file failure list with full error messages.

​
Making the check required

On GitLab, the Cloud-managed status is informational on every tier. AgentMark posts plain commit statuses, and GitLab’s merge-blocking mechanisms don’t consume them:
  • Pipelines must succeed (Settings → Merge requests → Merge checks) only watches real GitLab CI pipeline jobs, not externally posted statuses.
  • External Status Checks (Ultimate tier, Settings → Merge requests → Status checks) require the external service to respond through GitLab’s status-checks API. A commit status cannot satisfy an external status check, and AgentMark does not register as one.
To block merges on GitLab, use the CI job pattern in Hardcoded merge blocking via CI below: agentmark build failing as a pipeline job is a real pipeline failure, which Pipelines must succeed blocks on, on every GitLab tier.

​
Hardcoded merge blocking via CI

The Cloud-managed status checks above take zero config beyond installing the App or wiring a webhook, but they depend on AgentMark Cloud being reachable, and on GitLab they can’t block merge at all. For a fully self-hosted alternative that blocks merge on any provider regardless of tier, run agentmark build as a CI job in your own repository. The CLI compiles every prompt with the same compiler the runtime uses and exits non-zero on failure, which fails the pipeline and triggers the provider’s built-in “pipelines must succeed” merge guard. This pattern has three advantages over (or alongside) the Cloud-managed check:
  • The only merge-blocking path on GitLab. “Pipelines must succeed” blocks on pipeline failures, not externally posted statuses, and agentmark build failing IS a pipeline failure. Works on every GitLab tier, including Free.
  • Doesn’t depend on AgentMark Cloud uptime. The validation runs entirely inside your CI. If our webhook handler is degraded, your merge guard still works.
  • Faster feedback for big repos. No webhook round-trip; the CI job runs in seconds against the pushed commit directly.

​
GitLab CI

.gitlab-ci.yml
agentmark-build:
  image: node:22
  stage: test
  script:
    - npx --yes @agentmark-ai/cli build
  rules:
    - changes:
        - "agentmark/**/*"
        - "agentmark.json"
        - "**/*.prompt.mdx"
        - "**/*.mdx"
Then in Settings → Merge requests → Merge checks, enable Pipelines must succeed. From now on, an MR can’t merge until the agentmark-build job is green.

​
GitHub Actions

.github/workflows/agentmark-build.yml
name: AgentMark / Build
on:
  pull_request:
    paths:
      - "agentmark/**"
      - "agentmark.json"
      - "**/*.prompt.mdx"
      - "**/*.mdx"

jobs:
  build:
    name: "AgentMark / Build"
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 22
      - run: npx --yes @agentmark-ai/cli build
The name: on the job matters: GitHub’s required-status picker lists Actions checks by job display name, not the workflow name, so without it the check appears as build. Then in Settings → Branches → Branch protection rules, add a rule requiring the AgentMark / Build check name. Same picker, same UX as the App-based check: your CI job’s status takes the place of (or runs alongside) the App’s.
You can run both the Cloud-managed check and the self-hosted CI check in parallel. They use the same name (AgentMark / Build) but appear as separate entries in the required-status picker on GitHub; require both and the merge blocks if either fails. On GitLab they appear as separate statuses on the MR pipeline, and only the CI job blocks merge; the Cloud-managed status stays informational.

​
Re-running the check

There’s no “Re-run” button on either platform. The check is keyed to the commit SHA, so pushing a new commit (even an empty one) is the way to retry: 
git commit --allow-empty -m "Retry AgentMark check"
git push

​
Limitations

  • PRs from forks don’t get checks (GitHub). The check is posted from the push webhook, which forks don’t send to the upstream App. PRs from branches in the same repo do get checks. GitLab MRs from forked projects have the analogous limitation.
  • No retries on transient outages. If the status API itself errors when we post the result, we log it and move on; the deploy still runs. The check will sit at in progress (GitHub) / running (GitLab) until the provider’s timeout. You can re-push to recover.
  • One check per push, not per file. The aggregate conclusion comes from every prompt file in the change set.
  • No line-level annotations on GitLab. GitLab users get a categorized count in the status description; the full per-file list lives in the AgentMark Dashboard.

​
Have questions?

Reach out any time: