Adding Models

Custom model schemas give developers control over non-supported models in AgentMark:

1. Custom Model Integration: Define schemas for models that aren’t natively supported by AgentMark, including proprietary or third-party models

2. User Interface Control: Precisely specify which parameters platform users can modify through the UI, while keeping sensitive or advanced configurations under developer control

3. Cost Customization: Customize the costs for your custom models to enable proper tracking of logs/metrics/traces in AgentMark

​

Pulling Models

To add models to your project, use the pull-models command:

npx @agentmark/cli@latest pull-models

This interactive command will:

1. Show you available providers (like OpenAI, Anthropic)
2. Let you select which models you want to use
3. Add the appropriate model schemas to your agentmark.json file

​

Adding Custom Model Schemas

Each custom model schema allows you to:

• Set a user-friendly display label
• Configure available settings and their UI controls
• Define parameter constraints and defaults
• Control the order and presentation of settings

Here’s an example of a custom model schema:

{
  ...,
  "modelSchemas": {
    "my-custom-model": {
      "label": "My Custom Model",
      "settings": {
        "temperature": {
          "minimum": 0,
          "maximum": 1,
          "default": 0.7,
          "multipleOf": 0.01,
          "label": "Temperature",
          "order": 2,
          "type": "number",
          "ui": "slider"
        },
        "max_tokens": {
          "minimum": 16,
          "default": 4096,
          "maximum": 4096,
          "multipleOf": 1,
          "label": "Max Tokens",
          "order": 3,
          "type": "number",
          "ui": "slider"
        }
      }
    }
  }
}

​

Multiple Custom Models

You can define multiple custom models in your schema, each with its own configuration:

{
  "modelSchemas": {
    "proprietary-model-1": {
      "label": "Proprietary Model 1",
      "settings": { ... }
    },
    "third-party-model": {
      "label": "Third Party Model",
      "settings": { ... }
    },
    "custom-model": {
      "label": "My Custom Model",
      "settings": { ... }
    }
  }
}

​

Setting Properties

Each setting in a custom model schema can define:

• minimum/maximum: The allowed range for numeric values
• default: The initial value
• multipleOf: Step size for numeric inputs
• label: Display name in the UI
• order: Control the display order in forms
• type: The type of the setting (e.g. “number”, “string”, “boolean”)
• ui: The UI control to use for the setting (e.g. “slider”…more to come)

​

Custom Model Cost Configuration

You can customize the costs for your custom models to enable proper tracking in AgentMark:

{
  "modelSchemas": {
    "my-custom-model": {
      "label": "My Custom Model",
      "cost": {
        "inputCost": 0.01, // $0.01 per 1000 tokens
        "outputCost": 0.02, // $0.02 per 1000 tokens
        "unitScale": 1000
      }
    }
  }
}