## Create a custom model

**post** `/v5/models`

Create a custom model record in your account and begin deploying it through a supported serving vendor.

A model here is a record for a model you deploy and serve through Scale's own inference vendors: only the `launch` and `llmengine` vendors are accepted and any other vendor is rejected. This is distinct from `GET /v5/chat/completions/models`, which lists the models already available to call for chat completions rather than creating or managing these records. The call is asynchronous — the record is created in a deploying status, a deployment job is recorded, and a Temporal workflow is started to perform the deployment, so the model is not ready for inference when this returns. A model name must be unique per vendor within your account; if a model with the same name and vendor already exists the request fails unless `on_conflict` is set to `update`, in which case the existing model is updated instead.

### Body Parameters

- `model: object { name, vendor_configuration, model_metadata, 3 more }  or object { name, vendor_configuration, model_metadata, 3 more }  or object { model_type, model_vendor, name, 2 more }`

  Register a model already served by an external / proxy-served vendor
  (e.g. an OpenAI-compatible self-hosted model behind the inference proxy).

  Unlike launch/llmengine, no Scale-side deployment is performed: the record is
  created READY and is immediately callable via /v5/chat/completions. Accepted only
  when NATIVE_OPENAI_INFERENCE_GATEWAY is enabled. The discriminator (model_vendor)
  covers every vendor except launch/llmengine, and no vendor_configuration applies.

  - `Launch object { name, vendor_configuration, model_metadata, 3 more }`

    - `name: string`

      Unique name to reference your model

    - `vendor_configuration: LaunchVendorConfiguration`

      - `model_image: object { command, registry, repository, 9 more }`

        - `command: array of string`

        - `registry: string`

        - `repository: string`

        - `tag: string`

        - `env_vars: optional map[unknown]`

        - `healthcheck_route: optional string`

        - `predict_route: optional string`

        - `readiness_delay: optional number`

        - `request_schema: optional map[unknown]`

        - `response_schema: optional map[unknown]`

        - `streaming_command: optional array of string`

        - `streaming_predict_route: optional string`

      - `model_infra: object { cpus, endpoint_type, gpu_type, 9 more }`

        - `cpus: optional string or number`

          - `string`

          - `number`

        - `endpoint_type: optional "async" or "sync" or "streaming"`

          - `"async"`

          - `"sync"`

          - `"streaming"`

        - `gpu_type: optional "nvidia-tesla-t4" or "nvidia-ampere-a10" or "nvidia-ampere-a100" or 4 more`

          - `"nvidia-tesla-t4"`

          - `"nvidia-ampere-a10"`

          - `"nvidia-ampere-a100"`

          - `"nvidia-ampere-a100e"`

          - `"nvidia-hopper-h100"`

          - `"nvidia-hopper-h100-1g20gb"`

          - `"nvidia-hopper-h100-3g40gb"`

        - `gpus: optional number`

        - `high_priority: optional boolean`

        - `labels: optional map[string]`

        - `max_workers: optional number`

        - `memory: optional string`

        - `min_workers: optional number`

        - `per_worker: optional number`

        - `public_inference: optional boolean`

        - `storage: optional string`

    - `model_metadata: optional map[unknown]`

    - `model_type: optional "generic"`

      - `"generic"`

    - `model_vendor: optional "launch"`

      - `"launch"`

    - `on_conflict: optional "error" or "update"`

      - `"error"`

      - `"update"`

  - `Llmengine object { name, vendor_configuration, model_metadata, 3 more }`

    - `name: string`

      Unique name to reference your model

    - `vendor_configuration: LlmEngineVendorConfiguration`

      - `model: string`

      - `chat_template_override: optional string`

      - `checkpoint_path: optional string`

      - `cpus: optional number`

      - `default_callback_url: optional string`

      - `endpoint_type: optional string`

      - `gpu_type: optional string`

      - `gpus: optional number`

      - `high_priority: optional boolean`

      - `inference_framework: optional string`

      - `inference_framework_image_tag: optional string`

      - `labels: optional map[string]`

      - `max_workers: optional number`

      - `memory: optional string`

      - `min_workers: optional number`

      - `nodes_per_worker: optional number`

      - `num_shards: optional number`

      - `per_worker: optional number`

      - `post_inference_hooks: optional array of string`

      - `public_inference: optional boolean`

      - `quantize: optional string`

      - `source: optional string`

      - `storage: optional string`

    - `model_metadata: optional map[unknown]`

    - `model_type: optional "chat_completion"`

      - `"chat_completion"`

    - `model_vendor: optional "llmengine"`

      - `"llmengine"`

    - `on_conflict: optional "error" or "update"`

      - `"error"`

      - `"update"`

  - `HostedModelCreateRequest object { model_type, model_vendor, name, 2 more }`

    Register a model already served by an external / proxy-served vendor
    (e.g. an OpenAI-compatible self-hosted model behind the inference proxy).

    Unlike launch/llmengine, no Scale-side deployment is performed: the record is
    created READY and is immediately callable via /v5/chat/completions. Accepted only
    when NATIVE_OPENAI_INFERENCE_GATEWAY is enabled. The discriminator (model_vendor)
    covers every vendor except launch/llmengine, and no vendor_configuration applies.

    - `model_type: InferenceModelType`

      Type of model, for example `chat_completion`

      - `"generic"`

      - `"completion"`

      - `"chat_completion"`

    - `model_vendor: "openai" or "cohere" or "vertex_ai" or 7 more`

      Vendor to serve/create model

      - `"openai"`

      - `"cohere"`

      - `"vertex_ai"`

      - `"anthropic"`

      - `"azure"`

      - `"gemini"`

      - `"model_zoo"`

      - `"bedrock"`

      - `"xai"`

      - `"fireworks_ai"`

    - `name: string`

      Unique name to reference your model

    - `model_metadata: optional map[unknown]`

    - `on_conflict: optional "error" or "update"`

      - `"error"`

      - `"update"`

### Returns

- `InferenceModel object { id, created_at, created_by_identity_type, 10 more }`

  - `id: string`

    The unique identifier of the entity.

  - `created_at: string`

    The date and time when the entity was created in ISO format.

  - `created_by_identity_type: "user" or "service_account"`

    The type of identity that created the entity.

    - `"user"`

    - `"service_account"`

  - `created_by_user_id: string`

    The user who originally created the entity.

  - `model_type: InferenceModelType`

    - `"generic"`

    - `"completion"`

    - `"chat_completion"`

  - `model_vendor: InferenceModelVendor`

    - `"openai"`

    - `"cohere"`

    - `"vertex_ai"`

    - `"anthropic"`

    - `"azure"`

    - `"gemini"`

    - `"launch"`

    - `"llmengine"`

    - `"model_zoo"`

    - `"bedrock"`

    - `"xai"`

    - `"fireworks_ai"`

  - `name: string`

  - `status: "failed" or "ready" or "deploying" or "deployment_timeout"`

    - `"failed"`

    - `"ready"`

    - `"deploying"`

    - `"deployment_timeout"`

  - `model_availability: optional InferenceModelAvailability`

    - `"unknown"`

    - `"available"`

    - `"unavailable"`

  - `model_metadata: optional map[unknown]`

  - `object: optional "model"`

    - `"model"`

  - `status_reason: optional string`

  - `vendor_configuration: optional LaunchVendorConfiguration or LlmEngineVendorConfiguration`

    - `LaunchVendorConfiguration object { model_image, model_infra }`

      - `model_image: object { command, registry, repository, 9 more }`

        - `command: array of string`

        - `registry: string`

        - `repository: string`

        - `tag: string`

        - `env_vars: optional map[unknown]`

        - `healthcheck_route: optional string`

        - `predict_route: optional string`

        - `readiness_delay: optional number`

        - `request_schema: optional map[unknown]`

        - `response_schema: optional map[unknown]`

        - `streaming_command: optional array of string`

        - `streaming_predict_route: optional string`

      - `model_infra: object { cpus, endpoint_type, gpu_type, 9 more }`

        - `cpus: optional string or number`

          - `string`

          - `number`

        - `endpoint_type: optional "async" or "sync" or "streaming"`

          - `"async"`

          - `"sync"`

          - `"streaming"`

        - `gpu_type: optional "nvidia-tesla-t4" or "nvidia-ampere-a10" or "nvidia-ampere-a100" or 4 more`

          - `"nvidia-tesla-t4"`

          - `"nvidia-ampere-a10"`

          - `"nvidia-ampere-a100"`

          - `"nvidia-ampere-a100e"`

          - `"nvidia-hopper-h100"`

          - `"nvidia-hopper-h100-1g20gb"`

          - `"nvidia-hopper-h100-3g40gb"`

        - `gpus: optional number`

        - `high_priority: optional boolean`

        - `labels: optional map[string]`

        - `max_workers: optional number`

        - `memory: optional string`

        - `min_workers: optional number`

        - `per_worker: optional number`

        - `public_inference: optional boolean`

        - `storage: optional string`

    - `LlmEngineVendorConfiguration object { model, chat_template_override, checkpoint_path, 20 more }`

      - `model: string`

      - `chat_template_override: optional string`

      - `checkpoint_path: optional string`

      - `cpus: optional number`

      - `default_callback_url: optional string`

      - `endpoint_type: optional string`

      - `gpu_type: optional string`

      - `gpus: optional number`

      - `high_priority: optional boolean`

      - `inference_framework: optional string`

      - `inference_framework_image_tag: optional string`

      - `labels: optional map[string]`

      - `max_workers: optional number`

      - `memory: optional string`

      - `min_workers: optional number`

      - `nodes_per_worker: optional number`

      - `num_shards: optional number`

      - `per_worker: optional number`

      - `post_inference_hooks: optional array of string`

      - `public_inference: optional boolean`

      - `quantize: optional string`

      - `source: optional string`

      - `storage: optional string`

### Example

```http
curl https://api.egp.scale.com/v5/models \
    -H 'Content-Type: application/json' \
    -H "x-api-key: $SGP_API_KEY" \
    -d '{
          "name": "name",
          "vendor_configuration": {
            "model_image": {
              "command": [
                "string"
              ],
              "registry": "registry",
              "repository": "repository",
              "tag": "tag",
              "env_vars": {
                "foo": "bar"
              },
              "healthcheck_route": "healthcheck_route",
              "predict_route": "predict_route",
              "readiness_delay": 0,
              "request_schema": {
                "foo": "bar"
              },
              "response_schema": {
                "foo": "bar"
              },
              "streaming_command": [
                "string"
              ],
              "streaming_predict_route": "streaming_predict_route"
            },
            "model_infra": {
              "cpus": "string",
              "endpoint_type": "async",
              "gpu_type": "nvidia-tesla-t4",
              "gpus": 0,
              "high_priority": true,
              "labels": {
                "foo": "string"
              },
              "max_workers": 0,
              "memory": "memory",
              "min_workers": 0,
              "per_worker": 0,
              "public_inference": true,
              "storage": "storage"
            }
          },
          "model_metadata": {
            "foo": "bar"
          },
          "model_type": "generic",
          "model_vendor": "launch",
          "on_conflict": "error"
        }'
```

#### Response

```json
{
  "id": "id",
  "created_at": "2019-12-27T18:11:19.117Z",
  "created_by_identity_type": "user",
  "created_by_user_id": "created_by_user_id",
  "model_type": "generic",
  "model_vendor": "openai",
  "name": "name",
  "status": "failed",
  "model_availability": "unknown",
  "model_metadata": {
    "foo": "bar"
  },
  "object": "model",
  "status_reason": "status_reason",
  "vendor_configuration": {
    "model_image": {
      "command": [
        "string"
      ],
      "registry": "registry",
      "repository": "repository",
      "tag": "tag",
      "env_vars": {
        "foo": "bar"
      },
      "healthcheck_route": "healthcheck_route",
      "predict_route": "predict_route",
      "readiness_delay": 0,
      "request_schema": {
        "foo": "bar"
      },
      "response_schema": {
        "foo": "bar"
      },
      "streaming_command": [
        "string"
      ],
      "streaming_predict_route": "streaming_predict_route"
    },
    "model_infra": {
      "cpus": "string",
      "endpoint_type": "async",
      "gpu_type": "nvidia-tesla-t4",
      "gpus": 0,
      "high_priority": true,
      "labels": {
        "foo": "string"
      },
      "max_workers": 0,
      "memory": "memory",
      "min_workers": 0,
      "per_worker": 0,
      "public_inference": true,
      "storage": "storage"
    }
  }
}
```