Run inference with free-form payload

client.Inference.New(ctx, body) (*InferenceNewResponseUnion, error)

POST/v5/inference

Runs a model using a free-form, vendor-native request payload rather than a fixed OpenAI schema.

Use this endpoint when the target model does not fit the OpenAI chat, responses, or text-completion contracts: the args field is an arbitrary dict passed straight through to the selected vendor gateway, and the reply is returned inside response as arbitrary JSON (object, array, string, number, or boolean). Prefer /v5/chat/completions, /v5/responses, or /v5/completions when your request matches one of those OpenAI-standard shapes, since those return typed OpenAI response objects. The model is chosen from the model field formatted as vendor/name, which selects the per-vendor inference gateway. When the request enables streaming, the response is sent as server-sent events with each chunk wrapped as a generic_inference.chunk object; otherwise a single generic_inference object is returned.

ParametersExpand Collapse

body InferenceNewParams

Model param.Field[string]

model specified as vendor/name (ex. openai/gpt-5)

Args param.Field[map[string, any]]Optional

Arguments passed into model

InferenceConfiguration param.Field[LaunchInferenceConfiguration]Optional

Vendor specific configuration

ReturnsExpand Collapse

type InferenceNewResponseUnion interface{…}

One of the following:

type InferenceResponse struct{…}

Response InferenceResponseResponseUnion

One of the following:

type InferenceResponseResponseMap map[string, any]

type InferenceResponseResponseArray []any

string

float64

bool

Object InferenceResponseObjectOptional

type InferenceResponseChunk struct{…}

Response InferenceResponseChunkResponseUnion

One of the following:

type InferenceResponseChunkResponseMap map[string, any]

type InferenceResponseChunkResponseArray []any

string

float64

bool

Object InferenceResponseChunkObjectOptional

Run inference with free-form payload

package main

import (
  "context"
  "fmt"

  "github.com/scaleapi/sgp-dev-go"
  "github.com/scaleapi/sgp-dev-go/option"
)

func main() {
  client := sgpdev.NewClient(
    option.WithAPIKey("My API Key"),
    option.WithAccountID("My Account ID"),
  )
  inference, err := client.Inference.New(context.TODO(), sgpdev.InferenceNewParams{
    Model: "model",
  })
  if err != nil {
    panic(err.Error())
  }
  fmt.Printf("%+v\n", inference)
}

{
  "response": {
    "foo": "bar"
  },
  "object": "generic_inference"
}

Returns Examples

{
  "response": {
    "foo": "bar"
  },
  "object": "generic_inference"
}