API Reference

Copy Markdown

Open in Claude

Open in ChatGPT

Open in Cursor

---

Copy Markdown

View as Markdown

Inference

Run inference with free-form payload

client.inference.create(InferenceCreateParams { model, args, inference_configuration } body, RequestOptionsoptions?): InferenceCreateResponse

POST/v5/inference

ModelsExpand Collapse

InferenceResponse { response, object }

response: Record<string, unknown> | Array<unknown> | string | 2 more | null

One of the following:

Record<string, unknown>

Array<unknown>

string

number

boolean

object?: "generic_inference"

InferenceResponseChunk { response, object }

response: Record<string, unknown> | Array<unknown> | string | 2 more | null

One of the following:

Record<string, unknown>

Array<unknown>

string

number

boolean

object?: "generic_inference.chunk"

LaunchInferenceConfiguration { num_retries, timeout_seconds }

num_retries?: number

timeout_seconds?: number

InferenceCreateResponse = InferenceResponse { response, object } | InferenceResponseChunk { response, object }

One of the following:

InferenceResponse { response, object }

response: Record<string, unknown> | Array<unknown> | string | 2 more | null

One of the following:

Record<string, unknown>

Array<unknown>

string

number

boolean

object?: "generic_inference"

InferenceResponseChunk { response, object }

response: Record<string, unknown> | Array<unknown> | string | 2 more | null

One of the following:

Record<string, unknown>

Array<unknown>

string

number

boolean

object?: "generic_inference.chunk"