Model Endpoints¶

All classes here are returned by the get_model_endpoint method and provide a predict function.

AsyncEndpoint ¶

AsyncEndpoint(model_endpoint: ModelEndpoint, client: ModelEndpoint)

Bases: Endpoint

An asynchronous model endpoint.

Parameters:

Name	Type	Description	Default
`model_endpoint`	`ModelEndpoint`	ModelEndpoint object.	required
`client`		A LaunchClient object	required

predict(request: EndpointRequest) -> EndpointResponseFuture

Runs an asynchronous prediction request.

Parameters:

Name	Type	Description	Default
`request`	`EndpointRequest`	The `EndpointRequest` object that contains the payload.	required

Returns:

Name	Type	Description
	`EndpointResponseFuture`	An `EndpointResponseFuture` such the user can use to query the status of the request.
`Example`	`EndpointResponseFuture`
	`EndpointResponseFuture`	.. code-block:: python my_endpoint = AsyncEndpoint(...) f: EndpointResponseFuture = my_endpoint.predict(EndpointRequest(...)) result = f.get() # blocks on completion

predict_batch(requests: Sequence[EndpointRequest]) -> AsyncEndpointBatchResponse

(deprecated) Runs inference on the data items specified by urls. Returns a AsyncEndpointResponse.

Parameters:

Name	Type	Description	Default
`requests`	`Sequence[EndpointRequest]`	List of EndpointRequests. Request_ids must all be distinct.	required

Returns:

Type	Description
`AsyncEndpointBatchResponse`	an AsyncEndpointResponse keeping track of the inference requests made

SyncEndpoint(model_endpoint: ModelEndpoint, client: ModelEndpoint)

Bases: Endpoint

A synchronous model endpoint.

Parameters:

Name	Type	Description	Default
`model_endpoint`	`ModelEndpoint`	ModelEndpoint object.	required
`client`		A LaunchClient object	required

predict(request: EndpointRequest) -> EndpointResponse

Runs a synchronous prediction request.

Parameters:

Name	Type	Description	Default
`request`	`EndpointRequest`	The `EndpointRequest` object that contains the payload.	required

StreamingEndpoint(model_endpoint: ModelEndpoint, client: ModelEndpoint)

Bases: Endpoint

A synchronous model endpoint.

Parameters:

Name	Type	Description	Default
`model_endpoint`	`ModelEndpoint`	ModelEndpoint object.	required
`client`		A LaunchClient object	required

predict(request: EndpointRequest) -> EndpointResponseStream

Runs a streaming prediction request.

Parameters:

Name	Type	Description	Default
`request`	`EndpointRequest`	The `EndpointRequest` object that contains the payload.	required

Returns:

Type	Description
`EndpointResponseStream`	An `EndpointResponseStream` object that can be used to iterate through the stream.