🐍 Python Client Data Type Reference¶

Core Data Types¶

Classes that represent the main entities of SGP.

KnowledgeBase ¶

A data model representing a knowledge base.

Attributes:

Name	Type	Description
`knowledge_base_id`	`str`	The unique ID of the knowledge base
`knowledge_base_name`	`str`	The name of the knowledge base
`embedding_config`	`EmbeddingConfig`	The embedding configuration
`metadata`	`Optional[Dict[str, Any]]`	Metadata associated with the knowledge base
`created_at`	`str`	The timestamp at which the knowledge base was created
`updated_at`	`Optional[str]`	The timestamp at which the knowledge base was last updated

KnowledgeBaseUpload ¶

A data model representing a knowledge base upload.

Attributes:

Name	Type	Description
`upload_id`	`str`	Unique ID of the upload job
`data_source_config`	`DataSourceConfig`	Configuration for downloading data from source
`chunking_strategy_config`	`Optional[ChunkingStrategyConfig]`	Configuration for chunking the text content of each artifact
`created_at`	`str`	The timestamp at which the upload job started
`updated_at`	`str`	The timestamp at which the upload job was last updated
`status`	`UploadJobStatus`	Sync status
`status_reason`	`Optional[str]`	Reason for the upload job's status
`artifacts_status`	`Optional[ArtifactsStatus]`	Number of artifacts pending, completed, and failed
`artifacts`	`Optional[List[KnowledgeBaseArtifact]]`	List of info for each artifacts

KnowledgeBaseArtifact ¶

A data model representing an artifact in a knowledge base.

Attributes:

Name	Type	Description
`artifact_id`	`str`	Unique identifier for the artifact
`artifact_name`	`str`	Friendly name for the artifact
`artifact_uri`	`str`	Location (e.g. URI) of the artifact in the data source
`artifact_uri_public`	`Optional[str]`	Public Location (e.g. URI) of the artifact in the data source
`status`	`str`	Status of the artifact
`status_reason`	`Optional[str]`	Reason for the artifact's status
`source`	`ArtifactSource`	Data source of the artifact
`chunks_status`	`ChunksStatus`	Number of chunks pending, completed, and failed
`updated_at`	`Optional[datetime]`	Timestamp at which the artifact was last updated
`chunks`	`Optional[List[Chunk]]`	List of chunks associated with the artifact

Chunk ¶

A data model representing a chunk.

Attributes:

Name	Type	Description
`chunk_id`	`str`	The unique ID of the chunk
`text`	`str`	The text associated with the chunk
`score`	`Optional[float]`	A number between 0 and 1 representing how similar a chunk's embedding is to the query embedding. Higher numbers mean that this chunk is more similar.
`embedding`	`Optional[List[float]]`	The vector embedding of the text associated with the chunk
`metadata`	`Optional[Dict[str, Any]]`	Any additional key value pairs of information stored with the chunk

ModelTemplate ¶

This is a template for types of models that can be quickly customized by end users. It allows users to upload static docker images that can run specific types of models. These docker images will expose parameters that can be injected at ModelAlias creation time to customize the functionality. A common example of this is to use a HuggingFace LLM template, but swap out model weights for a finetuned model.

Attributes:

Name	Type	Description
`id`	`str`	The unique identifier of the entity.
`created_at`	`datetime`	The date and time when the entity was created in ISO format.
`account_id`	`str`	The ID of the account that owns the given entity.
`created_by_user_id`	`str`	The user who originally created the entity.
`name`	`str`	The name of the model template
`endpoint_type`	`ModelEndpointType`	The type of endpoint that the model template will create
`model_type`	`ModelType`	The type of model that the model template will create
`vendor_configuration`	`ModelVendorConfiguration`	The vendor configuration of the model template
`model_creation_parameters_schema`	`Optional[ParameterSchema]`	The field names and types of available parameter fields which may be specified during model creation
`model_request_parameters_schema`	`Optional[ParameterSchema]`	The field names and types of available parameter fields which may be specified in a model execution API's `model_request_parameters` field.

ApplicationSpec ¶

A data model representing an Application Spec.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the application spec
`name`	`str`	The name of the application
`description`	`str`	The description of the application

Question ¶

QuestionSet ¶

A data model representing a question set.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the question set
`name`	`str`	The name of the question set
`created_at`	`str`	The time the question set was created
`created_by_user_id`	`str`	The user ID of the user who created the question set
`account_id`	`str`	The account ID of the account that the question set is associated with
`questions`	`Optional[List[Question]]`	The questions in the question set

EvaluationConfig ¶

A data model representing an evaluation config.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the evaluation config
`evaluation_type`	`EvaluationType`	The type of evaluation. Currently, only `HUMAN` is supported.
`question_set_id`	`str`	The ID of the question set to use for the evaluation.
`account_id`	`str`	The account ID of the account that the evaluation config is associated with.
`created_at`	`datetime`	The time the evaluation config was created.
`created_by_user_id`	`str`	The user ID of the user who created the evaluation config.

EvaluationDataset ¶

A data model representing an evaluation dataset.

Attributes:

Name	Type	Description
`name`	`str`	The name of the evaluation dataset
`schema_type`	`TestCaseSchemaType`	The schema type of the evaluation dataset
`id`	`str`	The ID of the evaluation dataset
`created_at`	`datetime`	The time the evaluation dataset was created
`updated_at`	`datetime`	The time the evaluation dataset was last updated
`account_id`	`str`	The ID of the account that owns the evaluation dataset
`created_by_user_id`	`str`	The ID of the user that created the evaluation dataset
`version_num`	`Optional[int]`	The version number of the evaluation dataset

TestCase ¶

A data model representing a test case.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the test case
`evaluation_dataset_id`	`str`	The ID of the evaluation dataset that the test case belongs to
`schema_type`	`TestCaseSchemaType`	The schema type of the test case
`test_case_data`	`Union[GenerationTestCaseData]`	The data of the test case. In general, this can be thought of as the data to evaluate an application against.

Evaluation ¶

A data model representing an evaluation.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the evaluation
`name`	`str`	The name of the evaluation
`description`	`str`	The description of the evaluation
`status`	`EvaluationStatus`	The status of the evaluation
`application_spec_id`	`str`	The ID of the application spec that the evaluation is for
`evaluation_config_id`	`str`	The ID of the evaluation config
`tags`	`Optional[Dict[str, Any]]`	The tags of the evaluation represented as a dictionary of key value pairs
`created_at`	`datetime`	The time the evaluation was created

TestCaseResult ¶

A data model representing a test case result.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the test case result
`status`		The status of the test case result
`application_spec_id`	`str`	The ID of the application spec that the test case result is for
`evaluation_id`	`str`	The ID of the evaluation that the test case result is for
`evaluation_dataset_id`	`str`	The ID of the evaluation dataset that the test case result is for
`evaluation_dataset_version_num`	`str`	The version number of the evaluation dataset that the test case result is for
`test_case_id`	`str`	The ID of the test case that the test case result is for
`test_case_evaluation_data`	`GenerationTestCaseResultData`	A payload representing the data generated by the application described by the application spec when evaluated against the test case.
`test_case_evaluation_data_schema`	`TestCaseSchemaType`	The schema type of the `test_case_evaluation_data`
`result`	`Optional[Dict[str, Any]]`	The payload filled in when the evaluation of this test case result is completed. Examine this value to determine how the application performed on the test case.
`completed_at`	`Optional[datetime]`	The time the test case result was completed
`created_at`	`datetime`	The time the test case result was created
`annotated_by_user_id`	`Optional[str]`	The ID of the user that annotated the test case result
`time_spent_labeling_s`	`Optional[int]`	The time spent labeling the test case result

Enums¶

Classes that can be used to access enumerated values. It is generally recommended to use these enums instead of hard-coding strings in your code.

AgentAction ¶

An enum representing the different types of agent actions supported.

Attributes:

Name	Type	Description
`TOOL_REQUEST`		Denotes that the agent output contains a request for the user to use a tool.
`CONTENT`		Denotes that the agent output contains final content.

ArtifactSource ¶

An enum representing the different types of artifact sources supported.

Attributes:

Name	Type	Description
`S3`		Denotes that the artifact source is S3.
`CONFLUENCE`		Denotes that the artifact source is Confluence.
`SHAREPOINT`		Denotes that the artifact source is SharePoint.
`GOOGLE_DRIVE`		Denotes that the artifact source is Google Drive.
`LOCAL_CHUNKS`		Denotes that the artifact source comes from chunks directly uploaded via the API.

AuditStatus ¶

An enum representing the different possible audit statuses of a test case result.

Attributes:

Name	Type	Description
`UNAUDITED`		Denotes that the test case result has not been aduited yet.
`FIXED`		Denotes the test case result has been audited, and the mistakes have been corrected.
`APPROVED`		Denotes the test case has been audited and approved.

ChunkUploadStatus ¶

An enum representing the different types of chunk upload statuses supported.

Attributes:

Name	Type	Description
`PENDING`		Denotes that the chunk upload is pending.
`COMPLETED`		Denotes that the chunk upload is completed.
`FAILED`		Denotes that the chunk upload has failed.

ChunkingStrategy ¶

An enum representing the different types of chunking strategies supported.

Attributes:

Name	Type	Description
`CHARACTER`		Denotes that the chunking strategy is to chunk by character.

CrossEncoderModelName ¶

An enum representing the different types of cross encoder models supported.

Attributes:

Name	Type	Description
`CROSS_ENCODER_MS_MARCO_MINILM_L12_V2`		Denotes that the model is a cross encoder model.
`CROSS_ENCODER_MMARCO_MMINILMV2_L12_H384_V1`		Denotes that the model is a cross encoder model.

DataSource ¶

An enum representing the different types of data sources supported.

Attributes:

Name	Type	Description
`S3`	`str`	Denotes that the data source is S3.

DeduplicationStrategy ¶

An enum representing the different types of deduplication strategies supported.

Attributes:

Name	Type	Description
`OVERWRITE`		Denotes that the deduplication strategy is to overwrite.
`FAIL`		Denotes that the deduplication strategy is to fail.

EmbeddingModelName ¶

An enum representing the different types of embedding models supported.

Attributes:

Name	Type	Description
`SENTENCE_TRANSFORMERS_ALL_MINILM_L12_V2`		Denotes that the model is a sentence transformer model.
`SENTENCE_TRANSFORMERS_MULTI_QA_DISTILBERT_COS_V1`		Denotes that the model is a sentence transformer model.
`SENTENCE_TRANSFORMERS_PARAPHRASE_MULTILINGUAL_MPBET_BASE_V2`		Denotes that the model is a sentence transformer model.
`OPENAI_TEXT_EMBEDDING_ADA_002`		Denotes that the model is an openai text embedding model.
`OPENAI_TEXT_EMBEDDING_3_SMALL`		Denotes that the model is an openai text embedding model.
`OPENAI_TEXT_EMBEDDING_3_LARGE`		Denotes that the model is an openai text embedding model.
`COHERE_TEXT_EMBEDDING_ENGLISH_3`		Denotes that the model is a cohere text embedding model.
`COHERE_TEXT_EMBEDDING_ENGLISH_LIGHT_3`		Denotes that the model is a cohere text embedding model.
`COHERE_TEXT_EMBEDDING_MULTILINGUAL_3`		Denotes that the model is a cohere text embedding model.

EvaluationStatus ¶

An enum representing the different possible statuses of an Evaluation.

Attributes:

Name	Type	Description
`PENDING`	`str`	Denotes that an evaluation is pending.
`COMPLETED`	`str`	Denotes that an evaluation is completed.
`FAILED`	`str`	Denotes that an evaluation has failed.

EvaluationType ¶

An enum representing the different types of evaluations.

Currently only human evaluations are supported.

Attributes:

Name	Type	Description
`HUMAN`		Denotes that an evaluation is a human evaluation.

ExtraInfoSchemaType ¶

An enum representing the different types of extra info schemas.

Denotes the type of the "info" field in the ExtraInfo model.

Attributes:

Name	Type	Description
`STRING`	`str`	Denotes that the "info" field is a string.

QuestionType ¶

An enum representing the different types of questions.

This is used to specify the type of a question.

Attributes:

Name	Type	Description
`CATEGORICAL`	`str`	Denotes that a question is a categorical question.
`FREE_TEXT`	`str`	Denotes that a question is a free text question.

TestCaseSchemaType ¶

An enum representing the different test case schema types.

Attributes:

Name	Type	Description
`GENERATION`	`str`	If a TestCase specifies this schema type, it must have the following fields: `input` [Required] The input to the generation model. `expected_output` [Optional] The expected output of the generation model. `expected_extra_info` [Optional] The expected extra info of the generation model. If a TestCaseResult specifies this schema type, it must have the following fields: `generation_output` [Required] The output of the generation model. `generation_extra_info` [Required] The extra info of the generation model.

UploadJobStatus ¶

An enum representing the different types of upload job statuses supported.

Attributes:

Name	Type	Description
`RUNNING`		Denotes that the upload job is running.
`COMPLETED`		Denotes that the upload job is completed.
`FAILED`		Denotes that the upload job has failed.
`CANCELED`		Denotes that the upload job has been canceled.

Glossary¶

A glossary of all data types used in the SDK.

agents ¶

ActionContext ¶

The context of the action that the agent is taking.

Attributes:

Name	Type	Description
`content`	`Optional[str]`	The content of the final output of the agent when it no longer needs any tools.
`tool_request`	`Optional[ToolRequest]`	The tool request if the agent needs more information.

Tool ¶

A tool is a function that the Agent has at its disposal. This schema is used to tell the Agent about the tools that are available to it.

Attributes:

Name	Type	Description
`name`	`str`	Name of the tool. The tool name is the name the client wishes the Agent to use to refer to this function when it decides if it wants the user to use the tool or not. It must be unique amongst the set of tools provided in a single API call.
`description`	`str`	Description of the tool. Because some queries are complex and may require multiple tools to complete, it is important to make these descriptions as informative as possible. If a tool is not being chosen when it should, it is common practice to tune the description of the tool to make it more apparent to the agent when the tool can be used effectively.
`arguments`	`ToolArguments`	An JSON Schema-compliant schema for the tool arguments. To describe a function that accepts no parameters, provide the value `{"type": "object", "properties": {}}`. For more information on how to define a valid JSON Schema object, visit https://json-schema.org/understanding-json-schema/reference/object.html

ToolArguments ¶

An object where each key is the name of a keyword argument and each value is a schema used to validate that property. Each schema must have a type and description, but can also have a default value and examples.

For more information on how to define a valid property, visit https://json-schema.org/understanding-json-schema/reference/object.html

Attributes:

Name	Type	Description
`type`	`Literal['object']`	Type of argument. Currently only "object" is supported
`properties`	`Dict[str, ToolPropertyValue]`	An object where each key is the name of a keyword argument and each value is a schema used to validate that property. Each schema must have a type and description, but can also have a default value and examples. For more information on how to define a valid property, visit https://json-schema.org/understanding-json-schema/reference/object.html

ToolPropertyValue ¶

A schema used to validate a single keyword argument for a tool.

Attributes:

Name	Type	Description
`type`	`Literal['string', 'number', 'integer', 'boolean', 'object', 'array', 'null']`	The argument's type. The type is used to help the Agent generate valid arguments for the tool. For more information about types, see: https://json-schema.org/understanding-json-schema/reference/type.html#type-specific-keywords
`description`	`str`	Description of what the argument is used for. This description is used to help the Agent generate sensible arguments for the tool. It is very important that this description is succinct, clear, and accurate.
`default`	`Optional[str]`	A default value for the argument if unspecified.
`examples`	`Optional[List[str]]`	Example values demonstrating how the argument should look. This can be used to help the agent understand what a valid argument should look like.

ToolRequest ¶

A request to run a tool.

Attributes:

Name	Type	Description
`name`	`str`	Name of the tool that the AI wants the client to use.
`arguments`	`str`	Arguments to pass to the tool. The format must be a JSON Schema-compliant object serialized into a string.

application_specs ¶

ApplicationSpec ¶

A data model representing an Application Spec.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the application spec
`name`	`str`	The name of the application
`description`	`str`	The description of the application

chunks ¶

Chunk ¶

A data model representing a chunk.

Attributes:

Name	Type	Description
`chunk_id`	`str`	The unique ID of the chunk
`text`	`str`	The text associated with the chunk
`score`	`Optional[float]`	A number between 0 and 1 representing how similar a chunk's embedding is to the query embedding. Higher numbers mean that this chunk is more similar.
`embedding`	`Optional[List[float]]`	The vector embedding of the text associated with the chunk
`metadata`	`Optional[Dict[str, Any]]`	Any additional key value pairs of information stored with the chunk

CrossEncoderRankParams ¶

A data model representing the parameters needed for cross encoder ranking.

Attributes:

Name	Type	Description
`cross_encoder_model`	`Literal['cross-encoder/ms-marco-MiniLM-L-12-v2', 'cross-encoder/mmarco-mMiniLMv2-L12-H384-v1']`	The cross encoder model to use for ranking.

CrossEncoderRankStrategy ¶

A data model representing the cross encoder ranking strategy.

Attributes:

Name	Type	Description
`method`	`Literal['cross_encoder']`	The name of the rank strategy. Must be `cross_encoder`.
`params`	`CrossEncoderRankParams`	The parameters needed for ranking.

ModelRankParams ¶

A data model representing the parameters needed for ranking.

Attributes:

Name	Type	Description
`model_id`	`Optional[str]`	The ID of the model to use for ranking.
`base_model_name`	`Optional[str]`	The name of the base model to be used
`model_params`	`Optional[Dict[str, Any]]`	The parameters needed for the model.

ModelRankStrategy ¶

A data model representing the model ranking strategy.

Attributes:

Name	Type	Description
`method`	`Literal['model']`	The name of the rank strategy. Must be `model`.
`params`	`ModelRankParams`	The parameters needed for ranking.

RougeRankParams ¶

A data model representing the parameters needed for Rouge ranking.

Attributes:

Name	Type	Description
`metric`	`str`	Rouge type, can be n-gram based (e.g. rouge1, rouge2) or longest common subsequence (rougeL or rougeLsum)
`score`	`Literal['precision', 'recall', 'fmeasure']`	Metric to use from Rouge score

RougeRankStrategy ¶

A data model representing the Rouge ranking strategy.

Attributes:

Name	Type	Description
`method`	`Literal['rouge']`	The name of the rank strategy. Must be `rouge`.
`params`	`RougeRankParams`	The parameters needed for ranking.

completions ¶

Completion ¶

A data model representing a completion.

Attributes:

Name	Type	Description
`completion`	`CompletionContent`	The actual completion text and the finish reason.
`token_usage`	`Optional[TokenUsage]`	Token usage numbers. If streaming, this field is null until the stream completes, at which point it will be populated (if supported).

CompletionContent ¶

A data model representing the completion text and the finish reason.

Attributes:

Name	Type	Description
`text`	`str`	Completion text. If streaming, this field will contain each packet of text.
`finish_reason`	`Optional[str]`	Reason the LLM finished generating text.

ModelParameters ¶

A data model representing the configuration parameters for the completion model.

Attributes:

Name Type Description

temperature

float

What sampling temperature to use, between [0, 1]. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Setting temperature=0.0 will enable fully deterministic (greedy) sampling.

stop_sequences

Optional[List[str]]

List of up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.

max_tokens

Optional[int]

The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. If not, specified, max_tokens will be determined based on the model used:

Model API family	Model API default	SGP applied default
OpenAI Completions	`16`	`context window - prompt size`
OpenAI Chat Completions	`context window - prompt size`	`context window - prompt size`

TokenUsage ¶

A data model representing LLM token usage numbers.

Attributes:

Name	Type	Description
`prompt`	`Optional[int]`	Number of tokens in the prompt.
`completion`	`Optional[int]`	Number of tokens in the completion.
`total`	`int`	Total number of tokens in both the prompt and the completion.

embeddings ¶

evaluation_configs ¶

EvaluationConfig ¶

A data model representing an evaluation config.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the evaluation config
`evaluation_type`	`EvaluationType`	The type of evaluation. Currently, only `HUMAN` is supported.
`question_set_id`	`str`	The ID of the question set to use for the evaluation.
`account_id`	`str`	The account ID of the account that the evaluation config is associated with.
`created_at`	`datetime`	The time the evaluation config was created.
`created_by_user_id`	`str`	The user ID of the user who created the evaluation config.

evaluation_dataset_test_cases ¶

GenerationTestCaseData ¶

A data model representing the data of a Testcase with a GENERATION schema type.

Attributes:

Name	Type	Description
`input`	`str`	The input to the generation model
`expected_output`	`Optional[str]`	The expected output of the generation model
`expected_extra_info`	`Optional[ExtraInfo]`	The expected extra info of the generation model

TestCase ¶

A data model representing a test case.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the test case
`evaluation_dataset_id`	`str`	The ID of the evaluation dataset that the test case belongs to
`schema_type`	`TestCaseSchemaType`	The schema type of the test case
`test_case_data`	`Union[GenerationTestCaseData]`	The data of the test case. In general, this can be thought of as the data to evaluate an application against.

evaluation_datasets ¶

EvaluationDataset ¶

A data model representing an evaluation dataset.

Attributes:

Name	Type	Description
`name`	`str`	The name of the evaluation dataset
`schema_type`	`TestCaseSchemaType`	The schema type of the evaluation dataset
`id`	`str`	The ID of the evaluation dataset
`created_at`	`datetime`	The time the evaluation dataset was created
`updated_at`	`datetime`	The time the evaluation dataset was last updated
`account_id`	`str`	The ID of the account that owns the evaluation dataset
`created_by_user_id`	`str`	The ID of the user that created the evaluation dataset
`version_num`	`Optional[int]`	The version number of the evaluation dataset

evaluation_test_case_results ¶

GenerationTestCaseResultData ¶

A data model representing the data of a TestcaseResult with a GENERATION schema type.

Attributes:

Name	Type	Description
`generation_output`	`str`	The output of the generation model
`generation_extra_info`	`ExtraInfo`	The extra info of the generation model

TestCaseResult ¶

A data model representing a test case result.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the test case result
`status`		The status of the test case result
`application_spec_id`	`str`	The ID of the application spec that the test case result is for
`evaluation_id`	`str`	The ID of the evaluation that the test case result is for
`evaluation_dataset_id`	`str`	The ID of the evaluation dataset that the test case result is for
`evaluation_dataset_version_num`	`str`	The version number of the evaluation dataset that the test case result is for
`test_case_id`	`str`	The ID of the test case that the test case result is for
`test_case_evaluation_data`	`GenerationTestCaseResultData`	A payload representing the data generated by the application described by the application spec when evaluated against the test case.
`test_case_evaluation_data_schema`	`TestCaseSchemaType`	The schema type of the `test_case_evaluation_data`
`result`	`Optional[Dict[str, Any]]`	The payload filled in when the evaluation of this test case result is completed. Examine this value to determine how the application performed on the test case.
`completed_at`	`Optional[datetime]`	The time the test case result was completed
`created_at`	`datetime`	The time the test case result was created
`annotated_by_user_id`	`Optional[str]`	The ID of the user that annotated the test case result
`time_spent_labeling_s`	`Optional[int]`	The time spent labeling the test case result

evaluations ¶

Evaluation ¶

A data model representing an evaluation.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the evaluation
`name`	`str`	The name of the evaluation
`description`	`str`	The description of the evaluation
`status`	`EvaluationStatus`	The status of the evaluation
`application_spec_id`	`str`	The ID of the application spec that the evaluation is for
`evaluation_config_id`	`str`	The ID of the evaluation config
`tags`	`Optional[Dict[str, Any]]`	The tags of the evaluation represented as a dictionary of key value pairs
`created_at`	`datetime`	The time the evaluation was created

fine_tuning ¶

knowledge_base_artifacts ¶

ChunksStatus ¶

A data model representing the status of the chunks in an artifact.

Attributes:

Name	Type	Description
`chunks_completed`	`int`	Number of chunks uploaded successfully
`chunks_pending`	`int`	Number of chunks awaiting upload
`chunks_failed`	`int`	Number of chunks that failed upload

KnowledgeBaseArtifact ¶

A data model representing an artifact in a knowledge base.

Attributes:

Name	Type	Description
`artifact_id`	`str`	Unique identifier for the artifact
`artifact_name`	`str`	Friendly name for the artifact
`artifact_uri`	`str`	Location (e.g. URI) of the artifact in the data source
`artifact_uri_public`	`Optional[str]`	Public Location (e.g. URI) of the artifact in the data source
`status`	`str`	Status of the artifact
`status_reason`	`Optional[str]`	Reason for the artifact's status
`source`	`ArtifactSource`	Data source of the artifact
`chunks_status`	`ChunksStatus`	Number of chunks pending, completed, and failed
`updated_at`	`Optional[datetime]`	Timestamp at which the artifact was last updated
`chunks`	`Optional[List[Chunk]]`	List of chunks associated with the artifact

knowledge_base_chunks ¶

knowledge_base_uploads ¶

ArtifactsStatus ¶

A data model representing the status of the artifacts in a knowledge base.

Attributes:

Name	Type	Description
`artifacts_completed`	`int`	Number of artifacts uploaded successfully.
`artifacts_pending`	`int`	Number of artifacts awaiting upload.
`artifacts_uploading`	`int`	Number of artifacts with upload in progress.
`artifacts_failed`	`int`	Number of artifacts that failed upload.

AzureBlobStorageDataSourceAuthConfig ¶

A data model representing the configuration of an Azure Blob Storage data source.

Attributes:

Name	Type	Description
`source`	`Literal[AZURE_BLOB_STORAGE]`	The data source type. Must be 'azure_blob_storage'.
`blob_sas_token`	`str`	The SAS token for the Azure Blob Storage container

AzureBlobStorageDataSourceConfig ¶

A data model representing the configuration of an Azure Blob Storage data source.

Attributes:

Name	Type	Description
`source`	`Literal[AZURE_BLOB_STORAGE]`	The data source type. Must be 'azure_blob_storage'.
`container_url`	`str`	The SAS URL for the Azure Blob Storage container (a.k.a. Blob SAS URL)

CharacterChunkingStrategyConfig ¶

A data model representing the configuration of a character chunking strategy.

Attributes:

Name	Type	Description
`strategy`	`Literal[CHARACTER]`	The chunking strategy type. Must be 'character'.
`separator`	`Optional[str]`	Character designating breaks in input data. Text data will first be split into sections by this separator, then each section will be split into chunks of size `chunk_size`.
`chunk_size`	`Optional[int]`	Maximum number of characters in each chunk. If not specified, a chunk size of 1000 will be used.
`chunk_overlap`	`Optional[int]`	Number of characters to overlap between chunks. If not specified, an overlap of 200 will be used. For example if the chunk size is 3 and the overlap size is 1, and the text to chunk is 'abcde', the chunks will be 'abc', 'cde'.

ChunkToUpload ¶

A data model representing a local chunk.

Attributes:

Name	Type	Description
`text`	`str`	The text associated with the chunk
`chunk_position`	`int`	The position of the chunk in the artifact
`metadata`	`Dict[str, Any]`	Any additional key value pairs of information stored with the chunk

ChunkingStrategyConfig ¶

A type alias for a Union of all chunking strategy types.

Attributes:

Name	Type	Description
`__root__`	`Union[CharacterChunkingStrategyConfig, TokenChunkingStrategyConfig]`	Instead of directly using this class, please use the appropriate chunking strategy type for your use case.

DataSourceConfig ¶

A type alias for a Union of all data source types.

Attributes:

Name	Type	Description
`__root__`	`Union[S3DataSourceConfig, SharePointDataSourceConfig, GoogleDriveDataSourceConfig, AzureBlobStorageDataSourceConfig, LocalChunksSourceConfig]`	Instead of directly using this class, please use the appropriate data source type for your use case.

GoogleDriveDataSourceAuthConfig ¶

A data model representing the configuration of a Google Drive service account.

Attributes:

Name	Type	Description
`source`	`Literal[GOOGLE_DRIVE]`	The data source type. Must be 'GoogleDrive'.
`client_email`	`str`	The service account's client email
`private_key`	`str`	The service account's private_key
`token_uri`	`str`	The service account's token_uri
`client_id`	`str`	The service account's client_id

GoogleDriveDataSourceConfig ¶

A data model representing the configuration of a Google Drive data source.

Attributes:

Name	Type	Description
`source`	`Literal[GOOGLE_DRIVE]`	The data source type. Must be 'google_drive'.
`drive_id`	`str`	The ID of the Google Drive to retrieve contents from

KnowledgeBaseUpload ¶

A data model representing a knowledge base upload.

Attributes:

Name	Type	Description
`upload_id`	`str`	Unique ID of the upload job
`data_source_config`	`DataSourceConfig`	Configuration for downloading data from source
`chunking_strategy_config`	`Optional[ChunkingStrategyConfig]`	Configuration for chunking the text content of each artifact
`created_at`	`str`	The timestamp at which the upload job started
`updated_at`	`str`	The timestamp at which the upload job was last updated
`status`	`UploadJobStatus`	Sync status
`status_reason`	`Optional[str]`	Reason for the upload job's status
`artifacts_status`	`Optional[ArtifactsStatus]`	Number of artifacts pending, completed, and failed
`artifacts`	`Optional[List[KnowledgeBaseArtifact]]`	List of info for each artifacts

LocalChunksSourceConfig ¶

A data model representing the configuration of a local chunks data source.

Attributes:

Name	Type	Description
`source`	`Literal[LOCAL_CHUNKS]`	The data source type. Must be 'local_chunks'.
`artifact_name`	`str`	The file name assigned to the artifact, containing a file extension. Adding an extension is mandatory, to allow detecting file types for text extraction.
`artifact_uri`	`str`	A unique identifier for an artifact within the knowledge base, such as full path in a directory or file system.
`deduplication_strategy`	`Optional[DeduplicationStrategy]`	Action to take if an artifact with the same name already exists in the knowledge base. Can be either Overwrite (default) or Fail.

RemoteDataSourceConfig ¶

A type alias for a Union of all remote data source types.

Attributes:

Name	Type	Description
`__root__`	`Union[S3DataSourceConfig, SharePointDataSourceConfig, GoogleDriveDataSourceConfig, AzureBlobStorageDataSourceConfig]`	Instead of directly using this class, please use the appropriate data source type for your use case.

S3DataSourceAuthConfig ¶

A data model representing the configuration of a S3 data source.

Attributes:

Name	Type	Description
`source`	`Literal[S3]`	The data source type. Must be 'S3'.
`s3_role`	`Optional[str]`	Name of the role that a client will be initialized via AssumeRole of AWS sts
`external_id`	`Optional[str]`	External ID defined by the customer for the IAM role

S3DataSourceConfig ¶

A data model representing the configuration of a S3 data source.

Attributes:

Name	Type	Description
`source`	`Literal[S3]`	The data source type. Must be 's3'.
`s3_bucket`	`str`	The name of the S3 bucket where the data is stored
`s3_prefix`	`Optional[str]`	The prefix of the S3 bucket where the data is stored
`aws_region`	`str`	The AWS region where the S3 bucket is located
`aws_account_id`	`str`	The AWS account ID that owns the S3 bucket

SharePointDataSourceAuthConfig ¶

A data model representing the configuration of a SharePoint data source.

Attributes:

Name	Type	Description
`source`	`Literal[SHAREPOINT]`	The data source type. Must be 'sharepoint'.
`client_secret`	`str`	The secret for the app registration associated with this SharePoint site

SharePointDataSourceConfig ¶

A data model representing the configuration of a SharePoint data source.

Attributes:

Name	Type	Description
`source`	`Literal[SHAREPOINT]`	The data source type. Must be 'sharepoint'.
`client_id`	`str`	The client ID associated with this SharePoint site
`tenant_id`	`str`	The tenant ID that the SharePoint site is within
`site_id`	`str`	The site ID for this SharePoint site
`folder_path`	`Optional[str]`	The nested folder path to read files from the root of the site
`recursive`	`Optional[bool]`	Whether to recurse through the folder contents

TokenChunkingStrategyConfig ¶

A data model representing the configuration of a token chunking strategy.

Attributes:

Name	Type	Description
`strategy`	`Literal[TOKEN]`	The chunking strategy type. Must be 'token'.
`separator`	`Optional[str]`	Character designating breaks in input data. Text data will first be split into sections by this separator, then each section will be split into chunks of size `chunk_size`.
`target_chunk_size`	`Optional[int]`	Target number of tokens in each chunk. If not specified, a target chunk size of 200 will be used.
`max_chunk_size`	`Optional[int]`	Maximum number of tokens in each chunk. If not specified, a maximum chunk size of 200 will be used.
`chunk_overlap`	`Optional[int]`	Number of tokens to overlap between chunks. If not specified, an overlap of 0 will be used. Note this is only followed approximately.

knowledge_bases ¶

KnowledgeBase ¶

A data model representing a knowledge base.

Attributes:

Name	Type	Description
`knowledge_base_id`	`str`	The unique ID of the knowledge base
`knowledge_base_name`	`str`	The name of the knowledge base
`embedding_config`	`EmbeddingConfig`	The embedding configuration
`metadata`	`Optional[Dict[str, Any]]`	Metadata associated with the knowledge base
`created_at`	`str`	The timestamp at which the knowledge base was created
`updated_at`	`Optional[str]`	The timestamp at which the knowledge base was last updated

memory_strategy ¶

LastKMemoryStrategy ¶

Last K Memory Strategy. This strategy truncates the message history to the last k messages. It is the simplest way to prevent the model's context limit from being exceeded. However, this strategy only allows the model to have short term memory. For longer term memory, please use one of the other strategies.

Attributes:

Name	Type	Description
`name`	`Literal['last_k']`	Name of the memory strategy. Must be `last_k`.
`params`	`LastKMemoryStrategyParams`	Configuration parameters for the memory strategy.

LastKMemoryStrategyParams ¶

Parameters for the last k memory strategy

Attributes:

Name	Type	Description
`k`	`int`	The maximum number of previous messages to remember.

message ¶

AgentMessage ¶

Message from an agent

Attributes:

Name	Type	Description
`role`	`Literal['agent']`	The role of the message, must be "agent"
`content`	`Optional[str]`	Output of the agent if finished
`tool_request`	`Optional[ToolRequest]`	Request to run a tool

AssistantMessage ¶

Message from a non-agent AI.

Attributes:

Name	Type	Description
`role`	`Literal['assistant']`	The role of the message, must be "assistant"
`content`	`str`	The content of the message

Message ¶

A type alias for a Union of all message types.

Attributes:

Name	Type	Description
`__root__`	`Union[UserMessage, ToolMessage, AgentMessage, AssistantMessage, SystemMessage]`	Instead of directly using this class, please use the appropriate message type for your use case.

SystemMessage ¶

Message from the system. This is used if this message did not originate from the user or AI.

Attributes:

Name	Type	Description
`role`	`Literal['system']`	The role of the message, must be "system"
`content`	`str`	The content of the message

ToolMessage ¶

Message from a tool

Attributes:

Name	Type	Description
`role`	`Literal['tool']`	The role of the message, must be "tool"
`name`	`str`	Name of the tool
`content`	`str`	Output of calling the tool (JSON serialized to string)

ToolRequest ¶

Describes a request to run a tool

Attributes:

Name	Type	Description
`name`	`str`	Name of the tool
`arguments`	`str`	Arguments (JSON serializable string) to pass to the tool

UserMessage ¶

Message from the user

Attributes:

Name	Type	Description
`role`	`Literal['user']`	The role of the message, must be "user"
`content`	`str`	The content of the message

model_templates ¶

ModelTemplate ¶

This is a template for types of models that can be quickly customized by end users. It allows users to upload static docker images that can run specific types of models. These docker images will expose parameters that can be injected at ModelAlias creation time to customize the functionality. A common example of this is to use a HuggingFace LLM template, but swap out model weights for a finetuned model.

Attributes:

Name	Type	Description
`id`	`str`	The unique identifier of the entity.
`created_at`	`datetime`	The date and time when the entity was created in ISO format.
`account_id`	`str`	The ID of the account that owns the given entity.
`created_by_user_id`	`str`	The user who originally created the entity.
`name`	`str`	The name of the model template
`endpoint_type`	`ModelEndpointType`	The type of endpoint that the model template will create
`model_type`	`ModelType`	The type of model that the model template will create
`vendor_configuration`	`ModelVendorConfiguration`	The vendor configuration of the model template
`model_creation_parameters_schema`	`Optional[ParameterSchema]`	The field names and types of available parameter fields which may be specified during model creation
`model_request_parameters_schema`	`Optional[ParameterSchema]`	The field names and types of available parameter fields which may be specified in a model execution API's `model_request_parameters` field.

models ¶

AgentRequest ¶

Response schema for agents. See the Execute Agent REST API for more information.

Attributes:

Name	Type	Description
`memory_strategy`	`Optional[MemoryStrategy]`	The memory strategy to use for the agent. A memory strategy is a way to prevent the underlying LLM's context limit from being exceeded. Each memory strategy uses a different technique to condense the input message list into a smaller payload for the underlying LLM.
`tools`	`List[Tool]`	The list of specs of tools that the agent can use. Each spec must contain a `name` key set to the name of the tool, a `description` key set to the description of the tool, and an `arguments` key set to a JSON Schema compliant object describing the tool arguments. The name and description of each tool is used by the agent to decide when to use certain tools. Because some queries are complex and may require multiple tools to complete, it is important to make these descriptions as informative as possible. If a tool is not being chosen when it should, it is common practice to tune the description of the tool to make it more apparent to the agent when the tool can be used effectively.
`messages`	`List[Message]`	The list of messages in the conversation.
`instructions`	`Optional[str]`	The initial instructions to provide to the agent. Use this to guide the agent to act in more specific ways. For example, if you have specific rules you want to restrict the agent to follow you can specify them here. For example, if I want the agent to always use certain tools before others, I can write that rule in these instructions. Good prompt engineering is crucial to getting performant results from the agent. If you are having trouble getting the agent to perform well, try writing more specific instructions here before trying more expensive techniques such as swapping in other models or finetuning the underlying LLM.

AgentResponse ¶

Response schema for agents.

See the Execute Agent REST API for more information.

Attributes:

Name	Type	Description
`action`	`AgentAction`	The action that the agent performed.
`context`	`ActionContext`	Context object containing the output payload. This will contain a key for all actions that the agent can perform. However, only the key corresponding to the action that the agent performed have a populated value. The rest of the values will be `null`.

ChatCompletionRequest ¶

Request schema for chat completion models.

Attributes:

Name	Type	Description
`temperature`	`float`	What sampling temperature to use, between [0, 1]. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Setting temperature=0.0 will enable fully deterministic (greedy) sampling.
`stop_sequences`	`Optional[List[str]]`	List of up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
`max_tokens`	`Optional[int]`	The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. If not, specified, max_tokens will be determined based on the model used.
`messages`	`List[Message]`	List of messages for the chat completion to consider when generating a response.

ChatCompletionResponse ¶

Response schema for chat completion models.

Attributes:

Name	Type	Description
`message`	`Message`	The generated message from the chat completion model.
`finish_reason`	`Optional[str]`	The reason the chat completion finished.

CompletionRequest ¶

Request schema for completion models.

Attributes:

Name	Type	Description
`temperature`	`float`	What sampling temperature to use, between [0, 1]. Higher values like 0.8 will make the output more random, while lower values like 0.2 will make it more focused and deterministic. Setting temperature=0.0 will enable fully deterministic (greedy) sampling.
`stop_sequences`	`Optional[List[str]]`	List of up to 4 sequences where the API will stop generating further tokens. The returned text will not contain the stop sequence.
`max_tokens`	`Optional[int]`	The maximum number of tokens to generate in the completion. The token count of your prompt plus max_tokens cannot exceed the model's context length. If not, specified, max_tokens will be determined based on the model used.
`prompts`	`List[str]`	List of prompts to generate completions for.

CompletionResponse ¶

Response schema for completion models.

Attributes:

Name	Type	Description
`completions`	`List[Tuple[str, List[str]]]`	List of prompt, completion pairs.
`finish_reason`	`Optional[str]`	The reason the completion finished.

EmbeddingRequest ¶

Request schema for embedding models.

Attributes:

Name	Type	Description
`texts`	`List[str]`	List of texts to get embeddings for.

EmbeddingResponse ¶

Response schema for embedding models.

Attributes:

Name	Type	Description
`embeddings`	`List[Tuple[str, List[float]]]`	List of text, embedding pairs.

ModelDeployment ¶

Deployment of a model

Attributes:

Name	Type	Description
`id`	`str`	The unique identifier of the model deployment.

ModelInstance ¶

Entity for all models, including both self-hosted and 3rd party, base, and fine-tuned models.

Attributes:

Name	Type	Description
`id`	`str`	The unique identifier of the model.
`created_at`	`datetime`	The date and time when the entity was created in ISO format.
`account_id`	`str`	The ID of the account that owns the given entity.
`created_by_user_id`	`str`	The user who originally created the entity.
`name`	`str`	The name of the model
`description`	`Optional[str]`	The description of the model
`model_vendor`	`Optional[ModelVendor]`	The vendor of the model
`base_model_id`	`Optional[str]`	The ID of the base model
`base_model_metadata`	`Optional[Dict[str, Any]]`	Metadata for the base model
`model_template_id`	`Optional[str]`	The ID of the model template
`model_group_id`	`Optional[str]`	Model group that the entity belongs to

ParameterSchema ¶

The schema used to specify the parameters to be passed for model creation.

Attributes:

Name	Type	Description
`parameters`	`List[ParameterSchemaField]`	List of parameter schema fields.

ParameterSchemaField ¶

The schema used to specify the type and info about each parameter to be passed for model creation.

Attributes:

Name	Type	Description
`name`	`str`	Name of the parameter.
`type`	`Union[Literal['str'], Literal['int'], Literal['float'], Literal['bool']]`	Type of the parameter.
`description`	`str`	Description of the parameter.
`required`	`bool`	Whether the parameter is required or not.

RerankingRequest ¶

Request schema for reranking models.

Attributes:

Name	Type	Description
`query`	`str`	Query to rerank chunks against in order of relevance.
`chunks`	`List[str]`	List of chunks to rerank.

RerankingResponse ¶

Response schema for reranking models.

Attributes:

Name	Type	Description
`chunk_scores`	`List[float]`	List of scores for each chunk in the same order as the input chunks.

models_group ¶

ModelGroup ¶

Entity for grouping models which are tied to the base model. E.g.: gpt-4 can be a group containing all gpt-4 fine-tuned models

Attributes:

Name	Type	Description
`id`	`str`	The unique identifier of the entity.
`name`	`str`	The name of the group
`description`	`str`	Description of the group

question_sets ¶

QuestionSet ¶

A data model representing a question set.

Attributes:

Name	Type	Description
`id`	`str`	The ID of the question set
`name`	`str`	The name of the question set
`created_at`	`str`	The time the question set was created
`created_by_user_id`	`str`	The user ID of the user who created the question set
`account_id`	`str`	The account ID of the account that the question set is associated with
`questions`	`Optional[List[Question]]`	The questions in the question set

questions ¶

CategoricalCondition `module-attribute` ¶

CategoricalCondition = Dict[str, Union[str, List[str]]]

A mapping from question_id to either the exact value that must be selected, or a list of acceptable values. All key-value pairs in the mapping must be satisfied for the condition to be True. For questions with multi=True, the selected values include at least one of the acceptable values.

BaseQuestion ¶

A base class for questions.

Attributes:

Name	Type	Description
`id`	`str`	A unique id for this question. Each question_id will be represented as a key in the `result` field in the TestCaseResult after an annotation is completed. When examining how an application has performed on a specific question, users should look at the `result` field in the TestCaseResult for the key with the corresponding id.
`title`	`str`	The text displayed to annotators for this question.
`prompt`	`str`	The prompt displayed to annotators for this question.
`required`	`bool`	Whether this question is required. If `True`, the annotator must select at least one choice.
`conditions`	`Optional[List[CategoricalCondition]]`	Conditions that allow the question to be rendered. Example 1: `conditions=[{"accurate": "very", "complete": "yes"}]` means that the selected value for `accurate` must be `"very"` and the selected value for `complete` must be `"yes"`. Example 2: `conditions=[{"accurate": ["mostly", "very"]}, {"complete": "yes"}]` means that either the selected value for `accurate` must be `"mostly"` or `"very"`, or the selected value for `complete` must be `"yes"`.
`account_id`	`str`	The account ID of the account that the question is associated with.
`created_at`	`str`	The time the question was created.
`created_by_user_id`	`str`	The user ID of the user who created the question.

CategoricalChoice ¶

A choice for a categorical question.

This is only used in HUMAN evaluation type to specify a choice for a question that will be asked to users when they are evaluating generated outputs in the SGP annotation UI.

Attributes:

Name	Type	Description
`label`	`str`	The text displayed to annotators for this choice.
`value`	`Union[str, int, bool]`	The value reported in the TestCaseResult for this question if this choice is selected. If users would like to track the improvement of a model over time, it is recommended to use a numeric value for this field. A string value can be used if no ordering is desired.
`audit_required`	`bool`	Whether selecting this choice will flag the test case result. Defaulted to false.

CategoricalQuestion ¶

A categorical question.

This is only used in HUMAN evaluation type to specify a choice for a question that will be asked to users when they are evaluating generated outputs in the SGP annotation UI.

Attributes:

Name	Type	Description
`type`	`Literal[CATEGORICAL]`	The type of the question.
`choices`	`List[CategoricalChoice]`	The choices for the question.
`multi`	`bool`	Whether to allow multiple choices to be selected. If `True`, displays as a checkbox list. Otherwise, displays as a radio group.
`dropdown`	`Optional[bool]`	Whether to display options as a dropdown list. If `True`, displays as a dropdown. Otherwise, displays as a radio group. Currently, dropdown cannot be true if multi is true.

FreeTextQuestion ¶

A free text question.

This is only used in HUMAN evaluation type to specify a choice for a question that will be asked to users when they are evaluating generated outputs in the SGP annotation UI.

Attributes:

Name	Type	Description
`type`	`Literal[FREE_TEXT]`	The type of the question.