Evaluation Dashboards
Create Evaluation Dashboard
List Evaluation Dashboards
Get Evaluation Dashboard
Patch Evaluation Dashboard
Delete Evaluation Dashboard
ModelsExpand Collapse
class EvaluationDashboard: …
Widget results for this dashboard. Populated with ‘widget_results’ view.
Widget results for this dashboard. Populated with ‘widget_results’ view.
FK to evaluation_groups. Null if result is for a single evaluation.
widget: Optional[EvaluationDashboardWidget]Widget that this result is for
Widget that this result is for
query: Optional[Query]Structured query AST for metric computation (SeriesQuery or MetricQuery)
Structured query AST for metric computation (SeriesQuery or MetricQuery)
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
Widgets associated with this dashboard. Populated with ‘widgets’ view.
Widgets associated with this dashboard. Populated with ‘widgets’ view.
query: Optional[Query]Structured query AST for metric computation (SeriesQuery or MetricQuery)
Structured query AST for metric computation (SeriesQuery or MetricQuery)
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
Evaluation DashboardsWidgets
Add Widget to Dashboard
Update Dashboard Widget
Remove Widget from Dashboard
ModelsExpand Collapse
class EvaluationDashboardWidget: …
query: Optional[Query]Structured query AST for metric computation (SeriesQuery or MetricQuery)
Structured query AST for metric computation (SeriesQuery or MetricQuery)
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class EvaluationDashboardWidgetResult: …
FK to evaluation_groups. Null if result is for a single evaluation.
widget: Optional[EvaluationDashboardWidget]Widget that this result is for
Widget that this result is for
query: Optional[Query]Structured query AST for metric computation (SeriesQuery or MetricQuery)
Structured query AST for metric computation (SeriesQuery or MetricQuery)
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class EvaluationDashboardWidgetWithResult: …Response model for widget creation - includes widget and computed result
Response model for widget creation - includes widget and computed result
query: Optional[Query]Structured query AST for computation (SeriesQuery or MetricQuery)
Structured query AST for computation (SeriesQuery or MetricQuery)
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class Filter: …Filter clause with conditions connected by logical operators.
Conditions are evaluated left-to-right without precedence (no nesting/parentheses).
Example: condition1 AND condition2 OR condition3 evaluates as ((condition1 AND condition2) OR condition3)
Example:
{
“conditions”: [
{“column”: “score”, “operator”: ”>”, “value”: 0.5},
{“column”: “category”, “operator”: ”=”, “value”: “test”}
],
“logicalOperators”: [“AND”]
}
Filter clause with conditions connected by logical operators.
Conditions are evaluated left-to-right without precedence (no nesting/parentheses). Example: condition1 AND condition2 OR condition3 evaluates as ((condition1 AND condition2) OR condition3)
Example: { “conditions”: [ {“column”: “score”, “operator”: ”>”, “value”: 0.5}, {“column”: “category”, “operator”: ”=”, “value”: “test”} ], “logicalOperators”: [“AND”] }
class MetricQuery: …Query that returns a single metric value (used for metric widgets).
Used for widget type: metric.
Enforces exactly 1 aggregation in select.
Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent:
SELECT AVG(score) as average_score
FROM evaluation_items
Query that returns a single metric value (used for metric widgets).
Used for widget type: metric. Enforces exactly 1 aggregation in select. Returns: {“type”: “metric”, “data”: …}
Example SQL equivalent: SELECT AVG(score) as average_score FROM evaluation_items
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.
class SelectItem: …Column in SELECT clause
Column in SELECT clause
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
class SeriesQuery: …Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter.
Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent:
SELECT category, AVG(score) as avg_score, COUNT(*) as count
FROM evaluation_items
WHERE score > 0.5 AND category = ‘test’
GROUP BY category
ORDER BY avg_score DESC
LIMIT 100
Query that returns a series of records (used for table/bar/histogram/donut/scatter widgets).
Used for widget types: table, bar, histogram, donut, scatter. Returns: {“type”: “series”, “data”: […]}
Example SQL equivalent: SELECT category, AVG(score) as avg_score, COUNT(*) as count FROM evaluation_items WHERE score > 0.5 AND category = ‘test’ GROUP BY category ORDER BY avg_score DESC LIMIT 100
expression: ExpressionReference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionColumn: …Reference to a column from evaluation_items.data
Example:
{“type”: “COLUMN”, “column”: “category”}
Reference to a column from evaluation_items.data
Example: {“type”: “COLUMN”, “column”: “category”}
class ExpressionAggregation: …Aggregation function to apply
Examples:
{“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”}
{“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”}
{“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Aggregation function to apply
Examples: {“type”: “AGGREGATION”, “function”: “AVG”, “column”: “score”} {“type”: “AGGREGATION”, “function”: “COUNT”, “column”: ”*”} {“type”: “AGGREGATION”, “function”: “PERCENTILE”, “column”: “score”, “params”: {“percentile”: 95}}
Optional subset of evaluation IDs for per-aggregation filtering in evaluation group dashboards.
Optional subset of evaluation IDs to compute on. Only applicable for evaluation group dashboards. If omitted, computes on all evaluations in the group.
filter: Optional[Filter]Filter conditions (WHERE clause)
Filter conditions (WHERE clause)
When True, the widget computes against rows from only the most recent active evaluation in the group (by EvaluationORM.created_at). Only applicable for evaluation group dashboards. Composes with evaluation_ids (latest within the subset). Cannot be combined with per-aggregation evaluation_ids; the use case enforces these rules.