Scale API

Evaluations

evaluations

Methods

Claim Evaluation Task ->
Task
post/v4/evaluations/{evaluation_id}/claim-task

Claim Evaluation Task

Create Evaluation ->
Evaluation
post/v4/evaluations

Description

Creates a evaluation

Details

This API can be used to create a evaluation. To use this API, review the request schema and pass in all fields that are required to create a evaluation.

Delete Evaluation ->
GenericDeleteResponse
delete/v4/evaluations/{evaluation_id}

Description

Deletes a evaluation

Details

This API can be used to delete a evaluation by ID. To use this API, pass in the id that was returned from your Create Evaluation API call as a path parameter.

List Evaluations -> PageResponse<
EvaluationWithViews
>
get/v4/evaluations

Description

Lists all evaluations accessible to the user.

Details

This API can be used to list evaluations. If a user has access to multiple accounts, all evaluations from all accounts the user is associated with will be returned.

Get Evaluation ->
EvaluationWithViews
get/v4/evaluations/{evaluation_id}

Description

Gets the details of a evaluation

Details

This API can be used to get information about a single evaluation by ID. To use this API, pass in the id that was returned from your Create Evaluation API call as a path parameter.

Review the response schema to see the fields that will be returned.

Update Evaluation ->
Evaluation
patch/v4/evaluations/{evaluation_id}

Description

Updates a evaluation

Details

This API can be used to update the evaluation that matches the ID that was passed in as a path parameter. To use this API, pass in the id that was returned from your Create Evaluation API call as a path parameter.

Review the request schema to see the fields that can be updated.

Domain types

AnnotationConfig = { annotation_config_type, components, direction, 1 more... }
Evaluation = { id, account_id, application_spec_id, 15 more... }
EvaluationMetrics = { avg_labeling_time_per_test_case, avg_num_test_cases_labeled_per_day, evaluation_id, 9 more... }
EvaluationTraceSpan = { id, application_interaction_id, duration_ms, 8 more... }
EvaluationWithViews = { id, account_id, application_spec_id, 19 more... }
HybridEvaluationMetrics = { all_question_info }
MultiturnAnnotationConfig = { messages_loc, llm_prompt }
PaginatedEvaluations = { current_page, items, items_per_page, 1 more... }
SummarizationAnnotationConfig = { document_loc, summary_loc, expected_summary_loc, 1 more... }
Task = { id, account_id, priority, 7 more... }
TranslationAnnotationConfig = { original_text_loc, translation_loc, expected_translation_loc, 2 more... }
Evaluations

Contributor Metrics

evaluations.contributor_metrics

Methods

List Contributor Metrics -> PageResponse<
ContributorMetrics
>
get/v4/evaluations/{evaluation_id}/contributor-metrics

List Contributor Metrics

Get Contributor Metrics ->
ContributorMetrics
get/v4/evaluations/{evaluation_id}/contributor-metrics/{contributor_id}

Get Contributor Metrics

Domain types

ContributorMetrics = { annotated_by_user_id, evaluation_id, num_test_cases_fixed, 4 more... }
PaginatedContributorMetrics = { current_page, items, items_per_page, 1 more... }
Evaluations

Evaluation Metrics

evaluations.evaluation_metrics

Methods

Get Evaluation Metrics ->
EvaluationMetrics
get/v4/evaluations/{evaluation_id}/evaluation-metrics

Get Evaluation Metrics

Evaluations

Hybrid Eval Metrics

evaluations.hybrid_eval_metrics

Methods

Get Hybrid Evaluation Metrics ->
HybridEvaluationMetrics
get/v4/evaluations/{evaluation_id}/hybrid-eval-metrics

Get Hybrid Evaluation Metrics

Evaluations

Tasks

evaluations.tasks

Methods

Update Evaluation Task ->
Task
patch/v4/evaluations/{evaluation_id}/tasks/{task_id}

Update Evaluation Task

Evaluations

Test Case Results

evaluations.test_case_results

Methods

Batch Create Test Case Results -> Array<
TestCaseResult
>
post/v4/evaluations/{evaluation_id}/test-case-results/batch

Description

Creates a batch of test case results

Details

This API can be used to create multiple test case results so users do not have to the incur the cost of repeated network calls. To use this API, pass in a list of test case results in the request body.

Create Test Case Result ->
TestCaseResult
post/v4/evaluations/{evaluation_id}/test-case-results

Description

Creates a test case result

Details

This API can be used to create a test case result. To use this API, review the request schema and pass in all fields that are required to create a test case result.

List Test Case Results -> PageResponse<
TestCaseResultWithViews
>
get/v4/evaluations/{evaluation_id}/test-case-results

Description

Lists all test case results accessible to the user.

Details

This API can be used to list test case results. If a user has access to multiple accounts, all test case results from all accounts the user is associated with will be returned.

Get Test Case Result ->
TestCaseResultWithViews
get/v4/evaluations/{evaluation_id}/test-case-results/{test_case_result_id}

Description

Gets the details of a test case result

Details

This API can be used to get information about a single test case result by ID. To use this API, pass in the id that was returned from your Create Test Case Result API call as a path parameter.

Review the response schema to see the fields that will be returned.

Update Test Case Result ->
TestCaseResult
patch/v4/evaluations/{evaluation_id}/test-case-results/{test_case_result_id}

Description

Updates a test case result

Details

This API can be used to update the test case result that matches the ID that was passed in as a path parameter. To use this API, pass in the id that was returned from your Create Test Case Result API call as a path parameter.

Review the request schema to see the fields that can be updated.

Domain types

PaginatedTestCaseResultWithViews = { current_page, items, items_per_page, 1 more... }
TestCaseResult = { id, account_id, application_spec_id, 18 more... } | { id, account_id, application_spec_id, 18 more... }
TestCaseResultWithViews = { id, account_id, application_spec_id, 23 more... } | { id, account_id, application_spec_id, 23 more... }
EvaluationsTest Case Results

History

evaluations.test_case_results.history

Methods

List Historical Test Case Results -> PageResponse<
TestCaseResult
>
get/v4/evaluations/{evaluation_id}/test-case-results/history/{num}

List Historical Test Case Results

Get Historical Test Case Result ->
TestCaseResult
get/v4/evaluations/{evaluation_id}/test-case-results/{test_case_result_id}/history/{num}

Get Historical Test Case Result

Domain types

PaginatedTestCaseResults = { current_page, items, items_per_page, 1 more... }