Applications - Model Performance

The Application Model Performance endpoint returns the current best model's predictions on the validation data set, in addition to truth data based on user consensus subject-media associations.

Application Model Performance Request Body

Argument	Example	Definition
subject_uid string	"cat_123"	(required) Subject ID; filters truth data for the given subject_uid.
consensus string	"True"	(optional) True - Returns results for positive consensus subject-media associations. False - Returns results for negative consensus subject-media associations. Defaults to `null`, i.e. returns all consensus subject media.
abridged_media boolean	False	(optional) True - returns abridged media objects consisting solely of media_id. False - returns full Cogniac media objects. Defaults to False.
probability_lower float	0.25	(optional) Lower probability bound of records to return. Defaults to 0.
probability_upper float	0.75	(optional) Upper probability bound of records to return. Defaults to 1.
sort string	"loss"	(optional) One of 'assertion' or 'loss' to sort results by assertion probability or app_data error, respectively. Defaults to 'assertion'.
reverse boolean	True	(optional) True - return records in descending order. False - return records in ascending order. Defaults to False.
limit integer	25	(optional) Result limit per page. Defaults to 100.

Application Model Performance Response

The Retrieve Application Model Peformance endpoint returns a list of model performance objects.

Fields	Example	Definition
media object		A Cogniac media object.
model_id string	"Hpo-h-1ec0-to5dGrTAN9QeV6djep-CN_mtsv0_2000.tgz"	The ID of the current best model.
truth_data array		A list of subject-media association objects.
predictions map		A list of application subject-media assertion objects .
app_data_accuracy float	0.87	Accuracy of assertion app_data versus the truth data.
loss float	0.13	Error of assertion versus truth data.

GET /1/applications/{application_id}/modelPerformance
Host: https://api.cogniac.io

Example: Retrieve Application Model Performance

curl -X GET https://api.cogniac.io/1/applications/di71rG94/modelPerformance?subject_uid=cat_123&limit=10&reverse=true \
-H "Authorization: Bearer abcdefg.hijklmnop.qrstuvwxyz"

import requests
from pprint import pprint

my_headers = {"Authorization":"Bearer abcdefg.hijklmnop.qrstuvwxyz"}
resp = requests.get("https://api.cogniac.io/1/applications/di71rG94/modelPerformance?subject_uid=cat_123&limit=10&reverse=true",
                    headers=my_headers)

pprint(resp.json())

{
    "data": [
        {
            "app_data_accuracy": 0.0009971394103637232,
            "focus": null,
            "loss": 0.9990028605896363,
            "media": {...},
            "model_id": "Hpo-m-3ebb-yyR4vOynu8620k0MO9-GN-rf_224_224_mtsv1_2028.tgz",
            "predictions": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": null,
                    "focus": null,
                    "media_id": "901565ac34dab9b3cc1e356441a10ed7",
                    "subject_uid": "cat_123",
                    "uncal_prob": 0.9999996423721313
                }
            ],
            "truth_data": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": "False",
                    "focus": null,
                    "media_id": "901565ac34dab9b3cc1e356441a10ed7",
                    "subject_uid": "cat_123"
                }
            ]
        },
        {
            "app_data_accuracy": 0.9990034093483733,
            "focus": null,
            "loss": 0.000996590651626714,
            "media": {...},
            "model_id": "Hpo-m-3ebb-yyR4vOynu8620k0MO9-GN-rf_224_224_mtsv1_2028.tgz",
            "predictions": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": null,
                    "focus": null,
                    "media_id": "067d830b1ef3602c14e2fce67ea51fa9",
                    "subject_uid": "cat_123",
                    "uncal_prob": 0.9992589354515076
                }
            ],
            "truth_data": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": "True",
                    "focus": null,
                    "media_id": "067d830b1ef3602c14e2fce67ea51fa9",
                    "subject_uid": "cat_123"
                }
            ]
        },
        {
            "app_data_accuracy": 0.9990012135066857,
            "focus": null,
            "loss": 0.0009987864933143494,
            "media": {...},
            "model_id": "Hpo-m-3ebb-yyR4vOynu8620k0MO9-GN-rf_224_224_mtsv1_2028.tgz",
            "predictions": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": null,
                    "focus": null,
                    "media_id": "0a4badac8bcb0d64e704a39abaccd70a",
                    "subject_uid": "cat_123",
                    "uncal_prob": 1
                }
            ],
            "truth_data": [
                {
                    "app_data": null,
                    "app_data_type": null,
                    "consensus": "True",
                    "focus": null,
                    "media_id": "0a4badac8bcb0d64e704a39abaccd70a",
                    "subject_uid": "cat_123"
                }
            ]
        },
        ...
    }
]