Subjects - Media

Subject Media Associations

Subjects are associated with media by directly posting (uploading) media to that subject, or through an application-feedback context. The strength of a subject-media association is defined by its probability. Association probability of 1.0 denotes a very likely positive association; e.g. "this image contains a cat," and a probability of 0.0 denotes a very strong negative association; e.g. "this image does not contain a cat."

A subject-media association is represented as a JSON dictionary with the keys 'media' and 'subject' that contain meta data about the media object and subject association.

Subject Media Association Object

Field	Example	Description
media map		A Cogniac media object
subject map		Dictionary of the subject association including optional application-type based media association data. E.g. a bounding box for the subject.
focus map	{"box": {"x0": 0, "x1": 227, "y0": 0, "y1": 227}}	(optional) Map or object identifying the portion or subregion of the media, (e.g. a particular box area or range of frames) with which the subject is associated. If the entire media is processed, this field will be null.
other_media array		(optional) List of other media objects that may be associated with this subject-media association.

Subject Association Object

Each subject-media association object contains a dictionary defining the association meta data, including subject ID, probability of association, consensus, and optional application specific association data.

Field	Example	Description
subject_uid string	"cat_123"	The ID of the subject.
consensus string	"True"	(optional) True - the subject is conclusively positively associated with the media. False - the subject is conclusively negatively associated with the media. Note that if this field is null or "None", the subject has not been conclusively associated, positively or negatively, with the media.
app_data_type string	"box_set"	String representing the application-type-specific context of the subject-media association.
app_data object	{ "box": {"x0": 20, "y0": 56, "x1": 535, "y1": 249}, "probability": 0.95 }	Application-type-specific meta data for the subject-media association.
probability float	0.99	Probability of the subject media association. A subject media association probability of 1 corresponds with a definite positive association (the subject is present) and a probability of 0 corresponds with a definite negative association (the subject is not present).
updated_at float	1234567890	Unix timestamp; most recent update to the subject-media association through user feedback or model assertion.

📘
Application-Specific Subject-Media Data
Subject-media associations have optional additional application-specific data relative to the subject, box region areas in detection applications, and range of frames in detection full-frame applications, etc.
The Cogniac system supports the following application data types:
box_set - A list of box regions corresponding to the subject. A box region is defined as:
{
    “box”:            (dictionary of integers) “x0”, “x1”, “y0”, “y1”  
                          Values correspond to pixel offsets in the media item 
                          (x0, y0)  upper left corner of bounding box
                          (x1, y1) lower right corner of bounding box
                          Note that x1 must be greater than x0 
                          and y1 must be greater than y0

    “probability”:  (optional) (float) probability for box
}
segment_list - A list of video frame segments corresponding to the subject. A segment is defined as:
{
    “segment”:  (dictionary of integers) “f0”, “f1”
                       f0 & f1 correspond to the frame offsets in a video media 
                       (frame count starts at 0)
                       f0 is the first frame in the media item segment with the subject
                       f1 is the last frame in the media item segment with the subject 
                       Note that f1 must be greater than, or equal to f0

    “probabilities”:  [ordered list of per-frame model output probabilities]
                             The first list entry corresponds to prob for the frame at f0
                             The last list entry corresponds to prob for the frame at f1
                             Note the length of the list if present must equal (f1 - f0 + 1)
}
ocr - A string representing the characters detected.
7ZYA906
count - A float (>= 0) representing number of occurrences of the subject detected.
12

📘
app_data Probability
If an app_data value is present, the probability presents the association among the subject, the media and the app_data value. With the presence of an app_data value, the probability of 0 can represent two cases:

The subject is not present.

The subject is present, however the app_data is not correct.

For example, if app_data_type="ocr", the probability of 1 refers to the case that the text (app_data) is definitely correct and the subject is definitely present in the media. The probability of 0 can represent either:

The subject is not present.

The subject is present, however the text is not correct.

Upload Media to a Subject

To manually create a subject-media association (i.e. outside of the context of an application), existing Cogniac media objects can be directly posted to a subject.


media_id string	(required) The Cogniac media ID of the media to upload to the subject.
capture_tag string	(optional) Per-capture meta-tag
force_feedback boolean	(optional) True = Immediately surface the image for feedback. Bypasses any application-specific feedback rates. False = Rely on application-specific system-set or user-specified feedback rate per hour to surface the image for feedback or not. Defaults to False.
app_data_type string	(optional) String representing the application-type-specific context of the subject-media association.
app_data string	(optional) Application-type-specific meta data for the subject-media association.
focus map	Map identifying the portion or subregion of the media, (e.g. a particular box area or range of frames) with which the subject is associated. If the entire media is associated with the subject, this field should be null.
consensus boolean	(optional) False = Authoritatively assert a negative association between the media and subject. default to 'None'

👍
Best Practice: False Consensus with an app_data
Providing false consensus is to authoritatively assert a negative association between a subjet and media item. When app_data is present, in agreement with the probability definition in the Subject Association Object, the false consensus refers to the cases 1) when the subject is not present, or 2) when the subject is present, however the app_data is definitely not correct.
In fact, when the subject is not present in the media, the app_data becomes irrelevant. Hence, the best practice is to exclude app_data_type and app_data from the assertion when the subject is not present. Also, app_data_type and app_data should only be included in the message if the subject is present while the app_data is definitely not correct.

📘
Force Feedback
The Cogniac System will set an optimal rate of feedback requests to surface to the user based on application model performance and total subject-media consensus items. This dynamic feedback rate can be found in an application's settings as system_feedback_per_hour. Users also have the option to adjust their rate of feedback in an application's settings by adjusting requested_feedback_per_hour.
When posting media to a subject, the force_feedback field can be used to signify that the media should be surfaced for additional feedback. Setting force_feedback=True will surface 100% of uploaded media for feedback.
force_feedback=True is not recommended for large-scale streaming upload, instead use requested_feedback_per_hour in the context of an application to increase or decrease the rate of feedback requests on uploaded media.

POST /1/subjects/{subject_uid}/media
Host: https://api.cogniac.io

Example: Associate Media to a Subject

To associate media to a subject, the media item must exist in the Cogniac system. For more information on creating media objects see Media - Create.

curl -X POST https://api.cogniac.io/1/subjects/cat_123/media \
-H "Authorization: Bearer abcdefg.hijklmnop.qrstuvwxyz" \
-H "Content-Type: application/json" \
-d '{
  "media_id":"249i24hjkasd",
  "force_feedback":true
}'

import cogniac

# connect to the tenant
cc = cogniac.CogniacConnection(username="[email protected]", 
                               password="myPassword", 
                               tenant_id="63QhzFLc9tg4")

# first, create the media item
media = cc.create_media("cat.jpg")

# retrieve the subject with subject_uid
subject = cc.subject("cat_123")

# capture the media to the subject
# with the media object or media ID
# set force_feedback flag
subject.associate_media(media, force_feedback=True)

{
  "capture_id": "1234"
}

Gateway Subject Media

The gateway supports subject-media associations for its locally uploaded media. The capture process is identical, where the capture hostname is the IP address or hostname of the gateway.

POST /1/subjects/{subject_uid}/media
Host: http://{local gateway hostname}

Retrieving Subject Media Associations

Returns a list of subject-media associations for the given subject.

Argument	Description
limit integer	(optional) The maximum number of results to return. Defaults to 25, limited to 100 results.
reverse boolean	(optional) True - results will be sorted in descending order False - results will be sorted in ascending order. Defaults to False.
start unix timestamp	(optional) The timestamp of the oldest subject-media association record to return. Defaults to 0. (returns any age of subject media association)
end unix timestamp	(optional) The timestamp of the youngest subject-media association record to return. Defaults to the current timestamp.
duration integer	(optional) Duration (in seconds) from the current time to search for media. Note: Duration will be ignored if start and end arguments are present.
probability_upper float	(optional) Upper bound for media-subject association probability. Defaults to 1.
probability_lower float	(optional) Lower bound for media-subject association probability. Defaults to 0.
consensus string	(optional) Allows the results to be filtered based on the final determined subject-media association. 'True' - Returns positive subject-media associations. 'False' - Returns negative subject-media associations. 'None' - Returns subject-media associations where no consensus has been reached. Defaults to null, i.e. all subject-media associations will be returned regardless of consensus.
sort string	(optional) One of 'time' or 'probability', the field value by which to sort results. Defaults to 'time'.

GET /1/subjects/{subject_uid}/media
Host: https://api.cogniac.io

Example: Search for Subjects by Prefix

curl -X GET https://api.cogniac.io/1/subjects/cast_123/media?limit=10&reverse=true \
-H "Authorization: Bearer abcdefg.hijklmnop.qrstuvwxyz"

import cogniac

# connect to the tenant
cc = cogniac.CogniacConnection(username="[email protected]", 
                               password="myPassword", 
                               tenant_id="63QhzFLc9tg4")

# retrieve the subject with subject_uid
subject = cc.get_subject("cat_123")
smedia = subject.media_associations(limit=10, reverse=True)

# returns an iterator
# of subject-media association items
for sm in smedia:
  print sm

{
  "data": [
    {
      "media": {
        "media_id": "media_1",
        "media_url": "https://cogniac-media.s3.amazonaws.com/media_1",
        "resize_urls": {
          "454": "https://cogniac-media.s3.amazonaws.com/media_1_max_454",
          "750": "https://cogniac-media.s3.amazonaws.com/media_1_max_750"
        }
      },
      "subject": {
        "app_data": null,
        "app_data_type": null,
        "consensus": null,
        "media_id": "media_1",
        "probability": 0.5,
        "subject_uid": "cat_123",
        "updated_at": 1481661384.100544
      },
      "focus":{
        "box": {"x0": 50, "x1": 275, "y0": 634, "y1": 870}
      }
    },
    {
      "media": {
        "media_id": "media_2",
        "media_url": "https://cogniac-media.s3.amazonaws.com/media_2",
        "resize_urls": {
          "454": "https://cogniac-media.s3.amazonaws.com/media_2_max_454",
          "750": "https://cogniac-media.s3.amazonaws.com/media_2_max_750"
        }
      },
      "subject": {
        "app_data": null,
        "app_data_type": null,
        "consensus": null,
        "media_id": "media_2",
        "probability": 0.5,
        "subject_uid": "cat_123",
        "updated_at": 1481661507.608449
      },
      "focus":{} // focus can be empty
    }
  ],
  "paging": {
    "next": "http://127.0.0.1:5000/1/subjects/cat_123/media?sort=time&end=1483571015.26&reverse=False&start=0&limit=2&probability_upper=1.0&probability_lower=0.0&cursor=1481661507.608449"
  }
}

Deleting Subject Media Associations

Sometimes a False, or null consensus between a subject and media item is not sufficient, for example the case where an out-of-context media item is incorrectly associated with a subject. In this case it is best to completely remove the association record from the system.

The Delete Subject Media endpoint can be used to completely remove the association between a subject and media item.

Argument	Definition
media_id string	(required) The unique ID of the media object to disassociate with the subject.
focus object	(optional) Map identifying the portion or subregion of the media, (e.g. a particular box area or range of frames) to disassociate from the subject. If the entire media is to be disassociated with the subject, this field should be null.

🚧
Subject-Media-Focus
A unique subject-media association is a combination of a subject_uid, media_id, and focus area, where a null focus area implies the entire media item is the focus.
To disassociate media from a subject, both the media and focus area of the association must be passed. So, if an association between a subject and full media item is to be removed, focus=null should be passed. Not that this will not remove any subject associations with different focus areas within the media.
It is recommended to use the Retrieve Media Detections endpoint to retrieve all subject associations with a media item to determine how many associations require removal.

DELETE /1/subjects/{subject_uid}/media
Host: https://api.cogniac.io

Example: Disassociate Media from a Subject

curl -X DELETE https://api.cogniac.io/1/subjects/cat_123/media?media_id=249i24hjkasd \
-H "Authorization: Bearer abcdefg.hijklmnop.qrstuvwxyz"

import cogniac

# connect to the tenant
cc = cogniac.CogniacConnection(username="[email protected]", 
                               password="myPassword", 
                               tenant_id="63QhzFLc9tg4")

# retrieve the subject with subject_uid
subject = cc.subject("cat_123")

# disassociate the media by ID
subject.disassociate_media(media_id="249i24hjkasd")

HTTP 204 Code (with no body)

Subjects - Media

Subject Media Associations

Subject Media Association Object

Subject Association Object

📘
Application-Specific Subject-Media Data

📘
app_data Probability

Upload Media to a Subject

👍
Best Practice: False Consensus with an app_data

📘
Force Feedback

Example: Associate Media to a Subject

Gateway Subject Media

Retrieving Subject Media Associations

Example: Search for Subjects by Prefix

Deleting Subject Media Associations

🚧
Subject-Media-Focus

Example: Disassociate Media from a Subject

Subject Media Associations

Subject Media Association Object

Subject Association Object

📘Application-Specific Subject-Media Data

📘app_data Probability

Upload Media to a Subject

👍Best Practice: False Consensus with an app_data

📘Force Feedback

Example: Associate Media to a Subject

Gateway Subject Media

Retrieving Subject Media Associations

Example: Search for Subjects by Prefix

Deleting Subject Media Associations

🚧Subject-Media-Focus

Example: Disassociate Media from a Subject

📘
Application-Specific Subject-Media Data

📘
app_data Probability

👍
Best Practice: False Consensus with an app_data

📘
Force Feedback

🚧
Subject-Media-Focus