{"openapi":"3.1.0","info":{"title":"Expression Measurement API","version":"1.0.0"},"paths":{"/v1/expression/audio/file":{"post":{"operationId":"audio_measure","summary":"Upload audio","description":"Send the audio as the `file` part and, to change the defaults, settings as a JSON `config` part. The response holds every utterance found in the audio and every measurement taken, the same results a realtime session would stream for the same audio.\n\n## How measurements are produced\n\nThe server detects when speech starts and stops. Each continuous stretch of speech is an **utterance**, listed in `utterances` with its span in the audio.\n\n- **During speech:** a measurement is taken every `measurement_timer_ms` milliseconds of speech, with `trigger` set to `interval`.\n- **When speech ends:** a final measurement is taken with `trigger` set to `speech_ended`, unless the previous measurement ended less than one second earlier. Every utterance gets at least one measurement, and audio that ends during speech ends the utterance.\n- **Span:** each measurement covers the most recent 10 seconds of the utterance, or the whole utterance if it is shorter, so consecutive measurements can overlap. `audio_start_ms` and `audio_end_ms` give the span, counted from the start of the audio.\n\n## Limits\n\n| Limit | Value |\n|---|---|\n| Request size | 25MB, the whole multipart body |\n| Audio format | 16-bit PCM, 16 kHz, mono, as a WAV file or headerless samples |\n\nThe server does not convert audio, so send it in the format above.\n\n- **Too large:** a request over the size limit is rejected with `audio_too_long`.\n- **Unsupported formats:** MP3, FLAC, Ogg, MP4 or M4A, and RIFF files other than WAV, are rejected with `unsupported_media_type`. Convert them before sending.\n- **Wrong WAV format:** a WAV file at another sample rate, channel count or sample format, one that uses the extensible WAV header, or one that is truncated or malformed is rejected with `audio_decode_failed`.\n- **Anything else** is read as headerless samples rather than rejected, so check the format before sending. An empty file, or one with an odd number of bytes, is rejected with `audio_decode_failed`.","tags":["audio"],"responses":{"200":{"description":"Every utterance and measurement in the audio.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AudioFileResponse"}}}},"400":{"description":"Something in the request needs correcting: a malformed body or parameter, or a value the endpoint does not accept. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"401":{"description":"The request carried no valid credential. Check the API key or access token you sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticationError"}}}},"403":{"description":"Your organization is not permitted to use this endpoint. Contact support.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"405":{"description":"The endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"413":{"description":"The request exceeds a size limit. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"415":{"description":"The media is in a format the endpoint does not accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"500":{"description":"The server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"503":{"description":"The service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}}},"requestBody":{"content":{"multipart/form-data":{"schema":{"type":"object","properties":{"config":{"$ref":"#/components/schemas/AudioFileConfig"},"file":{"type":"string","format":"binary","description":"The audio, as 16-bit PCM at 16 kHz, mono: either a WAV file or the headerless little-endian samples. A WAV file at another sample rate, channel count or sample format is rejected rather than converted."}},"required":["file"]}}}},"security":[{"ApiKey":[]}]}},"/v1/expression/video/file":{"post":{"operationId":"video_measure","summary":"Upload images","description":"Send one or more images, each as a `file` part, and, to change the defaults, settings as a JSON `config` part. The response holds one result per image, in the order the parts were sent. An image with no faces is a normal result with an empty `faces` array.\n\n## Detection and measurement\n\nThe server detects the faces in each image and lists them in `faces`. It measures the largest of them, up to the limit below, and lists those first, largest first.\n\n- A face is skipped, and is not listed, if its detection confidence is below `face.threshold` or its bounding box is under `face.min_size` pixels on its shorter side.\n- Each measured face carries its bounding box, its detection confidence, a `face_id`, and two sets of scores: `expressions` for emotional expressions such as amusement or interest, and `descriptions` for visible facial actions such as a smile or a jaw drop.\n- The faces that were not measured follow, most confident first, up to 32 faces in all. Each carries its bounding box and detection confidence, with `face_id`, `expressions` and `descriptions` null.\n- `face_id` tracks a face across the images of one request. A face normally keeps its ID while it is measured, and can get it back after it leaves the view or is no longer among the largest faces. IDs do not identify a person.\n- **Across requests:** IDs start again from 0 in every request. To keep them consistent, set `return_face_state` and pass the returned `face_state` as `face_state` in the next request.\n\n## Limits\n\n| Limit | Value |\n|---|---|\n| Request size | 25MB, the whole multipart body |\n| Image size | 2MB |\n| Pixels per image | 8,294,400 (the area of a 3840x2160 image) |\n| Faces measured per image | 2 by default, the largest; contact support to raise it |\n| Faces listed per image | 32 |\n\nEvery image is checked before any is measured. One bad image fails the whole request, which then returns no results:\n\n- **Not a JPEG:** rejected with `unsupported_media_type`.\n- **Over the image size or pixel limit:** rejected with `image_too_large`.\n- **Cannot be decoded:** rejected with `image_decode_failed`, possibly after earlier images were measured.","tags":["video"],"responses":{"200":{"description":"One result per image, in the order sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/VideoFileResponse"}}}},"400":{"description":"Something in the request needs correcting: a malformed body or parameter, or a value the endpoint does not accept. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"401":{"description":"The request carried no valid credential. Check the API key or access token you sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticationError"}}}},"403":{"description":"Your organization is not permitted to use this endpoint. Contact support.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"405":{"description":"The endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"413":{"description":"The request exceeds a size limit. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"415":{"description":"The media is in a format the endpoint does not accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"500":{"description":"The server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"503":{"description":"The service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}}},"requestBody":{"content":{"multipart/form-data":{"schema":{"type":"object","properties":{"config":{"$ref":"#/components/schemas/VideoFileConfig"},"file":{"type":"array","items":{"type":"string","format":"binary"},"description":"The images, one JPEG per `file` part. Results are numbered by the order of the parts."}},"required":["file"]}}}},"security":[{"ApiKey":[]}]}},"/v1/expression/run":{"get":{"operationId":"runs_list","summary":"List runs","description":"Your organization's runs, newest first, one page at a time. The filters combine.","tags":["runs"],"parameters":[{"name":"page_size","in":"query","description":"How many runs to return.","required":false,"schema":{"type":"integer","minimum":1,"maximum":200,"default":50}},{"name":"cursor","in":"query","description":"The `next_cursor` of the previous page. Pass it back unchanged.","required":false,"schema":{"type":"string"}},{"name":"user_id","in":"query","description":"Only runs made by this user.","required":false,"schema":{"type":"string"}},{"name":"status","in":"query","description":"Only runs with this status.","required":false,"schema":{"$ref":"#/components/schemas/RunStatus"}},{"name":"mode","in":"query","description":"Only runs that reached the API this way.","required":false,"schema":{"$ref":"#/components/schemas/RunMode"}},{"name":"endpoint","in":"query","description":"Only runs that measured this kind of media.","required":false,"schema":{"$ref":"#/components/schemas/RunEndpoint"}},{"name":"started_before","in":"query","description":"Only runs that started before this time, in RFC 3339 format.","required":false,"schema":{"type":"string","format":"date-time"}},{"name":"started_after","in":"query","description":"Only runs that started at or after this time, in RFC 3339 format. Consecutive time ranges that share a boundary neither skip a run nor return one twice.","required":false,"schema":{"type":"string","format":"date-time"}}],"responses":{"200":{"description":"One page of runs.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/RunsPage"}}}},"400":{"description":"Something in the request needs correcting: a malformed body or parameter, or a value the endpoint does not accept. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"401":{"description":"The request carried no valid credential. Check the API key or access token you sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticationError"}}}},"405":{"description":"The endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"500":{"description":"The server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"503":{"description":"The service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}}},"security":[{"ApiKey":[]}]}},"/v1/expression/run/{run_id}":{"get":{"operationId":"runs_get","summary":"Get run","description":"The run's status, when it started and ended, what it received and stored, and the configuration it ran under.","tags":["runs"],"parameters":[{"name":"run_id","in":"path","description":"The run's ID: the `session_id` of a realtime session or the `run_id` of a file response.","required":true,"schema":{"type":"string","format":"uuid"}}],"responses":{"200":{"description":"The run.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/Run"}}}},"401":{"description":"The request carried no valid credential. Check the API key or access token you sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticationError"}}}},"404":{"description":"No run with this ID exists in your organization.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"405":{"description":"The endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"500":{"description":"The server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"503":{"description":"The service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}}},"security":[{"ApiKey":[]}]}},"/v1/expression/run/{run_id}/events":{"get":{"operationId":"runs_list-events","summary":"Get run events","description":"The problems the run reported, in order: frames it rejected and failures it hit. A run that went well has none. Only realtime sessions record them.","tags":["runs"],"parameters":[{"name":"run_id","in":"path","description":"The run's ID: the `session_id` of a realtime session or the `run_id` of a file response.","required":true,"schema":{"type":"string","format":"uuid"}},{"name":"page_size","in":"query","description":"How many entries to return.","required":false,"schema":{"type":"integer","minimum":1,"maximum":200,"default":50}},{"name":"cursor","in":"query","description":"The `next_cursor` of the previous page. Pass it back unchanged.","required":false,"schema":{"type":"string"}}],"responses":{"200":{"description":"One page of the run's event log.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/RunEventsPage"}}}},"400":{"description":"Something in the request needs correcting: a malformed body or parameter, or a value the endpoint does not accept. `code` says which.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"401":{"description":"The request carried no valid credential. Check the API key or access token you sent.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/AuthenticationError"}}}},"404":{"description":"No run with this ID exists in your organization.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"405":{"description":"The endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"500":{"description":"The server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}},"503":{"description":"The service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","content":{"application/json":{"schema":{"$ref":"#/components/schemas/ErrorBody"}}}}},"security":[{"ApiKey":[]}]}}},"tags":[{"name":"audio","description":"Measure expression in an audio file."},{"name":"video","description":"Measure expression in JPEG images."},{"name":"runs","description":"The record of past runs: their status, configuration and problems."}],"servers":[{"url":"https://api.cloud.hume.ai","description":"Production"}],"components":{"schemas":{"AudioFileConfig":{"type":"object","properties":{"measurement_timer_ms":{"type":"integer","minimum":3000,"maximum":10000,"default":3000,"description":"How often to measure within an utterance, in milliseconds of speech."}},"description":"Settings for an audio request, sent as the JSON `config` part.","title":"AudioFileConfig"},"AudioFileResponseReceived":{"type":"object","properties":{"audio_duration_ms":{"type":"integer","minimum":0,"description":"The duration of the audio, in milliseconds."}},"required":["audio_duration_ms"],"description":"What the server took from the request.","title":"AudioFileResponseReceived"},"AudioFileUtterance":{"type":"object","properties":{"utterance_id":{"type":"integer","minimum":0,"description":"Identifies the utterance within the run, counting from 0."},"audio_start_ms":{"type":"integer","minimum":0,"description":"Where the utterance started, in milliseconds from the start of the audio."},"audio_end_ms":{"type":"integer","minimum":0,"description":"Where the utterance ended, in milliseconds from the start of the audio."}},"required":["utterance_id","audio_start_ms","audio_end_ms"],"description":"An utterance: one continuous stretch of speech, with its full extent in the audio.","title":"AudioFileUtterance"},"MeasurementTrigger":{"type":"string","enum":["interval","speech_ended"],"description":"Why the measurement was produced.\n\n- `interval`: the measurement interval (`measurement_timer_ms`) elapsed while speech continued.\n- `speech_ended`: the utterance ended, or the audio ended while speech was in progress.\n\nA final measurement is skipped when the previous one ended less than one second earlier, so the last measurement of an utterance may carry the trigger `interval`. Where an utterance ended is reported separately: as an `utterance.end` message in a realtime session, and in `utterances` in an audio file response.","title":"MeasurementTrigger"},"AudioExpressionName":{"type":"string","description":"The name of an expression that can be detected in speech; see the [full list of audio expressions](/expression-measurement/docs/audio/measurements#expressions). Names are lowercase and may contain spaces and punctuation. New names may be added over time; treat a name you do not recognize as a new prediction.","title":"AudioExpressionName"},"AudioExpressionScore":{"type":"object","properties":{"name":{"$ref":"#/components/schemas/AudioExpressionName"},"probability":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"The probability that this expression is present, rounded to four decimal places."}},"required":["name","probability"],"description":"A scored expression detected in the speech.","title":"AudioExpressionScore"},"VoiceAttributeName":{"type":"string","description":"The name of a quality of the voice or the audio, such as a manner of speaking, a vocal timbre or a background sound; see the [full list of voice attributes](/expression-measurement/docs/audio/measurements#voice-attributes). Names are lowercase and may contain spaces and punctuation. New names may be added over time; treat a name you do not recognize as a new prediction.","title":"VoiceAttributeName"},"VoiceAttributeScore":{"type":"object","properties":{"name":{"$ref":"#/components/schemas/VoiceAttributeName"},"probability":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"The probability that this quality is present, rounded to four decimal places."}},"required":["name","probability"],"description":"A scored quality of the voice or the audio.","title":"VoiceAttributeScore"},"AudioFileMeasurement":{"type":"object","properties":{"measurement_id":{"type":"integer","minimum":0,"description":"Sequential identifier of the measurement within the run, starting at 0."},"utterance_id":{"type":"integer","minimum":0,"description":"The utterance this measurement belongs to."},"audio_start_ms":{"type":"integer","minimum":0,"description":"Start of the window measured, in milliseconds from the start of the audio. Never earlier than the start of the utterance."},"audio_end_ms":{"type":"integer","minimum":0,"description":"End of the window measured, in milliseconds from the start of the audio."},"trigger":{"$ref":"#/components/schemas/MeasurementTrigger"},"expressions":{"type":"array","items":{"$ref":"#/components/schemas/AudioExpressionScore"},"description":"Expressions detected in the speech, such as `joy`, `anger` and `chuckling`, in descending order of probability. Scores with equal probability are ordered by the model's underlying score, then by name. Only expressions judged present are included, each against a threshold set for that expression, so the array may be empty. `probability` is the measured chance that the expression is truly present."},"voice_attributes":{"type":"array","items":{"$ref":"#/components/schemas/VoiceAttributeScore"},"description":"Qualities of the voice and the audio, such as `husky`, `fast` and `background music`, in descending order of probability. Scores with equal probability are ordered by the model's underlying score, then by name. Only scores of 0.725 or above are included, so the array may be empty. `probability` is calibrated: the expected intensity a human rater would give the quality."}},"required":["measurement_id","utterance_id","audio_start_ms","audio_end_ms","trigger","expressions","voice_attributes"],"description":"Expression measurements for a window of an utterance. The window spans `audio_start_ms` to `audio_end_ms` and covers up to the most recent 10 seconds of the utterance. An interval measurement taken as speech stops can end shortly after the utterance does. Consecutive measurements of an utterance overlap, except interval measurements at a `measurement_timer_ms` of 10000, which meet end to end.","title":"AudioFileMeasurement"},"AudioFileResponse":{"type":"object","properties":{"request_id":{"type":"string","description":"Identifies this request in Hume's records. Quote it when contacting support."},"run_id":{"type":"string","format":"uuid","description":"Identifies the run this request was recorded as. Pass it to the run endpoints to look the run up."},"received":{"$ref":"#/components/schemas/AudioFileResponseReceived","description":"What the server took from the request."},"utterances":{"type":"array","items":{"$ref":"#/components/schemas/AudioFileUtterance"},"description":"Every utterance found, in order."},"measurements":{"type":"array","items":{"$ref":"#/components/schemas/AudioFileMeasurement"},"description":"Every measurement, in order. `utterance_id` says which utterance each belongs to."}},"required":["request_id","run_id","received","utterances","measurements"],"description":"The response to an audio request.","title":"AudioFileResponse"},"ErrorBodyType":{"type":"string","enum":["error"],"title":"ErrorBodyType"},"RequestErrorCode":{"type":"string","enum":["message_invalid","config_invalid","audio_decode_failed","image_decode_failed","invalid_request","forbidden","not_found","method_not_allowed","audio_too_long","image_too_large","unsupported_media_type","internal_error","server_busy"],"description":"Identifies what went wrong. The HTTP status carried with it says how the request failed.\n\nSome of these names are also error codes of the realtime API, where they name the same kind of problem.\n\n**`400 Bad Request`**, something in the request needs correcting.\n\n- `message_invalid`: the multipart body could not be read, is missing its `file` part, or carries a part with another name. Audio takes exactly one `file` part; video takes one or more. File endpoints only.\n- `config_invalid`: the `config` part is not valid JSON, names a field the endpoint does not recognize, holds a value outside its range, or appears more than once. File endpoints only.\n- `audio_decode_failed`: the audio could not be decoded: a WAV file that is truncated or malformed, or at another sample rate, channel count or sample format, or with the extensible WAV header, or a file that is empty or is not a whole number of 16-bit samples. Audio only.\n- `image_decode_failed`: an image could not be decoded. Video only.\n- `invalid_request`: a query parameter has a value the endpoint does not accept, or `cursor` is not one the endpoint issued. Run endpoints only.\n\n**`403 Forbidden`**\n\n- `forbidden`: your organization is not permitted to use this endpoint. Contact support. File endpoints only.\n\n**`404 Not Found`**\n\n- `not_found`: no run with this ID exists in your organization, or no endpoint exists at this path.\n\n**`405 Method Not Allowed`**\n\n- `method_not_allowed`: the endpoint does not accept this HTTP method. The `Allow` header lists the methods it does accept.\n\n**`413 Content Too Large`**, the request exceeds a size limit.\n\n- `audio_too_long`: the request body is over the size limit. Audio only.\n- `image_too_large`: the request body or one image is over the size limit, or an image has more pixels than the endpoint accepts. Video only.\n\n**`415 Unsupported Media Type`**, the media is in a format the endpoint does not accept.\n\n- `unsupported_media_type`: the audio is in a format the server recognizes but does not accept, such as MP3 or FLAC, or an image is not a JPEG. File endpoints only.\n\n**`500 Internal Server Error`**\n\n- `internal_error`: the server failed while serving the request. Retry it; if it keeps failing, contact support, quoting the `request_id`.\n\n**`503 Service Unavailable`**\n\n- `server_busy`: the service is too busy to serve the request now. Retry it after the number of seconds in the `Retry-After` header.","title":"RequestErrorCode"},"ErrorBody":{"type":"object","properties":{"type":{"$ref":"#/components/schemas/ErrorBodyType"},"code":{"$ref":"#/components/schemas/RequestErrorCode"},"message":{"type":"string","description":"What went wrong, for a person. Its wording may change; switch on `code`."},"request_id":{"type":"string","description":"Identifies the request in Hume's records. Quote it when contacting support."}},"required":["type","code","message","request_id"],"description":"The body of every error response other than a `401`. `type` is always `error`; the HTTP status says how the request failed and `code` says why.","title":"ErrorBody"},"AuthenticationError":{"type":"object","properties":{"message":{"type":"string","description":"What went wrong, for a person."},"request_id":{"type":"string","description":"Identifies the request. Sent only when an API key was rejected."}},"required":["message"],"description":"The body of a `401` response, sent when the request carried no valid API key or access token. It is answered before the request reaches the service, so it carries neither `type` nor `code`.","title":"AuthenticationError"},"VideoFileConfigFace":{"type":"object","properties":{"threshold":{"type":"number","format":"double","minimum":0,"maximum":1,"default":0.9,"description":"Minimum detection confidence for a face to be measured. Lower values include more faces at the risk of false detections. This scale is specific to this API; thresholds tuned for other face detection APIs do not carry over."},"min_size":{"type":"integer","minimum":1,"maximum":4294967295,"default":60,"description":"Minimum face size to measure, as the shorter side of the face's bounding box in pixels of the submitted image."},"return_face_state":{"type":"boolean","default":false,"description":"Return a `face_state` with the response, so a later request can continue `face_id` numbering from the faces this one saw."}},"description":"Face detection settings.","title":"VideoFileConfigFace"},"VideoFileConfig":{"type":"object","properties":{"face":{"$ref":"#/components/schemas/VideoFileConfigFace","description":"Face detection settings."},"face_state":{"type":"string","description":"The `face_state` from an earlier response, so `face_id` continues from the faces that request saw. A state is accepted for 24 hours after it was issued. One that has expired, was altered, or was issued to another organization is ignored, and numbering starts again from 0."}},"description":"Settings for a video request, sent as the JSON `config` part.","title":"VideoFileConfig"},"VideoFileResponseReceived":{"type":"object","properties":{"frames":{"type":"integer","minimum":0,"description":"The number of images measured."}},"required":["frames"],"description":"What the server took from the request.","title":"VideoFileResponseReceived"},"BoundingBox":{"type":"array","items":{"type":"number","format":"double"},"description":"The location of the face in the submitted image, as `[x0, y0, x1, y1]`: the top-left and bottom-right corners in pixels, with the origin at the top-left of the image. Coordinates are in the pixels as stored in the file: EXIF orientation is not applied.","title":"BoundingBox"},"VideoExpressionName":{"type":"string","description":"The name of an expression that can be detected on a face, from a vocabulary that differs from that of audio expressions; see the [full list of video expressions](/expression-measurement/docs/video/measurements#expressions). Names are lowercase and may contain spaces and punctuation. New names may be added over time; treat a name you do not recognize as a new prediction.","title":"VideoExpressionName"},"VideoExpressionScore":{"type":"object","properties":{"name":{"$ref":"#/components/schemas/VideoExpressionName"},"probability":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"The probability that this expression is present, rounded to four decimal places."}},"required":["name","probability"],"description":"A scored expression detected on the face.","title":"VideoExpressionScore"},"VideoDescriptionName":{"type":"string","description":"The name of a visible facial action; see the [full list of video descriptions](/expression-measurement/docs/video/measurements#descriptions). Names are lowercase and may contain spaces and punctuation. New names may be added over time; treat a name you do not recognize as a new prediction.","title":"VideoDescriptionName"},"VideoDescriptionScore":{"type":"object","properties":{"name":{"$ref":"#/components/schemas/VideoDescriptionName"},"probability":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"The probability that this action is present, rounded to four decimal places."}},"required":["name","probability"],"description":"A scored facial action.","title":"VideoDescriptionScore"},"Face":{"type":"object","properties":{"face_id":{"type":["integer","null"],"minimum":0,"description":"Tracks this face across the images of a realtime session or a file request. IDs are assigned from 0 in the order faces are first measured, largest first within an image. A face normally keeps its ID while it is measured, and can get it back after it leaves the view or is no longer among the largest faces. The server remembers up to 32 faces, so a face that returns after many others have been measured may receive a new ID. IDs restart in every session and every file request, unless a file request continues them with a `face_state`, and do not identify a person. Null for a face that was not measured."},"bbox":{"$ref":"#/components/schemas/BoundingBox"},"confidence":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"Confidence that the bounding box contains a face. Always at or above the configured `threshold`. Unrelated to the expression scores."},"expressions":{"type":["array","null"],"items":{"$ref":"#/components/schemas/VideoExpressionScore"},"description":"Expressions detected on the face, such as `amusement`, `interest` and `surprise (positive)`, in descending order of probability, then by name. Only scores above 0.1 are included, so the array may be empty. `probability` is the chance a human rater would name the expression for this face, independently for each expression. Null for a face that was not measured."},"descriptions":{"type":["array","null"],"items":{"$ref":"#/components/schemas/VideoDescriptionScore"},"description":"Visible facial actions, such as `smile`, `jaw drop` and `wide-eyed`, in descending order of probability, then by name. Only scores above 0.1 are included, so the array may be empty. Null for a face that was not measured."}},"required":["face_id","bbox","confidence","expressions","descriptions"],"description":"A face detected in the image, with its expression measurements if it was measured. Only the largest faces in an image are measured; the others carry their bounding box and detection confidence, with `face_id`, `expressions` and `descriptions` null.","title":"Face"},"VideoFileMeasurement":{"type":"object","properties":{"frame_id":{"type":"integer","minimum":0,"description":"Identifies the image within the run, counting from 0. A file request numbers its images by the order of the `file` parts; a realtime session numbers only the frames it answered with a measurement."},"faces":{"type":"array","items":{"$ref":"#/components/schemas/Face"},"description":"The faces detected in the image: the measured faces first, largest first, then the faces that were not measured, in descending order of confidence."}},"required":["frame_id","faces"],"description":"Expression measurements for every face in one image. `faces` is empty when no face was detected; this is a normal result, not an error.","title":"VideoFileMeasurement"},"VideoFileResponse":{"type":"object","properties":{"request_id":{"type":"string","description":"Identifies this request in Hume's records. Quote it when contacting support."},"run_id":{"type":"string","format":"uuid","description":"Identifies the run this request was recorded as. Pass it to the run endpoints to look the run up."},"received":{"$ref":"#/components/schemas/VideoFileResponseReceived","description":"What the server took from the request."},"measurements":{"type":"array","items":{"$ref":"#/components/schemas/VideoFileMeasurement"},"description":"One result per image, in the order the `file` parts were sent. An image with no faces still has a result, with `faces` empty."},"face_state":{"type":"string","description":"Present when `return_face_state` was set. Pass it unchanged as `face_state` in a later request, within 24 hours, to continue `face_id` numbering."}},"required":["request_id","run_id","received","measurements"],"description":"The response to a video request.","title":"VideoFileResponse"},"RunStatus":{"type":"string","enum":["live","ended","errored"],"description":"Where a run is in its life: `live` while it is in progress, `ended` once it completed, and `errored` if it ended because of a failure or because the connection was lost. A live run the server has not heard from in five minutes is also reported `errored`, and reads `live` again if the server hears from it.","title":"RunStatus"},"RunMode":{"type":"string","enum":["realtime","file"],"description":"How the media reached the API: `realtime` over a WebSocket session, or `file` in a single request.","title":"RunMode"},"RunEndpoint":{"type":"string","enum":["audio","video"],"description":"What the run measured: `audio` for audio sent to an audio endpoint, `video` for images sent to a video endpoint.","title":"RunEndpoint"},"RunSummary":{"type":"object","properties":{"run_id":{"type":"string","format":"uuid","description":"The run's ID: the `session_id` of a realtime session, or the `run_id` of a file response."},"user_id":{"type":"string","description":"The ID of the Hume user who made the run."},"status":{"$ref":"#/components/schemas/RunStatus"},"endpoint":{"$ref":"#/components/schemas/RunEndpoint","description":"What the run measured."},"mode":{"$ref":"#/components/schemas/RunMode","description":"How the media reached the API."},"started_at":{"type":"string","format":"date-time","description":"When the run started."},"ended_at":{"type":["string","null"],"format":"date-time","description":"When the run ended. Null while it is in progress. For a run reported `errored` because the server stopped hearing from it, the last time the server heard from it."},"bytes_received":{"type":["integer","null"],"description":"Bytes of media the server took in. A realtime video frame counts once it passes the send-rate check, even if it is then rejected as invalid. Null until the run ends, and for a run whose end was never recorded."},"bytes_recorded":{"type":["integer","null"],"description":"How far the server got in storing the run's media, in bytes. Media that could not be stored is included, so this can exceed what was kept. Null when the run's media was not being stored, or when the server has not reported how much it stored, as early in a live run."},"metered_seconds":{"type":"number","format":"double","minimum":0,"description":"Billable seconds for the run. For audio, this is the duration of the audio the server took in. For video, each frame or image the server took in counts as one third of a second, regardless of frame rate, including a realtime frame rejected as invalid after it passed the send-rate check. A run that took in any media is billed at least 10 seconds, applied when the run ends. Updated while a realtime run is in progress. A file request that fails is not billed, and a run recorded for it reports 0."}},"required":["run_id","user_id","status","endpoint","mode","started_at","ended_at","bytes_received","bytes_recorded","metered_seconds"],"description":"A run, as the list returns it.","title":"RunSummary"},"RunsPage":{"type":"object","properties":{"runs":{"type":"array","items":{"$ref":"#/components/schemas/RunSummary"},"description":"The runs, newest first."},"next_cursor":{"type":"string","description":"Absent on the last page. Pass it back as `cursor` for the next one."}},"required":["runs"],"description":"One page of runs.","title":"RunsPage"},"AudioConfig":{"type":"object","properties":{"type":{"type":"string","enum":["audio/pcm"]},"encoding":{"type":"string","enum":["s16le"],"description":"The sample encoding. `s16le` is signed 16-bit little-endian PCM."},"sample_rate":{"type":"integer","description":"The sample rate in Hz."},"channels":{"type":"integer"}},"required":["type","encoding","sample_rate","channels"],"description":"The audio format.","title":"AudioConfig"},"AudioRunConfig":{"type":"object","properties":{"schema":{"type":"string","enum":["v4"],"description":"Identifies the layout of this configuration. Check it if you store configurations, so one in a later layout is recognized rather than misread."},"audio":{"$ref":"#/components/schemas/AudioConfig"},"measurement_timer_ms":{"type":"integer","minimum":3000,"maximum":10000,"description":"How often the run measured within an utterance, in milliseconds of speech."}},"required":["schema","audio","measurement_timer_ms"],"description":"The configuration an audio run ran under.","title":"AudioRunConfig"},"ImageConfig":{"type":"object","properties":{"type":{"type":"string","enum":["image/jpeg"]}},"required":["type"],"description":"The image format.","title":"ImageConfig"},"FaceConfig":{"type":"object","properties":{"threshold":{"type":"number","format":"double","minimum":0,"maximum":1,"description":"Minimum detection confidence for a face to be measured. Lower values include more faces at the risk of false detections. This scale is specific to this API; thresholds tuned for other face detection APIs do not carry over. Reported rounded to four decimal places."},"min_size":{"type":"integer","minimum":1,"maximum":4294967295,"description":"Minimum face size to measure, as the shorter side of the face's bounding box in pixels of the submitted image."}},"required":["threshold","min_size"],"description":"The face detection settings.","title":"FaceConfig"},"VideoRunConfig":{"type":"object","properties":{"schema":{"type":"string","enum":["v4"],"description":"Identifies the layout of this configuration. Check it if you store configurations, so one in a later layout is recognized rather than misread."},"image":{"$ref":"#/components/schemas/ImageConfig"},"face":{"$ref":"#/components/schemas/FaceConfig"}},"required":["schema","image","face"],"description":"The configuration a video run ran under.","title":"VideoRunConfig"},"RunConfig":{"oneOf":[{"$ref":"#/components/schemas/AudioRunConfig"},{"$ref":"#/components/schemas/VideoRunConfig"}],"description":"The configuration the run ran under, including defaults for any settings the request did not set.","title":"RunConfig"},"Run":{"type":"object","properties":{"run_id":{"type":"string","format":"uuid","description":"The run's ID: the `session_id` of a realtime session, or the `run_id` of a file response."},"user_id":{"type":"string","description":"The ID of the Hume user who made the run."},"status":{"$ref":"#/components/schemas/RunStatus"},"endpoint":{"$ref":"#/components/schemas/RunEndpoint","description":"What the run measured."},"mode":{"$ref":"#/components/schemas/RunMode","description":"How the media reached the API."},"started_at":{"type":"string","format":"date-time","description":"When the run started."},"ended_at":{"type":["string","null"],"format":"date-time","description":"When the run ended. Null while it is in progress. For a run reported `errored` because the server stopped hearing from it, the last time the server heard from it."},"close_code":{"type":["integer","null"],"description":"The WebSocket close code the session ended with, or 1011 if the connection was lost without a close frame. Only realtime runs that have ended have one, and not all of those: it is absent when the client's close frame carried no code, or when the server stopped before recording how the session closed."},"close_reason":{"type":["string","null"],"description":"Why the session closed: `client_request`, `server_shutdown`, `idle_timeout` or `error` when the server closed it, `error` when the connection was lost without a close frame, and otherwise the reason text from the client's close frame. Only realtime runs that have ended have one, and not all of those: it is absent when the client's close frame carried no code, or when the server stopped before recording how the session closed."},"bytes_received":{"type":["integer","null"],"description":"Bytes of media the server took in. A realtime video frame counts once it passes the send-rate check, even if it is then rejected as invalid. Null until the run ends, and for a run whose end was never recorded."},"bytes_recorded":{"type":["integer","null"],"description":"How far the server got in storing the run's media, in bytes. Media that could not be stored is included, so this can exceed what was kept. Null when the run's media was not being stored, or when the server has not reported how much it stored, as early in a live run."},"metered_seconds":{"type":"number","format":"double","minimum":0,"description":"Billable seconds for the run. For audio, this is the duration of the audio the server took in. For video, each frame or image the server took in counts as one third of a second, regardless of frame rate, including a realtime frame rejected as invalid after it passed the send-rate check. A run that took in any media is billed at least 10 seconds, applied when the run ends. Updated while a realtime run is in progress. A file request that fails is not billed, and a run recorded for it reports 0."},"config":{"$ref":"#/components/schemas/RunConfig","description":"The configuration the run ran under, including defaults for any settings the request did not set."}},"required":["run_id","user_id","status","endpoint","mode","started_at","ended_at","close_code","close_reason","bytes_received","bytes_recorded","metered_seconds","config"],"description":"A run in full.","title":"Run"},"RunEventKind":{"type":"string","enum":["invalid_audio_frame","invalid_image_frame","rate_limited","inference_failed","message_too_large","video_recording_abandoned"],"description":"What a run's event log records. Every kind is something the run refused or that went wrong, so a run that went well has no events.\n\n- `invalid_audio_frame`: an audio frame was rejected. Audio only.\n- `invalid_image_frame`: an image frame was rejected, or face detection failed on a frame and the session reported `internal_error`. Video only.\n- `rate_limited`: media arrived faster than the send rate and a frame was rejected.\n- `inference_failed`: a frame was skipped because its measurement failed, which the session reported as `internal_error`. Video only.\n- `message_too_large`: a message exceeded the size limit and the session ended.\n- `video_recording_abandoned`: the server stopped storing the run's frames or images; results were still delivered. Video only, and only on runs recorded before the server stopped emitting it.\n\nOnly realtime sessions record events. A file request that hits any other problem fails as a whole and reports it in the response instead.","title":"RunEventKind"},"RunEventDetail":{"type":"object","properties":{},"description":"Details, depending on `kind`: the reason a frame was rejected (`detail`), the size limit a message exceeded (`limit_bytes`), whether a measurement failure was temporary and how many failed in a row (`retryable`, `consecutive`), or why recording stopped (`reason`). Absent for `rate_limited`.","title":"RunEventDetail"},"RunEvent":{"type":"object","properties":{"seq":{"type":"integer","minimum":1,"description":"The entry's position in the log."},"at":{"type":"string","format":"date-time","description":"When the event was recorded."},"kind":{"$ref":"#/components/schemas/RunEventKind"},"detail":{"$ref":"#/components/schemas/RunEventDetail","description":"Details, depending on `kind`: the reason a frame was rejected (`detail`), the size limit a message exceeded (`limit_bytes`), whether a measurement failure was temporary and how many failed in a row (`retryable`, `consecutive`), or why recording stopped (`reason`). Absent for `rate_limited`."}},"required":["seq","at","kind"],"description":"One entry in a run's event log.","title":"RunEvent"},"RunEventsPage":{"type":"object","properties":{"events":{"type":"array","items":{"$ref":"#/components/schemas/RunEvent"},"description":"The entries, in `seq` order."},"at_cap":{"type":"boolean","description":"Whether some kind of event used up its per-run logging limit. Any later events of that kind were not logged, though there may have been none. When `gaps` is true, a false value cannot be relied on. `capped_kinds` names the kinds."},"capped_kinds":{"type":"array","items":{"$ref":"#/components/schemas/RunEventKind"},"description":"The kinds that used up their limit. Empty when none did."},"gaps":{"type":"boolean","description":"Whether entries are missing from the log: events that were numbered and never stored."},"next_cursor":{"type":"string","description":"Absent on the last page. Pass it back as `cursor` for the next one."}},"required":["events","at_cap","capped_kinds","gaps"],"description":"One page of a run's event log.","title":"RunEventsPage"}},"securitySchemes":{"ApiKey":{"type":"apiKey","in":"header","name":"X-Hume-Api-Key"}}}}