Image upload

The image upload endpoint measures emotional expression on faces in one or more JPEG images. Send the images in one request, and the response has one result per image, listing the faces found in it. A realtime session returns the same measurements for the same images.

Request

The request body has one or more file parts and an optional config part. Authenticate with a credential header, as described in Authentication.

The request below uploads two images, lowers the detection threshold, and asks for a face_state to continue face tracking in a later request.

curl https://api.cloud.hume.ai/v1/expression/video/file \
  -H "X-Hume-Api-Key: $HUME_API_KEY" \
  -F "[email protected];type=image/jpeg" \
  -F "[email protected];type=image/jpeg" \
  -F 'config={
    "face": {
      "threshold": 0.8,
      "return_face_state": true
    }
  };type=application/json'

Images

Each file part holds one complete JPEG image.

The server checks every image before measuring any. An image that is not a JPEG, or that exceeds the size or pixel limit, fails the request with unsupported_media_type or image_too_large. After earlier images are measured, an image that cannot be decoded can still fail the request with image_decode_failed. A request that fails in any of these ways returns no result for any image and is recorded as an errored run.

Detection

The server scans each image for faces and excludes any whose detection confidence is below face.threshold or whose bounding box is shorter than face.min_size pixels on its shorter side. It measures the largest of the remaining faces, up to 2 by default, and lists them first in faces, largest first. The faces that were not measured follow, most confident first, up to 32 faces in all.

A face that was not measured carries its bbox and confidence, with face_id, expressions, and descriptions set to null. The limit on measured faces is set for your organization and does not appear in responses. To raise it, contact support.

Response

The response below is for two images. Three faces appear in the first, and the two largest are measured. One of the measured faces remains in the second.

{
  "request_id": "8b27e4d1c05f4a93b6e0d7a2c91f3e58",
  "run_id": "01926f3e-397d-7ceb-9f52-a864c0102a44",
  "received": {
    "frames": 2
  },
  "measurements": [
    {
      "frame_id": 0,
      "faces": [
        {
          "face_id": 0,
          "bbox": [412.5, 118, 604.25, 371.75],
          "confidence": 0.96875,
          "expressions": [
            {
              "name": "amusement",
              "probability": 0.6231
            },
            {
              "name": "joy",
              "probability": 0.4418
            }
          ],
          "descriptions": [
            {
              "name": "smile",
              "probability": 0.8125
            }
          ]
        },
        {
          "face_id": 1,
          "bbox": [98, 140.25, 251.5, 348.75],
          "confidence": 0.9375,
          "expressions": [
            {
              "name": "concentration",
              "probability": 0.5108
            }
          ],
          "descriptions": [
            {
              "name": "jaw drop",
              "probability": 0.2266
            }
          ]
        },
        {
          "face_id": null,
          "bbox": [702, 160.5, 790.25, 276],
          "confidence": 0.90625,
          "expressions": null,
          "descriptions": null
        }
      ]
    },
    {
      "frame_id": 1,
      "faces": [
        {
          "face_id": 0,
          "bbox": [420.75, 121.5, 611, 374.25],
          "confidence": 0.953125,
          "expressions": [
            {
              "name": "amusement",
              "probability": 0.5742
            },
            {
              "name": "joy",
              "probability": 0.4016
            }
          ],
          "descriptions": [
            {
              "name": "smile",
              "probability": 0.7695
            }
          ]
        }
      ]
    }
  ],
  "face_state": "v1.Qm9keSBvZiB0aGUgc2VhbGVkIGlkZW50aXR5IGNhY2hlLCBiYXNlNjR1cmwgd2l0aG91dCBwYWRkaW5n"
}

Matching results to images

measurements holds one result per image, in the order the file parts were sent. Each result’s frame_id is the position of its image’s file part, starting at 0. An image with no faces still has a result, with an empty faces array.

Face tracking

face_id links the same face across the images of one request. IDs are assigned from 0 in the order faces are first measured, largest first within an image. A face normally keeps its ID while it is measured, and can get it back after it leaves the view or is no longer among the largest faces. The server remembers up to 32 faces, so a face that returns after many others have been measured may receive a new ID. IDs do not identify a person.

IDs restart at 0 in every request. To continue numbering across requests, set face.return_face_state to true, and pass the face_state from the response as face_state in the next request’s config.

{
  "face": {
    "return_face_state": true
  },
  "face_state": "v1.Qm9keSBvZiB0aGUgc2VhbGVkIGlkZW50aXR5IGNhY2hlLCBiYXNlNjR1cmwgd2l0aG91dCBwYWRkaW5n"
}

1. Treat face_state as opaque. Pass it back unchanged.
2. A state is accepted for 24 hours after it was issued. A state that has expired, was altered, or was issued to another organization is ignored, and numbering starts again from 0.

Configuration

To change the detection settings, send a config part. Fields you omit keep their defaults.

{
  "face": {
    "threshold": 0.8,
    "min_size": 40
  }
}

Errors

A failed request returns an error body with a code, a message, and a request_id. A 401 carries only a message, and a request_id when an API key was rejected. HTTP errors describes the format.

Next steps