> This page is for Expression Measurement.

> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://dev.hume.ai/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://dev.hume.ai/_mcp/server.

# For coding agents

**This page gives a coding agent what it needs to build with the Expression Measurement API: what the API does, one complete request and response, the realtime session flow and the rules that code most often gets wrong.** Point your agent to this page before it writes code that calls the API.

## Set up your agent

Add this to your project's agent instructions, such as `CLAUDE.md` or `AGENTS.md`:

#### Agent instructions

```markdown wordWrap showLineNumbers={false}
Before writing code that calls the Expression Measurement API, read https://dev.hume.ai/expression-measurement/docs/for-coding-agents.md. For any other Expression Measurement page, find it in https://dev.hume.ai/expression-measurement/llms.txt and fetch its Markdown by adding .md to the page URL.
```

<table>
  <thead>
    <tr>
      <th width="25%">
        Resource
      </th>

      <th>
        Use
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        Markdown pages
      </td>

      <td>
        Add 

        `.md`

         to the URL of any page, such as 

        `https://dev.hume.ai/expression-measurement/docs/for-coding-agents.md`

        .
      </td>
    </tr>

    <tr>
      <td>
        `llms.txt`
      </td>

      <td>
        `https://dev.hume.ai/expression-measurement/llms.txt`

         lists every Expression Measurement page with a one-line description.
      </td>
    </tr>

    <tr>
      <td>
        MCP server
      </td>

      <td>
        `https://dev.hume.ai/_mcp/server`

         gives an MCP client a tool that searches all of dev.hume.ai and returns an answer with links to its sources. Fetch a cited page's 

        `.md`

         for exact field names and message shapes. In Claude Code, run 

        `claude mcp add --transport http hume-docs https://dev.hume.ai/_mcp/server`

        .
      </td>
    </tr>
  </tbody>
</table>

## What the API does

1. **The API measures emotional expression in speech and in faces.** The audio endpoints detect utterances in speech and measure each window of speech. The video endpoints detect faces in JPEG images and measure the largest faces in each one.
2. **Each measurement is lists of scores, not text.** A score pairs a `name`, such as `amusement`, with a `probability` from 0 to 1. Each array includes only the scores that pass its own cutoff.
3. **Upload endpoints measure media you already have, and realtime endpoints measure media as it streams.** An upload is one `multipart/form-data` request that returns every measurement in its response. A realtime session is a WebSocket that returns each measurement as soon as it is produced. For the same audio, both produce the same measurements.
4. **Every upload request and realtime session is recorded as a run.** The run endpoints return a run's status, configuration and event log, but not its measurements.

## Make a request

An audio upload is a `multipart/form-data` `POST` to `https://api.cloud.hume.ai/v1/expression/audio/file`, with the API key in the `X-Hume-Api-Key` header and up to two parts:

1. `file`: one audio file, as a 16 kHz mono 16-bit PCM WAV file or as headerless samples in that format.
2. `config`: an optional JSON object of settings, such as `measurement_timer_ms`, the interval between measurements.

#### cURL

```bash
curl https://api.cloud.hume.ai/v1/expression/audio/file \
  -H "X-Hume-Api-Key: $HUME_API_KEY" \
  -F "file=@speech.wav;type=audio/wav" \
  -F 'config={
    "measurement_timer_ms": 5000
  };type=application/json'
```

#### Python

```python
from hume_expression_measurement import AudioFileConfig, ExpressionMeasurementClient

client = ExpressionMeasurementClient()

with open("speech.wav", "rb") as audio:
    response = client.audio.measure(
        file=("speech.wav", audio, "audio/wav"),
        config=AudioFileConfig(measurement_timer_ms=5000),
    )

print(response.model_dump_json(indent=2))
```

#### Node.js

```javascript
import { ExpressionMeasurementClient } from "@humeai/expression-measurement-node";

const client = new ExpressionMeasurementClient();

const response = await client.audio.measure({
  file: {
    path: "speech.wav",
    contentType: "audio/wav",
  },
  config: {
    measurementTimerMs: 5000,
  },
});

console.log(JSON.stringify(response, null, 2));
```

The response lists every utterance with its span, and every measurement with the `utterance_id` it belongs to. Times are in milliseconds from the start of the audio. This response is for 5 seconds of audio containing one utterance:

#### 200 OK

```json
{
  "request_id": "3f9a1c07e2b84d56a0c7e19b5d2f8a64",
  "run_id": "01926f3c-d33c-7c9f-9c91-4502d80834d5",
  "received": {
    "audio_duration_ms": 5000
  },
  "utterances": [
    {
      "utterance_id": 0,
      "audio_start_ms": 98,
      "audio_end_ms": 4510
    }
  ],
  "measurements": [
    {
      "measurement_id": 0,
      "utterance_id": 0,
      "audio_start_ms": 98,
      "audio_end_ms": 3098,
      "trigger": "interval",
      "expressions": [
        {
          "name": "interest",
          "probability": 0.4286
        },
        {
          "name": "curiosity",
          "probability": 0.308
        }
      ],
      "voice_attributes": [
        {
          "name": "clear",
          "probability": 0.9612
        },
        {
          "name": "calm",
          "probability": 0.9148
        }
      ]
    },
    {
      "measurement_id": 1,
      "utterance_id": 0,
      "audio_start_ms": 98,
      "audio_end_ms": 4510,
      "trigger": "speech_ended",
      "expressions": [
        {
          "name": "amusement",
          "probability": 0.5
        },
        {
          "name": "joy",
          "probability": 0.3593
        }
      ],
      "voice_attributes": [
        {
          "name": "cheerful",
          "probability": 0.9275
        },
        {
          "name": "bright",
          "probability": 0.8864
        }
      ]
    }
  ]
}
```

An image upload to `https://api.cloud.hume.ai/v1/expression/video/file` takes one or more JPEG images as `file` parts, and returns one result per image with the faces found in it. See [Image upload](/expression-measurement/docs/video/upload).

<table>
  <thead>
    <tr>
      <th width="20%">
        SDK
      </th>

      <th>
        Package
      </th>

      <th>
        Client
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        Python
      </td>

      <td>
        `hume-expression-measurement`

        , imported as 

        `hume_expression_measurement`
      </td>

      <td>
        `ExpressionMeasurementClient()`

         reads the key from 

        `HUME_API_KEY`

        . 

        `AsyncExpressionMeasurementClient`

         has the same methods, awaited. A failed request raises 

        `ApiError`

         from 

        `hume_expression_measurement.core`

        .
      </td>
    </tr>

    <tr>
      <td>
        Node.js
      </td>

      <td>
        `@humeai/expression-measurement-node`
      </td>

      <td>
        `new ExpressionMeasurementClient()`

         reads the key from 

        `HUME_API_KEY`

        . Fields are camelCase, such as 

        `measurementTimerMs`

         and 

        `audioStartMs`

        .
      </td>
    </tr>
  </tbody>
</table>

## Stream in realtime

The realtime endpoints are `wss://api.cloud.hume.ai/v1/expression/audio/realtime` and `wss://api.cloud.hume.ai/v1/expression/video/realtime`. Both use the same session protocol:

1. **Open the WebSocket with the API key in the `X-Hume-Api-Key` header.** The server sends `session.created` with the `session_id`, which is also the session's run ID.
2. **To change the defaults, send `session.update` before the first media frame.** The server confirms with `session.updated`.
3. **Send media as binary frames.** On the audio endpoint, send headerless 16 kHz mono 16-bit PCM in frames of about 100 ms. On the video endpoint, send one JPEG image per frame.
4. **Read each `measurement.result`.** The audio endpoint also sends `utterance.start` and `utterance.end` around each stretch of speech. The video endpoint replies to every frame with a `measurement.result` or an `error`.
5. **Send `session.close`, then read until `session.closed` arrives.** Results still in progress are delivered before it, and it totals what the server received and produced.

See [Sessions](/expression-measurement/docs/sessions) for the full protocol.

## Score arrays

Each array's `probability` denotes a different quantity, and each array has its own cutoff.

<table>
  <thead>
    <tr>
      <th width="22%">
        Array
      </th>

      <th>
        `probability`

         is
      </th>

      <th width="30%">
        Included when
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        Voice 

        `expressions`
      </td>

      <td>
        The measured chance that the expression is truly present, according to human raters
      </td>

      <td>
        Judged present, against a threshold set for each expression
      </td>
    </tr>

    <tr>
      <td>
        Voice 

        `voice_attributes`
      </td>

      <td>
        Calibrated to the expected intensity a human rater would give the quality
      </td>

      <td>
        0.725 or above, except 

        `nasal`

         and 

        `breathy`

        , which are never included
      </td>
    </tr>

    <tr>
      <td>
        Face 

        `expressions`
      </td>

      <td>
        The chance a human rater would name the expression for this face
      </td>

      <td>
        Above 0.1
      </td>
    </tr>

    <tr>
      <td>
        Face 

        `descriptions`
      </td>

      <td>
        The chance that the facial action is present
      </td>

      <td>
        Above 0.1
      </td>
    </tr>
  </tbody>
</table>

## Rules for calling code

### Scores

1. **Treat each score as independent.** Each `probability` is its own estimate for one name. Scores do not sum to 1, several names can score highly at once and they are not meant to be compared with each other.
2. **Handle empty arrays.** An empty array means that no score passed the cutoff, so a measurement may list fewer scores than expected, or none.
3. **Do not hardcode the list of names.** New names may be added, so treat a name you do not recognize as a new prediction. The audio and video endpoints use different expression names.
4. **Read a face's `confidence` as detection, not expression.** It is the detector's confidence that the bounding box contains a face.
5. **Store the measurements you need.** The run endpoints return a run's status, configuration and event log, but not its measurements.

### Audio

1. **Send 16 kHz mono 16-bit PCM.** Audio is not resampled or downmixed. A WAV file in another format is rejected with `audio_decode_failed`, and MP3, FLAC and other containers with `unsupported_media_type`. Convert with `ffmpeg -i recording.mp3 -acodec pcm_s16le -ac 1 -ar 16000 speech.wav`.
2. **Keep an audio upload within 25 MB**, a little over 13 minutes of audio. A larger request is rejected with `audio_too_long`.
3. **Find where an utterance ends from `utterances` or `utterance.end`, not from `trigger`.** The final measurement of an utterance can be skipped, so its last measurement can have a `trigger` of `interval`.
4. **Stream audio no faster than 1 second of audio per second.** A faster frame can be rejected with `rate_limited`; wait briefly and send the same frame again. To measure a recording faster than it plays, upload it instead.

### Video

1. **Send JPEG images of up to 2 MB and 3840x2160 pixels.** Other formats are rejected.
2. **Handle unmeasured faces.** Only the largest faces are measured, 2 by default, and up to 32 are listed. A face that was not measured has `face_id`, `expressions` and `descriptions` set to `null`.
3. **Match image upload results to images by position.** `measurements` holds one result per image, in the order the `file` parts were sent.
4. **Match realtime replies to frames by position, not by `frame_id`.** Keep a queue of sent frames and remove the oldest for each `measurement.result`, and for each `error` whose code is `invalid_image_frame`, `rate_limited` or `internal_error`. `frame_id` advances only when a frame produces a result.
5. **Stream at most 3 frames per second.** A faster frame can be rejected with `rate_limited`. From a live camera, drop it and send the next frame. From a recording, wait briefly and send the same frame again.
6. **Do not use `face_id` to identify a person.** It links a face across the images of one request or session and restarts at 0 in each. To continue numbering across image uploads, pass `face_state` from one response to the next request.

### Sessions

1. **Include every setting you want in each `session.update`.** It replaces the configuration rather than merging into it. The configuration locks at the first media frame, and a later `session.update` or an unrecognized field ends the session with `config_invalid`.
2. **Close with `session.close` and read until `session.closed`.** Closing the WebSocket instead loses the results in progress.
3. **Keep media flowing.** A session closes after 30 seconds without a media frame the server can use. To pause for longer, end the session and open a new one.
4. **Open a new connection after any session ends.** A session cannot be resumed. A close that was not preceded by `session.closed` means the session was lost. The Node.js SDK's socket connects again after a server restart or a dropped connection, and so does the Python SDK's `reconnecting` helper. Each reconnect starts a new session whose IDs and times start again from 0.

### Requests and errors

1. **Keep the API key on a server you control.** Browsers cannot set the header on a WebSocket handshake, so relay media from a browser through your server.
2. **Send only documented fields.** A `config` part or `session.update` with an unrecognized field is rejected with `config_invalid`. In responses and messages, ignore fields you do not recognize.
3. **Branch on the error's `code`, and on the status when there is no `code`.** A `401` is answered before the request reaches the service and carries no `code`. A realtime handshake rejected with `401` or `403` sends no `error` message, so read the status from the handshake result.
4. **Fix a `4xx` before resending, retry a `500` and wait for the seconds in `Retry-After` before retrying a `503`.** A realtime `error` carries `retryable`, which is `false` for every error that ends the session. See [Errors](/expression-measurement/docs/errors).

## Where to go next

<table>
  <thead>
    <tr>
      <th width="45%">
        To
      </th>

      <th>
        Read
      </th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td>
        Run a first upload and session from a script
      </td>

      <td>
        [Python quickstart](/expression-measurement/docs/quickstart/python)

        , 

        [Node.js quickstart](/expression-measurement/docs/quickstart/nodejs)
      </td>
    </tr>

    <tr>
      <td>
        Upload or stream audio
      </td>

      <td>
        [Audio upload](/expression-measurement/docs/audio/upload)

        , 

        [Audio realtime](/expression-measurement/docs/audio/realtime)
      </td>
    </tr>

    <tr>
      <td>
        Upload images or stream video
      </td>

      <td>
        [Image upload](/expression-measurement/docs/video/upload)

        , 

        [Video realtime](/expression-measurement/docs/video/realtime)
      </td>
    </tr>

    <tr>
      <td>
        Find every name an array can contain
      </td>

      <td>
        [Audio measurements](/expression-measurement/docs/audio/measurements)

        , 

        [Video measurements](/expression-measurement/docs/video/measurements)
      </td>
    </tr>

    <tr>
      <td>
        Read scores and what they denote
      </td>

      <td>
        [Scores](/expression-measurement/docs/scores)
      </td>
    </tr>

    <tr>
      <td>
        Follow the realtime protocol
      </td>

      <td>
        [Sessions](/expression-measurement/docs/sessions)
      </td>
    </tr>

    <tr>
      <td>
        Check the exact request, response and message schemas
      </td>

      <td>
        [API reference](/expression-measurement/reference/audio/measure)
      </td>
    </tr>

    <tr>
      <td>
        Look up past requests and sessions
      </td>

      <td>
        [Runs](/expression-measurement/docs/runs)
      </td>
    </tr>

    <tr>
      <td>
        Check limits and handle errors
      </td>

      <td>
        [Limits](/expression-measurement/docs/limits)

        , 

        [Errors](/expression-measurement/docs/errors)
      </td>
    </tr>
  </tbody>
</table>