Evaluating many files

To evaluate many recordings, send one request per file, a few at a time, and save each response as soon as it arrives. A job that stops partway can then be run again and pick up where it left off, without paying twice for the files that already have answers.

Plan the job

1. Send every file the same questions in the same order. All the questions in a request are answered together, so an answer can change when other questions are added, removed or reordered. Answers are comparable across files only when the request is the same.
2. Try the questions on a few files you know before running the whole set. See Check your questions on audio you know.
3. Estimate the input tokens with Estimate input tokens. It takes the request’s context and questions with an audio duration in milliseconds, and returns an estimate of the input tokens for a file of that duration. Each decisions response gives the input tokens it is billed as received.input_tokens, which can differ slightly from the estimate. The estimate counts toward the rate limit like any other request.

Prepare the files

A request takes one MP3 or WAV file of 5 seconds to 5 minutes and up to 25,000,000 bytes. See Limits.

The model hears all audio as 16 kHz, 16-bit, one channel. A higher sample rate, a greater bit depth or a second channel does not improve answers, and only makes the file larger. 5 minutes of 16 kHz, 16-bit mono WAV is 9.6 MB, well under the size limit.

The script below converts every file in recordings/ to 16 kHz, 16-bit mono WAV, and splits any recording longer than 5 minutes into parts. It writes the files to audio/, named after the recording with a part number, along with a CSV file for each recording that lists its parts with their start and end times in seconds. ffmpeg can end a part slightly after -segment_time, so 299 seconds keeps every part under 5 minutes. The script cuts at fixed times and mixes the channels of a stereo recording into one. To cut at pauses or turn ends, or to send one speaker’s channel, see Preparing your audio. The last part of a recording can be shorter than 5 seconds, which is rejected with audio_too_short; skip it, or split the recording at a different length so that no part is that short.

mkdir -p audio
for recording in recordings/*; do
  name=$(basename "${recording%.*}")
  ffmpeg -i "$recording" -ac 1 -ar 16000 -sample_fmt s16 \
    -f segment -segment_time 299 \
    -segment_list "audio/$name.csv" \
    "audio/${name}_%03d.wav"
done

Long recordings

The model answers each part of a split recording as if it were the whole recording, and nothing combines the answers across parts.

1. Write questions that hold for any part. “Does the agent talk over the caller?” can be answered from any 5 minutes of a call. “Is the caller’s issue resolved by the end of the call?” can be answered only from the last part.
2. Place spans in the recording. A span’s times are measured from the start of its part, so add the part’s start time from the CSV file.
3. Combine the answers for each question in your code. For example, to learn whether something happens at any point in the call, take the highest noul across the parts.

The Python and TypeScript SDKs can split a recording and send each part for you. See Long recordings in the Python SDK README and the TypeScript SDK README.

Send the requests

Every API key in an organization shares a limit of 60 requests per minute. See Limits. Cap how many requests run at once, so that a job stays under the limit and leaves room for other work in your organization. The scripts below send 4 requests at a time. If many requests return 429, lower -P in the cURL script or CONCURRENCY in the Python and TypeScript scripts.

Each script sends every WAV file in audio/ and saves each response to results/, named after the file. A file whose result already exists is skipped, so running a script again sends only the files that have no answers. The scripts read your API key from HUME_API_KEY. For the cURL script, it must be exported, so the commands that xargs starts can read it.

cat > request.json <<'EOF'
{
  "context": "A call in which a voice AI agent helps a caller reschedule an appointment.",
  "questions": {
    "talks_over_caller": {
      "type": "noul",
      "instructions": "Does the agent talk over the caller?"
    },
    "agent_naturalness": {
      "type": "score",
      "instructions": "How natural does the agent sound?",
      "criteria": ["Not at all", "2", "3", "4", "Extremely"]
    }
  }
}
EOF
mkdir -p results
find audio -name '*.wav' -print0 | xargs -0 -n 1 -P 4 sh -c '
  name=$(basename "$1" .wav)
  [ -f "results/$name.json" ] && exit 0
  status=$(curl -sS --retry 2 \
    -H "X-Hume-Api-Key: $HUME_API_KEY" \
    -F "request=<request.json;type=application/json" \
    -F "file=@$1" \
    -o "results/$name.response" \
    -w "%{http_code}" \
    https://api.cloud.hume.ai/v0/decisions/file)
  if [ "$status" = 200 ]; then
    mv "results/$name.response" "results/$name.json"
    rm -f "results/$name.error.json"
    exit 0
  fi
  [ -f "results/$name.response" ] && mv "results/$name.response" "results/$name.error.json"
  echo "$1: HTTP $status" >&2
  case "$status" in 401|402|403) exit 255 ;; esac
' sh

Handle failed requests

What to do with a failed request depends on its status. See Handling errors.

1. 429, 503 and 500 are retried. curl --retry 2 and both SDKs retry a request twice, waiting the number of seconds in the Retry-After header when the response has one. For a long job, allow more retries with --retry 5 in cURL, request_options={"max_retries": 5} in Python, or { maxRetries: 5 } as the second argument to create in TypeScript.
2. 400, 413 and 415 fail again if the file is sent unchanged. The scripts save the error body to results/<file>.error.json and move on to the next file. Its code and param name what to correct, and its request_id identifies the request when you contact support.
3. 401, 402 and 403 stop the script, since every remaining request would fail the same way. Correct the credential or the organization’s account, then run the script again.
4. A timeout or a failed connection has no status and no error body. The scripts print the error for the file and move on to the next file.

A file whose request failed, after its retries or with a timeout, has no result, so running the script again sends it again.

Store the results

Each result holds the run_id, the answers and the billed input tokens for one file:

{
  "run_id": "01a18e2a-b2c0-71e8-8d58-92d4c14a8125",
  "answers": {
    "talks_over_caller": {
      "type": "noul",
      "noul": 0.0191
    },
    "agent_naturalness": {
      "type": "score",
      "score": 2.0616,
      "confidence": 0.1275,
      "probabilities": {
        "0": 0.1585,
        "1": 0.1756,
        "2": 0.2362,
        "3": 0.305,
        "4": 0.1246
      },
      "legend": {
        "0": "Not at all",
        "1": "2",
        "2": "3",
        "3": "4",
        "4": "Extremely"
      }
    }
  },
  "received": {
    "input_tokens": 25419
  }
}

The TypeScript script saves the response as the SDK returns it, with camelCase field names such as runId in place of run_id.

Keep the request you sent with the results, so you can check your thresholds again later against the same answers. See Storing answers.

Find errored runs

Each request that reaches the model is recorded as a run. A run errors when inference fails, or when the connection closes before the response arrives, as happens to the requests in flight when you interrupt a script. A request rejected with a 4xx status never reaches the model and has no run, so its error body in results/ is the only record of it.

To list the runs that errored during a job, filter by status and by the times the job started and ended, and pass each page’s next_cursor back as cursor until a page has none. See Listing runs. The cURL script uses jq to read each page and to encode the cursor for the URL.

query="status=errored&started_after=2026-10-26T17:00:00Z&started_before=2026-10-26T19:00:00Z"
url="https://api.cloud.hume.ai/v0/decisions/run?$query"
while [ -n "$url" ]; do
  page=$(curl -sS "$url" -H "X-Hume-Api-Key: $HUME_API_KEY")
  echo "$page" | jq -c '.runs[] | {run_id, started_at, file_bytes: .received.file_bytes}'
  cursor=$(echo "$page" | jq -r '.next_cursor // empty | @uri')
  url=${cursor:+"https://api.cloud.hume.ai/v0/decisions/run?$query&cursor=$cursor"}
done

An errored run returned no answers, and a run does not record which file it evaluated. To resend the files, run the script again: it sends every file that has no result.

Next steps