> This page is for Audio Decisions. > For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://dev.hume.ai/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://dev.hume.ai/_mcp/server. # Evaluating many files **To evaluate many recordings, send one request per file, a few at a time, and save each response as soon as it arrives.** A job that stops partway can then be run again and pick up where it left off, without paying twice for the files that already have answers. ## Plan the job 1. **Send every file the same questions in the same order.** All the questions in a request are answered together, so an answer can change when other questions are added, removed or reordered. Answers are comparable across files only when the request is the same. 2. **Try the questions on a few files you know** before running the whole set. See [Check your questions on audio you know](/audio-decisions/docs/writing-questions#check-your-questions-on-audio-you-know). 3. **Estimate the input tokens** with [Estimate input tokens](/audio-decisions/reference/decisions/input-tokens). It takes the request's `context` and `questions` with an audio duration in milliseconds, and returns an estimate of the input tokens for a file of that duration. Each decisions response gives the input tokens it is billed as `received.input_tokens`, which can differ slightly from the estimate. The estimate counts toward the rate limit like any other request. ## Prepare the files A request takes one MP3 or WAV file of 5 seconds to 5 minutes and up to 25,000,000 bytes. See [Limits](/audio-decisions/docs/limits#audio). The model hears all audio as 16 kHz, 16-bit, one channel. A higher sample rate, a greater bit depth or a second channel does not improve answers, and only makes the file larger. 5 minutes of 16 kHz, 16-bit mono WAV is 9.6 MB, well under the size limit. The script below converts every file in `recordings/` to 16 kHz, 16-bit mono WAV, and splits any recording longer than 5 minutes into parts. It writes the files to `audio/`, named after the recording with a part number, along with a CSV file for each recording that lists its parts with their start and end times in seconds. ffmpeg can end a part slightly after `-segment_time`, so 299 seconds keeps every part under 5 minutes. The script cuts at fixed times and mixes the channels of a stereo recording into one. To cut at pauses or turn ends, or to send one speaker's channel, see [Preparing your audio](/audio-decisions/docs/preparing-your-audio). The last part of a recording can be shorter than 5 seconds, which is rejected with `audio_too_short`; skip it, or split the recording at a different length so that no part is that short. #### Shell ```bash mkdir -p audio for recording in recordings/*; do name=$(basename "${recording%.*}") ffmpeg -i "$recording" -ac 1 -ar 16000 -sample_fmt s16 \ -f segment -segment_time 299 \ -segment_list "audio/$name.csv" \ "audio/${name}_%03d.wav" done ``` ### Long recordings The model answers each part of a split recording as if it were the whole recording, and nothing combines the answers across parts. 1. **Write questions that hold for any part.** "Does the agent talk over the caller?" can be answered from any 5 minutes of a call. "Is the caller's issue resolved by the end of the call?" can be answered only from the last part. 2. **Place spans in the recording.** A span's times are measured from the start of its part, so add the part's start time from the CSV file. 3. **Combine the answers for each question in your code.** For example, to learn whether something happens at any point in the call, take the highest `noul` across the parts. A 12-minute call split into parts of 299 seconds Parts Part 1 Part 2 Part 3 Part 2 `audio_start_ms` 92000 1:32 Call 6:31 0:00 4:59 9:58 12:00 Part 2 starts at 4:59, so a span 1:32 into part 2 is at 6:31 in the call. The Python and TypeScript SDKs can split a recording and send each part for you. See Long recordings in the [Python SDK README](https://github.com/HumeAI/hume-audio-decisions-python-sdk#long-recordings) and the [TypeScript SDK README](https://github.com/HumeAI/hume-audio-decisions-typescript-sdk#long-recordings). ## Send the requests Every API key in an organization shares a limit of 60 requests per minute. See [Limits](/audio-decisions/docs/limits#rate-limits). Cap how many requests run at once, so that a job stays under the limit and leaves room for other work in your organization. The scripts below send 4 requests at a time. If many requests return `429`, lower `-P` in the cURL script or `CONCURRENCY` in the Python and TypeScript scripts. Each script sends every WAV file in `audio/` and saves each response to `results/`, named after the file. A file whose result already exists is skipped, so running a script again sends only the files that have no answers. The scripts read your API key from `HUME_API_KEY`. For the cURL script, it must be exported, so the commands that `xargs` starts can read it. #### cURL ```bash maxLines={0} cat > request.json <<'EOF' { "context": "A call in which a voice AI agent helps a caller reschedule an appointment.", "questions": { "talks_over_caller": { "type": "noul", "instructions": "Does the agent talk over the caller?" }, "agent_naturalness": { "type": "score", "instructions": "How natural does the agent sound?", "criteria": ["Not at all", "2", "3", "4", "Extremely"] } } } EOF mkdir -p results find audio -name '*.wav' -print0 | xargs -0 -n 1 -P 4 sh -c ' name=$(basename "$1" .wav) [ -f "results/$name.json" ] && exit 0 status=$(curl -sS --retry 2 \ -H "X-Hume-Api-Key: $HUME_API_KEY" \ -F "request=&2 case "$status" in 401|402|403) exit 255 ;; esac ' sh ``` #### Python ```python maxLines={0} import json import sys import threading from concurrent.futures import ThreadPoolExecutor from pathlib import Path import httpx from hume_audio_decisions import ( DecisionsClient, DecisionsRequest, Error, Question_Noul, Question_Score, ) from hume_audio_decisions.core import ApiError AUDIO = Path("audio") RESULTS = Path("results") CONCURRENCY = 4 client = DecisionsClient() stop = threading.Event() request = DecisionsRequest( context="A call in which a voice AI agent helps a caller reschedule an appointment.", questions={ "talks_over_caller": Question_Noul( instructions="Does the agent talk over the caller?", ), "agent_naturalness": Question_Score( instructions="How natural does the agent sound?", criteria=["Not at all", "2", "3", "4", "Extremely"], ), }, ) def evaluate(path: Path) -> None: result = RESULTS / f"{path.stem}.json" error_result = RESULTS / f"{path.stem}.error.json" if stop.is_set() or result.exists(): return try: with path.open("rb") as audio: response = client.decisions.create( request=request, file=(path.name, audio, "audio/wav"), ) except ApiError as error: body = error.body.dict() if isinstance(error.body, Error) else error.body error_result.write_text(json.dumps(body)) print(f"{path}: HTTP {error.status_code}", file=sys.stderr) if error.status_code in (401, 402, 403): stop.set() return except httpx.TransportError as error: print(f"{path}: {error!r}", file=sys.stderr) return result.write_text(json.dumps(response.dict())) error_result.unlink(missing_ok=True) RESULTS.mkdir(exist_ok=True) with ThreadPoolExecutor(max_workers=CONCURRENCY) as executor: list(executor.map(evaluate, sorted(AUDIO.glob("*.wav")))) ``` #### TypeScript ```typescript maxLines={0} import { createReadStream, existsSync, mkdirSync, readdirSync, rmSync, writeFileSync, } from "node:fs"; import { basename } from "node:path"; import { type Decisions, DecisionsClient, DecisionsError, } from "@humeai/audio-decisions"; const CONCURRENCY = 4; const client = new DecisionsClient(); let stopped = false; const request: Decisions.DecisionsRequest = { context: "A call in which a voice AI agent helps a caller reschedule an appointment.", questions: { talks_over_caller: { type: "noul", instructions: "Does the agent talk over the caller?", }, agent_naturalness: { type: "score", instructions: "How natural does the agent sound?", criteria: ["Not at all", "2", "3", "4", "Extremely"], }, }, }; async function evaluate(file: string): Promise { const name = basename(file, ".wav"); if (existsSync(`results/${name}.json`)) return; try { const response = await client.decisions.create({ file: createReadStream(`audio/${file}`), request, }); writeFileSync(`results/${name}.json`, JSON.stringify(response)); rmSync(`results/${name}.error.json`, { force: true }); } catch (error) { if (!(error instanceof DecisionsError)) throw error; if (error.body !== undefined) { writeFileSync(`results/${name}.error.json`, JSON.stringify(error.body)); } const reason = error.statusCode === undefined ? error.message : `HTTP ${error.statusCode}`; console.error(`audio/${file}: ${reason}`); if ([401, 402, 403].includes(error.statusCode ?? 0)) stopped = true; } } const files = readdirSync("audio") .filter((file) => file.endsWith(".wav")) .sort(); async function worker(): Promise { let file = files.shift(); while (file !== undefined && !stopped) { await evaluate(file); file = files.shift(); } } mkdirSync("results", { recursive: true }); await Promise.all(Array.from({ length: CONCURRENCY }, worker)); ``` ## Handle failed requests What to do with a failed request depends on its status. See [Handling errors](/audio-decisions/docs/errors#handling-errors). 1. **`429`, `503` and `500`** are retried. `curl --retry 2` and both SDKs retry a request twice, waiting the number of seconds in the `Retry-After` header when the response has one. For a long job, allow more retries with `--retry 5` in cURL, `request_options={"max_retries": 5}` in Python, or `{ maxRetries: 5 }` as the second argument to `create` in TypeScript. 2. **`400`, `413` and `415`** fail again if the file is sent unchanged. The scripts save the error body to `results/.error.json` and move on to the next file. Its `code` and `param` name what to correct, and its `request_id` identifies the request when you contact support. 3. **`401`, `402` and `403`** stop the script, since every remaining request would fail the same way. Correct the credential or the organization's account, then run the script again. 4. **A timeout or a failed connection** has no status and no error body. The scripts print the error for the file and move on to the next file. A file whose request failed, after its retries or with a timeout, has no result, so running the script again sends it again. ## Store the results Each result holds the `run_id`, the answers and the billed input tokens for one file: #### results/call_017_000.json ```json { "run_id": "01a18e2a-b2c0-71e8-8d58-92d4c14a8125", "answers": { "talks_over_caller": { "type": "noul", "noul": 0.0191 }, "agent_naturalness": { "type": "score", "score": 2.0616, "confidence": 0.1275, "probabilities": { "0": 0.1585, "1": 0.1756, "2": 0.2362, "3": 0.305, "4": 0.1246 }, "legend": { "0": "Not at all", "1": "2", "2": "3", "3": "4", "4": "Extremely" } } }, "received": { "input_tokens": 25419 } } ``` The TypeScript script saves the response as the SDK returns it, with camelCase field names such as `runId` in place of `run_id`. Keep the request you sent with the results, so you can check your thresholds again later against the same answers. See [Storing answers](/audio-decisions/docs/interpreting-answers#storing-answers). ## Find errored runs Each request that reaches the model is recorded as a [run](/audio-decisions/docs/runs). A run errors when inference fails, or when the connection closes before the response arrives, as happens to the requests in flight when you interrupt a script. A request rejected with a `4xx` status never reaches the model and has no run, so its error body in `results/` is the only record of it. To list the runs that errored during a job, filter by status and by the times the job started and ended, and pass each page's `next_cursor` back as `cursor` until a page has none. See [Listing runs](/audio-decisions/docs/runs#listing-runs). The cURL script uses jq to read each page and to encode the cursor for the URL. #### cURL ```bash maxLines={0} query="status=errored&started_after=2026-10-26T17:00:00Z&started_before=2026-10-26T19:00:00Z" url="https://api.cloud.hume.ai/v0/decisions/run?$query" while [ -n "$url" ]; do page=$(curl -sS "$url" -H "X-Hume-Api-Key: $HUME_API_KEY") echo "$page" | jq -c '.runs[] | {run_id, started_at, file_bytes: .received.file_bytes}' cursor=$(echo "$page" | jq -r '.next_cursor // empty | @uri') url=${cursor:+"https://api.cloud.hume.ai/v0/decisions/run?$query&cursor=$cursor"} done ``` #### Python ```python maxLines={0} from datetime import datetime, timezone from hume_audio_decisions import DecisionsClient client = DecisionsClient() cursor = None while True: page = client.runs.list( status="errored", started_after=datetime(2026, 10, 26, 17, tzinfo=timezone.utc), started_before=datetime(2026, 10, 26, 19, tzinfo=timezone.utc), cursor=cursor, ) for run in page.runs: print(run.run_id, run.started_at.isoformat(), run.received.file_bytes) if page.next_cursor is None: break cursor = page.next_cursor ``` #### TypeScript ```typescript maxLines={0} import { DecisionsClient } from "@humeai/audio-decisions"; const client = new DecisionsClient(); let cursor: string | undefined; do { const page = await client.runs.list({ status: "errored", startedAfter: new Date("2026-10-26T17:00:00Z"), startedBefore: new Date("2026-10-26T19:00:00Z"), cursor, }); for (const run of page.runs) { console.log(run.runId, run.startedAt.toISOString(), run.received.fileBytes); } cursor = page.nextCursor; } while (cursor !== undefined); ``` An errored run returned no answers, and a run does not record which file it evaluated. To resend the files, run the script again: it sends every file that has no result. ## Next steps #### [Interpreting answers](/audio-decisions/docs/interpreting-answers) Learn how to act on the stored answers with thresholds you set on your own audio.