Audio Decisions API

Send an audio file with questions written in plain language, and get back probabilities that your code can threshold, rank or combine. For example, ask “How natural does the voice sound?” on a five-point scale, and the answer gives the probability of each level.

A decision model for audio

The Audio Decisions API runs a decision model for audio. The model takes a recording and your questions as input, and answers each question with probabilities instead of text. It returns no transcript, reply or explanation; your code does the reasoning with the numbers.

Because the model takes the audio itself as input, a question can be about:

1. How speech sounds, such as a caller’s frustration or a voice agent’s naturalness.
2. What is said, such as when a caller asks to cancel their service.
3. Other sounds, such as background noise or a phone ringing.

Use it where people would otherwise listen to recordings to label them, or where a transcript would lose how something was said: evaluating text-to-speech output, testing voice agents, reviewing support calls and labeling speech corpora.

How a request works

A request is a multipart/form-data POST with your credential in a header and two parts, sent in this order:

This request evaluates a simulated call with a voice AI agent. It asks two questions about the same audio: whether the agent talks over the caller, and how natural the agent sounds on a five-point scale.

curl https://api.cloud.hume.ai/v0/decisions/file \
  -H "X-Hume-Api-Key: $HUME_API_KEY" \
  -F 'request={
    "context": "A simulated test call in which a voice AI agent helps a caller reschedule an appointment. Both voices are synthetic, and the agent speaks first.",
    "questions": {
      "talks_over_caller": {
        "type": "noul",
        "instructions": "Does the agent talk over the caller?"
      },
      "agent_naturalness": {
        "type": "score",
        "instructions": "How natural does the agent sound?",
        "criteria": ["Not at all", "2", "3", "4", "Extremely"]
      }
    }
  };type=application/json' \
  -F file=@test_call.wav

The response keys each answer by the ID of its question. It also carries the run_id of the model run that produced the answers, and received, which holds the input tokens the request is billed:

{
  "run_id": "019a1f3c-7d2e-7b41-9c58-2e6f4a1d8b03",
  "answers": {
    "talks_over_caller": {
      "type": "noul",
      "noul": 0.1274
    },
    "agent_naturalness": {
      "type": "score",
      "score": 2.7195,
      "confidence": 0.4229,
      "probabilities": {
        "0": 0.0218,
        "1": 0.0874,
        "2": 0.2463,
        "3": 0.4385,
        "4": 0.206
      },
      "legend": {
        "0": "Not at all",
        "1": "2",
        "2": "3",
        "3": "4",
        "4": "Extremely"
      }
    }
  },
  "received": {
    "input_tokens": 11234
  }
}

The noul answer estimates that 13% of raters would say the agent talks over the caller. The score answer gives the probability of each level, with “4” the most likely, and its score of 2.72 is the mean level, between “3” and “4”. legend maps each level index to the text you gave it.

A probability estimates the share of human raters who would give that answer, so a value near 0.5 means raters would split on the audio, not that the model is unsure.

Primitives

Each question’s type names its primitive, which sets the criteria the question takes and the form of its answer. One request can mix primitives.

Combine answers in code

Ask each question for one focused judgment, and combine the answers in your code. “Did the agent handle the call well?” depends on several things, so ask about each one: the agent’s tone, how frustrated the caller sounds and whether the caller asks to cancel.

Send all the questions about a recording in one request, and send the same questions in the same order for every recording you compare, since an answer can change when the other questions change. Then combine the answers with thresholds and weights you choose, such as flagging a call for review when the probability that the agent is curt crosses a threshold.

Endpoints

Every endpoint is served from https://api.cloud.hume.ai.

Each request that reaches the model is recorded as a run, which can take a moment to appear. Under heavy load a run may not be recorded. The run endpoints return a run’s status and the request it evaluated, but not its answers, so store the run_id and answers you need.

SDKs

The Python and TypeScript SDKs cover every endpoint with typed requests and responses. They read your API key from HUME_API_KEY, retry requests that are rate limited or fail on the server, and include audio helpers that split recordings longer than 5 minutes and send one request per part.

Compatibility

The API version is the path prefix, /v0/. Within a version:

1. The server may add fields to the responses it sends. Ignore fields you do not recognize.
2. New answer types may be added. Treat an answer with an unrecognized type as unknown, not as an error.
3. Requests are validated strictly. An unrecognized field is rejected with question_invalid inside a question, and with message_invalid elsewhere in the request part.
4. The model can be updated without a change to the API. Answers for the same audio and questions can change after an update.

Next steps