Use cases | Hume API

Each card is an example question to send the Audio Decisions API. Pick an industry or search the cards, then open a card to copy its question as JSON, and adapt the wording and options to your recordings.

Foundationalchoice

Which language is spoken?

List the languages you expect as options, such as English, Japanese, Arabic, French and Russian.

{
  "questions": {
    "language_spoken": {
      "type": "choice",
      "instructions": "Which language is spoken?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Foundationalscore

How fast does the speaker talk?

Use levels from very slow to very fast. The answer is not a count of words per minute.

{
  "questions": {
    "speaking_rate": {
      "type": "score",
      "instructions": "How fast does the speaker talk?",
      "criteria": ["Very slow", "Slow", "Moderate", "Fast", "Very fast"]
    }
  }
}

Speech and language

Foundationalnoul

Does the speaker say a filler such as “uh” or “um”?

The answer does not say why the speaker hesitates.

{
  "questions": {
    "filler_words": {
      "type": "noul",
      "instructions": "Does the speaker say a filler such as \"uh\" or \"um\"?"
    }
  }
}

Speech and language

Foundationalchoice

Which accent does the speaker have?

List the accents you expect as options, such as Mainstream US English and South African English. An accent is not proof of nationality or first language.

{
  "questions": {
    "accent": {
      "type": "choice",
      "instructions": "Which accent does the speaker have?",
      "criteria": [
        {
          "name": "Mainstream US English"
        },
        {
          "name": "South African English"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Foundationalnoul

Is this speech artificially generated?

Ask how natural the speech sounds as a separate question.

{
  "questions": {
    "synthetic_speech": {
      "type": "noul",
      "instructions": "Is this speech artificially generated?"
    }
  }
}

Speakers and voices

Foundationalchoice

Which emotion does the speaker’s voice convey?

List the emotions as options, such as sad, happy, fearful and disgusted. The answer is the emotion expressed, not what the speaker feels.

{
  "questions": {
    "expressed_emotion": {
      "type": "choice",
      "instructions": "Which emotion does the speaker's voice convey?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Disgusted"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Foundationalchoice

Which word carries the most stress?

Put the sentence in context, such as “Technology has changed how we view art today.”, and list the candidate words as options.

{
  "context": "Sentence: Technology has changed how we view art today.",
  "questions": {
    "emphasized_word": {
      "type": "choice",
      "instructions": "Which word carries the most stress?",
      "criteria": [
        {
          "name": "Technology"
        },
        {
          "name": "changed"
        },
        {
          "name": "view"
        },
        {
          "name": "art"
        },
        {
          "name": "today"
        }
      ]
    }
  }
}

Emotion and expression

Foundationalnoul

Is there noise behind the speaker?

Ask what kind of noise it is as a separate question.

{
  "questions": {
    "background_noise": {
      "type": "noul",
      "instructions": "Is there noise behind the speaker?"
    }
  }
}

Acoustic environment

Foundationalchoice

Which kind of sound is most prominent under the voice?

List the kinds you expect as options, such as hiss, hum, other people talking and music. The answer names the sound, not its cause.

{
  "questions": {
    "kind_of_background_noise": {
      "type": "choice",
      "instructions": "Which kind of sound is most prominent under the voice?",
      "criteria": [
        {
          "name": "Hiss"
        },
        {
          "name": "Hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Foundationalscore

How intrusive is the background noise?

Use levels from not noticeable to very intrusive. The answer is a listener’s judgment, not a decibel level.

{
  "questions": {
    "noise_intrusiveness": {
      "type": "score",
      "instructions": "How intrusive is the background noise?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Foundationalscore

How would most listeners rate the sound quality of this speech?

Use levels from bad to excellent, and compare answers only between recordings of the same kind, such as phone calls with phone calls.

{
  "questions": {
    "sound_quality": {
      "type": "score",
      "instructions": "How would most listeners rate the sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Foundationalnoul

Does anyone laugh?

Ask about each other vocal sound, such as a cough, as a separate question.

{
  "questions": {
    "laughter": {
      "type": "noul",
      "instructions": "Does anyone laugh?"
    }
  }
}

Non-speech events

Foundationalchoice

Which kind of vocal sound is most prominent?

List the sounds you expect as options, such as a cough, a breath, a cry and a burp.

{
  "questions": {
    "kind_of_vocal_sound": {
      "type": "choice",
      "instructions": "Which kind of vocal sound is most prominent?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Cry"
        },
        {
          "name": "Burp"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Foundationalspan

When does someone laugh?

The answer lists the time spans where laughter may occur, with times measured from the start of the file.

{
  "context": "Times are relative to the start of the supplied recording.",
  "questions": {
    "when_laughter_occurs": {
      "type": "span",
      "instructions": "When does someone laugh?"
    }
  }
}

Non-speech events

Foundationalnoul

Can running water be heard?

Name the sound in the question. The answer says whether the sound is heard, not what caused it.

{
  "questions": {
    "a_named_sound": {
      "type": "noul",
      "instructions": "Can running water be heard?"
    }
  }
}

Non-speech events

Foundationalchoice

Which of the listed sounds is most prominent?

List the sounds that could occur in your recordings as options, such as a dog barking, a vacuum cleaner and an alarm bell.

{
  "questions": {
    "which_listed_sound": {
      "type": "choice",
      "instructions": "Which of the listed sounds is most prominent?",
      "criteria": [
        {
          "name": "Dog barking"
        },
        {
          "name": "Vacuum cleaner"
        },
        {
          "name": "Alarm bell"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Foundationalspan

When is a car horn heard?

Name the sound in full in the question. Times are measured from the start of the file you send.

{
  "questions": {
    "when_a_sound_occurs": {
      "type": "span",
      "instructions": "When is a car horn heard?"
    }
  }
}

Non-speech events

Foundationalnoul

Do two people speak at the same time?

Send the excerpt you want checked. Overlap is not always an interruption.

{
  "questions": {
    "overlapping_speech": {
      "type": "noul",
      "instructions": "Do two people speak at the same time?"
    }
  }
}

Conversation dynamics

Foundationalchoice

Does the second speaker take over the turn, or does the first speaker carry on?

Send one excerpt in which someone speaks during another person’s turn.

{
  "questions": {
    "interruption_or_acknowledgment": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or does the first speaker carry on?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        }
      ]
    }
  }
}

Conversation dynamics

Foundationalnoul

Is there a pause of half a second or more within one person’s turn?

The answer does not say why the speaker paused.

{
  "questions": {
    "pause_within_a_turn": {
      "type": "noul",
      "instructions": "Is there a pause of half a second or more within one person's turn?"
    }
  }
}

Conversation dynamics

Foundationalscore

How natural does the speech sound?

Use levels from bad to excellent, and rank voices or model versions by the answer. Natural-sounding speech is not proof that a person spoke it.

{
  "questions": {
    "naturalness": {
      "type": "score",
      "instructions": "How natural does the speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Foundationalscore

How well does the assistant’s reply address what the user said?

Put the earlier conversation in context. The answer judges relevance, not whether the reply is correct.

{
  "context": "Earlier conversation:\nUser: Find me a quiet café near the station.\nAssistant: There are two nearby. Do you want one with outdoor seating?",
  "questions": {
    "relevance_of_a_reply": {
      "type": "score",
      "instructions": "How well does the assistant's reply address what the user said?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Foundationalscore

How many of the 10 Mandarin characters in the sentence can a listener understand?

Use five count ranges as the levels: 0 to 2, 3 to 4, 5 to 6, 7 to 8 and 9 to 10.

{
  "questions": {
    "intelligibility": {
      "type": "score",
      "instructions": "How many of the 10 Mandarin characters in the sentence can a listener understand?",
      "criteria": ["0 to 2", "3 to 4", "5 to 6", "7 to 8", "9 to 10"]
    }
  }
}

Communication quality

Foundationalchoice

What is the user asking the assistant to do?

List the requests your assistant handles as options, such as checking the weather, adding a calendar event and playing music.

{
  "questions": {
    "routing_a_request": {
      "type": "choice",
      "instructions": "What is the user asking the assistant to do?",
      "criteria": [
        {
          "name": "Check the weather"
        },
        {
          "name": "Add a calendar event"
        },
        {
          "name": "Play music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Foundationalchoice

Which do you hear first, the cough or the laugh?

Name both sounds in the question and give an option for each, so audio where they overlap or one is missing can go to “None of these or unclear”.

{
  "questions": {
    "order_of_two_sounds": {
      "type": "choice",
      "instructions": "Which do you hear first, the cough or the laugh?",
      "criteria": [
        {
          "name": "Cough first"
        },
        {
          "name": "Laugh first"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Foundationalchoice

After which word does the speaker pause?

Put the transcript in context, and list the candidate words as options.

{
  "context": "Transcript: Well, honestly, no, I haven't decided yet.",
  "questions": {
    "pause_after_a_word": {
      "type": "choice",
      "instructions": "After which word does the speaker pause?",
      "criteria": [
        {
          "name": "Well"
        },
        {
          "name": "honestly"
        },
        {
          "name": "no"
        }
      ]
    }
  }
}

Temporal and contextual signals

Foundationalspan

When is the word “night” spoken?

The answer lists the time spans where the word may be spoken, with the start and end of each.

{
  "questions": {
    "when_a_word_is_spoken": {
      "type": "span",
      "instructions": "When is the word \"night\" spoken?"
    }
  }
}

Temporal and contextual signals

Foundationalspan

When is trend watching discussed?

Ask it of the whole recording, and take the earliest span returned as the point the topic first comes up.

{
  "questions": {
    "when_a_topic_comes_up": {
      "type": "span",
      "instructions": "When is trend watching discussed?"
    }
  }
}

Temporal and contextual signals

Foundationalchoice

Does this voice sound male, female, or neither?

Describe how listeners are likely to perceive a designed voice’s gender presentation, using the tested choices.

{
  "questions": {
    "perceived_gender_presentation_of_a_designed": {
      "type": "choice",
      "instructions": "Does this voice sound male, female, or neither?",
      "criteria": [
        {
          "name": "Male"
        },
        {
          "name": "Female"
        },
        {
          "name": "Neither"
        }
      ]
    }
  }
}

Speakers and voices

Foundationalchoice

Does the speaker’s vocal tone fit both the words and the situation described?

Whether the vocal tone fits the words and the situation.

{
  "context": "A customer says, \"My order is late and I'm frustrated.\" The clip is the assistant saying, \"I'm sorry about the delay; I'll check it now.\" Judge the audible delivery against those words and that situation, without inferring sincerity or private feelings.",
  "questions": {
    "emotion_word_incongruence": {
      "type": "choice",
      "instructions": "Does the speaker's vocal tone fit both the words and the situation described?",
      "criteria": [
        {
          "name": "Fits both"
        },
        {
          "name": "Conflicts with the words"
        },
        {
          "name": "Does not fit the situation"
        },
        {
          "name": "Fits neither"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Foundationalnoul

Is music audible in the background of this recording?

Whether music is audible in the environment.

{
  "questions": {
    "background_music": {
      "type": "noul",
      "instructions": "Is music audible in the background of this recording?"
    }
  }
}

Acoustic environment

Foundationalchoice

How many distinct speakers are audible in this recording?

Count the distinct speakers in the selected audio.

{
  "context": "Count distinct voices, not the number of turns.",
  "questions": {
    "number_of_speakers": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this recording?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Foundationalchoice

Did the agent give the requested refund arrival time?

Whether the reply supplies the information the person asked for.

{
  "context": "Customer: \"When will my refund arrive?\" Judge only what the reply provides: approval alone is not an arrival time. Yes = arrival time provided; Partially = some but incomplete arrival-time information; No = requested time not provided; Unclear = insufficient audio or context.",
  "questions": {
    "did_the_response_answer_the_question": {
      "type": "choice",
      "instructions": "Did the agent give the requested refund arrival time?",
      "criteria": [
        {
          "name": "Yes"
        },
        {
          "name": "Partially"
        },
        {
          "name": "No"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Foundationalnoul

Is the group noticeably more involved here than in calm, matter-of-fact discussion?

A selected window’s apparent involvement compared with calm discussion.

{
  "context": "This is a clip from a research group's meeting (headset microphones mixed together).",
  "questions": {
    "meeting_engagement": {
      "type": "noul",
      "instructions": "Is the group noticeably more involved here than in calm, matter-of-fact discussion?"
    }
  }
}

Social and behavioral signals

AI/ML researchscore

How would most listeners rate the technical sound quality of this speech?

Rank speech clips for a listening check before deciding which belong in training or evaluation.

{
  "questions": {
    "prioritize_low_quality_recordings_for_review": {
      "type": "score",
      "instructions": "How would most listeners rate the technical sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

AI/ML researchnoul

Do two speakers talk at the same time in this excerpt?

Route clips with simultaneous speech to a dedicated annotation or evaluation slice.

{
  "questions": {
    "separate_overlapping_speech_for_transcription": {
      "type": "noul",
      "instructions": "Do two speakers talk at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

AI/ML researchnoul

Is laughter audible in this excerpt?

Find candidate clips containing a chosen non-speech vocalization for annotation and dataset sampling.

{
  "questions": {
    "mine_vocal_burst_examples_for_training": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

AI/ML researchchoice

Which listed emotion does the voice convey?

Propose vocal-expression labels that annotators can verify and compare with existing labels.

{
  "questions": {
    "label_expressed_emotion_for_a_speech_dataset": {
      "type": "choice",
      "instructions": "Which listed emotion does the voice convey?",
      "criteria": [
        {
          "name": "Angry"
        },
        {
          "name": "Disgusted"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Happiness"
        },
        {
          "name": "Sadness"
        },
        {
          "name": "Surprised"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

AI/ML researchscore

How natural does this speech sound?

Rank generated samples for listening review when comparing voice models or checkpoints.

{
  "questions": {
    "compare_naturalness_across_generated_speech_runs": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

AI/ML researchnoul

Was the supplied sentence read correctly and clearly?

Find generated samples that need a transcript or listening check before evaluation.

{
  "context": "The script for this generated utterance is: \"Save your work before closing the window.\" Check the supplied audio against that sentence.",
  "questions": {
    "check_generated_speech_against_its_script": {
      "type": "noul",
      "instructions": "Was the supplied sentence read correctly and clearly?"
    }
  }
}

Communication quality

AI/ML researchchoice

Which supplied word carries the strongest emphasis?

Compare generated or performed emphasis with the word assigned in a prosody experiment.

{
  "context": "The spoken sentence is \"Send the replacement today.\"",
  "questions": {
    "check_word_emphasis_in_controlled_speech": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Send"
        },
        {
          "name": "The"
        },
        {
          "name": "Replacement"
        },
        {
          "name": "Today"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

AI/ML researchnoul

Are filled pauses such as “uh” or “um” audible in this excerpt?

Collect clips containing “uh” or “um” for a disfluency annotation queue.

{
  "questions": {
    "find_examples_of_filled_pauses": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible in this excerpt?"
    }
  }
}

Speech and language

AI/ML researchchoice

How many distinct speakers are audible in this clip?

Find clips whose existing speaker-count label may be wrong, for listening review before a speaker-separation study.

{
  "context": "Count audible speaking voices within this clip only; people who are present but silent do not count.",
  "questions": {
    "check_the_speaker_count_label_on_a_clip": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this clip?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

AI/ML researchchoice

Which conversational move does this utterance make?

Flag utterances whose existing dialogue-act label may be wrong before the labels are used for training.

{
  "questions": {
    "check_dialogue_act_labels_in_a_call_corpus": {
      "type": "choice",
      "instructions": "Which conversational move does this utterance make?",
      "criteria": [
        {
          "name": "Opening the call or introducing themselves"
        },
        {
          "name": "Giving instructions"
        },
        {
          "name": "Disagreeing or saying no"
        },
        {
          "name": "Agreeing or saying yes"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

AI/ML researchscore

How directly does the agent’s reply address the question “How do I reset my password?”

Queue off-topic responses for error analysis in a voice-agent evaluation set.

{
  "context": "The user asks \"How do I reset my password?\" The supplied audio contains the agent's reply. Judge relevance to that question, not factual correctness or successful execution.",
  "questions": {
    "check_whether_a_voice_agent_reply_answers": {
      "type": "score",
      "instructions": "How directly does the agent's reply address the question \"How do I reset my password?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Intent and outcomes

AI/ML researchnoul

Does the replacement line sound like the same voice as the surrounding speech?

Queue possible voice changes within a generated utterance for a listening review.

{
  "context": "A replacement sentence is joined between two surrounding sentences. The replacement begins with \"Your order arrives tomorrow.\" Compare the voice before, during and after that sentence.",
  "questions": {
    "check_voice_continuity_in_generated_samples": {
      "type": "noul",
      "instructions": "Does the replacement line sound like the same voice as the surrounding speech?"
    }
  }
}

Speakers and voices

AI/ML researchscore

How strongly would listeners agree that the vocal delivery fits the supplied direction for this line?

Compare how well generated takes follow a controlled expressive direction.

{
  "context": "The line is \"Thanks for waiting; I can help you now.\" The speaking direction is calm and reassuring, without sounding rushed. Rate the audible delivery against that direction.",
  "questions": {
    "check_delivery_against_a_speaking_direction": {
      "type": "score",
      "instructions": "How strongly would listeners agree that the vocal delivery fits the supplied direction for this line?",
      "criteria": ["Do not agree at all", "Disagree", "Neutral", "Agree", "Fully agree"]
    }
  }
}

Emotion and expression

AI/ML researchchoice

Does the second speaker take over the turn, or briefly acknowledge while the first continues?

Review selected interjections before using them as dialogue-behavior labels.

{
  "questions": {
    "separate_backchannels_from_floor_taking": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment without taking the floor"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

AI/ML researchchoice

Which listed language is spoken in this clip?

Route a bounded multilingual collection to the appropriate annotation queue.

{
  "context": "The collection is documented to contain single-language clips in English, Japanese, Arabic, French or Russian.",
  "questions": {
    "sort_recordings_by_spoken_language": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this clip?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

AI/ML researchspan

Find the intervals where two people speak at the same time.

Suggest candidate intervals for annotators to mark overlapping speech more precisely.

{
  "context": "Return candidate intervals relative to this audio.",
  "questions": {
    "find_overlapping_speech_within_a_recording": {
      "type": "span",
      "instructions": "Find the intervals where two people speak at the same time."
    }
  }
}

Conversation dynamics

AI/ML researchspan

Find the parts of this interview discussing how the participant uses voice assistants.

Retrieve candidate excerpts about a specified research topic for human coding.

{
  "questions": {
    "find_a_topic_passage_for_a_research_corpus": {
      "type": "span",
      "instructions": "Find the parts of this interview discussing how the participant uses voice assistants."
    }
  }
}

Temporal and contextual signals

AI/ML researchscore

How fast does the speaker talk in this clip?

Rank clips by coarse speaking rate and set slow, moderate and fast slice thresholds for balanced testing.

{
  "questions": {
    "create_speaking_rate_evaluation_slices": {
      "type": "score",
      "instructions": "How fast does the speaker talk in this clip?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

AI/ML researchspan

Find where “Friday” is spoken in this recording.

Locate candidate instances of a target word to inspect a speech-alignment dataset.

{
  "context": "Return candidate intervals relative to this clip.",
  "questions": {
    "find_a_spoken_target_for_alignment_review": {
      "type": "span",
      "instructions": "Find where \"Friday\" is spoken in this recording."
    }
  }
}

Temporal and contextual signals

AI/ML researchchoice

Which listed sound can be heard?

Propose an event class for a bounded sound-event corpus.

{
  "questions": {
    "label_a_sound_from_a_known_class_inventory": {
      "type": "choice",
      "instructions": "Which listed sound can be heard?",
      "criteria": [
        {
          "name": "Speech"
        },
        {
          "name": "Dog"
        },
        {
          "name": "Vacuum cleaner"
        },
        {
          "name": "Alarm bell ringing"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

AI/ML researchchoice

Is “Friday” spoken before or after “Monday”?

Compare a generated utterance with a test that deliberately varies the order of two words.

{
  "questions": {
    "check_word_order_in_a_speech_generation_test": {
      "type": "choice",
      "instructions": "Is \"Friday\" spoken before or after \"Monday\"?",
      "criteria": [
        {
          "name": "Friday before Monday"
        },
        {
          "name": "Friday after Monday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

AI/ML researchspan

Find the laughter in this recording.

Suggest laughter intervals that a data pipeline can export with surrounding context for review.

{
  "context": "Timestamps are relative to this recording.",
  "questions": {
    "find_laughter_to_cut_annotation_clips": {
      "type": "span",
      "instructions": "Find the laughter in this recording."
    }
  }
}

Non-speech events

AI/ML researchchoice

Which listed background sound is audible underneath the voice?

Group clips by a specified audible noise type to compare model performance under different conditions.

{
  "questions": {
    "build_background_noise_evaluation_slices": {
      "type": "choice",
      "instructions": "Which listed background sound is audible underneath the voice?",
      "criteria": [
        {
          "name": "Hiss"
        },
        {
          "name": "Hum"
        },
        {
          "name": "Background chatter"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

AI/ML researchscore

How intrusive is the background sound?

Keep separate evaluation slices for unobtrusive and intrusive background sound.

{
  "questions": {
    "measure_background_interference": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

AI/ML researchchoice

Which delivery day does the speaker ask for?

Compare a spoken command’s requested value with the existing annotation.

{
  "questions": {
    "check_a_requested_value_in_spoken_command_data": {
      "type": "choice",
      "instructions": "Which delivery day does the speaker ask for?",
      "criteria": [
        {
          "name": "Monday"
        },
        {
          "name": "Tuesday"
        },
        {
          "name": "Thursday"
        },
        {
          "name": "Friday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Advertisingchoice

Which word carries the strongest emphasis?

Put the line in context, such as “The offer ends on Friday.”, and list its words as options. Compare the answer with the word the brief calls for.

{
  "context": "Line: The offer ends on Friday.",
  "questions": {
    "emphasis_in_a_voice_over_take": {
      "type": "choice",
      "instructions": "Which word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "offer"
        },
        {
          "name": "ends"
        },
        {
          "name": "on"
        },
        {
          "name": "Friday"
        }
      ]
    }
  }
}

Emotion and expression

Advertisingscore

Does the replacement line sound like the same voice as the surrounding ad?

Send the original excerpt, the replacement line and the original again in one file. Use levels from definitely different to definitely the same.

{
  "context": "The listening file plays an original voice-over excerpt, the replacement line, then the original excerpt again. Check whether the voice sounds consistent across the edit, where recording conditions or audio processing may differ.",
  "questions": {
    "voice_match_after_a_pickup": {
      "type": "score",
      "instructions": "Does the replacement line sound like the same voice as the surrounding ad?",
      "criteria": ["Definitely different", "Probably different", "Unsure", "Probably the same", "Definitely the same"]
    }
  }
}

Speakers and voices

Advertisingchoice

Which of these recording problems is most audible?

List the problems as options, such as background noise, muffled sound, dropouts and clipping. Use the answer to decide what to inspect during cleanup.

{
  "questions": {
    "recording_problem": {
      "type": "choice",
      "instructions": "Which of these recording problems is most audible?",
      "criteria": [
        {
          "name": "Background noise"
        },
        {
          "name": "Muffled or tinny sound"
        },
        {
          "name": "Dropouts or interruptions"
        },
        {
          "name": "Clipping"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Communication quality

Advertisingnoul

Is laughter audible in this clip?

Ask it of each clip in your asset library, and listen to the matches to choose one.

{
  "questions": {
    "laughter_in_an_asset_library": {
      "type": "noul",
      "instructions": "Is laughter audible in this clip?"
    }
  }
}

Non-speech events

Advertisingchoice

Which listed discount is spoken in the excerpt?

List the discounts across your ad versions as options, such as 10, 15 and 20 percent. The answer is what the audio says, not whether the offer is still valid.

{
  "questions": {
    "discount_in_an_ad": {
      "type": "choice",
      "instructions": "Which listed discount is spoken in the excerpt?",
      "criteria": [
        {
          "name": "10 percent"
        },
        {
          "name": "15 percent"
        },
        {
          "name": "20 percent"
        },
        {
          "name": "25 percent"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Advertisingscore

How natural does this speech sound?

Compare how natural the generated speech sounds before choosing a delivery.

{
  "questions": {
    "naturalness_of_a_generated_ad_read": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Advertisingnoul

Is there a quiet pause of half a second or more within the sentence “The offer is available through Friday”?

Check whether an unexpected quiet gap interrupts the flow of a recorded sentence.

{
  "questions": {
    "a_pause_inside_the_offer_line": {
      "type": "noul",
      "instructions": "Is there a quiet pause of half a second or more within the sentence \"The offer is available through Friday\"?"
    }
  }
}

Conversation dynamics

Advertisingscore

How intrusive is the background sound?

Describe distracting background sound before deciding whether to request a cleaner recording.

{
  "questions": {
    "background_intrusion_in_a_creator_recorded": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Advertisingnoul

Is laughter audible in this excerpt?

Check for an audible production cue.

{
  "questions": {
    "a_laugh_cue_in_a_recorded_spot": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Advertisingspan

Find where the word “Acme” is spoken in this ad excerpt.

Give an editor a candidate interval to replay.

{
  "context": "Interval times are relative to the start of the excerpt, not the original full recording.",
  "questions": {
    "find_a_spoken_brand_name_in_an_ad_excerpt": {
      "type": "span",
      "instructions": "Find where the word \"Acme\" is spoken in this ad excerpt."
    }
  }
}

Temporal and contextual signals

Advertisingchoice

Does the customer’s reply to “Was the product easy to set up?” explicitly agree, explicitly disagree, or give a mixed answer?

Avoid editing a spoken disagreement into apparent agreement.

{
  "context": "The excerpt includes an interviewer asking \"Was the product easy to set up?\" and the customer's reply. Classify only what the reply explicitly says; mixed or qualified answers remain mixed rather than becoming a positive endorsement.",
  "questions": {
    "agreement_or_objection_in_a_testimonial": {
      "type": "choice",
      "instructions": "Does the customer's reply to \"Was the product easy to set up?\" explicitly agree, explicitly disagree, or give a mixed answer?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Mixed or qualified"
        },
        {
          "name": "Neither or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Advertisingnoul

Do two voices speak at the same time in this excerpt?

Flag a candidate take that needs listening or a mix adjustment.

{
  "questions": {
    "an_offer_line_under_another_voice": {
      "type": "noul",
      "instructions": "Do two voices speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Advertisingchoice

Which emotion is expressed by the voice in this excerpt?

Describe the vocal expression without claiming to know the customer’s feelings.

{
  "questions": {
    "emotion_conveyed_by_a_testimonial_excerpt": {
      "type": "choice",
      "instructions": "Which emotion is expressed by the voice in this excerpt?",
      "criteria": [
        {
          "name": "Happy"
        },
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Advertisingscore

How much do you agree that the generated voice delivers this line as a light, self-deprecating joke?

Compare a generated performance with an explicit creative brief.

{
  "context": "The line is \"Our plants have a better social life than I do.\" The brief calls for a light, self-deprecating joke rather than a serious complaint.",
  "questions": {
    "comedic_delivery_in_a_generated_spot": {
      "type": "score",
      "instructions": "How much do you agree that the generated voice delivers this line as a light, self-deprecating joke?",
      "criteria": ["Do not agree at all", "Disagree", "Neutral", "Agree", "Fully agree"]
    }
  }
}

Social and behavioral signals

Advertisingchoice

From the first sentence to the second, does the voice speed up, slow down, or keep the same speed?

Check a planned rate transition in the supplied take.

{
  "questions": {
    "check_speaking_rate_changes_between_ad_lines": {
      "type": "choice",
      "instructions": "From the first sentence to the second, does the voice speed up, slow down, or keep the same speed?",
      "criteria": [
        {
          "name": "Speeds up"
        },
        {
          "name": "Slows down"
        },
        {
          "name": "Keeps the same speed"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Advertisingscore

Is the speaking rate slow, moderate or fast?

Give concrete delivery feedback on a recording the campaign produces.

{
  "questions": {
    "pace_against_a_voiceover_brief": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Advertisingnoul

Are filled pauses such as “uh” or “um” audible in this excerpt?

Keep the words and audible hesitations distinguishable when preparing a verbatim testimonial quote.

{
  "questions": {
    "filled_pauses_in_a_testimonial_recording": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible in this excerpt?"
    }
  }
}

Speech and language

Advertisingchoice

Which listed accent does the voice sound closest to?

Check a concrete delivery attribute against a supplied recording brief.

{
  "questions": {
    "an_accent_specified_in_the_voice_brief": {
      "type": "choice",
      "instructions": "Which listed accent does the voice sound closest to?",
      "criteria": [
        {
          "name": "Mainstream US English"
        },
        {
          "name": "South African English"
        },
        {
          "name": "Neither or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Advertisingspan

Find the laughter in this dialogue excerpt.

Locate laughter for replay before choosing a dialogue cut.

{
  "context": "Interval times are relative to the start of the excerpt.",
  "questions": {
    "finding_the_laugh_for_a_dialogue_edit": {
      "type": "span",
      "instructions": "Find the laughter in this dialogue excerpt."
    }
  }
}

Non-speech events

Advertisingchoice

Does the spoken closing line ask the listener to act, state information, or make a commitment?

Organize an existing audio archive by the explicit function of its closing lines.

{
  "questions": {
    "what_the_call_to_action_says": {
      "type": "choice",
      "instructions": "Does the spoken closing line ask the listener to act, state information, or make a commitment?",
      "criteria": [
        {
          "name": "Asks the listener to act"
        },
        {
          "name": "States information"
        },
        {
          "name": "Makes a commitment"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Advertisingscore

How much of the speech in this excerpt can a listener understand?

Identify a passage where the spoken words may be difficult to make out in the mix.

{
  "questions": {
    "speech_audibility_in_a_finished_mix": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Advertisingnoul

Is music audible beneath the speech?

Avoid an accidental double music layer.

{
  "questions": {
    "music_already_present_in_a_voiceover": {
      "type": "noul",
      "instructions": "Is music audible beneath the speech?"
    }
  }
}

Acoustic environment

Contact centerschoice

Which emotion does the caller’s voice convey?

List emotions such as sad, angry and happy, and add “Neutral” and “None of these or unclear”. The answer is how the caller sounds, not whether they will cancel.

{
  "questions": {
    "callers_emotion": {
      "type": "choice",
      "instructions": "Which emotion does the caller's voice convey?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Contact centerschoice

Does the agent take over the turn, or does the caller carry on?

Send the moment the agent speaks during the caller’s explanation, with the speech just before and after. Say who is who in context.

{
  "context": "A support call. The agent answers the call, and the caller explains a billing problem.",
  "questions": {
    "acknowledgment_or_takeover": {
      "type": "choice",
      "instructions": "Does the agent take over the turn, or does the caller carry on?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the caller continues"
        }
      ]
    }
  }
}

Conversation dynamics

Contact centersnoul

Does the audio cut out or skip?

Send the audio as the listener received it, not a local microphone recording. The answer says whether audio drops out, not why.

{
  "questions": {
    "dropouts_on_a_call": {
      "type": "noul",
      "instructions": "Does the audio cut out or skip?"
    }
  }
}

Acoustic environment

Contact centerschoice

Which listed request is the caller making?

List the requests your line handles as options, such as cancel an order and return an item, and add “None of these or unclear”. Send that answer to a person.

{
  "questions": {
    "routing_a_caller": {
      "type": "choice",
      "instructions": "Which listed request is the caller making?",
      "criteria": [
        {
          "name": "Cancel an order"
        },
        {
          "name": "Ask about delivery"
        },
        {
          "name": "Return an item"
        },
        {
          "name": "Change the delivery address"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Contact centersscore

Does the reply address what the caller actually asked?

Send the caller’s question with the agent’s full reply. The answer judges relevance, not whether the policy is right or the request was completed.

{
  "context": "Assess relevance to \"How do I return an item?\", separately from whether the policy is correct or the return was completed.",
  "questions": {
    "agents_reply_on_topic": {
      "type": "score",
      "instructions": "Does the reply address what the caller actually asked?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Contact centerschoice

Which listed language is spoken in this voicemail?

Suggest a language queue for a person to confirm before returning the call.

{
  "questions": {
    "language_of_an_unassigned_voicemail": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this voicemail?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "Unclear or another language"
        }
      ]
    }
  }
}

Speech and language

Contact centerschoice

Which listed vocal sound best matches the non-speech sound in the selected passage?

Replace a generic non-speech note with a candidate audible-event label.

{
  "questions": {
    "a_vocal_event_in_an_unclear_transcript_note": {
      "type": "choice",
      "instructions": "Which listed vocal sound best matches the non-speech sound in the selected passage?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Cry"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Contact centersnoul

Does the caller pause for at least half a second and then continue speaking?

Identify a temporary pause within the caller’s turn.

{
  "questions": {
    "reviewing_pauses_within_a_callers_turn": {
      "type": "noul",
      "instructions": "Does the caller pause for at least half a second and then continue speaking?"
    }
  }
}

Conversation dynamics

Contact centersscore

How natural does this speech sound?

Review how a generated reply sounds in an actual customer exchange.

{
  "questions": {
    "naturalness_of_a_voice_assistants_service_reply": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Contact centerschoice

Which supplied word carries the emphasis?

Locate the emphasized word before interpreting what the customer meant.

{
  "context": "The customer says, \"I requested a refund.\" Listen to that recorded sentence and identify the stressed word rather than assuming intent from emphasis.",
  "questions": {
    "emphasis_in_a_disputed_service_request": {
      "type": "choice",
      "instructions": "Which supplied word carries the emphasis?",
      "criteria": [
        {
          "name": "I"
        },
        {
          "name": "Requested"
        },
        {
          "name": "A"
        },
        {
          "name": "Refund"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Contact centerschoice

Did the reply give the requested refund arrival time?

Check for the specific information requested rather than a merely related reply.

{
  "context": "Customer: \"When will my refund arrive?\" Judge whether the agent's reply supplies an arrival time; approval alone is not a date.",
  "questions": {
    "whether_a_refund_question_received_an_answer": {
      "type": "choice",
      "instructions": "Did the reply give the requested refund arrival time?",
      "criteria": [
        {
          "name": "Yes"
        },
        {
          "name": "Partly"
        },
        {
          "name": "No"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Contact centersnoul

Is music audible underneath the agent’s speech?

Check whether the support attachment captures the reported mixing problem.

{
  "questions": {
    "hold_music_continuing_over_speech": {
      "type": "noul",
      "instructions": "Is music audible underneath the agent's speech?"
    }
  }
}

Acoustic environment

Contact centerschoice

How many distinct voices are audible in this excerpt?

Describe how many voices can be heard before asking who is participating.

{
  "questions": {
    "more_than_one_caller_on_a_support_line": {
      "type": "choice",
      "instructions": "How many distinct voices are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Contact centersnoul

Do two speakers talk at the same time in this excerpt?

Check whether competing voices coincide with the part the customer missed.

{
  "questions": {
    "overlapping_speech_around_an_unclear_instruction": {
      "type": "noul",
      "instructions": "Do two speakers talk at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Contact centerschoice

How does the customer respond to the agent’s instruction “Please email us a photo of your receipt”?

Distinguish explicit agreement or disagreement from a question, instruction or unclear reply.

{
  "context": "The excerpt contains the agent saying \"Please email us a photo of your receipt\" and the customer's reply. Classify that reply only; do not infer consent, completed action or agreement from tone.",
  "questions": {
    "what_the_customers_reply_explicitly_does": {
      "type": "choice",
      "instructions": "How does the customer respond to the agent's instruction \"Please email us a photo of your receipt\"?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Asks a question or gives a different instruction"
        },
        {
          "name": "Mixed, something else or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Contact centerschoice

Which supplied tool should the assistant select first?

Illustrate a tool-selection decision without executing it.

{
  "context": "Supplied tools: transfer_to_agent — connect to a human; look_up_return_policy — retrieve the policy without changing an order; get_order_status — retrieve delivery status. Choose a name, not arguments or an action to execute.",
  "questions": {
    "first_tool_for_an_explicit_support_request": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "transfer_to_agent"
        },
        {
          "name": "look_up_return_policy"
        },
        {
          "name": "get_order_status"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Contact centersscore

Is the speaking rate slow, moderate or fast?

Describe the pace of an instruction a caller asked to hear again.

{
  "context": "Describe the pace of the agent's original instruction on how to return an item; do not equate the rate with whether the customer understood it.",
  "questions": {
    "rate_of_a_spoken_return_instruction": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Contact centersnoul

Is laughter audible in this excerpt?

Preserve an audible event in the transcript without interpreting it.

{
  "questions": {
    "a_nonverbal_event_in_a_call_transcript": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Contact centerschoice

Which listed sound is audible underneath the voice?

Describe an interfering sound without guessing where the caller is.

{
  "questions": {
    "what_is_audible_behind_the_customer": {
      "type": "choice",
      "instructions": "Which listed sound is audible underneath the voice?",
      "criteria": [
        {
          "name": "Hiss or static"
        },
        {
          "name": "Low hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Contact centersspan

Find the first occurrence of the word “Friday” in this delivery-date exchange.

Jump back to the quoted date instead of relying on an unchecked service note.

{
  "context": "Interval times are relative to the supplied one-minute excerpt.",
  "questions": {
    "replaying_a_stated_delivery_day": {
      "type": "span",
      "instructions": "Find the first occurrence of the word \"Friday\" in this delivery-date exchange."
    }
  }
}

Temporal and contextual signals

Contact centersspan

Find the parts of this service-call excerpt where return conditions are discussed.

Find the explanation a reviewer wants to replay.

{
  "context": "Interval times are relative to the start of the supplied excerpt.",
  "questions": {
    "replaying_the_explanation_of_return_conditions": {
      "type": "span",
      "instructions": "Find the parts of this service-call excerpt where return conditions are discussed."
    }
  }
}

Temporal and contextual signals

Corporate trainingscore

Does the instructor’s reply address the learner’s question?

Send the question with the full reply, and use levels from off-topic to directly relevant. The answer judges relevance, not whether the instructions are correct.

{
  "context": "The audio includes the learner asking \"How do I save a draft?\" and the instructor's reply. Judge relevance to that question independently of whether the software instructions are factually correct or complete.",
  "questions": {
    "instructors_answer_on_topic": {
      "type": "score",
      "instructions": "Does the instructor's reply address the learner's question?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Corporate trainingscore

How fast does the instructor talk?

Ask it of each section of a recorded walkthrough to find passages worth a closer listen. Fast speech is not always a problem.

{
  "questions": {
    "pace_of_a_walkthrough": {
      "type": "score",
      "instructions": "How fast does the instructor talk?",
      "criteria": ["Very slow", "Slow", "Moderate", "Fast", "Very fast"]
    }
  }
}

Speech and language

Corporate trainingnoul

Is the training instruction read exactly as scripted?

Put the script in context, such as “Save your work, close the application, then restart the computer.” The answer checks the reading, not the procedure.

{
  "context": "Script: Save your work, close the application, then restart the computer.",
  "questions": {
    "instruction_read_as_scripted": {
      "type": "noul",
      "instructions": "Is the training instruction read exactly as scripted?"
    }
  }
}

Communication quality

Corporate trainingscore

How strongly would listeners agree that the delivery fits the direction “calm and matter-of-fact”?

Help the learner compare an audible performance with the exercise direction.

{
  "context": "The learner says, \"I can help you work through the next step.\" The direction is calm and matter-of-fact, not sarcastic or celebratory. Judge the recording, not how the learner actually feels.",
  "questions": {
    "tone_in_a_customer_service_practice_reply": {
      "type": "score",
      "instructions": "How strongly would listeners agree that the delivery fits the direction \"calm and matter-of-fact\"?",
      "criteria": ["Do not agree at all", "Disagree", "Neutral", "Agree", "Fully agree"]
    }
  }
}

Emotion and expression

Corporate trainingspan

Find the section where the instructor demonstrates how to export the report.

Give the learner a candidate place to replay the worked example.

{
  "context": "Times are relative to the supplied audio.",
  "questions": {
    "find_the_worked_example_in_a_recorded_lesson": {
      "type": "span",
      "instructions": "Find the section where the instructor demonstrates how to export the report."
    }
  }
}

Temporal and contextual signals

Corporate trainingchoice

Which conversational move does this reply make?

Name the observable move for discussion during the exercise.

{
  "questions": {
    "the_conversational_move_in_a_practice_reply": {
      "type": "choice",
      "instructions": "Which conversational move does this reply make?",
      "criteria": [
        {
          "name": "Opening the call or introducing themselves"
        },
        {
          "name": "Giving instructions"
        },
        {
          "name": "Disagreeing or saying no"
        },
        {
          "name": "Agreeing or saying yes"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Corporate trainingnoul

Is background noise audible behind the speaker?

Note audible background sound in a submitted practice recording before giving feedback.

{
  "questions": {
    "noise_in_a_submitted_practice_recording": {
      "type": "noul",
      "instructions": "Is background noise audible behind the speaker?"
    }
  }
}

Acoustic environment

Corporate trainingspan

Find where the word “export” is spoken in this lesson excerpt.

Give a learner a candidate place to replay a term they missed.

{
  "context": "Interval times are relative to the start of the excerpt.",
  "questions": {
    "locating_a_named_command_in_a_lesson_recording": {
      "type": "span",
      "instructions": "Find where the word \"export\" is spoken in this lesson excerpt."
    }
  }
}

Temporal and contextual signals

Corporate trainingchoice

How many distinct speaking voices are audible in this excerpt?

Clarify whether an excerpt contains a single explanation or several audible speakers.

{
  "context": "Count voices in the selected excerpt; do not count silent attendees.",
  "questions": {
    "voices_in_a_workshop_recording": {
      "type": "choice",
      "instructions": "How many distinct speaking voices are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Corporate trainingscore

How natural does this speech sound?

Describe the sound of a generated lesson passage independently of its content.

{
  "questions": {
    "naturalness_of_generated_course_narration": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Corporate trainingchoice

Which listed help request does the learner make?

Identify a stated learning-support request without inferring ability or motivation.

{
  "questions": {
    "the_help_a_learner_explicitly_asks": {
      "type": "choice",
      "instructions": "Which listed help request does the learner make?",
      "criteria": [
        {
          "name": "Repeat a step"
        },
        {
          "name": "Explain a term"
        },
        {
          "name": "Show another example"
        },
        {
          "name": "Fix an access problem"
        }
      ]
    }
  }
}

Intent and outcomes

Corporate trainingnoul

Are filled pauses such as “uh” or “um” audible in this excerpt?

Give the presenter an observable feature of their own delivery.

{
  "questions": {
    "filled_pauses_in_a_presentation_rehearsal": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible in this excerpt?"
    }
  }
}

Speech and language

Corporate trainingchoice

Which word in “Send the replacement today” carries the emphasis in this take?

Check which word is emphasized in a customer-service practice line.

{
  "context": "Identify the actual emphasis in the recorded take.",
  "questions": {
    "emphasis_on_the_key_instruction": {
      "type": "choice",
      "instructions": "Which word in \"Send the replacement today\" carries the emphasis in this take?",
      "criteria": [
        {
          "name": "Send"
        },
        {
          "name": "The"
        },
        {
          "name": "Replacement"
        },
        {
          "name": "Today"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Corporate trainingnoul

Do two people speak at the same time in this excerpt?

Flag an excerpt with overlapping voices so a reviewer can replay it before transcribing the learner’s question.

{
  "questions": {
    "overlapping_voices_during_a_workshop_question": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Corporate trainingscore

How intrusive is the background sound?

Describe an audio problem that makes a recorded explanation harder to follow.

{
  "questions": {
    "background_interference_in_a_remote_workshop": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Corporate trainingnoul

Is the speaker audibly inhaling in this excerpt?

Avoid an abrupt-sounding join without asking the model to make the final cut.

{
  "questions": {
    "an_inhalation_near_a_proposed_narration_join": {
      "type": "noul",
      "instructions": "Is the speaker audibly inhaling in this excerpt?"
    }
  }
}

Non-speech events

Corporate trainingnoul

Is laughter audible in this excerpt?

Preserve an audible event without labeling the audience’s reaction.

{
  "questions": {
    "laughter_in_a_workshop_transcript": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Corporate trainingchoice

Does the learner explicitly agree or disagree with the proposed recap?

Keep a learner’s disagreement from becoming an apparent agreement in the recap.

{
  "context": "The facilitator says, \"So you wanted to offer a refund first,\" and the learner replies. Classify the learner's explicit response.",
  "questions": {
    "an_explicit_correction_in_a_role_play_debrief": {
      "type": "choice",
      "instructions": "Does the learner explicitly agree or disagree with the proposed recap?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Mixed or qualified"
        },
        {
          "name": "Neither or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Corporate trainingchoice

Does the interjection take over the turn, or does the first speaker carry on?

Select a concrete turn-taking moment to discuss with the trainee.

{
  "questions": {
    "taking_over_versus_acknowledging_in_a_role_play": {
      "type": "choice",
      "instructions": "Does the interjection take over the turn, or does the first speaker carry on?",
      "criteria": [
        {
          "name": "It interrupts and takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Corporate trainingchoice

Does the intonation sound monotone or expressive?

Give the recording’s author an observable delivery cue rather than a personality score.

{
  "questions": {
    "flat_or_lively_delivery_in_a_course_introduction": {
      "type": "choice",
      "instructions": "Does the intonation sound monotone or expressive?",
      "criteria": [
        {
          "name": "Monotone"
        },
        {
          "name": "Expressive"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Corporate trainingchoice

Which delivery day does the speaker ask for?

Check whether the trainee’s spoken request conveyed the day they intended.

{
  "questions": {
    "the_requested_day_in_a_practice_request": {
      "type": "choice",
      "instructions": "Which delivery day does the speaker ask for?",
      "criteria": [
        {
          "name": "Monday"
        },
        {
          "name": "Tuesday"
        },
        {
          "name": "Thursday"
        },
        {
          "name": "Friday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Enterprise meetingschoice

Which decision is stated in this part of the meeting?

List the decisions under discussion as options, and add “None of these or unclear”. The answer is the decision stated, not the likely outcome.

{
  "questions": {
    "decision_reached": {
      "type": "choice",
      "instructions": "Which decision is stated in this part of the meeting?",
      "criteria": [
        {
          "name": "Hand over on Tuesday"
        },
        {
          "name": "Hand over on Thursday"
        },
        {
          "name": "Keep the date undecided"
        },
        {
          "name": "Cancel the handover"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Enterprise meetingschoice

Which action is explicitly assigned in this excerpt?

List candidate actions as options, such as sending the checklist, and add “None of these or unclear”. Confirm the owner and deadline before adding it to a task list.

{
  "questions": {
    "action_item_assigned": {
      "type": "choice",
      "instructions": "Which action is explicitly assigned in this excerpt?",
      "criteria": [
        {
          "name": "Send the checklist"
        },
        {
          "name": "Schedule a walkthrough"
        },
        {
          "name": "Update the contact list"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Enterprise meetingsscore

Does the reply address the question that was asked?

Send a question such as “Who will own the handover checklist?” with its reply. A reply about the deadline concerns the project but not the question.

{
  "context": "Judge relevance to \"Who will own the handover checklist?\", not whether ownership was formally assigned. A statement about the deadline may be relevant to the project but does not necessarily address who owns the work.",
  "questions": {
    "reply_on_topic": {
      "type": "score",
      "instructions": "Does the reply address the question that was asked?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Enterprise meetingsnoul

Does the remote participant’s audio cut out or skip?

Send the meeting recording of the remote participant. The answer says whether audio drops out, not how well they presented.

{
  "questions": {
    "remote_participants_audio": {
      "type": "noul",
      "instructions": "Does the remote participant's audio cut out or skip?"
    }
  }
}

Acoustic environment

Enterprise meetingsspan

Find the parts of the meeting where the budget is discussed.

Find the relevant parts of a planning conversation to replay.

{
  "context": "Interval times are relative to the start of the supplied audio.",
  "questions": {
    "find_the_budget_discussion_in_a_recorded_meeting": {
      "type": "span",
      "instructions": "Find the parts of the meeting where the budget is discussed."
    }
  }
}

Temporal and contextual signals

Enterprise meetingsnoul

Do people speak at the same time in this excerpt?

Flag a passage that needs closer replay before attributing words.

{
  "questions": {
    "crosstalk_around_a_proposed_quotation": {
      "type": "noul",
      "instructions": "Do people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Enterprise meetingschoice

How does the selected reply respond to the proposal?

Capture the reply without turning it into an inference about team consensus.

{
  "context": "The excerpt contains a proposal and the reply immediately following it. Classify that reply only. A request for clarification is a question, not automatically disagreement; mixed or unclear responses do not establish consensus.",
  "questions": {
    "an_explicit_response_to_a_proposal": {
      "type": "choice",
      "instructions": "How does the selected reply respond to the proposal?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Asks a question"
        },
        {
          "name": "Qualified, something else or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Enterprise meetingschoice

How many distinct voices are audible in this excerpt?

Describe an excerpt that sounds like more than a solo update.

{
  "context": "Count audible speakers in this excerpt, not silent attendees or identities.",
  "questions": {
    "how_many_voices_join_the_discussion": {
      "type": "choice",
      "instructions": "How many distinct voices are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Enterprise meetingsnoul

Can you hear a switch between Mandarin and English?

Flag mixed-language passages for the appropriate transcription pass.

{
  "questions": {
    "a_bilingual_passage_needing_transcription": {
      "type": "noul",
      "instructions": "Can you hear a switch between Mandarin and English?"
    }
  }
}

Speech and language

Enterprise meetingschoice

Does the second speaker take over the turn, or does the first speaker carry on?

Distinguish a brief agreement sound from a change of speaker.

{
  "questions": {
    "acknowledgment_versus_taking_the_floor": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or does the first speaker carry on?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Enterprise meetingschoice

Which listed sound best describes the unexpected event in this excerpt?

Describe an audible interruption without identifying whose device produced it.

{
  "questions": {
    "a_background_event_in_the_meeting_recording": {
      "type": "choice",
      "instructions": "Which listed sound best describes the unexpected event in this excerpt?",
      "criteria": [
        {
          "name": "Speech"
        },
        {
          "name": "Dog"
        },
        {
          "name": "Vacuum cleaner"
        },
        {
          "name": "Alarm bell ringing"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Enterprise meetingsspan

Find the first occurrence of the word “Tuesday” in this meeting excerpt.

Let the note’s reader check the quoted date in context.

{
  "context": "Interval times are relative to the supplied one-minute excerpt.",
  "questions": {
    "replaying_the_quoted_deadline": {
      "type": "span",
      "instructions": "Find the first occurrence of the word \"Tuesday\" in this meeting excerpt."
    }
  }
}

Temporal and contextual signals

Enterprise meetingsscore

Rate the background sound in this passage, from very intrusive to not noticeable.

Describe the background sound when participants say a passage was hard to hear.

{
  "questions": {
    "background_interference_in_a_remote_meeting": {
      "type": "score",
      "instructions": "Rate the background sound in this passage, from very intrusive to not noticeable.",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Enterprise meetingschoice

Which listed language is spoken in this excerpt?

Choose the language needed for an appropriate transcript pass.

{
  "context": "The supplied passage is known to use one of the five listed languages.",
  "questions": {
    "language_of_a_passage_in_a_shared_recording": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Enterprise meetingsnoul

Is laughter audible in this excerpt?

Keep the transcript faithful without judging employee engagement or intent.

{
  "questions": {
    "laughter_in_a_verbatim_meeting_excerpt": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Enterprise meetingschoice

Which supplied word carries the strongest emphasis?

Locate stress in the actual meeting sentence before interpreting the speaker’s meaning.

{
  "context": "A participant says, \"We can ship Monday.\"",
  "questions": {
    "emphasis_in_a_spoken_deadline_proposal": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "We"
        },
        {
          "name": "Can"
        },
        {
          "name": "Ship"
        },
        {
          "name": "Monday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Enterprise meetingsscore

How much of the speech in this excerpt can a listener understand?

Separate an uncertain recording from a confirmed quotation in the notes.

{
  "questions": {
    "whether_a_meeting_quotation_is_intelligible": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Enterprise meetingsnoul

Is there a quiet pause of half a second or more inside this person’s turn?

Describe an audible gap without guessing why the speaker paused.

{
  "questions": {
    "a_quiet_pause_before_a_participant_continues": {
      "type": "noul",
      "instructions": "Is there a quiet pause of half a second or more inside this person's turn?"
    }
  }
}

Conversation dynamics

Entertainmentchoice

Which listed emotion does the voice convey?

List emotions such as sad, fearful and happy, and compare the answer with the direction. The answer is the performed expression, not the actor’s feelings.

{
  "questions": {
    "emotion_in_a_performed_line": {
      "type": "choice",
      "instructions": "Which listed emotion does the voice convey?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Entertainmentchoice

Which listed language is spoken in this excerpt?

List the languages of your localized versions as options, and compare the answer with the track’s label. Check tracks in other languages separately.

{
  "context": "The supplied track is from the English, Japanese, Arabic, French or Russian localized versions.",
  "questions": {
    "language_of_a_dubbed_track": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Entertainmentscore

How much of the speech in this excerpt can a listener understand?

Flag hard-to-understand dialogue for a listening check before mix revision.

{
  "questions": {
    "dialogue_audibility_in_the_mix": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Entertainmentspan

Find where the word “midnight” is spoken in this recording.

Find a spoken cue to replay before aligning an edit.

{
  "questions": {
    "find_a_spoken_cue_in_a_recording": {
      "type": "span",
      "instructions": "Find where the word \"midnight\" is spoken in this recording."
    }
  }
}

Temporal and contextual signals

Entertainmentnoul

Is music audible behind the dialogue?

Check whether a purported dialogue-only excerpt still contains audible music.

{
  "questions": {
    "music_in_a_dialogue_only_export": {
      "type": "noul",
      "instructions": "Is music audible behind the dialogue?"
    }
  }
}

Acoustic environment

Entertainmentscore

How intrusive is the background sound behind the dialogue?

Flag a mix passage for human listening without diagnosing a technical cause.

{
  "questions": {
    "dialogue_against_scene_effects": {
      "type": "score",
      "instructions": "How intrusive is the background sound behind the dialogue?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Entertainmentscore

How much do you agree that the generated voice delivers this joke in a fitting comedic way?

Get a rough listener-style read on whether a generated take delivers a joke in a fitting way.

{
  "context": "The line is \"I started a band called The Waiting List. We haven't got a gig yet.\" Judge whether the audible delivery suits a joke.",
  "questions": {
    "check_comedic_delivery_of_a_generated_joke_take": {
      "type": "score",
      "instructions": "How much do you agree that the generated voice delivers this joke in a fitting comedic way?",
      "criteria": ["Do not agree at all", "Disagree", "Neutral", "Agree", "Fully agree"]
    }
  }
}

Social and behavioral signals

Entertainmentnoul

Are filled pauses such as “uh” or “um” audible?

Describe an audible feature of an improvised take without treating it as a flaw.

{
  "questions": {
    "filled_pauses_in_an_improvised_scene": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Entertainmentscore

How fast is the speaking rate in this line?

Check a concrete direction without interpreting the actor’s actual state.

{
  "questions": {
    "pace_in_a_performed_line": {
      "type": "score",
      "instructions": "How fast is the speaking rate in this line?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Entertainmentchoice

Does the second speaker take over the turn, or briefly acknowledge while the first continues?

Check one performed interruption against the scene’s conversational staging.

{
  "questions": {
    "a_character_taking_over_another_characters_turn": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Entertainmentchoice

Which listed vocal sound is the cue in this excerpt?

Describe a short non-speech cue before comparing it with the script.

{
  "questions": {
    "the_type_of_vocal_cue_in_a_scene": {
      "type": "choice",
      "instructions": "Which listed vocal sound is the cue in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Cry"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Entertainmentnoul

Is laughter audible in this excerpt?

Check for a specific recorded performance cue.

{
  "questions": {
    "check_that_a_scripted_laugh_was_recorded": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Entertainmentchoice

Which listed request does the character make?

Keep a dialogue annotation faithful to what a character asks for.

{
  "questions": {
    "the_explicit_request_in_a_scene": {
      "type": "choice",
      "instructions": "Which listed request does the character make?",
      "criteria": [
        {
          "name": "Open the door"
        },
        {
          "name": "Call a taxi"
        },
        {
          "name": "Keep a secret"
        },
        {
          "name": "Wait outside"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Entertainmentchoice

How many distinct speakers are audible in this excerpt?

Check the audible cast size of a selected passage, not the identity of the cast.

{
  "context": "Count voices speaking in the supplied passage, including any off-screen voices, not silent performers; do not identify characters.",
  "questions": {
    "speaking_voices_in_a_scene_excerpt": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Entertainmentchoice

Does the reply to “Let’s leave town tonight.” explicitly agree or disagree with that proposal?

Check whether the dialogue contains an explicit agreement before describing the scene that way.

{
  "context": "The scene excerpt includes one character proposing \"Let's leave town tonight.\" and the other character's reply. Treat a qualified or mixed reply as unresolved rather than assuming cooperation.",
  "questions": {
    "agreement_or_objection_in_a_scene_exchange": {
      "type": "choice",
      "instructions": "Does the reply to \"Let's leave town tonight.\" explicitly agree or disagree with that proposal?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Mixed, neither or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Entertainmentscore

How natural does this speech sound?

Review the perceived naturalness of generated character dialogue when choosing takes.

{
  "questions": {
    "review_naturalness_of_synthetic_character": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Entertainmentchoice

Which decision is explicitly reached in this exchange?

Check a scene log against the spoken decision.

{
  "questions": {
    "the_decision_stated_in_a_scene": {
      "type": "choice",
      "instructions": "Which decision is explicitly reached in this exchange?",
      "criteria": [
        {
          "name": "Meet at the station"
        },
        {
          "name": "Meet at the café"
        },
        {
          "name": "Meet at the bridge"
        },
        {
          "name": "Do not meet again"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Entertainmentchoice

Is the laugh before or after the spoken word “finally”?

Check whether a recorded laugh occurs on the specified side of a scripted word.

{
  "questions": {
    "check_a_scripted_laugh_cue_against_the_spoken": {
      "type": "choice",
      "instructions": "Is the laugh before or after the spoken word \"finally\"?",
      "criteria": [
        {
          "name": "Before"
        },
        {
          "name": "After"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Entertainmentchoice

Which word in “You took my ticket.” carries the strongest emphasis?

Identify the performed stress to compare a take with the supplied script direction.

{
  "context": "Identify the word actually emphasized in the supplied take.",
  "questions": {
    "compare_a_dialogue_takes_emphasis": {
      "type": "choice",
      "instructions": "Which word in \"You took my ticket.\" carries the strongest emphasis?",
      "criteria": [
        {
          "name": "You"
        },
        {
          "name": "Took"
        },
        {
          "name": "My"
        },
        {
          "name": "Ticket"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Entertainmentnoul

Do the two voices speak at the same time in this closing-phrase excerpt?

Check the presence of a planned conversational event.

{
  "questions": {
    "a_scripted_simultaneous_line": {
      "type": "noul",
      "instructions": "Do the two voices speak at the same time in this closing-phrase excerpt?"
    }
  }
}

Conversation dynamics

Entertainmentscore

On a scale from definitely different to definitely the same, how does the replacement line’s voice compare with the surrounding dialogue?

Check perceived voice continuity across a dialogue edit.

{
  "context": "The listening file plays an original dialogue excerpt, the replacement line, then the original excerpt again. Compare how consistent the voice sounds across the edit.",
  "questions": {
    "check_voice_continuity_after_replacing_2": {
      "type": "score",
      "instructions": "On a scale from definitely different to definitely the same, how does the replacement line's voice compare with the surrounding dialogue?",
      "criteria": ["Definitely different", "Probably different", "Unsure", "Probably the same", "Definitely the same"]
    }
  }
}

Speakers and voices

Financial serviceschoice

Which listed request is the caller making?

List the topics your line handles as options, such as changing an address and paying a bill. The answer neither authenticates the caller nor changes the account.

{
  "questions": {
    "routing_a_banking_request": {
      "type": "choice",
      "instructions": "Which listed request is the caller making?",
      "criteria": [
        {
          "name": "Change an address"
        },
        {
          "name": "Use a card abroad"
        },
        {
          "name": "Pay a bill"
        },
        {
          "name": "Report a problem with the app"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Financial servicesnoul

Is the transaction reference number read back correctly?

Put the expected number in context, such as “four six two, nine one seven”, and send the agent’s readout.

{
  "context": "Expected reference number: four six two, nine one seven",
  "questions": {
    "reference_number_read_back": {
      "type": "noul",
      "instructions": "Is the transaction reference number read back correctly?"
    }
  }
}

Communication quality

Financial servicesscore

Does the reply address what the customer asked?

Send the customer’s question about a fee with the full reply. The answer judges relevance, not whether the fee or the explanation is correct.

{
  "context": "Judge relevance, not whether the fee is lawful, the explanation is true or a refund is owed.",
  "questions": {
    "reply_to_a_fee_question": {
      "type": "score",
      "instructions": "Does the reply address what the customer asked?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Financial servicesnoul

Can an echo of the speech be heard?

Send the audio as the caller heard it. The answer says whether an echo is heard, not what the account information was.

{
  "questions": {
    "echo_on_a_banking_call": {
      "type": "noul",
      "instructions": "Can an echo of the speech be heard?"
    }
  }
}

Acoustic environment

Financial serviceschoice

How does the caller respond to the callback offer, including whether an agreement sounds hesitant?

Distinguish the caller’s stated response from how an agreement sounds.

{
  "questions": {
    "agreement_and_hesitant_sounding_delivery": {
      "type": "choice",
      "instructions": "How does the caller respond to the callback offer, including whether an agreement sounds hesitant?",
      "criteria": [
        {
          "name": "Agrees, with hesitant-sounding delivery"
        },
        {
          "name": "Agrees, without clear hesitant-sounding delivery"
        },
        {
          "name": "Declines"
        },
        {
          "name": "Asks a question"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Financial serviceschoice

Which supplied word carries the strongest emphasis?

Locate the stressed word before asking the customer to clarify its meaning.

{
  "context": "The customer says, \"I paid that bill yesterday.\" Identify the audible stress, without deciding whether the payment actually happened.",
  "questions": {
    "emphasis_in_a_disputed_transaction_account": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "I"
        },
        {
          "name": "Paid"
        },
        {
          "name": "That"
        },
        {
          "name": "Bill"
        },
        {
          "name": "Yesterday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Financial serviceschoice

How many distinct speakers are audible in this excerpt?

Count audible voices before the person handling the call asks who is present.

{
  "questions": {
    "several_voices_on_the_customer_side_of_a_call": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Financial serviceschoice

Which emotion is conveyed by the voice in this excerpt?

Describe vocal expression separately from the merits of the complaint.

{
  "questions": {
    "emotion_expressed_during_a_service_complaint": {
      "type": "choice",
      "instructions": "Which emotion is conveyed by the voice in this excerpt?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Financial servicesscore

Is the speaking rate slow, moderate or fast?

Describe the pace of the customer’s spoken explanation, not their competence or emotional state.

{
  "questions": {
    "speaking_rate_while_explaining_an_account": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Financial serviceschoice

Does the representative take over the turn, or briefly acknowledge while the customer continues?

Describe one exchange where a customer and representative begin speaking together.

{
  "context": "The representative is the speaker who begins speaking at the start of the excerpt.",
  "questions": {
    "acknowledgment_or_taking_over_in_a_banking_call": {
      "type": "choice",
      "instructions": "Does the representative take over the turn, or briefly acknowledge while the customer continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Financial servicesspan

Find where the explanation of the fee begins.

Find the part of a call a reviewer wants to replay.

{
  "questions": {
    "finding_the_explanation_of_a_fee": {
      "type": "span",
      "instructions": "Find where the explanation of the fee begins."
    }
  }
}

Temporal and contextual signals

Financial servicesscore

How intrusive is the background sound?

Flag a noisy message for closer listening before a callback request is written down.

{
  "questions": {
    "noise_in_a_callback_voicemail": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Financial servicesnoul

Do two people speak at the same time in this excerpt?

Check whether simultaneous speech is present where part of the exchange was missed.

{
  "questions": {
    "overlapping_voices_during_an_address_explanation": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Financial servicesspan

Find the first occurrence of “Friday” in this recording.

Locate the date mention so its surrounding qualification can be checked.

{
  "context": "Times are relative to the supplied one-minute excerpt.",
  "questions": {
    "replaying_a_refund_time_statement": {
      "type": "span",
      "instructions": "Find the first occurrence of \"Friday\" in this recording."
    }
  }
}

Temporal and contextual signals

Financial servicesnoul

Are filled pauses such as “uh” or “um” audible?

Describe audible hesitations without treating them as evidence about the account or caller.

{
  "questions": {
    "filled_pauses_in_an_explanation": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Financial serviceschoice

Which supplied tool should the assistant select first?

Illustrate the next administrative function without executing an account change.

{
  "context": "Tools: transfer_to_staff connects them to staff; explain_address_change describes the steps; get_opening_hours returns service hours. Select only a tool name.",
  "questions": {
    "selecting_a_first_service_tool": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "transfer_to_staff"
        },
        {
          "name": "explain_address_change"
        },
        {
          "name": "get_opening_hours"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Financial serviceschoice

Which listed language is spoken in this excerpt?

Identify a likely language so the caller can be offered communication support.

{
  "context": "The call is known to use one of the five listed languages.",
  "questions": {
    "language_support_for_a_banking_call": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Financial serviceschoice

Which listed vocal sound is audible in this excerpt?

Describe a non-word event without interpreting its cause or the customer’s state.

{
  "questions": {
    "a_vocal_sound_in_a_recorded_customer_account": {
      "type": "choice",
      "instructions": "Which listed vocal sound is audible in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Other vocal sound"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Financial servicesnoul

Is laughter audible in this excerpt?

Preserve a non-word event when preparing a verbatim quotation.

{
  "questions": {
    "laughter_in_a_customer_interview_quotation": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Gamingchoice

Which listed action is the player asking for?

List your voice menu as options, such as open the map, show the quest log and pause the game. Handle commands outside the menu separately.

{
  "questions": {
    "voice_command": {
      "type": "choice",
      "instructions": "Which listed action is the player asking for?",
      "criteria": [
        {
          "name": "Open the map"
        },
        {
          "name": "Show the quest log"
        },
        {
          "name": "Pause the game"
        },
        {
          "name": "Open audio settings"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Gamingchoice

Is the teammate asking someone to act, reporting an event or offering to act?

List the three as options, and add “None of these or unclear”. The answer classifies what is said, not teamwork or intent.

{
  "questions": {
    "teammate_callout": {
      "type": "choice",
      "instructions": "Is the teammate asking someone to act, reporting an event or offering to act?",
      "criteria": [
        {
          "name": "Asks someone to act"
        },
        {
          "name": "Reports an event"
        },
        {
          "name": "Offers to act"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Gamingnoul

Do two players speak at the same time in this excerpt?

The answer says whether speech overlaps, not who was at fault.

{
  "questions": {
    "overlapping_callouts": {
      "type": "noul",
      "instructions": "Do two players speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Gamingchoice

Which emotion does the actor’s delivery convey in this take?

Ask it of each take of a line such as “Get back! The bridge is falling!”, and keep the takes that match the brief. The answer is the performed expression.

{
  "questions": {
    "choosing_a_performance_take": {
      "type": "choice",
      "instructions": "Which emotion does the actor's delivery convey in this take?",
      "criteria": [
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Sad"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Gamingnoul

Is laughter audible in this clip?

Search a game’s sound library by asking it of every candidate clip, then audition the matches.

{
  "questions": {
    "laughter_for_a_game_asset": {
      "type": "noul",
      "instructions": "Is laughter audible in this clip?"
    }
  }
}

Non-speech events

Gamingscore

How natural does the speech sound?

Flag a draft generated line for revision before selecting the tutorial recording.

{
  "questions": {
    "naturalness_of_a_generated_tutorial_take": {
      "type": "score",
      "instructions": "How natural does the speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Gamingchoice

Which listed language is spoken in this voice message?

Help a player decide which language support or translation option to request.

{
  "context": "For this example, the voice message is known to use one of the five listed languages.",
  "questions": {
    "language_in_a_multiplayer_voice_message": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this voice message?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Gamingscore

Is the speaking rate slow, moderate or fast?

Describe how quickly a teammate speaks during a recorded exchange, without equating pace with skill or urgency.

{
  "context": "Rate the pace of the selected voice-chat passage, separately from microphone quality, word recognition or whether the player made the right decision.",
  "questions": {
    "speaking_rate_in_a_teammates_callout": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Gamingchoice

How many distinct speaking voices are audible in this excerpt?

Distinguish one audible speaker from several without assigning voices to player accounts.

{
  "questions": {
    "how_many_voices_are_in_a_shared_microphone_clip": {
      "type": "choice",
      "instructions": "How many distinct speaking voices are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Gamingchoice

Does the intonation sound monotone or expressive?

Check whether a designed voice candidate matches the character brief.

{
  "questions": {
    "a_deliberately_flat_fictional_guide_voice": {
      "type": "choice",
      "instructions": "Does the intonation sound monotone or expressive?",
      "criteria": [
        {
          "name": "Monotone"
        },
        {
          "name": "Expressive"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Gamingchoice

Which supplied function should the game companion select first?

Distinguish a request for location information from immediately starting a match.

{
  "context": "Available functions: find_nearest_checkpoint returns the nearest checkpoint; show_all_checkpoints lists every checkpoint without selecting the nearest; start_match begins a match; open_settings opens preferences. The question selects a function only; it does not execute it.",
  "questions": {
    "selecting_a_companion_function_for_a_spoken": {
      "type": "choice",
      "instructions": "Which supplied function should the game companion select first?",
      "criteria": [
        {
          "name": "find_nearest_checkpoint"
        },
        {
          "name": "show_all_checkpoints"
        },
        {
          "name": "start_match"
        },
        {
          "name": "open_settings"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Gamingscore

How directly does the companion’s reply address the question “How do I reopen the map?”

Separate an on-topic reply from one that answers a different question.

{
  "context": "Judge relevance, not whether the control instructions are accurate for the game version.",
  "questions": {
    "whether_the_companion_reply_addresses": {
      "type": "score",
      "instructions": "How directly does the companion's reply address the question \"How do I reopen the map?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Gamingscore

How intrusive is the background sound in this clip?

Describe distracting background sound when teammates say a voice is difficult to hear.

{
  "questions": {
    "background_intrusion_in_a_players_voice_chat": {
      "type": "score",
      "instructions": "How intrusive is the background sound in this clip?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Gamingchoice

Which supplied word carries the strongest emphasis?

Identify which word in a recorded callout was stressed when reviewing a match.

{
  "context": "The recorded teammate says, \"Defend the left gate.\" Describe the audible emphasis, not the correct tactic or the speaker's intention.",
  "questions": {
    "the_word_stressed_in_a_tactical_callout": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Defend"
        },
        {
          "name": "The"
        },
        {
          "name": "Left"
        },
        {
          "name": "Gate"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Gamingchoice

Which listed vocal sound is audible in this excerpt?

Identify a selected vocal sound before a dialogue editor decides whether to trim it.

{
  "questions": {
    "a_sound_marked_in_a_character_take": {
      "type": "choice",
      "instructions": "Which listed vocal sound is audible in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Other vocal sound"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Gamingspan

Where is laughter audible in this excerpt?

Mark where laughter occurs in a text account of the exchange without guessing its meaning.

{
  "context": "Times are measured from the start of the supplied excerpt.",
  "questions": {
    "marking_laughter_in_a_saved_multiplayer": {
      "type": "span",
      "instructions": "Where is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Gamingscore

How would most listeners rate the technical sound quality of this speech?

Describe the received voice signal when a player reports a call-quality problem.

{
  "context": "Rate the received voice-chat speech recording, separately from game sound design or the player's communication skill.",
  "questions": {
    "technical_quality_of_a_received_chat_message": {
      "type": "score",
      "instructions": "How would most listeners rate the technical sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Gamingspan

Where is the word “retreat” spoken in this voice-chat excerpt?

Find where a known callout is spoken so the player can replay the surrounding exchange.

{
  "context": "Times are measured from the start of the excerpt, not from game-state events.",
  "questions": {
    "finding_a_known_callout_in_a_match_recording": {
      "type": "span",
      "instructions": "Where is the word \"retreat\" spoken in this voice-chat excerpt?"
    }
  }
}

Temporal and contextual signals

Gamingchoice

Which word is spoken first: “wait” or “go”?

Check an assembled recording against the intended cue sequence.

{
  "questions": {
    "order_of_two_tutorial_cues": {
      "type": "choice",
      "instructions": "Which word is spoken first: \"wait\" or \"go\"?",
      "criteria": [
        {
          "name": "Wait"
        },
        {
          "name": "Go"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Healthcarescore

How intrusive is the background noise?

Send the audio the listener heard on the clinic or telehealth call. The answer rates the noise, not the medical content.

{
  "questions": {
    "noise_on_a_telehealth_call": {
      "type": "score",
      "instructions": "How intrusive is the background noise?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Healthcarenoul

Do two people speak at the same time in this excerpt?

Send the part of the exchange that is hard to follow. Both people may simply have started speaking at once.

{
  "questions": {
    "overlap_in_a_follow_up_call": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Healthcarechoice

Which listed request is the caller making?

List your service flows as options, such as booking an appointment, changing one and asking about a bill. The answer is the stated request, not a motive.

{
  "questions": {
    "routing_an_appointment_request": {
      "type": "choice",
      "instructions": "Which listed request is the caller making?",
      "criteria": [
        {
          "name": "Book an appointment"
        },
        {
          "name": "Change an appointment"
        },
        {
          "name": "Ask about a bill"
        },
        {
          "name": "Request clinic hours"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Healthcarescore

Does the reply address what the caller actually asked?

Send a question such as “Do you have Saturday appointments?” with the full reply. The answer judges relevance, not whether the availability is right.

{
  "context": "Judge relevance to \"Does the clinic have Saturday appointments?\", separately from whether the availability information is accurate or complete.",
  "questions": {
    "reply_to_a_patients_question": {
      "type": "score",
      "instructions": "Does the reply address what the caller actually asked?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Healthcarechoice

Which of the listed languages is spoken in this excerpt?

Use the answer to suggest language support, then confirm with the caller. Calls in unlisted languages need separate handling.

{
  "context": "For this example, the clinic-call excerpt is known to use one of the five listed languages.",
  "questions": {
    "language_of_a_clinic_call": {
      "type": "choice",
      "instructions": "Which of the listed languages is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Healthcarescore

How would most listeners rate the technical sound quality of this speech?

Describe recording quality when a participant says the connection is difficult to hear.

{
  "questions": {
    "technical_quality_of_a_clinic_or_telehealth_call": {
      "type": "score",
      "instructions": "How would most listeners rate the technical sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Healthcarenoul

Is laughter audible in this excerpt?

Preserve an audible event in the quote record without inferring satisfaction, humor, diagnosis or who laughed.

{
  "questions": {
    "laughter_in_a_patient_experience_quote": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Healthcarechoice

Did the reply give the requested appointment arrival time?

Spot a specific unanswered administrative question before closing the conversation.

{
  "context": "Caller: \"What time should I arrive for my appointment?\" The staff reply follows in the recording. Judge whether the requested arrival time was actually supplied, not whether the reply was friendly or on-topic.",
  "questions": {
    "whether_the_caller_received_the_information_they": {
      "type": "choice",
      "instructions": "Did the reply give the requested appointment arrival time?",
      "criteria": [
        {
          "name": "Yes"
        },
        {
          "name": "Partly"
        },
        {
          "name": "No"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Healthcarechoice

How much of the spoken instruction can be understood from this excerpt?

Identify a spoken passage that may need repetition or clarification.

{
  "questions": {
    "how_understandable_an_instruction_sounds": {
      "type": "choice",
      "instructions": "How much of the spoken instruction can be understood from this excerpt?",
      "criteria": [
        {
          "name": "All or nearly all"
        },
        {
          "name": "Most"
        },
        {
          "name": "About half"
        },
        {
          "name": "Very little"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Communication quality

Healthcarechoice

How does the caller respond to the offered appointment?

Record an observable response without treating it as legal/medical consent or proof of unspoken willingness.

{
  "context": "The excerpt contains the appointment offer and the caller's reply. Classify the caller's reply only.",
  "questions": {
    "an_explicit_response_to_an_appointment_offer": {
      "type": "choice",
      "instructions": "How does the caller respond to the offered appointment?",
      "criteria": [
        {
          "name": "Accepts the offered appointment"
        },
        {
          "name": "Declines the offered appointment"
        },
        {
          "name": "Asks a question or requests a different slot"
        },
        {
          "name": "Mixed, something else or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Healthcarenoul

Is there a quiet pause of half a second or more within the patient’s turn?

Identify whether a quiet gap occurs while the patient still has the turn.

{
  "questions": {
    "a_quiet_pause_inside_the_patients_turn": {
      "type": "noul",
      "instructions": "Is there a quiet pause of half a second or more within the patient's turn?"
    }
  }
}

Conversation dynamics

Healthcarechoice

Does the vocal delivery fit the supplied words, or sound noticeably different?

Find a passage worth clarifying with the patient without calling it deception or suppressed emotion.

{
  "context": "The patient says, \"I'm fine with that plan.\" Compare the audible delivery with those words; do not infer hidden reluctance or consent.",
  "questions": {
    "when_the_words_and_vocal_delivery_seem_different": {
      "type": "choice",
      "instructions": "Does the vocal delivery fit the supplied words, or sound noticeably different?",
      "criteria": [
        {
          "name": "Delivery fits the words"
        },
        {
          "name": "Delivery seems different"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Healthcarechoice

Which word carries the strongest emphasis?

Locate the stressed word before asking what the patient meant, rather than assigning a meaning to the stress.

{
  "context": "The patient says, \"I didn't think that would happen.\" Identify audible emphasis, not the patient's intention.",
  "questions": {
    "emphasis_in_a_patients_account": {
      "type": "choice",
      "instructions": "Which word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "I"
        },
        {
          "name": "Didn't"
        },
        {
          "name": "Think"
        },
        {
          "name": "That"
        },
        {
          "name": "Would"
        },
        {
          "name": "Happen"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Healthcarechoice

Which listed background sound is audible underneath the voice?

Give a concrete description of the sound without guessing its source.

{
  "questions": {
    "the_kind_of_noise_interfering_with_a_call": {
      "type": "choice",
      "instructions": "Which listed background sound is audible underneath the voice?",
      "criteria": [
        {
          "name": "Hiss or steady static"
        },
        {
          "name": "Low hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Healthcarespan

Where in this excerpt do the patient and clinic staff discuss follow-up appointment arrangements?

Find the passages to replay rather than generate a clinical summary.

{
  "questions": {
    "finding_the_discussion_of_follow_up_arrangements": {
      "type": "span",
      "instructions": "Where in this excerpt do the patient and clinic staff discuss follow-up appointment arrangements?"
    }
  }
}

Temporal and contextual signals

Healthcarechoice

Which offered appointment does “that one” refer to in the caller’s reply?

Avoid putting an unsupported date reference into a callback summary.

{
  "context": "The selected recording runs from the offers of Tuesday morning and Thursday afternoon through the caller's reply using \"that one.\" Use No clear reference if the exchange does not resolve the reference.",
  "questions": {
    "which_offered_appointment_the_caller_means": {
      "type": "choice",
      "instructions": "Which offered appointment does \"that one\" refer to in the caller's reply?",
      "criteria": [
        {
          "name": "Tuesday morning"
        },
        {
          "name": "Thursday afternoon"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Healthcarespan

Where does coughing occur in this excerpt?

Help someone find an audible event for a closer listen, without measuring a symptom.

{
  "questions": {
    "a_replay_window_for_a_coughing_sound": {
      "type": "span",
      "instructions": "Where does coughing occur in this excerpt?"
    }
  }
}

Non-speech events

Healthcarechoice

How many distinct speakers are audible in this excerpt?

Clarify how many voices are audible before asking who is speaking.

{
  "questions": {
    "more_than_one_voice_on_the_patient_side": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Healthcarenoul

Is background music audible behind the caller?

Describe whether music is part of the background in a difficult-to-hear patient call.

{
  "questions": {
    "music_behind_the_caller": {
      "type": "noul",
      "instructions": "Is background music audible behind the caller?"
    }
  }
}

Acoustic environment

Healthcarechoice

Does the second speaker take over the turn, or briefly acknowledge while the first speaker continues?

Make one turn-taking event clear enough to discuss without scoring the clinician’s empathy.

{
  "questions": {
    "acknowledgment_or_taking_over_during_a_patient": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first speaker continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Healthcarescore

Is the speaking rate slow, moderate or fast?

Describe how quickly the patient speaks in a selected passage, without treating pace as a symptom.

{
  "context": "The selected passage is the patient's explanation of why they called. Rate its pace—not its clarity, the patient's distress or medical urgency.",
  "questions": {
    "speaking_rate_during_a_patients_explanation": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Healthcarechoice

How does the selected patient reply contribute to the exchange?

Describe what the person says or asks rather than infer whether they care or understand.

{
  "questions": {
    "observable_participation_in_a_consultation": {
      "type": "choice",
      "instructions": "How does the selected patient reply contribute to the exchange?",
      "criteria": [
        {
          "name": "Asks a question"
        },
        {
          "name": "Adds information or elaborates"
        },
        {
          "name": "Gives a brief acknowledgment only"
        },
        {
          "name": "Mixed or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Healthcarenoul

Does this clip appear to contain artificially generated speech?

Support a manual review of an unusual call recording without deciding who is entitled to care.

{
  "questions": {
    "possible_synthetic_speech_on_the_clinic_phone": {
      "type": "noul",
      "instructions": "Does this clip appear to contain artificially generated speech?"
    }
  }
}

Speakers and voices

Healthcarechoice

Which emotion is conveyed by the speaker’s voice in this excerpt?

Describe the vocal expression in a selected patient utterance, not the patient’s private feelings.

{
  "questions": {
    "emotion_expressed_while_discussing_a_concern": {
      "type": "choice",
      "instructions": "Which emotion is conveyed by the speaker's voice in this excerpt?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Healthcarechoice

Which listed vocal sound is audible in this excerpt?

Make an audio note more specific without turning it into a clinical finding.

{
  "questions": {
    "naming_a_vocal_sound_in_a_consultation_recording": {
      "type": "choice",
      "instructions": "Which listed vocal sound is audible in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Cry"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Healthcarechoice

Which supplied tool should the assistant select first?

Illustrate an administrative next step without authorizing or executing it.

{
  "context": "Available tools: find_appointment_options — list available slots without changing a booking; transfer_to_staff — connect the caller to clinic staff; get_clinic_hours — return opening hours. Select a tool name only; this question does not execute a transfer or alter a booking.",
  "questions": {
    "choosing_the_first_tool_for_a_clinic_phone": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "find_appointment_options"
        },
        {
          "name": "transfer_to_staff"
        },
        {
          "name": "get_clinic_hours"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Healthcarechoice

After which supplied word does the speaker pause?

Create a focused replay point for a clinician’s clarification question.

{
  "context": "The consultation clip contains \"I felt better after lunch yesterday.\" Use the supplied transcript and candidate words to locate the audible pause; do not infer why it occurred or what caused the reported experience.",
  "questions": {
    "locating_a_pause_within_the_patients_sentence": {
      "type": "choice",
      "instructions": "After which supplied word does the speaker pause?",
      "criteria": [
        {
          "name": "Felt"
        },
        {
          "name": "Better"
        },
        {
          "name": "After"
        },
        {
          "name": "Lunch"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Healthcarescore

How natural does this speech sound?

Review how the generated reply sounds during an actual administrative exchange.

{
  "questions": {
    "naturalness_of_a_clinic_voice_assistants_reply": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Healthcarenoul

Are filled pauses such as “uh” or “um” audible in this excerpt?

Describe audible filled pauses in the patient’s spoken account, without interpreting their cause.

{
  "questions": {
    "filled_pauses_while_describing_a_concern": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible in this excerpt?"
    }
  }
}

Speech and language

Insurancechoice

Which listed request is the caller making?

List service topics as options, such as policy documents, payments and claim status. The answer does not decide the request or assess coverage.

{
  "questions": {
    "policy_service_request": {
      "type": "choice",
      "instructions": "Which listed request is the caller making?",
      "criteria": [
        {
          "name": "Request policy documents"
        },
        {
          "name": "Ask about a payment"
        },
        {
          "name": "Check claim status"
        },
        {
          "name": "Change contact details"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Insurancechoice

Which supplied tool should the assistant select first?

List tool names as options, such as get_required_document_list and get_claim_status, each with a description of what it does.

{
  "context": "The claim is already identified. Tools: get_required_document_list returns its requested-document checklist; get_claim_status returns status without that checklist; send_policy_copy sends policy wording, not the checklist; contact_claims_team requests human help. This question only selects a function.",
  "questions": {
    "choosing_a_tool": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "get_required_document_list",
          "description": "Returns the checklist of documents requested for a claim."
        },
        {
          "name": "get_claim_status",
          "description": "Returns a claim's status, without the document checklist."
        },
        {
          "name": "send_policy_copy",
          "description": "Sends the policy wording, not the checklist."
        },
        {
          "name": "contact_claims_team",
          "description": "Asks a person on the claims team for help."
        },
        {
          "name": "None of these or unclear",
          "description": "No listed tool fits the request."
        }
      ]
    }
  }
}

Intent and outcomes

Insurancechoice

What does “those” refer to in this recorded exchange?

List the earlier items as options, such as the photographs and the receipts, and add “None of these or unclear”. On that answer, ask the caller instead of guessing.

{
  "context": "Earlier in the recorded call, the caller mentions photographs of the damage and receipts for the repairs. The representative later says, \"Please upload those by Friday.\" Use None of these or unclear if the spoken context does not resolve what \"those\" refers to.",
  "questions": {
    "what_a_reference_points_to": {
      "type": "choice",
      "instructions": "What does \"those\" refer to in this recorded exchange?",
      "criteria": [
        {
          "name": "The photographs"
        },
        {
          "name": "The receipts"
        },
        {
          "name": "Both groups of documents"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Insurancenoul

Do two people speak at the same time in this excerpt?

Send the exchange where the caller asks a question while the agent is explaining. On a yes, ask for unclear wording to be repeated.

{
  "questions": {
    "overlap_about_documents": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Insurancechoice

Does the caller state “I will send a report” as their next step, or say it as a question about the instruction?

Keep a questioned instruction from being recorded as a confirmed next step.

{
  "context": "The representative says, \"Please send the damage report.\" The caller replies, \"I will send a report.\" Listen to whether the caller states a next step or questions the instruction.",
  "questions": {
    "clarify_whether_a_document_next_step_is_being": {
      "type": "choice",
      "instructions": "Does the caller state \"I will send a report\" as their next step, or say it as a question about the instruction?",
      "criteria": [
        {
          "name": "States the next step"
        },
        {
          "name": "Questions the instruction"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Insurancescore

How directly does the representative’s reply address the question “Can I upload photographs of the damage?”

Distinguish relevance from correctness or a completed submission.

{
  "context": "Judge whether the representative's reply concerns the question \"Can I upload photographs of the damage?\"; do not verify the policy, legal requirements or whether anything was uploaded.",
  "questions": {
    "whether_a_reply_addresses_the_document_question": {
      "type": "score",
      "instructions": "How directly does the representative's reply address the question \"Can I upload photographs of the damage?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Insurancescore

How would most listeners rate the technical sound quality of this speech?

Describe the audio when a caller cannot hear an explanation clearly.

{
  "questions": {
    "technical_quality_of_an_insurance_service_call": {
      "type": "score",
      "instructions": "How would most listeners rate the technical sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Insurancescore

How would most listeners rate the background sound, from very intrusive to not noticeable?

Flag a difficult recording for closer listening before preparing its callback note.

{
  "questions": {
    "background_intrusion_in_a_service_voicemail": {
      "type": "score",
      "instructions": "How would most listeners rate the background sound, from very intrusive to not noticeable?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Insurancechoice

When the representative’s voice comes in during the claimant’s explanation, does the representative take over the turn, or briefly acknowledge while the claimant continues?

Describe one moment where another voice enters before the first turn ends.

{
  "context": "The claimant is the speaker explaining which documents they already sent; the representative is the voice entering during that explanation.",
  "questions": {
    "acknowledgment_or_taking_over_a_callers": {
      "type": "choice",
      "instructions": "When the representative's voice comes in during the claimant's explanation, does the representative take over the turn, or briefly acknowledge while the claimant continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Insurancechoice

Which emotion is conveyed by the voice in this excerpt?

Describe audible expression separately from the claim’s merits.

{
  "questions": {
    "emotion_expressed_in_a_claim_service": {
      "type": "choice",
      "instructions": "Which emotion is conveyed by the voice in this excerpt?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Insurancechoice

Which listed language is spoken in this excerpt?

Identify a likely language before offering an appropriate communication channel.

{
  "context": "The call is known to use one of the five listed languages.",
  "questions": {
    "language_support_for_a_policyholder": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Insurancechoice

Which supplied word carries the strongest emphasis?

Identify the word a caller stresses before asking what they want clarified.

{
  "context": "The caller asks, \"Do you need the original receipt?\" Identify the stressed word without inferring the intended meaning of the emphasis.",
  "questions": {
    "emphasis_in_a_document_question": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Do"
        },
        {
          "name": "You"
        },
        {
          "name": "Need"
        },
        {
          "name": "The"
        },
        {
          "name": "Original"
        },
        {
          "name": "Receipt"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Insurancechoice

How many distinct speakers are audible in this excerpt?

Clarify how many voices can be heard before a person confirms who is participating.

{
  "questions": {
    "several_voices_in_a_policy_service_call": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Insurancenoul

Are filled pauses such as “uh” or “um” audible?

Describe audible hesitation without treating it as proof about the reported event.

{
  "questions": {
    "filled_pauses_in_a_claim_service_explanation": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Insurancespan

Find where the document-submission deadline is discussed.

Locate a candidate replay passage for someone checking the wording of the conversation.

{
  "context": "Times are relative to the excerpt.",
  "questions": {
    "finding_where_a_submission_deadline_was": {
      "type": "span",
      "instructions": "Find where the document-submission deadline is discussed."
    }
  }
}

Temporal and contextual signals

Insurancenoul

Is laughter audible in this excerpt?

Preserve an audible event in the verbatim transcript before quoting the speaker.

{
  "questions": {
    "laughter_in_a_recorded_service_experience": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Insurancechoice

Which listed vocal sound is audible in this excerpt?

Describe a non-word transcript marker without interpreting it as evidence about the event.

{
  "questions": {
    "a_vocal_sound_in_a_recorded_account": {
      "type": "choice",
      "instructions": "Which listed vocal sound is audible in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Cry"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Insurancenoul

Is a word said twice in a row in this excerpt?

Describe an audible repetition without deciding why it occurred.

{
  "questions": {
    "repeated_words_in_a_callers_account": {
      "type": "noul",
      "instructions": "Is a word said twice in a row in this excerpt?"
    }
  }
}

Speech and language

Insurancescore

How natural does this speech sound?

Review the sound of generated speech in an administrative exchange.

{
  "questions": {
    "naturalness_of_an_insurance_assistants_reply": {
      "type": "score",
      "instructions": "How natural does this speech sound?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Market researchchoice

Which listed change does the participant ask for first?

List the features introduced in the session as options, such as larger print and an easier opening. The answer codes only what is said.

{
  "questions": {
    "requested_change": {
      "type": "choice",
      "instructions": "Which listed change does the participant ask for first?",
      "criteria": [
        {
          "name": "Larger print"
        },
        {
          "name": "Easier opening"
        },
        {
          "name": "Smaller package"
        },
        {
          "name": "A resealable closure"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Market researchscore

How positive or negative is the opinion expressed?

Use levels from very negative to very positive. The answer separates praise from criticism, not purchase intent.

{
  "questions": {
    "opinion_of_a_product": {
      "type": "score",
      "instructions": "How positive or negative is the opinion expressed?",
      "criteria": ["Very negative", "Negative", "Neutral", "Positive", "Very positive"]
    }
  }
}

Emotion and expression

Market researchchoice

Which prototype does “that one” refer to in the reply?

Send the exchange in which the prototypes are discussed, list them as options, and add “None of these or unclear”.

{
  "context": "The selected exchange includes discussion of a blue box and a white pouch, followed by the participant saying \"that one.\" Use the audible preceding context; leave the reference unresolved if the exchange does not distinguish them.",
  "questions": {
    "which_prototype_is_meant": {
      "type": "choice",
      "instructions": "Which prototype does \"that one\" refer to in the reply?",
      "criteria": [
        {
          "name": "Blue box"
        },
        {
          "name": "White pouch"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Market researchnoul

Do two people speak at the same time in this excerpt?

Check unclear passages before attributing a quotation to a participant.

{
  "questions": {
    "overlap_in_a_focus_group": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Market researchspan

Find where the sustained packaging discussion begins.

Find a candidate passage for human review.

{
  "questions": {
    "finding_the_packaging_discussion": {
      "type": "span",
      "instructions": "Find where the sustained packaging discussion begins."
    }
  }
}

Temporal and contextual signals

Market researchchoice

Does the participant’s reply explicitly agree or disagree with the moderator’s suggestion that the package is easy to open?

Keep an explicit objection from being summarized as agreement.

{
  "context": "The excerpt contains the moderator's suggestion that the package is easy to open and the participant's reply. Classify that reply only.",
  "questions": {
    "an_explicit_response_to_a_suggestion": {
      "type": "choice",
      "instructions": "Does the participant's reply explicitly agree or disagree with the moderator's suggestion that the package is easy to open?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Mixed, hedged, neither or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Market researchnoul

Is music audible beneath the participant’s speech?

Record an audible condition that may complicate transcription.

{
  "questions": {
    "music_beneath_a_remote_interview": {
      "type": "noul",
      "instructions": "Is music audible beneath the participant's speech?"
    }
  }
}

Acoustic environment

Market researchscore

How much of the speech in this excerpt can a listener understand?

Avoid presenting an uncertain interpretation as a confirmed quotation.

{
  "questions": {
    "whether_a_quotation_is_understandable": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Market researchchoice

Which listed sample does the participant ask to see again?

Select a stated next step without inferring which product the participant prefers.

{
  "context": "The session offered a blue box, white pouch, green bottle and red tin. The selected reply supplies the request; count only a sample the participant explicitly asks to see again, and use the fallback if it names several or none of them.",
  "questions": {
    "the_sample_explicitly_requested": {
      "type": "choice",
      "instructions": "Which listed sample does the participant ask to see again?",
      "criteria": [
        {
          "name": "Blue box"
        },
        {
          "name": "White pouch"
        },
        {
          "name": "Green bottle"
        },
        {
          "name": "Red tin"
        },
        {
          "name": "None, several or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Market researchscore

How intrusive is the background sound in this interview passage?

Describe background sound in a difficult-to-hear interview passage.

{
  "questions": {
    "interference_in_an_interview_passage": {
      "type": "score",
      "instructions": "How intrusive is the background sound in this interview passage?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Market researchchoice

Which listed vocal sound is audible in this excerpt?

Replace a generic non-speech marker with a candidate sound label.

{
  "questions": {
    "the_type_of_vocal_sound_in_an_interview_note": {
      "type": "choice",
      "instructions": "Which listed vocal sound is audible in this excerpt?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Laugh"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Market researchchoice

Which of the listed languages is spoken in this excerpt?

Find the appropriate language-support path before transcription.

{
  "context": "For this example, the interview is known to use one of the five listed languages.",
  "questions": {
    "language_in_an_incoming_interview_recording": {
      "type": "choice",
      "instructions": "Which of the listed languages is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Market researchnoul

Are filled pauses such as “uh” or “um” audible?

Flag a transcript detail for human confirmation.

{
  "questions": {
    "fillers_in_a_verbatim_interview_quote": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Market researchnoul

Is there a quiet pause of half a second or more within the participant’s answer, before another person speaks?

Keep an audible gap separate from an interpretation of the participant’s reaction.

{
  "questions": {
    "a_quiet_pause_within_an_interview_answer": {
      "type": "noul",
      "instructions": "Is there a quiet pause of half a second or more within the participant's answer, before another person speaks?"
    }
  }
}

Conversation dynamics

Market researchchoice

Which word in “It works for me” carries the strongest emphasis?

Keep a stressed qualification visible when choosing a quote.

{
  "questions": {
    "emphasis_in_a_participants_qualification": {
      "type": "choice",
      "instructions": "Which word in \"It works for me\" carries the strongest emphasis?",
      "criteria": [
        {
          "name": "It"
        },
        {
          "name": "Works"
        },
        {
          "name": "For"
        },
        {
          "name": "Me"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Market researchchoice

How many distinct speaking voices are audible?

Help distinguish a single response from multi-voice discussion.

{
  "context": "Count distinct audible voices, not silent people; the count does not identify participants.",
  "questions": {
    "a_solo_interview_or_a_group_exchange": {
      "type": "choice",
      "instructions": "How many distinct speaking voices are audible?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Market researchscore

How directly does the participant’s reply address the question “What was difficult about opening the package?”

Distinguish a relevant answer from a change of subject when organizing responses.

{
  "context": "Judge whether the participant's reply addresses \"What was difficult about opening the package?\", not whether the opinion is correct or sufficiently detailed.",
  "questions": {
    "whether_an_interview_response_addresses": {
      "type": "score",
      "instructions": "How directly does the participant's reply address the question \"What was difficult about opening the package?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Market researchnoul

Is laughter audible in this excerpt?

Preserve an audible nonverbal event without interpreting it as delight, discomfort or agreement.

{
  "questions": {
    "laughter_in_a_research_quotation": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Market researchchoice

Does the moderator’s overlapping “right” take over the turn or briefly acknowledge the participant?

Help preserve a continuous quotation without treating every interjection as a new answer.

{
  "questions": {
    "acknowledgment_or_taking_the_floor": {
      "type": "choice",
      "instructions": "Does the moderator's overlapping \"right\" take over the turn or briefly acknowledge the participant?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment without taking the floor"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Market researchchoice

Which conversational move does the participant’s reply to the moderator’s feature explanation make?

Describe the participant’s observable response, not their level of engagement.

{
  "questions": {
    "questions_versus_short_acknowledgments": {
      "type": "choice",
      "instructions": "Which conversational move does the participant's reply to the moderator's feature explanation make?",
      "criteria": [
        {
          "name": "Asks a question"
        },
        {
          "name": "Adds information"
        },
        {
          "name": "Briefly acknowledges"
        },
        {
          "name": "Mixed or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Media productionnoul

Is laughter audible in this clip?

Ask it of candidate clips for a scene or montage, then listen to the matches to pick one.

{
  "questions": {
    "laughter_for_a_scene": {
      "type": "noul",
      "instructions": "Is laughter audible in this clip?"
    }
  }
}

Non-speech events

Media productionnoul

Are filled pauses such as “uh” or “um” audible?

Flag takes to listen to before choosing which pauses to keep in a quotation.

{
  "questions": {
    "fillers_in_an_interview": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Media productionnoul

Do two people speak at the same time in this passage?

Check whether the interviewer overlaps an answer before cutting it as a quotation.

{
  "questions": {
    "crosstalk_in_an_answer": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this passage?"
    }
  }
}

Conversation dynamics

Media productionscore

How intrusive is the background sound?

Use levels from not noticeable to very intrusive. An editor still decides whether to use the take.

{
  "questions": {
    "background_in_a_take": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Media productionspan

Find the parts of the interview discussing bridge renovation.

Find relevant sections to replay while assembling a rough cut.

{
  "context": "Times are relative to the supplied audio.",
  "questions": {
    "finding_a_discussion_within_a_long_interview": {
      "type": "span",
      "instructions": "Find the parts of the interview discussing bridge renovation."
    }
  }
}

Temporal and contextual signals

Media productionchoice

After which supplied word is there an audible pause?

Suggest a word-relative listening point for a shorter quotation.

{
  "context": "The interview passage says, \"We finished the repairs on Friday. Then we opened the doors for the first time.\" Locate the audible pause relative to the supplied words.",
  "questions": {
    "a_pause_near_a_proposed_interview_cut": {
      "type": "choice",
      "instructions": "After which supplied word is there an audible pause?",
      "criteria": [
        {
          "name": "Repairs"
        },
        {
          "name": "Friday"
        },
        {
          "name": "Opened"
        },
        {
          "name": "Doors"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Media productionchoice

Which supplied word carries the emphasis?

Check a concrete performance direction before selecting the take.

{
  "context": "Supplied narration: \"The museum opens today.\" Identify the prominence actually heard.",
  "questions": {
    "emphasis_in_a_narration_line": {
      "type": "choice",
      "instructions": "Which supplied word carries the emphasis?",
      "criteria": [
        {
          "name": "The"
        },
        {
          "name": "Museum"
        },
        {
          "name": "Opens"
        },
        {
          "name": "Today"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Media productionchoice

Which listed nonverbal vocal sound is audible?

Replace a generic marker with a bounded vocal-sound label for human confirmation.

{
  "questions": {
    "a_nonverbal_cue_in_a_podcast_recording": {
      "type": "choice",
      "instructions": "Which listed nonverbal vocal sound is audible?",
      "criteria": [
        {
          "name": "Cough"
        },
        {
          "name": "Breath"
        },
        {
          "name": "Other vocal sound"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Media productionchoice

Does the second voice take over the turn, or briefly acknowledge while the first speaker continues?

Distinguish a brief interjection from a change of speaker turn before shortening an interview.

{
  "questions": {
    "an_interviewers_acknowledgment_within_an_answer": {
      "type": "choice",
      "instructions": "Does the second voice take over the turn, or briefly acknowledge while the first speaker continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Media productionchoice

Which listed emotion does the voice convey?

Describe how a quoted line sounds without assigning the interviewee a private feeling.

{
  "questions": {
    "expression_in_a_selected_interview_answer": {
      "type": "choice",
      "instructions": "Which listed emotion does the voice convey?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Media productionchoice

Which of the listed languages is spoken in this clip?

Identify which language track an interview clip belongs with.

{
  "context": "The clip is known, for this example, to use one of the five listed languages.",
  "questions": {
    "language_in_an_interview_recording": {
      "type": "choice",
      "instructions": "Which of the listed languages is spoken in this clip?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Media productionscore

Does the replacement line sound like the same voice as the surrounding narration?

Check perceived voice continuity after replacing a narration line.

{
  "context": "The listening file plays an original narration excerpt, the replacement line, then the original excerpt again. Compare the perceived voice continuity across the edit.",
  "questions": {
    "check_voice_continuity_after_replacing": {
      "type": "score",
      "instructions": "Does the replacement line sound like the same voice as the surrounding narration?",
      "criteria": ["Definitely different", "Probably different", "Unsure", "Probably the same", "Definitely the same"]
    }
  }
}

Speakers and voices

Media productionscore

How much of the speech in this excerpt can be understood?

Check a passage whose words are hard to make out before it is quoted.

{
  "questions": {
    "understandable_words_in_a_field_interview": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can be understood?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Media productionchoice

Which listed background sound is audible underneath the voice?

Name the audible noise when discussing an audio problem in a take.

{
  "questions": {
    "the_kind_of_background_sound_in_a_location": {
      "type": "choice",
      "instructions": "Which listed background sound is audible underneath the voice?",
      "criteria": [
        {
          "name": "Hiss or static"
        },
        {
          "name": "Low hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Media productionnoul

Is laughter audible in this excerpt?

Add a descriptive sound note without interpreting why anyone laughed.

{
  "questions": {
    "laughter_missing_from_a_podcast_transcript": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Media productionchoice

Does the guest’s reply explicitly agree or disagree with the description?

Avoid presenting an explicit objection as endorsement.

{
  "context": "The excerpt contains the interviewer's description of the project and the guest's reply. Classify that reply.",
  "questions": {
    "an_explicit_response_in_an_interview": {
      "type": "choice",
      "instructions": "Does the guest's reply explicitly agree or disagree with the description?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Mixed, qualified, neither or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Media productionchoice

How many distinct speakers are audible in this excerpt?

Describe the audible speaker count before separating interview turns in an edit.

{
  "context": "Count the voices heard in the selected passage, not the people visible on camera or silent participants.",
  "questions": {
    "voices_in_a_panel_interview_excerpt": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Media productionchoice

Which decision does the speaker say the garden group reached?

Keep the clip log faithful to the decision the speaker actually reports.

{
  "questions": {
    "a_decision_reported_in_an_interview": {
      "type": "choice",
      "instructions": "Which decision does the speaker say the garden group reached?",
      "criteria": [
        {
          "name": "Move the garden to a new site"
        },
        {
          "name": "Set up a shared volunteer rota"
        },
        {
          "name": "Grow vegetables for a food bank"
        },
        {
          "name": "Extend the opening hours"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Media productionnoul

Is background music audible behind the speech?

Check whether an interview excerpt already carries a music bed before adding another soundtrack.

{
  "questions": {
    "music_already_mixed_into_an_interview_quote": {
      "type": "noul",
      "instructions": "Is background music audible behind the speech?"
    }
  }
}

Acoustic environment

Media productionchoice

Which supplied tool should the assistant select first?

Illustrate choosing an editing-support tool without changing the project.

{
  "context": "Available tools: search_transcripts — search recorded interview transcripts; play_clip — play a clip already selected; export_sequence — export an existing edit. Select a tool name only; do not execute it.",
  "questions": {
    "an_editors_spoken_request_to_a_media_assistant": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "search_transcripts"
        },
        {
          "name": "play_clip"
        },
        {
          "name": "export_sequence"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Media productionscore

How natural does the synthesized narration sound?

Give perceptual feedback on a generated voice performance.

{
  "questions": {
    "naturalness_of_generated_narration": {
      "type": "score",
      "instructions": "How natural does the synthesized narration sound?",
      "criteria": ["Very unnatural", "Unnatural", "Neither natural nor unnatural", "Natural", "Very natural"]
    }
  }
}

Communication quality

Media productionscore

Is the speaking rate slow, moderate or fast?

Describe the pace when a narrated section feels rushed or slow.

{
  "questions": {
    "pace_of_an_explainer_narration": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Media searchnoul

Is laughter audible in this excerpt?

An editor still checks who is laughing and whether the moment suits the sequence.

{
  "questions": {
    "laughter_for_a_montage": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Media searchchoice

Which listed emotion does the voice convey?

List emotions such as sad and happy, and add “Neutral” and “None of these or unclear”. The answer is the audible expression, not the speaker’s feelings.

{
  "questions": {
    "happy_sounding_reaction": {
      "type": "choice",
      "instructions": "Which listed emotion does the voice convey?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Disgusted"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Media searchnoul

Can a running water tap be heard?

Ask it of ambience recordings before an editor picks the usable part.

{
  "questions": {
    "running_tap": {
      "type": "noul",
      "instructions": "Can a running water tap be heard?"
    }
  }
}

Non-speech events

Media searchspan

Find the parts of this interview discussing the renovation.

Find the relevant parts of an interview to replay.

{
  "context": "Interval times are relative to the supplied audio's beginning.",
  "questions": {
    "find_a_topic_within_an_interview": {
      "type": "span",
      "instructions": "Find the parts of this interview discussing the renovation."
    }
  }
}

Temporal and contextual signals

Media searchspan

Find where “Friday” is spoken in this recording.

Open the relevant part of a candidate recording for listening.

{
  "questions": {
    "find_a_quoted_word_in_a_recording": {
      "type": "span",
      "instructions": "Find where \"Friday\" is spoken in this recording."
    }
  }
}

Temporal and contextual signals

Media searchchoice

Which supplied tool should the assistant select first?

Illustrate a search assistant’s next step with explicit tool definitions.

{
  "context": "Available tools: search_transcripts — search episode transcripts; play_selected_episode — play an already selected episode; show_saved_clips — list saved clips. Choose only a tool name; no tool is executed by this question.",
  "questions": {
    "choosing_a_first_action_for_a_spoken_media": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "search_transcripts"
        },
        {
          "name": "play_selected_episode"
        },
        {
          "name": "show_saved_clips"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Media searchchoice

Which listed sound can be heard?

Narrow a sound-library candidate to a known kind of sound.

{
  "questions": {
    "an_environmental_sound_among_supplied_types": {
      "type": "choice",
      "instructions": "Which listed sound can be heard?",
      "criteria": [
        {
          "name": "Speech"
        },
        {
          "name": "Dog"
        },
        {
          "name": "Vacuum cleaner"
        },
        {
          "name": "Alarm bell ringing"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Non-speech events

Media searchscore

How much of the speech in this excerpt can be understood?

Check whether the words in a promising search result can be made out.

{
  "questions": {
    "understandable_words_in_a_candidate_interview": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can be understood?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Media searchscore

Is the speaking rate slow, moderate or fast?

Find candidate passages with a requested delivery pace.

{
  "context": "Tag the selected candidate's pace, not its importance or the speaker's emotional state.",
  "questions": {
    "a_fast_spoken_passage_in_a_clip_collection": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Media searchchoice

Does the second speaker take over the turn, or briefly acknowledge while the first continues?

Find an example of a particular conversational event, not a rude person.

{
  "questions": {
    "a_turn_takeover_rather_than_a_brief": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Media searchchoice

Which series is the person asking to find?

Pass the named catalogue value into a separate search step.

{
  "context": "The recorded request asks for one of these catalogue series: \"City Walks,\" \"Night Kitchen,\" \"Game Notes\" or \"Garden Hour.\"",
  "questions": {
    "a_title_named_in_a_spoken_search_request": {
      "type": "choice",
      "instructions": "Which series is the person asking to find?",
      "criteria": [
        {
          "name": "City Walks"
        },
        {
          "name": "Night Kitchen"
        },
        {
          "name": "Game Notes"
        },
        {
          "name": "Garden Hour"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Media searchnoul

Is music audible in this excerpt?

Avoid selecting a quote that already contains an unwanted music bed.

{
  "questions": {
    "finding_a_quote_without_background_music": {
      "type": "noul",
      "instructions": "Is music audible in this excerpt?"
    }
  }
}

Acoustic environment

Media searchchoice

Which listed accent does the voice most resemble?

Narrow candidate recordings for the supplied production brief.

{
  "context": "Describe the clip's audible accent rather than a person's background.",
  "questions": {
    "accent_for_a_supplied_voiceover_brief": {
      "type": "choice",
      "instructions": "Which listed accent does the voice most resemble?",
      "criteria": [
        {
          "name": "Mainstream US English"
        },
        {
          "name": "South African English"
        },
        {
          "name": "Neither or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Media searchchoice

How many distinct speakers are audible in this clip?

Distinguish candidate clip formats before choosing an interview excerpt.

{
  "context": "Count only voices in the supplied candidate; do not identify people or infer who is present but silent.",
  "questions": {
    "a_single_voice_or_multi_voice_candidate": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this clip?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Media searchchoice

Which listed language is spoken in this clip?

Narrow a known multilingual collection before listening to search results.

{
  "context": "For this example, the archive candidates are known to contain speech in English, Japanese, Arabic, French or Russian.",
  "questions": {
    "language_tags_for_unlabelled_archive_clips": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this clip?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Media searchchoice

Does the reply explicitly refuse the request?

Find a literal conversational move without inferring whether the speaker is unwilling in private.

{
  "context": "The excerpt includes the request and the reply. If the reply is qualified, indirect or unclear, choose Mixed or unclear.",
  "questions": {
    "an_explicit_refusal_in_a_candidate_exchange": {
      "type": "choice",
      "instructions": "Does the reply explicitly refuse the request?",
      "criteria": [
        {
          "name": "Explicit refusal"
        },
        {
          "name": "No explicit refusal"
        },
        {
          "name": "Mixed or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Media searchnoul

Do two speakers talk at the same time in this excerpt?

Flag candidates that need careful replay before a clean quote is selected.

{
  "questions": {
    "a_quote_without_competing_speakers": {
      "type": "noul",
      "instructions": "Do two speakers talk at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Media searchscore

How would most listeners rate the technical sound quality of this speech?

Prioritize technically clearer candidates for a human listen.

{
  "context": "The candidate is a phone or VoIP interview recording. Rate the technical quality of the speech signal, separately from what the speaker says.",
  "questions": {
    "sound_quality_of_a_phone_interview_excerpt": {
      "type": "score",
      "instructions": "How would most listeners rate the technical sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Media searchchoice

Which listed conversational move does the speaker make?

Narrow candidate excerpts for an editor to confirm against the brief.

{
  "questions": {
    "a_clip_containing_an_explicit_offer_or_promise": {
      "type": "choice",
      "instructions": "Which listed conversational move does the speaker make?",
      "criteria": [
        {
          "name": "Asking someone to act"
        },
        {
          "name": "Reporting information"
        },
        {
          "name": "Offering or promising an action"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Media searchchoice

Which listed background sound is audible underneath the voice?

Add an audible-type tag to distinguish noisy candidates.

{
  "questions": {
    "the_noise_type_in_an_archive_candidate": {
      "type": "choice",
      "instructions": "Which listed background sound is audible underneath the voice?",
      "criteria": [
        {
          "name": "Hiss or static"
        },
        {
          "name": "Low hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Media searchchoice

Which supplied word carries the strongest emphasis?

Find candidate delivery that matches a specific spoken emphasis.

{
  "context": "The supplied sentence is \"This is our home\". Identify the audible emphasis.",
  "questions": {
    "a_clip_emphasizing_a_target_word": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "This"
        },
        {
          "name": "Is"
        },
        {
          "name": "Our"
        },
        {
          "name": "Home"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Media searchscore

How would most listeners rate the naturalness of this generated narration?

Compare a speech-quality impression across candidate generated recordings.

{
  "questions": {
    "natural_sounding_generated_narration_candidates": {
      "type": "score",
      "instructions": "How would most listeners rate the naturalness of this generated narration?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Communication quality

Moderationchoice

Does the speaker ask for an action, report information or offer an action?

Send the reply with the nearby conversation, and add “None of these or unclear” as an option. The answer does not decide whether a rule was broken.

{
  "questions": {
    "act_in_a_reported_reply": {
      "type": "choice",
      "instructions": "Does the speaker ask for an action, report information or offer an action?",
      "criteria": [
        {
          "name": "Asks someone to act"
        },
        {
          "name": "Reports information"
        },
        {
          "name": "Offers or promises an action"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Moderationchoice

How does the other person explicitly respond to “Please stop”?

List options such as agrees, refuses and apologizes, and add “None of these or unclear”. The answer labels the reply, not whether the conduct stopped.

{
  "context": "The selected short exchange includes \"Please stop\" and the other person's reply immediately afterward. Classify that reply only.",
  "questions": {
    "reply_to_a_request_to_stop": {
      "type": "choice",
      "instructions": "How does the other person explicitly respond to \"Please stop\"?",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly refuses"
        },
        {
          "name": "Apologizes"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Moderationnoul

Do two people speak at the same time in this excerpt?

Check before a reviewer attributes the reported phrase. The answer does not say who said it.

{
  "questions": {
    "overlap_around_a_reported_phrase": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Moderationspan

Find where “leave me alone” is spoken in this recording.

Locate the quoted passage without assuming it is a policy violation.

{
  "context": "Return intervals measured from the beginning of the supplied audio.",
  "questions": {
    "find_a_reported_phrase_in_a_recording": {
      "type": "span",
      "instructions": "Find where \"leave me alone\" is spoken in this recording."
    }
  }
}

Temporal and contextual signals

Moderationchoice

Which of the listed sounds is audible in this excerpt?

Describe an audible event so a reviewer can check it in context.

{
  "questions": {
    "identify_a_non_speech_sound_in_a_reported_clip": {
      "type": "choice",
      "instructions": "Which of the listed sounds is audible in this excerpt?",
      "criteria": [
        {
          "name": "Screaming"
        },
        {
          "name": "Crying"
        },
        {
          "name": "Gunshot-like sound"
        },
        {
          "name": "Several of these"
        },
        {
          "name": "Another sound or unclear"
        }
      ]
    }
  }
}

Non-speech events

Moderationchoice

Which review label best describes this exchange under the supplied rule?

Organize a reported exchange for human review under an explicitly supplied rule.

{
  "context": "Supplied example platform rule: \"Repeatedly directing personal insults at another participant after they have asked for it to stop is disallowed. Quoting an insult to report it is not itself a violation of this rule.\"",
  "questions": {
    "triage_a_reported_exchange_against_a_supplied": {
      "type": "choice",
      "instructions": "Which review label best describes this exchange under the supplied rule?",
      "criteria": [
        {
          "name": "Matches the rule's prohibited conduct"
        },
        {
          "name": "Does not match the rule's prohibited conduct"
        },
        {
          "name": "Insufficient context"
        }
      ]
    }
  }
}

Social and behavioral signals

Moderationscore

How directly does the reply address the question “Why are you messaging me again?”

Check whether a reply addresses the preceding question in a reported exchange.

{
  "context": "In this audio conversation between two participants, one asks, \"Why are you messaging me again?\" Assess whether the other participant's reply addresses the question, not whether it is true, appropriate or a policy violation.",
  "questions": {
    "whether_a_reply_addresses_a_question": {
      "type": "score",
      "instructions": "How directly does the reply address the question \"Why are you messaging me again?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Moderationscore

How intrusive is the background sound?

Describe the audible interference rather than guessing the words through it.

{
  "questions": {
    "background_interference_in_a_reported_clip": {
      "type": "score",
      "instructions": "How intrusive is the background sound?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Moderationnoul

Are filled pauses such as “uh” or “um” audible?

Check whether a verbatim quotation omits an audible filled pause.

{
  "questions": {
    "a_filled_pause_in_a_disputed_quotation": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Moderationchoice

Which supplied word carries the strongest emphasis?

Locate stress without assigning sarcasm or hostile intent.

{
  "context": "The attached clip contains \"Please leave me alone.\" Identify a stressed candidate word, not the meaning or policy status of the sentence.",
  "questions": {
    "emphasis_in_the_reported_sentence": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Please"
        },
        {
          "name": "Leave"
        },
        {
          "name": "Me"
        },
        {
          "name": "Alone"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Moderationchoice

Which listed delivery style is audible?

Add an audible delivery description for replay without treating it as aggression.

{
  "questions": {
    "audible_delivery_style_in_a_reported_passage": {
      "type": "choice",
      "instructions": "Which listed delivery style is audible?",
      "criteria": [
        {
          "name": "Shouting"
        },
        {
          "name": "Whispering"
        },
        {
          "name": "Laughing while speaking"
        },
        {
          "name": "Ordinary speaking voice"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Moderationnoul

Does this clip appear to contain artificially generated speech?

Keep audio-origin uncertainty separate from the truth of a report.

{
  "questions": {
    "synthetic_origin_question_for_an_audio": {
      "type": "noul",
      "instructions": "Does this clip appear to contain artificially generated speech?"
    }
  }
}

Speakers and voices

Moderationnoul

Is laughter audible in this excerpt?

Preserve an event for context without labelling it as mockery or enjoyment.

{
  "questions": {
    "a_nonverbal_event_omitted_from_a_report": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Moderationchoice

Does the second speaker take over the turn, or briefly acknowledge while the first continues?

Describe one turn transition without interpreting courtesy or hostility.

{
  "questions": {
    "a_takeover_or_acknowledgment_around_a_reported": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Moderationscore

How much of the speech in this excerpt can a listener understand?

Keep a difficult-to-hear attachment from being treated as a verified quote.

{
  "questions": {
    "can_the_reported_words_be_made_out": {
      "type": "score",
      "instructions": "How much of the speech in this excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Moderationchoice

After which supplied word does the speaker pause?

Locate a pause relative to supplied words without interpreting why it occurred.

{
  "context": "Supplied line: \"I never said you should leave.\"",
  "questions": {
    "where_a_speaker_pauses_within_the_quoted_line": {
      "type": "choice",
      "instructions": "After which supplied word does the speaker pause?",
      "criteria": [
        {
          "name": "Never"
        },
        {
          "name": "Said"
        },
        {
          "name": "You"
        },
        {
          "name": "Should"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Temporal and contextual signals

Moderationspan

Find the first audible laughter in this reported excerpt.

Help a person replay a sound that a text report leaves out.

{
  "questions": {
    "find_laughter_in_a_reported_recording": {
      "type": "span",
      "instructions": "Find the first audible laughter in this reported excerpt."
    }
  }
}

Non-speech events

Moderationnoul

Is background music audible underneath the speech?

Describe an audio obstacle without guessing the words it covers.

{
  "questions": {
    "music_behind_the_reported_words": {
      "type": "noul",
      "instructions": "Is background music audible underneath the speech?"
    }
  }
}

Acoustic environment

Moderationchoice

Which listed language is spoken in this reported clip?

Suggest a language-skilled reviewer without interpreting the clip as a policy violation.

{
  "questions": {
    "language_for_a_human_report_review": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this reported clip?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "Unclear or another language"
        }
      ]
    }
  }
}

Speech and language

Moderationchoice

How many distinct voices are audible in this excerpt?

Provide limited listening context for the reported recording.

{
  "questions": {
    "how_many_voices_need_to_be_heard": {
      "type": "choice",
      "instructions": "How many distinct voices are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Recruitingchoice

Which action is explicitly assigned in this excerpt?

List candidate actions as options, such as sending the interview schedule, and add “None of these or unclear”. A coordinator confirms the owner and deadline.

{
  "questions": {
    "follow_up_from_a_planning_call": {
      "type": "choice",
      "instructions": "Which action is explicitly assigned in this excerpt?",
      "criteria": [
        {
          "name": "Send the interview schedule"
        },
        {
          "name": "Book a meeting room"
        },
        {
          "name": "Send the meeting link"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Recruitingscore

How intrusive is the background noise?

Send a short excerpt from a phone or video interview, and use levels from not noticeable to very intrusive.

{
  "questions": {
    "noise_in_an_interview_recording": {
      "type": "score",
      "instructions": "How intrusive is the background noise?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Communication quality

Recruitingscore

How directly does the candidate’s reply address the interviewer’s question?

Send the question with the full reply. The answer judges whether the reply stays on the question, not whether it is true or shows fit for the role.

{
  "context": "The interviewer asks, \"What steps did you take when the project requirements changed?\" Check whether the candidate's recorded reply stays on that question, not whether the account is true or demonstrates suitability for the job.",
  "questions": {
    "candidates_reply_on_topic": {
      "type": "score",
      "instructions": "How directly does the candidate's reply address the interviewer's question?",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Recruitingnoul

Do two people speak at the same time in this excerpt?

Check a passage before its transcript goes into interview notes.

{
  "questions": {
    "overlap_before_transcription": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Recruitingspan

Find the parts of this recording where the recruiter explains the next interview stage and how it will be scheduled.

Find the logistical instructions in a recruiting recording so they are easy to revisit.

{
  "context": "Search the supplied excerpt and return intervals relative to its start.",
  "questions": {
    "find_the_interview_next_steps": {
      "type": "span",
      "instructions": "Find the parts of this recording where the recruiter explains the next interview stage and how it will be scheduled."
    }
  }
}

Temporal and contextual signals

Recruitingnoul

Are filled pauses such as “uh” or “um” audible in this excerpt?

Flag a careers-video excerpt for an editor to listen to before deciding whether to trim it.

{
  "questions": {
    "fillers_in_a_recruiting_video": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible in this excerpt?"
    }
  }
}

Speech and language

Recruitingchoice

Which of the listed languages is spoken in this excerpt?

Help a recruiting content team organize multilingual recordings before preparing captions.

{
  "context": "This excerpt belongs to a collection recorded in English, Japanese, Arabic, French or Russian.",
  "questions": {
    "language_labels_for_careers_videos": {
      "type": "choice",
      "instructions": "Which of the listed languages is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Recruitingscore

How strongly would listeners agree that the voice expresses the intended upbeat, welcoming tone for this line?

Help a recruiting content producer review a recorded welcome against the intended tone.

{
  "context": "The recorded line is \"Welcome! Here is what to expect when you apply.\" The production direction is upbeat and welcoming. Review the recorded delivery against that direction.",
  "questions": {
    "delivery_against_a_careers_video_brief": {
      "type": "score",
      "instructions": "How strongly would listeners agree that the voice expresses the intended upbeat, welcoming tone for this line?",
      "criteria": ["Do not agree at all", "Disagree", "Neutral", "Agree", "Fully agree"]
    }
  }
}

Emotion and expression

Recruitingchoice

How does the reply respond to the proposed time “Would Thursday at 2 p.m. work for your interview?”

Help a coordinator distinguish an explicit acceptance from a question before confirming a booking.

{
  "context": "The excerpt contains \"Would Thursday at 2 p.m. work for your interview?\" and the reply. Classify the explicit reply.",
  "questions": {
    "a_reply_to_a_proposed_interview_time": {
      "type": "choice",
      "instructions": "How does the reply respond to the proposed time \"Would Thursday at 2 p.m. work for your interview?\"",
      "criteria": [
        {
          "name": "Explicitly agrees"
        },
        {
          "name": "Explicitly disagrees"
        },
        {
          "name": "Asks a question"
        },
        {
          "name": "Qualified, something else or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Recruitingchoice

How many distinct speakers are audible in this excerpt?

Help an editor check whether a selected panel excerpt contains one voice or several before making a short clip.

{
  "context": "Count audible voices, not attendees.",
  "questions": {
    "single_speaker_clips_from_a_careers_panel": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Recruitingnoul

Does this audio have noise behind the talker?

Help a coordinator identify a recording that may need closer listening before transcription.

{
  "questions": {
    "background_noise_in_a_recorded_interview": {
      "type": "noul",
      "instructions": "Does this audio have noise behind the talker?"
    }
  }
}

Acoustic environment

Recruitingchoice

Which listed request does the speaker explicitly make?

Help a coordinator sort incoming voice messages before replying.

{
  "questions": {
    "an_explicit_scheduling_request": {
      "type": "choice",
      "instructions": "Which listed request does the speaker explicitly make?",
      "criteria": [
        {
          "name": "Reschedule an interview"
        },
        {
          "name": "Ask for the meeting link"
        },
        {
          "name": "Ask what to prepare"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Recruitingnoul

Does a speaker or listener laugh in this excerpt?

Help an editor find promising excerpts for a careers-event highlight reel.

{
  "questions": {
    "laughter_in_a_careers_event_recording": {
      "type": "noul",
      "instructions": "Does a speaker or listener laugh in this excerpt?"
    }
  }
}

Non-speech events

Social medianoul

Are filled pauses such as “uh” or “um” audible?

Flag clips for a listen before editing. The answer says whether fillers are present, not where.

{
  "questions": {
    "fillers_in_a_clip": {
      "type": "noul",
      "instructions": "Are filled pauses such as \"uh\" or \"um\" audible?"
    }
  }
}

Speech and language

Social medianoul

Do two people speak at the same time in this excerpt?

Flag street-interview passages for a closer listen before captioning. The answer does not say who said which words.

{
  "questions": {
    "crosstalk_before_captioning": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Social mediachoice

Which listed language is spoken in this excerpt?

List the languages your clips use as options, and have a person confirm the tag before captioning.

{
  "context": "The clip is known to use one of the five listed languages.",
  "questions": {
    "language_tag": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Social mediascore

How would most listeners rate the sound quality of this speech?

Check a recording before it is posted. The answer rates the sound, not which defect causes it.

{
  "questions": {
    "quality_before_posting": {
      "type": "score",
      "instructions": "How would most listeners rate the sound quality of this speech?",
      "criteria": ["Bad", "Poor", "Fair", "Good", "Excellent"]
    }
  }
}

Acoustic environment

Social mediaspan

Find where the main battery-life discussion begins.

Find a candidate chapter location for human verification.

{
  "questions": {
    "finding_a_discussion_for_a_chapter_marker": {
      "type": "span",
      "instructions": "Find where the main battery-life discussion begins."
    }
  }
}

Temporal and contextual signals

Social mediachoice

Does the speaker ask for an action, report information, commit to an action, or express a reaction?

Make a follow-up list from explicit conversational moves, without assuming an offer will be fulfilled.

{
  "questions": {
    "a_request_report_or_commitment": {
      "type": "choice",
      "instructions": "Does the speaker ask for an action, report information, commit to an action, or express a reaction?",
      "criteria": [
        {
          "name": "Asks for an action"
        },
        {
          "name": "Reports information"
        },
        {
          "name": "Commits to an action"
        },
        {
          "name": "Expresses a reaction"
        },
        {
          "name": "Mixed or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Social medianoul

Is background music audible underneath the speech?

Avoid unintentionally layering two music beds.

{
  "questions": {
    "existing_music_in_a_submitted_clip": {
      "type": "noul",
      "instructions": "Is background music audible underneath the speech?"
    }
  }
}

Acoustic environment

Social mediachoice

Does the second voice take over, or briefly acknowledge while the first speaker continues?

Describe a conversational event before cutting an exchange into a short clip.

{
  "questions": {
    "an_interjection_in_a_podcast_excerpt": {
      "type": "choice",
      "instructions": "Does the second voice take over, or briefly acknowledge while the first speaker continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; the first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Social mediascore

Is the speaking rate slow, moderate or fast?

Describe how quickly a creator speaks in a short how-to clip.

{
  "context": "Rate the selected passage's pace separately from whether the instructions are correct, intelligible or easy to learn.",
  "questions": {
    "pace_of_a_spoken_explanation": {
      "type": "score",
      "instructions": "Is the speaking rate slow, moderate or fast?",
      "criteria": ["Slow", "Moderate", "Fast"]
    }
  }
}

Speech and language

Social mediascore

How noticeable is the background sound, on a scale from very intrusive to not noticeable?

Describe the interference without guessing what the speaker said through it.

{
  "questions": {
    "background_sound_in_a_difficult_to_hear": {
      "type": "score",
      "instructions": "How noticeable is the background sound, on a scale from very intrusive to not noticeable?",
      "criteria": ["Not noticeable", "Slightly noticeable", "Noticeable but not intrusive", "Somewhat intrusive", "Very intrusive"]
    }
  }
}

Acoustic environment

Social medianoul

Is laughter audible in this excerpt?

Preserve a nonverbal event in the text without interpreting it as agreement.

{
  "questions": {
    "laughter_omitted_from_an_interview_transcript": {
      "type": "noul",
      "instructions": "Is laughter audible in this excerpt?"
    }
  }
}

Non-speech events

Social mediascore

How directly does the creator’s reply address the audience question “How did you fix the loose handle on this pan?”

Separate an on-topic response clip from one that addresses something else.

{
  "context": "Supplied audience question: \"How did you fix the loose handle on this pan?\" Judge relevance to that question, not whether the repair advice is safe, true or complete.",
  "questions": {
    "whether_a_recorded_reply_answers_the_same_topic": {
      "type": "score",
      "instructions": "How directly does the creator's reply address the audience question \"How did you fix the loose handle on this pan?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Social mediachoice

How many distinct speaking voices are audible?

Help distinguish a solo quotation from a discussion clip.

{
  "context": "Count distinct voices, not turns.",
  "questions": {
    "one_voice_or_a_conversation": {
      "type": "choice",
      "instructions": "How many distinct speaking voices are audible?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Social mediachoice

Which listed series does the speaker ask to hear about?

Organize explicit content requests without inferring interests from someone’s voice.

{
  "context": "The supplied series menu is Garden Notes, City Walks, Kitchen Trials and Weekend Repairs; classify only the named request. Choose \"None, several or unclear\" when no listed series, several series or an unclear title is requested.",
  "questions": {
    "which_series_was_requested": {
      "type": "choice",
      "instructions": "Which listed series does the speaker ask to hear about?",
      "criteria": [
        {
          "name": "Garden Notes"
        },
        {
          "name": "City Walks"
        },
        {
          "name": "Kitchen Trials"
        },
        {
          "name": "Weekend Repairs"
        },
        {
          "name": "None, several or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Social mediachoice

Which listed purpose does this audio message state?

Sort a spoken request by the action explicitly requested, without inferring the listener’s interests.

{
  "questions": {
    "routing_an_explicit_audience_request": {
      "type": "choice",
      "instructions": "Which listed purpose does this audio message state?",
      "criteria": [
        {
          "name": "Explain how something works"
        },
        {
          "name": "Suggest a topic"
        },
        {
          "name": "Invite a collaboration"
        },
        {
          "name": "Correct a stated detail"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Social mediaspan

Find the first occurrence of “refund” in this recording.

Find a spoken word for replay while checking a caption.

{
  "context": "Times are relative to the excerpt.",
  "questions": {
    "locating_a_word_needed_for_a_caption_check": {
      "type": "span",
      "instructions": "Find the first occurrence of \"refund\" in this recording."
    }
  }
}

Temporal and contextual signals

Social mediascore

How much of the speech in this short excerpt can a listener understand?

Describe how much is understandable before treating a guessed phrase as a quotation.

{
  "questions": {
    "how_much_of_a_hard_to_hear_passage_is": {
      "type": "score",
      "instructions": "How much of the speech in this short excerpt can a listener understand?",
      "criteria": ["Almost none", "Some", "Most", "All"]
    }
  }
}

Communication quality

Social mediascore

How positive or negative is the opinion expressed in this clip?

Index the expressed opinion, separately from the reviewer’s private feelings or future purchases.

{
  "questions": {
    "the_opinion_expressed_in_a_review": {
      "type": "score",
      "instructions": "How positive or negative is the opinion expressed in this clip?",
      "criteria": ["Very negative", "Negative", "Neutral", "Positive", "Very positive"]
    }
  }
}

Emotion and expression

Social mediaspan

Find the first audible laughter in this interview excerpt.

Suggest a replay interval for an editor to inspect.

{
  "context": "Return a candidate interval relative to the supplied excerpt.",
  "questions": {
    "finding_a_reaction_moment": {
      "type": "span",
      "instructions": "Find the first audible laughter in this interview excerpt."
    }
  }
}

Non-speech events

Social mediachoice

Which supplied word carries the strongest emphasis?

Keep the stressed word visible when deciding how to present a spoken quotation.

{
  "context": "The creator says, \"Try it before buying.\" Identify the actual stress rather than deciding whether the statement is sarcastic or sincere.",
  "questions": {
    "emphasis_in_a_quote_used_for_a_caption": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Try"
        },
        {
          "name": "It"
        },
        {
          "name": "Before"
        },
        {
          "name": "Buying"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Utilitiesnoul

Are the visit date and time read back correctly?

Put the booked time in context, such as “Tuesday, between two and four in the afternoon”. The answer checks the readout, not the booking.

{
  "context": "Booked visit: Tuesday, between two and four in the afternoon",
  "questions": {
    "visit_time_read_back": {
      "type": "noul",
      "instructions": "Are the visit date and time read back correctly?"
    }
  }
}

Communication quality

Utilitieschoice

Which listed request is the caller making?

List requests such as changing a visit, asking about a bill and submitting a meter reading as options. The answer is the stated request, not an inferred need.

{
  "questions": {
    "utility_request": {
      "type": "choice",
      "instructions": "Which listed request is the caller making?",
      "criteria": [
        {
          "name": "Change a visit"
        },
        {
          "name": "Ask about a bill"
        },
        {
          "name": "Submit a meter reading"
        },
        {
          "name": "Ask about opening hours"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Utilitiesnoul

Do two people speak at the same time in this excerpt?

Send the part where the service address is said. A person still confirms any unclear details.

{
  "questions": {
    "overlap_on_an_address": {
      "type": "noul",
      "instructions": "Do two people speak at the same time in this excerpt?"
    }
  }
}

Conversation dynamics

Utilitiesnoul

Does the speech sound distorted or clipped?

The answer says whether distortion is heard, not whether the visit details are correct.

{
  "questions": {
    "distortion_on_a_utility_call": {
      "type": "noul",
      "instructions": "Does the speech sound distorted or clipped?"
    }
  }
}

Acoustic environment

Utilitieschoice

Does “Send a reading” sound like a question about the instruction or a restatement of it?

Identify a questioning repetition that may need a clearer explanation.

{
  "context": "The agent says, \"Please send a meter reading.\" The customer replies, \"Send a reading.\" Listen to the customer's delivery.",
  "questions": {
    "clarify_a_repeated_meter_reading_instruction": {
      "type": "choice",
      "instructions": "Does \"Send a reading\" sound like a question about the instruction or a restatement of it?",
      "criteria": [
        {
          "name": "Questions the instruction"
        },
        {
          "name": "Restates the instruction"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Social and behavioral signals

Utilitieschoice

Which listed sound is audible underneath the caller’s voice?

Give a concrete audio description rather than infer conditions at the property.

{
  "questions": {
    "noise_underneath_a_customers_service_report": {
      "type": "choice",
      "instructions": "Which listed sound is audible underneath the caller's voice?",
      "criteria": [
        {
          "name": "Hiss or static"
        },
        {
          "name": "Low hum"
        },
        {
          "name": "Other people talking"
        },
        {
          "name": "Music"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Acoustic environment

Utilitiesnoul

Is coughing audible in this recording?

Describe an audible interruption in a service announcement, not a health condition.

{
  "questions": {
    "a_coughing_sound_during_an_outage_update": {
      "type": "noul",
      "instructions": "Is coughing audible in this recording?"
    }
  }
}

Non-speech events

Utilitieschoice

How many distinct speakers are audible in this excerpt?

Count the voices heard before a person confirms who is speaking.

{
  "questions": {
    "several_voices_on_the_customer_side_of_a_service": {
      "type": "choice",
      "instructions": "How many distinct speakers are audible in this excerpt?",
      "criteria": [
        {
          "name": "1"
        },
        {
          "name": "2"
        },
        {
          "name": "3"
        },
        {
          "name": "4 or more"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speakers and voices

Utilitiesscore

How likely is the new generated update to be in the same voice as the existing introduction?

Check whether a new generated update sounds consistent with the phone line’s existing voice.

{
  "context": "The clip plays the introduction, the update and the introduction again. Judge perceived voice continuity, not whether the update's factual content is correct.",
  "questions": {
    "voice_continuity_across_changing_outage": {
      "type": "score",
      "instructions": "How likely is the new generated update to be in the same voice as the existing introduction?",
      "criteria": ["Definitely different", "Probably different", "Unsure", "Probably the same", "Definitely the same"]
    }
  }
}

Speakers and voices

Utilitieschoice

Does the second speaker take over the turn, or briefly acknowledge while the first continues?

Describe a selected turn-taking event in a customer–utility conversation.

{
  "questions": {
    "acknowledgment_or_taking_over_a_service_report": {
      "type": "choice",
      "instructions": "Does the second speaker take over the turn, or briefly acknowledge while the first continues?",
      "criteria": [
        {
          "name": "Takes over the turn"
        },
        {
          "name": "Brief acknowledgment; first speaker continues"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Conversation dynamics

Utilitiesnoul

Is music audible underneath the agent’s speech?

Find a reproducible audio example for a service-platform issue, without identifying its cause.

{
  "questions": {
    "music_continuing_after_hold": {
      "type": "noul",
      "instructions": "Is music audible underneath the agent's speech?"
    }
  }
}

Acoustic environment

Utilitiesscore

How directly does the reply address the question “How do I submit a meter reading?”

Distinguish relevance from whether a reading was actually received.

{
  "context": "Judge relevance, not the reading's correctness, the bill or successful submission.",
  "questions": {
    "whether_the_reply_addresses_the_meter_reading": {
      "type": "score",
      "instructions": "How directly does the reply address the question \"How do I submit a meter reading?\"",
      "criteria": ["Off-topic", "Mostly off-topic", "Partly relevant", "Mostly relevant", "Directly relevant"]
    }
  }
}

Communication quality

Utilitieschoice

Which supplied word carries the strongest emphasis?

Identify what is stressed before a person clarifies the customer’s question.

{
  "context": "The customer asks, \"Is the visit this Tuesday?\" Locate stress rather than confirming the date or explaining the customer's intention.",
  "questions": {
    "emphasis_in_a_question_about_a_visit": {
      "type": "choice",
      "instructions": "Which supplied word carries the strongest emphasis?",
      "criteria": [
        {
          "name": "Is"
        },
        {
          "name": "The"
        },
        {
          "name": "Visit"
        },
        {
          "name": "This"
        },
        {
          "name": "Tuesday"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Utilitieschoice

Which supplied tool should the assistant select first?

Choose the function matching the explicit request for availability, rather than prematurely changing a booking.

{
  "context": "The visit is already identified, scheduled and changeable. Tools: list_available_visit_slots returns available times; get_booking_status returns existing status; confirm_visit_change commits a chosen slot; contact_scheduler requests human help. Choose the first function without executing it.",
  "questions": {
    "choosing_a_rescheduling_function": {
      "type": "choice",
      "instructions": "Which supplied tool should the assistant select first?",
      "criteria": [
        {
          "name": "list_available_visit_slots"
        },
        {
          "name": "get_booking_status"
        },
        {
          "name": "confirm_visit_change"
        },
        {
          "name": "contact_scheduler"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Intent and outcomes

Utilitieschoice

Which listed language is spoken in this excerpt?

Identify a likely language before offering communication assistance.

{
  "context": "The call is known to use one of the five listed languages.",
  "questions": {
    "language_support_for_a_utility_service_call": {
      "type": "choice",
      "instructions": "Which listed language is spoken in this excerpt?",
      "criteria": [
        {
          "name": "English"
        },
        {
          "name": "Japanese"
        },
        {
          "name": "Arabic"
        },
        {
          "name": "French"
        },
        {
          "name": "Russian"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Speech and language

Utilitiesspan

Find the first occurrence of “Tuesday” in this recording.

Find the date statement for human replay instead of relying only on a tentative transcript.

{
  "context": "Times are relative to the supplied one-minute excerpt.",
  "questions": {
    "replaying_a_stated_visit_date": {
      "type": "span",
      "instructions": "Find the first occurrence of \"Tuesday\" in this recording."
    }
  }
}

Temporal and contextual signals

Utilitiesspan

Find where the engineer visit is discussed.

Offer a candidate passage to replay when checking what was said about the visit.

{
  "context": "Return candidate intervals relative to the supplied excerpt.",
  "questions": {
    "finding_where_a_visit_arrangement_was_discussed": {
      "type": "span",
      "instructions": "Find where the engineer visit is discussed."
    }
  }
}

Temporal and contextual signals

Utilitiesnoul

Is a word said twice in a row in this excerpt?

Check for an audible repetition when preparing a verbatim transcript, without interpreting the customer’s state.

{
  "questions": {
    "repeated_words_in_a_customers_service_report": {
      "type": "noul",
      "instructions": "Is a word said twice in a row in this excerpt?"
    }
  }
}

Speech and language

Utilitieschoice

Which listed emotion does the customer’s voice express?

Describe vocal expression without using it to prioritize an outage or account decision.

{
  "questions": {
    "emotion_expressed_during_a_service_complaint_2": {
      "type": "choice",
      "instructions": "Which listed emotion does the customer's voice express?",
      "criteria": [
        {
          "name": "Sad"
        },
        {
          "name": "Fearful"
        },
        {
          "name": "Angry"
        },
        {
          "name": "Happy"
        },
        {
          "name": "Neutral"
        },
        {
          "name": "None of these or unclear"
        }
      ]
    }
  }
}

Emotion and expression

Next steps

Writing questions

Learn how to write questions that get useful answers, and which questions to avoid.

Language spoken

Speaking rate

Filler words

Accent

Synthetic speech

Expressed emotion

Emphasized word

Background noise

Kind of background noise

Noise intrusiveness

Sound quality

Laughter

Kind of vocal sound

When laughter occurs

A named sound

Which listed sound

When a sound occurs

Overlapping speech

Interruption or acknowledgment

Pause within a turn

Naturalness

Relevance of a reply

Intelligibility

Routing a request

Order of two sounds

Pause after a word

When a word is spoken

When a topic comes up

Perceived gender presentation of a designed voice

Emotion-word incongruence

Background music

Number of speakers

Did the response answer the question?

Meeting engagement

Prioritize low-quality recordings for review

Separate overlapping speech for transcription review

Mine vocal-burst examples for training and evaluation

Label expressed emotion for a speech dataset

Compare naturalness across generated-speech runs

Check generated speech against its script

Check word emphasis in controlled speech

Find examples of filled pauses

Check the speaker-count label on a clip

Check dialogue-act labels in a call corpus

Check whether a voice-agent reply answers the question

Check voice continuity in generated samples

Check delivery against a speaking direction

Separate backchannels from floor-taking

Sort recordings by spoken language

Find overlapping speech within a recording

Find a topic passage for a research corpus

Create speaking-rate evaluation slices

Find a spoken target for alignment review

Label a sound from a known class inventory

Check word order in a speech-generation test

Find laughter to cut annotation clips

Build background-noise evaluation slices

Measure background interference

Check a requested value in spoken-command data

Emphasis in a voice-over take

Voice match after a pickup

Recording problem

Laughter in an asset library

Discount in an ad

Naturalness of a generated ad read

A pause inside the offer line

Background intrusion in a creator-recorded demonstration

A laugh cue in a recorded spot

Find a spoken brand name in an ad excerpt

Agreement or objection in a testimonial interview

An offer line under another voice

Emotion conveyed by a testimonial excerpt

Comedic delivery in a generated spot

Check speaking-rate changes between ad lines

Pace against a voiceover brief

Filled pauses in a testimonial recording

An accent specified in the voice brief

Finding the laugh for a dialogue edit

What the call to action says

Speech audibility in a finished mix