Skip to navigation

LiveKit

Add Hume Audio Decisions to a LiveKit voice agent with a small plugin, so the LLM can reply to how the caller sounded.

This page shows you how to write a small plugin that adds Hume Audio Decisions to a LiveKit agent, so the LLM can reply to how the caller sounded. Your plugin copies the caller’s audio, cuts a clip when the caller stops speaking, sends the clip to Hume Audio Decisions, and adds the answers to the caller’s turn before the LLM replies. The plugin takes the same inline approach as Pipecat’s PerceptionReader, described in Choose whether the reply waits.

Official LiveKit support is coming soon.

What the plugin does

StepIn a LiveKit agent
Copy the caller’s audioRead the caller’s audio track alongside the agent’s own input, and keep a rolling buffer of it. Leave the frames the agent receives unchanged, so speech-to-text gets them without delay.
Cut a clipWhen VAD reports the end of speech, take the latest speech from the buffer, extended to 5 seconds with earlier speech from the same turn, as 16 kHz, 16-bit mono WAV. See Choose which audio to send.
Send the requestFrom a background task, with one client reused for every clip and a wait limit of about 2 seconds after the caller stops.
Add the answers to the turnIn the agent’s on_user_turn_completed hook, wait for the answer up to the wait limit, then put the answer at the head of the caller’s message, such as [frustrated 0.82]. If the limit passes, reply without the answer.
Choose questionsStart with one question, such as “Does the caller sound frustrated?”, and explain the answer line in the system prompt. See Choose questions.

Build it

Next steps