Real-time measurement streaming

WebSocket-based streaming facilitates continuous data flow between your application and Hume’s models, providing immediate feedback and insights.

Key features

• Real-time data processing: Leveraging WebSockets, this API allows for the streaming of data to Hume’s models, enabling instant analysis and response. This feature is particularly beneficial for applications requiring immediate processing, such as live interaction systems or real-time monitoring tools.
• Persistent, two-way communication: Unlike traditional request-response models, the WebSocket-based streaming maintains an open connection for two-way communication between the client and server. This facilitates an ongoing exchange of data, allowing for a more interactive and responsive user experience.
• High throughput and low latency: The API is optimized for high performance, supporting high-volume data streaming with minimal delay. This ensures that applications can handle large streams of data efficiently, without sacrificing speed or responsiveness.

Applications and use cases

WebSockets are ideal for a wide range of applications that benefit from real-time data analysis and interaction. Examples include:

• Live customer service tools: enhance customer support with real-time sentiment analysis and automated, emotionally intelligent responses
• Interactive educational platforms: provide immediate feedback and adaptive learning experiences based on real-time student input
• Health and wellness apps: support live mental health and wellness monitoring, offering instant therapeutic feedback or alerts based on the user’s vocal or textual expressions
• Entertainment and gaming: create more immersive and interactive experiences by responding to user inputs and emotions in real time

Getting started with WebSocket streaming

Integrating WebSocket-based streaming into your application involves establishing a WebSocket connection with Hume AI’s servers and streaming data directly to the models for processing.

Streaming is built for analysis of audio, video, and text streams. By connecting to WebSocket endpoints you can get near real-time feedback on the expressive and emotional content of your data.

Install the Hume Python SDK

First, ensure you have installed the SDK using pip or another package manager.

$
pip install "hume"

Emotional language from text

This example uses our Emotional Language model to perform sentiment analysis on a children’s nursery rhyme.

If you haven’t already, grab your API key.

1
import asyncio
2
from hume import AsyncHumeClient
3
from hume.expression_measurement.stream import Config
4
from hume.expression_measurement.stream.socket_client import StreamConnectOptions
5
from hume.expression_measurement.stream.types import StreamLanguage
6
7
samples = [
8
    "Mary had a little lamb,",
9
    "Its fleece was white as snow."
10
    "Everywhere the child went,"
11
    "The little lamb was sure to go."
12
]
13
14
async def main():
15
    client = AsyncHumeClient(api_key="<YOUR_API_KEY>")
16
17
    model_config = Config(language=StreamLanguage())
18
19
    stream_options = StreamConnectOptions(config=model_config)
20
21
    async with client.expression_measurement.stream.connect(options=stream_options) as socket:
22
        for sample in samples:
23
            result = await socket.send_text(sample)
24
            print(result.language.predictions[0].emotions)
25
26
if __name__ == "__main__":
27
    asyncio.run(main())

Your result should look something like this:

1
[
2
  {'name': 'Admiration', 'score': 0.06379243731498718},
3
  {'name': 'Adoration', 'score': 0.07222934812307358},
4
  {'name': 'Aesthetic Appreciation', 'score': 0.02808445133268833},
5
  {'name': 'Amusement', 'score': 0.027589013800024986},
6
  ......
7
  {'name': 'Surprise (positive)', 'score': 0.030542362481355667},
8
  {'name': 'Sympathy', 'score': 0.03246130049228668},
9
  {'name': 'Tiredness', 'score': 0.03606246039271355},
10
  {'name': 'Triumph', 'score': 0.01235896535217762}
11
]

Facial expressions from an image

This example uses our Facial Expression model to get expression measurements from an image.

1
import asyncio
2
from hume import AsyncHumeClient
3
from hume.expression_measurement.stream import Config
4
from hume.expression_measurement.stream.socket_client import StreamConnectOptions
5
from hume.expression_measurement.stream.types import StreamFace
6
7
async def main():
8
    client = AsyncHumeClient(api_key="<YOUR_API_KEY>")
9
10
    model_config = Config(face=StreamFace())
11
12
    stream_options = StreamConnectOptions(config=model_config)
13
14
    async with client.expression_measurement.stream.connect(options=stream_options) as socket:
15
        result = await socket.send_file("<YOUR_IMAGE_FILEPATH>")
16
        print(result)
17
18
if __name__ == "__main__":
19
    asyncio.run(main())

Speech prosody from an audio or video file

This example uses our Speech Prosody model to get expression measurements from an audio or video file.

1
import asyncio
2
from hume import AsyncHumeClient
3
from hume.expression_measurement.stream import Config
4
from hume.expression_measurement.stream.socket_client import StreamConnectOptions
5
6
async def main():
7
    client = AsyncHumeClient(api_key="<YOUR_API_KEY>")
8
9
    model_config = Config(prosody={})
10
11
    stream_options = StreamConnectOptions(config=model_config)
12
13
    async with client.expression_measurement.stream.connect(options=stream_options) as socket:
14
        result = await socket.send_file("YOUR_AUDIO_OR_VIDEO_FILEPATH")
15
        print(result)
16
17
if __name__ == "__main__":
18
    asyncio.run(main())

Streaming with your own WebSockets client

To call the API from your own WebSockets client you’ll need the API endpoint, a JSON message, and an API key header/param. More information can be found in the Expression Measurement API reference.

To get started, you can use a WebSocket client of your choice to connect to the models endpoint:

url wss://api.hume.ai/v0/stream/models 

Make sure you configure the socket connection headers with your personal API key

1
X-Hume-Api-Key: <YOUR API KEY>

And finally, send the following JSON message on the socket:

1
{
2
    "models": {
3
        "language": {}
4
    },
5
    "raw_text": true,
6
    "data": "Mary had a little lamb"
7
}

You should receive a JSON response that looks something like this:

1
{
2
  "language": {
3
    "predictions": [
4
      {
5
        "text": "Mary",
6
        "position": { "begin": 0, "end": 4 },
7
        "emotions": [
8
          { "name": "Anger", "score": 0.012025930918753147 },
9
          { "name": "Joy", "score": 0.056471485644578934 },
10
          { "name": "Sadness", "score": 0.031556881964206696 },
11
        ]
12
      },
13
      {
14
        "text": "had",
15
        "position": { "begin": 5, "end": 8 },
16
        "emotions": [
17
          { "name": "Anger", "score": 0.0016927534015849233 },
18
          { "name": "Joy", "score": 0.02388327568769455 },
19
          { "name": "Sadness", "score": 0.018137391656637192 },
20
          ...
21
        ]
22
      },
23
      ...
24
    ]
25
  }
26
}

Sending images or audio

The WebSocket endpoints of the Expression Measurement API require that you encode your media using base64. Here’s a quick example of base64 encoding data in Python:

1
import base64
2
from pathlib import Path
3
4
def encode_data(filepath: Path) -> str:
5
    with Path(filepath).open('rb') as fp:
6
        bytes_data = base64.b64encode(fp.read())
7
        encoded_data = bytes_data.decode("utf-8")
8
    return encoded_data
9
10
filepath = "<PATH TO YOUR MEDIA>"
11
encoded_data = encode_data(filepath)
12
print(encoded_data)

API limits

• WebSocket duration limit: connections are subject to a default timeout after one (1) minute of inactivity to ensure unused connections are released.
• WebSocket message payload size limit: the size limit for a given payload depends on the type of content being transmitted and its dimensions.
  • Video: 5000 milliseconds (5 seconds)
  • Audio: 5000 milliseconds (5 seconds)
  • Image: 3,000 x 3,000 pixels
  • Text: 10,000 characters
• Request rate limit: HTTP requests (e.g. WebSocket handshake endpoint) are limited to fifty (50) requests per second.

FAQ