Guide to using function calling with the Empathic Voice Interface (EVI).
EVI simplifies the integration of external APIs through function calling. Developers can integrate custom functions that are invoked dynamically based on the user’s input, enabling more useful conversations. There are two key concepts for using function calling with EVI, Tools and Configurations (Configs):
-
Tools are resources that EVI uses to do things, like search the web or call external APIs. For example, tools can check the weather, update databases, schedule appointments, or take actions based on what occurs in the conversation. While the tools can be user-defined, Hume also offers natively implemented tools, like web search, which are labeled as “built-in” tools.
-
Configurations enable developers to customize an EVI’s behavior and incorporate these custom tools. Setting up an EVI configuration allows developers to seamlessly integrate their tools into the voice interface. A configuration includes prompts, user-defined tools, and other settings.
Currently, our function calling feature only supports OpenAI models. Function calling is not available if you are using your own custom language model. We plan to support more function calling LLMs in the future.
The focus of this guide is on creating a Tool and a Configuration that allows EVI to use the Tool. Additionally, this guide details the message flow of function calls within a session, and outlines the expected responses when function calls fail. Refer to our Configuration Guide for detailed, step-by-step instructions on how to create and use an EVI Configuration.
Explore this sample project for an example of how Tool use could be implemented in practice.
Setup
For EVI to leverage tools or call functions, a configuration must be created with the tool’s definition. Our step-by-step guide below walks you through creating a tool and a configuration.
Create a Tool
We will first create a Tool with a specified function. In this case, we will create a tool for getting the weather. Create this tool by making a POST request to /tools with the following request body:
The parameters
field must contain a valid JSON schema.
Record the value in the id
field, as we will use it to specify the newly created Tool in the next step.
Create a Configuration
Next we will create an EVI Configuration called Weather Assistant Config, and include the created Tool by making a POST request to /configs with the following request body:
Ensure your tool definitions conform to the language model’s schema. The specified language model will be the one to execute the function calls.
Function calling
In this section we will go over the end-to-end flow of a function call within a chat session. This flow will be predicated on having specified the
Weather Assistant Config
when establishing a connection with EVI. See our Configuration Guide
for details on how to apply your configuration when connecting.
Currently, EVI does not support parallel function calling. Only one function call can be processed at a time.
Invoke function call
With EVI configured to use the get_current_weather
Tool, we can now ask it: “what is the weather in New York?” We can expect EVI to respond with a user_message
and a tool_call
message:
Next, extract the tool_call_id
from the tool_call
message to be used in the next step. Then, you will need to pass the parameters from the tool_call
message to your function to retrieve the weather for the designated city in the specified format.
While EVI will send a message to indicate when to invoke your function and
which parameters to pass into it, you will need to define the function itself
in your code. For the sake of this example, you can define a function which
actually calls a weather API, or simply hard code a return value like: 60F
.
Send function call result
Upon receiving the response from your function, we will then send a tool_response
message containing the result. The specified tool_call_id
should match the one received in
the tool_call
message in the previous step.
EVI responds
After the interface receives the tool_response
message, it will then send an assistant_message
containing the response generated from the reported result of the function call:
Using built-in tools
User-defined tools allow EVI to identify when a function should be invoked, but you implement the function itself. On the other hand, Hume also provides built-in tools that are natively integrated. This means that you don’t need to define the function; EVI handles both determining when the function needs to be called and invoking it.
One such example of a built-in tool we provide is Web search. Web search equips EVI with the ability to search the web for up-to-date information.
This section explains how to specify built-in tools in your configurations and details the message flow you can expect when EVI uses a built-in tool during a chat session.
Specify built-in tool in EVI configuration
Let’s begin by creating a configuration which includes the built-in web search tool by making a POST request to /configs with the following request body:
EVI uses built-in tool
Now that we’ve created an EVI configuration which includes the built-in web search tool, let’s review the message flow for when web search is invoked.
Interruptibility
Function calls can be interrupted to cancel them or to resend them with updated parameters.
Canceling a function call
Just as EVI is able to infer when to make a function call, it can also infer from the user’s input when to cancel one. Here is an overview of what the message flow would look like:
Updating a function call
Sometimes we don’t necessarily want to cancel the function call, and instead want to update the parameters. EVI can infer the difference. Below is a sample flow of interrupting the interface to update the parameters of the function call:
Handling errors
It’s possible for tool use to fail. For example, it can fail if the tool_response
message content was not in UTF-8 format or if the function call response timed out. This
section outlines how to specify fallback content to be used by EVI to communicate a failure, as well as the message flow for when a function call failure occurs.
Specifying fallback content
When defining your Tool, you can specify fallback content within the Tool’s fallback_content
field. When the Tool fails to generate content, the text in this
field will be sent to the LLM in place of a result. To accomplish this, let’s update the Tool we created during setup to include fallback content. We can accomplish
this by publishing a new version of the Tool via a POST request to /tools/{id}:
Failure message flow
This section outlines the sort of messages that can be expected when Tool use fails. After sending a tool-response
message, we will know an error, or failure,
occurred when we receive the tool_error
message:
Let’s cover another type of failure scenario: what if the weather API the function was using was down? In this case we would send EVI a tool_error
message.
When sending the tool_error
message we can specify fallback_content
more specific to the error our function throws. This is what the message flow would be
for this type of failure: