Function calling with Arcee Caller and the OpenAI API

Transcript

Hi everybody, this is Julien from Arcee. The ability to call code functions from your AI workflows is very important. Language models cannot do everything right. For deterministic steps, like calling an API, running some calculation, or extracting data from a data store, you just need to call a function or an API that lives in your IT platform. In this video, I'm going to show you how you can easily perform function calling with one of our recent models, which is called Caller. We released it just a couple of weeks ago. To do that, we'll use the OpenAI client and the OpenAI function calling API, which you probably already know. So let's get started. The Caller model is part of a collection we published just a couple of weeks ago. These models are available as SaaS models within our inference platform, which is called the model engine. I did a recent video on this; I'll put the link in the video description. If you want to give them a shot, just go to models.arcee.ai and let's select Caller large. We can quickly test it in the sandbox, although that's not what we're trying to do today. You can chat with this model and the other models. Pricing is pay per token, and these models are API-based, so you can use them with curl or Python, which is exactly what we're going to do in the demo. Okay, so let's switch to a notebook and look at how we can invoke Python code from our AI workflows. This notebook is running on my local machine, so first, we need to install a couple of dependencies for this example. As mentioned, we're going to use the OpenAI client. The model engine requires HTTP2, so we need to install that as well. In this example, we're going to use the Yahoo Finance API to perform external API calls. So we need to install this package as well. I've already done that. We import all these dependencies, and the first step is to create an OpenAI client to invoke the model. First, we set the endpoint, which is the URL I just showed you. We set the API key, which you can create in the model engine. Now, let's work on the external functions that we would like the model to use to answer some of our prompts. In this case, I have three functions all based on the Yahoo Finance API: `get_stock_price`, `get_ceo_name`, and `get_company_summary`. We can just run this and quickly check that the code works. So, the Google closing price, the name of the Ford CEO, and a long company summary for Verizon. Great. So we can talk to the Yahoo Finance API, and this code runs. Now, how do we connect these functions to our function calling model? That's where we start using the API for function calling in OpenAI. The API is documented on the OpenAI website, and I will add the link to the description. For each of the functions, we need to define the tool name, which is a code function, the function name, a description, which is very important because that's what the model will use to understand what this function is capable of and help the model match your prompts to the right function. Then, we define the parameters. In this case, two strings: the company name and the stock symbol, both of which are required. We do the same for the `get_ceo_name` function and the `get_company_summary` function. As you can see, it's a simple API, and it's very easy to add all these tools that the model can consider in answering your prompts. Let's run this cell and see how we actually call the model. I wrote a small function to keep everything in the same place. It is based on the completion API. We can see everything here: the name of the model, Caller, the actual message, the question we'll ask, the list of tools we just defined, and the tool behavior of the model. We set it to automatic, which means the model will decide if it should call an external tool and which one. There are different values here, but maybe our prompt doesn't work with any of the tools provided. So let's see what the model can do. We have a typical text generation prompt, with the only difference being that we're passing tools to the model. The model will try to match the prompt to one of the tools. Then, we need to look at whether the model actually called a tool or not. We'll see this in the `tool_calls` variable. If this is defined in the response, it means a tool was called. Technically, it could call multiple tools, but in this example, it's only going to call one. We take the first one, and the response has the function name and the function arguments. This tells us which function the model picked and the parameters it needs to call the function. Then, it's just a matter of extracting the actual function name and parameters and performing a function call locally, as the code is defined in the namespace of this notebook. I extract the information returned by the model and call the function. If no tool was invoked or if the function name wasn't defined locally, I return an empty response. So, the code is pretty simple: text generation, look at the response, check if a tool was called, extract the function name and arguments, and turn those JSON parameters into an actual function call, returning the result. Let's run this cell and try an example. Let's start with a simple one: who's the CEO of General Motors? This should map very easily to a tool. We invoke the Caller model, it figures out that it needs to call the `get_ceo_name` function with the arguments, we extract the function name and parameters, and call the Yahoo API to get the answer. Pretty cool stuff. We didn't pass the stock symbol, the ticker code, in the prompt. The prompt just said, "Who's the CEO of General Motors?" The model decided to call `get_ceo_name` with General Motors because that's obvious, and it knew the stock symbol for General Motors is GM. This is another cool benefit of combining language models with code: you can use the model's implicit knowledge to fill in some parameters. Let's try another one. What about McDonald's? Same story: figure out which function to call, extract the function name and parameters, build a Python function call to the Yahoo Finance API, and return the results. We could go on like this. Let's try a prompt that's not so obvious, like where are the main offices of 3M, which doesn't obviously match one of our functions. In this case, the model wasn't able to match our prompt to one of the tools. It lets us know by returning an empty response. Maybe you want a very disciplined model that uses your functions and nothing else, or maybe you want to do a little more. In this modified function, we start the same: call the tools and return the results. Then, we pass the response to another model, in this case, Virtuoso large, a high-performance general-purpose language model. I'll ask the model to answer the question using its own knowledge if the tool response was empty, or to augment the tool's response with its own knowledge without contradicting it. Let's give this a shot. We'll call the Caller model once and try the prompt we didn't answer originally. The workflow will be: call the Caller model, no tool is called, take that result, and pass it to Virtuoso. The tool result provided is not useful for answering the question about the location of 3M's main offices, but Virtuoso can provide information based on its own knowledge. Perfect. It gives the answer: 3M's headquarters are located in Maplewood, Minnesota, US. Now you understand how the workflow works. You can give transparency to your user, saying that you still answered the question, but it didn't come from internal knowledge; it came from the model's knowledge. Let's try this one and see what happens. We should get the company summary. Call `get_company_summary` with the arguments, and we get the result from the Yahoo Finance API. Now, let's do the same with the double invocation and see what happens. We're still calling the Yahoo API, but the response should be different because Virtuoso will take that data and apply its own magic to it. It's breaking it down, organizing it in a nicer way, and probably adding more information. You can tweak this behavior with the prompt you're passing. You could say, "Summarize it," "Translate it," or use retrieval-augmented generation, etc. The possibilities are endless. This shows you can use a specialized model to call external functions and then feed the result to another model specialized for text generation to build a better answer with more context. Instead of having one single model that tries to do everything, use the best models for the job. Here, I used Virtuoso large, but we could try Virtuoso small and see how that works. It should be faster and a little cheaper. We get a shorter, more compact answer, which might be what you're looking for. So, that's what I wanted to show you today. A nice little demo on function calling. The code is very simple and generic, and you can easily adapt it to use the right model for the job and have models collaborate to build better answers, giving your users maximum satisfaction and quality. All right, my friends, that's it for today. I'm still on the road, but I'll see you soon with more content. Until next time, you know what to do. Keep rocking.