Arcee AI webinar build agentic workflows with Arcee Orchestra

Transcript

My name is Julien. I'm the chief evangelist for Arcee. We have quite a team of Arceevengers. Shouldn't I say Transformers? No, we're human still. So we're the Arceevengers. We have Lucas, who is the lead of Arcee Labs, our research arm. We have Andrew, one of our solution architects working closely with all our customers to build great SLM-based solutions. And you can't see him, but he is here. We have Chris, another one of our solutions architects, and he is anxiously waiting for your questions. So fire away, and Chris will answer everything. No pressure, Chris. Okay, so today we're going to spend some time discussing a new platform that we built, right, guys? It's called Arcee Maestro. And I guess Lucas will tell us later why we picked that name. So Arcee Maestro is really our super exciting launch from just a couple of weeks ago. The reason we're so excited is because it combines what Arcee has been doing since the company was started about a year and a half ago—small language models based on open-source architectures—with new technology that lets all of you out there rely on those small language models to be agentic AI and agentic workflows. Andrew will do a bunch of cool demos. That's the plan today. The plan is to show you what we're building, talk about SLMs, talk about agents, talk about Arcee, and not only talk about it, but also show it to you and answer as many questions as we can. Okay, so get all the questions going. This is really about conversation and hopefully helping you learn a few things. So let's get started. Lucas, I guess by now folks understand what Arcee has been doing, right? Arcee is building small language models, most of them open source, and we put them on Hugging Face. They tend to sit very high on the OpenLLM leaderboard. But there's more to it, right? So why are we moving now into Maestro and agents? What's the story there? And did we lose Lucas? Oh, he's back. All right. Having fun with Wi-Fi. Do you need me to repeat my question? By now, folks out there know about Arcee as a model builder. We put our cool models on the Hugging Face hub, most of them are open source, and they get evaluated on the leaderboard. They rank very high, often number one, and stay there for a while. We get a lot of good feedback on those models. You contributed to all of them, the Supernova Lights, Virtuoso Lights, and the latest one we've released, the Blitz, among others. That's the starting point. Now, why are we moving into agents? What are agents? What is Maestro? What's the big picture here? Arcee is first and foremost a company that wants to enable enterprises to get genuine value from AI. The way that happens has changed as the capabilities of these models have become better. When we initially released those models open source, we started doing a lot of that last June or July. The idea was, if we can make these models good at X, Y, and Z domains, imagine them for your data and your use case. We had a lot of customers come to us interested in helping us deliver tools to train their models. But what we found towards the end was that a lot of companies were coming to us with use cases they thought necessitated a model, like voice processing, sentiment analysis, sales summarizing, market sales lead finding. While you could train a model to be good at that, a vast majority of it is in the implementation. One of the big things with Arcee is we want these models to run in your environment. That's why they're small language models. They should be able to run locally in your own cloud or system, giving you complete control over the data. So we said, we can still help people make amazing models. We can also continue to make our own models trained specifically to do integration-heavy tasks. That's where the idea for Maestro began. Instead of training a custom model for every customer, which can take six to eight weeks, Maestro uses one of our strong models in your environment, connected to all your tools, and can take many actions using those tools that are repeatable, redundant, and reliable. Maestro is not currently an open-ended agent platform, though we are exploring that and doing research towards how to do it well. The goal is to take the time to value for enterprises from a few months to a couple of days. A lot of customers have learned that you can't solve your business problem end-to-end with just a model. People would come to us and say, "We want a model," but their experience was with Chad GPT, which has many layers on top. Regardless of whether it was just model training, there's always an engineering and implementation side. We're still in tech, still building products, whether customer-facing or not. There's no silver bullet. You have data, code, and processes based on Salesforce, HubSpot, Jira, or relational databases. The idea that you can just take a language model and throw everything away to achieve what you were doing before with complex IT processes is a pipe dream. Whatever code you have today is still useful. We love the power of language models for data pattern extraction or conversational applications, but if you need to call an API to get something done with 100% reliability, don't train a model. Do it programmatically. That's why those workflows are so important. Gen AI is incredibly powerful, and we're very early in its stages. These models, both large and small, will be capable of much more, but they are not infallible. Maestro, by nature, is an orchestration platform and an agent platform. With these workflows, we're taking a little bit of agency away from the models and saying, "Look, let's have you do exactly what you're good at." Read this text, transform it into the required format, use the best interpreter, and then do everything else programmatically. The calling of APIs, manipulation in code, and knowledge retrieval are all done programmatically. This makes the output much more reliable and tangible. Andrew can show some of that. A lot of value in Maestro comes from all those integrations. I'm sure we'll see some in Andrew's demo. Let's not just talk about the integrations right now, unless you wanted to say a couple of words. How do you see the role of those integrations? What's the value they bring? If a customer came to us and said, "I need an invoice process," we could train a model to do that well, but we also need to give it those invoices. It wouldn't help if we tried to cut time to value from months to days if every customer had to write their own integration or tool into the system. That was a massive undertaking here at Arcee with some partners, and we have over 250 different integrations. Whether it's Discord, QuickBooks, or any enterprise tool, we're not asking you to replace it. We want to work well with it. Within each, there are hundreds of actions that can be taken. This allows us at Arcee to say yes to a lot more. For example, something that used to be scoped for four people for a month can now be done by one person in two days. Before we jump to the demos and let Andrew show us more of what Maestro is capable of, can we talk a little bit about the models themselves? Generally, what kind of models would you want to use in those agentic workflows? Do they need to be different models compared to the ones we usually build? Would it make sense for customers to have third-party models thrown into the mix? What's the big picture there? We have started to train many new models to do well in Maestro at varying sizes and speeds. What Andrew's going to show today is in our hosted cloud, but we do offer NVPC and other custom solutions as needed. It didn't make sense to say, "I'm going to put this software in your environment, but I'll host the models myself," especially for highly regulated industries like medicine, finance, and government. We want to give people that option. The whole reason for adding our own models and designing Maestro around our own SLMs is to put the entire thing in their environment, including the models. We trained many new models to do well in Maestro, from our largest flagship model, Virtuoso Large, which is very close to GPT-4, down to the RC Light and RC Blitz models we just released. They have all been trained with our world-class distillation pipeline, using raw API data to ensure they can parse important information from API responses. We've also trained them on a lot of raw API data because it doesn't help if the models can't understand what's important in the data they're processing. We've learned that some APIs are self-explanatory, while others are weird and require specific training. Large models like Opus or even Sonnet can typically infer importance, but for smaller models to maintain performance, we had to train them specifically for these tasks. We also want to make Maestro as easy to use as possible, so we allow Sonnet, GPT-4, and other models. You do what you're most comfortable with, and we ensure the option is available. Freedom of choice. Andrew, thank you, Lucas, for sharing your views on agents and Maestro. Now let's show Maestro. Andrew works with customers day in, day out, so he's one of the best people to show us Maestro. We also have Chris, our solution architect, answering questions. Keep him busy, okay? We don't want him to slack. All your questions, please send them to Chris. Andrew, what are you going to show us? We're done with the philosophical part. We want to see the platform in action. Andrew, tell us what you're going to show us. We're going to see a couple of things. First, we'll look at what we consider agentic and non-agentic actions within Maestro as we interact with the workflows or automations we build. We'll also take a look at how to build a workflow and then see a couple of different examples to show how the different pieces can integrate together. This is the chat interface you see when you log in to Arcee Maestro. It's similar to many applications because we didn't want to reinvent the wheel in terms of how you communicate with models and interact with your workflows. In this UI, you can directly interact with different models. For example, here we're interacting with our Virtuoso Large model, a large general-purpose model. You can set typical parameters and all the fun stuff you can do with models. But you also have the ability to enable tools, which are the workflows you build. When you interact with this system, the model is aware of the selected tools, allowing you to directly invoke them. Let's start with a simple example. We'll look at an order status example. When I sent this request without any tools selected, the answer was very generic. It's like asking a general-purpose person on the street about an order status. But when I select the tool and give the same prompt, the model recognizes it can help and starts the workflow, providing a response. This is where we see the difference between non-agentic, where you can't access external systems, and agentic, where the model can invoke automations. Some viewers might not be familiar with this, but there's no magic. When you enable tools, the model can add two and two together based on the tool descriptions and figure out the best tool to use. Under the hood, something happens, and that's what we'll show you. One of the great things is that Maestro creates the tool descriptions for you. You ask the same question, and Maestro figures out how to best answer it. When we were building Maestro, we decided to do it all from scratch to make it easy to summarize the workflow and automatically describe it by AI. The user doesn't need to worry about whether their description was enough for the model to understand. We handle all that. Let's take a common example: doing research. I enabled a research automation tool and asked it to research the stock market performance today and global news. It pulled real-time articles from the web, analyzed them, and provided a summary in a very consumable fashion. Let's take a look at how this actually works. This is an example of a workflow. While it looks complex, it's actually very easy to put together. We use a drag-and-drop UI to build these workflows, enabling both technical and non-technical people to create automations. There are nodes you can use, including specific models from the Arcee suite or models you bring in. There's the ability to run code for flexibility, do retrieval-augmented generation, and use built-in integrations. This workflow is really drag-and-drop. Andrew has a bunch of code boxes, but you can build complex workflows without code boxes, just using models and integrations. You don't need to write a line of code. You just drag and drop, enter your prompts, and connect the nodes. This is very user-friendly, and we have colleagues at Arcee who are not software engineers building workflows for their job functions. You don't need to be a software engineer or a data scientist to use this tool. One highlight in this specific workflow is the different model nodes. Some are labeled senior analysts, some are junior, distinguishing between model sizes. For detailed analysis, we use a larger model, and for simpler tasks, we use a smaller, more performant model. This balances accuracy and performance. We also see parallel execution, which is great for latency. Customers often want to be notified about automations, whether through a messaging platform or email. The integrations we have built make this very easy. For example, in this workflow, an email is sent to Gmail, and a report is created. The email includes a one to two sentence summary of each article, and a more detailed report is attached. This allows you to quickly read through and focus on the most interesting articles. Every one of those integration boxes, like Gmail and Google Docs, is about filling parameters for the underlying API. You don't need to write code for these APIs. This is a huge time-saver. One upcoming feature is a natural language option within code nodes. If you want to do something custom, you can use a code node, but you can also use a model to facilitate it. This is what we're calling "vibe coding," where you can rely on model abilities to understand data structures and build complex workflows without writing code. Let's take another example. This one pulls a YouTube video, extracts the transcript, and generates a blog post based on the content. I use this workflow in real life. If you go to my Substack, you'll see blog posts in Chinese and Hindi, languages I don't speak. This workflow runs very nicely. There's no code box; it's just models and integrations. If I wanted to add more languages, it would be as easy as adding more boxes in parallel. The input is as simple as the ID of the video. This can be applied to any system where you have records you want to process, whether for automations, identifying trends, or summarizing data. Once this finishes running, I can pull the Google documents that were created and show you what they look like. For example, when cleaning up captions, we use a smaller model, but for writing the blog, we use a larger model to ensure technical accuracy and include all the necessary components. This shows the dynamic use of different models within a single execution. Here's one of the blogs created two minutes ago. It includes all the information from the video, which was about our Virtuoso Large model. This demonstrates what models are great at—storytelling. The raw data from the video and captions is transformed into a well-structured blog post. Models are great writers and have been trained on technical documents, so they know what a blog post should look like. The whole idea behind Maestro is to handle repetitive, mundane tasks, allowing people to focus on creativity, engineering, and problem-solving. One automation I use every day is for customer-facing roles, where I take notes from meetings, look at the CRM, and update different systems. I used to spend a significant part of my day on this, but now I have workflows that automate the entire process with a single button click. This allows teams to focus on more important tasks, like building new features or demos. Maestro just launched, and now we get to have fun. We have the ability to do genuine R&D to deliver experiences like Etena, which you can't get elsewhere. We will always clearly mark research previews or beta features so you know what's safe for production use. One goal was to design the workflow representation in a way that's easier for a model to understand and generate. While it's not hard to get a model to generate workflows, getting a model to generate good workflows is different. The model needs to ask clarifying questions to get a usable workflow. This is another layer of automation and acceleration. We have over 200 integrations available, and each has many actions. For example, Slack has hundreds of actions. This provides a lot of functionality. Even if it took only 30 minutes to do one action, piecing together 10 actions would take a substantial amount of time, which you can now do in seconds. The integrations available are a big piece of Maestro. Now, I'll show you a video that Lucas talked about. This is about a workflow generator that helps you create workflows more easily. It's like Clippy 2.0, where you say, "Make me a workflow for forwarding every marketing email to my Slack channel." It realizes you don't have a marketing channel and creates one. It gives you a wireframe of what the workflow will do. The idea is to make the process much easier and more accessible. The ultimate future for Maestro is to make it super easy to use. If an organization buys Maestro, I want everyone from HR to IT to sales to be able to use it well. Real workflows and business operations are complex, and security, compliance, privacy, and cost are always important. We come from those backgrounds, and we understand that people don't want models; they want solutions to their business problems. If AI is involved, great, but it needs to work. We're excited about solving real problems for real people, and the workflow generator is another step in that direction. We need to wrap up. Thank you to all our viewers for spending time with us. If you want to learn more, visit our website, read our product pages, and book a demo. Chances are, Andrew, Chris, or someone from the team will connect with you to understand your business problem and build a demo that makes sense for you. Thank you, Lucas, for your insights and leading this amazing effort. I can't wait to see what we build and release together. Andrew, thank you for the demos and clear explanations. Chris, thank you for answering questions. Thank you to our marketing team for setting up this event and giving us a chance to show you the cool stuff we're building. Stay in touch, connect on LinkedIn, and we'll see you soon with more Arceevengers goodness. Thank you, everyone. Have a great day.