Sneak peek Arcee SuperNova on Amazon Bedrock

Transcript

Hi, everybody. This is Julien from Arcee. In this video, I'm going to give you a sneak peek of what's coming. I don't do that very often, but I got permission. Trust me. You are going to see Arcee Supernova, our newest 70 billion parameter model that outperforms the largest open-source and closed models as a custom model import in Amazon Bedrock, which is pretty awesome. So I can't wait. Let me show you. This is the Bedrock console, and the particular feature that allows me to import Supernova to Bedrock is called imported models. This is in preview, but it's already been demoed on YouTube publicly by AWS engineers. So before somebody in Seattle wants to send black choppers my way, go check out that video. Actually, I'll put the link in the video description for extra safety. Click on this, and well, this is actually very straightforward. It allows you to import a custom model in different architectures. Supernova is based on the LAMA-3 architecture, which is supported. Just put the model weights in a S3 bucket and import it. It takes a while because it is a big model. I've done that. Once you've done that, you have an imported model. You are able to open it in the playground. So why don't we ask a question? There we go. I don't know what you think, but I think this is pretty fast for a 70 billion parameter model. You can go and tweak all the parameters and also the number of generated tokens if my face isn't in the way. Maximum length. But that's not really what you want to do. What you want to do is use it in a notebook. Once again, it is very straightforward. Using Boto3, the model ID becomes the ARN of the imported model, not one of the actual model names as with the other models. This is a LAMA-3 model, so I'm using the LAMA prompt. The rest is just very simple, just invoke the model, and because we have streaming, we're going to do streaming too. So here goes nothing. Okay, ready? Bam. This is fast, right? Time to first token is actually very, very fast. Want to try something else? Who cares about deep learning? That's a much more interesting question. There we go. See? So let's see where we go with this. The paint is very fresh. This is still a preview feature, but if you're interested in running Arcee Supernova, which is really the best 70 billion model today and a great alternative to the closed models like the OpenAI models, please get in touch. This is certainly something we'd like to deploy with you. It's your first offer. Ping me. All right. Well, that's it for today. Very cool demo. Very excited about running this cool model in Bedrock. Until next time, keep rocking.