AWS re Invent 2021 A first look at SageMaker Studio Lab

Transcript

Hi everybody, this is Julien from Hugging Face. In this video, we're going to look at SageMaker StudioLab, a new launch from AWS reInvent. This is a really cool tool, something that a lot of us have been waiting for for a long time, as we can easily create notebook environments to experiment on CPU or GPU without having to sign up for an AWS account and without any setup complexity. So this is really cool, and I'm going to demonstrate it with Hugging Face models. Let's get started. StudioLab is in preview. You should go to the page, sign up for it, and you should get access pretty quickly. I got access in a matter of hours, so it should be fast. Then, of course, you can go and sign in. Note that you are not using your AWS account here. You're using your email, and that's it. So if you have no AWS account, that works. Super easy. No credit card required. It's all free, by the way. We can create different runtimes. You can create CPU runtimes or GPU runtimes. We'll start with a CPU runtime and get some resources here. Yes, we see Hugging Face, which is pretty cool. What do we have here? We can click and see a quick tour notebook that we'll be able to run once we have an environment. Let's first create this environment. I'll just click on start runtime. It takes a few seconds to spin up, and then we can go and open notebooks. Okay, it took about 20 to 30 seconds. Now my CPU runtime is running, and I can use it for 12 hours in a row. That should be more than enough for your working day. If you're done prior to that, you can stop it and release resources. If you let it run for 12 hours, it shuts down automatically, and you won't lose any files, as you will see. Files are persisted. Let's copy that to the project and see what happens. Animations. Copy from GitHub. Just take the notebook. Let me zoom in a little bit. That was simple. It's imported. We can pick our kernel. There's a default Python kernel. We'll stick to that one for now, and I'll show you in a minute how you can create your own kernel. We could run this example, but it's a little long, so I'll run some of my own examples. First, I like to figure out what this environment is. So let me fire up a terminal. What do we have here? How much memory do we have, first of all? Yes, we have 15 gigs, which is not bad at all. What about CPUs? We have four cores, reasonably fast ones too. So we have four vCPUs and 16 gigs, which is pretty good. I've got the instance types here. Let me zoom in. It's equivalent to a T3 XL. It's kind of better than what you got in Studio. I think Studio by default was T3 medium, if I'm not wrong. Anyway, that's not bad. We have what we need here to work. Let's run a quick demo. I've got a very simple pipeline demo with Hugging Face. I've created these notebooks before, and as you see, they're still around. Even though my environment, my runtime was shut down, my files are persisted. Although it's probably safe to commit them to Git or save them somewhere, I haven't lost anything so far. If I run CUDA here, it's not going to work because we don't have a GPU. I could go and pip install what I need. Let's do that. It should take a second, and then I'll just run my pipeline. See how that goes. As you can see, it's JupyterLab, and it's super simple. That worked. I'm not 99.85% positive so far, but okay. We have the debugger integration here as well, which could come in handy. So far, I really like the fact that nothing about IAM, nothing about S3 buckets is required. You don't need to know the first thing about AWS to use this, which is really how it should be. Well done on that. Let's try and create a Conda environment. For the record, this is going to be a minor complaint. We have a getting started doc here, which is generally quite useful. It tells you how to install packages, some sample notebooks, etc. It tells you how to create an environment, except the command they give you doesn't work. I'll prove that. If you're stuck here, I'll fix it in a second, but if you try and do this... Nope, it won't work. What you should do instead is this. Okay, and that one works. Let's create an environment called Transformers PT for PyTorch with Python 3.9. Yes, please install all that good stuff. If you're not familiar with Conda, this is a really good way to keep your dependencies separate. So if you want a TensorFlow environment, that's one environment. If you want a PyTorch environment, that's another one. If you want a clean, let's say, SKLearn environment, that's the third one. If you're new to this, please read a little bit about Conda and create these things correctly. It'll save you a lot of headaches. We can just activate this environment now. Okay, now I can go and install some of my dependencies. All my dependencies have been added, and I'm going to install IPy kernel so that I can add this new Conda environment as a kernel in my Jupyter Notebooks. Now this should add it to Jupyter. So let's see if I have to restart the environment or not. Yes, it looks like I have to restart it. So let me shut down the environment and restart it again. I just stopped the CPU runtime, launched it again, and I can see my new environment. My new Conda hub has been updated, so I should be able to use it here. Okay, that worked. Just make sure you create your environment right. Make sure you add it to Jupyter, and you're good to go. I strongly encourage you to use these. Otherwise, you'll just make a mess of your Python environment, which is what I do all the time. Here's the CPU environment. Let's stop this and try a GPU runtime. My GPU runtime is up, and we can see we're limited to four hours here, which should be enough for experimentation and testing. We can't train for more than four hours. Let's open the project. Oh yeah, let's check that we have a GPU now. We have a Tesla T4 with 16 gigs. We could open a terminal again and take a peek at this. Yes, let's look at our GPU. CUDA 11.4, Tesla T4, 16 gigs. That's a nice GPU. How much memory do we have on this machine? We still have 16 gigs. What about the CPU? How many do we have here? Yeah, it's the same. So we still have four vCPUs. Still four vCPUs, 16 gigs, and a T4, which is pretty much what you would get with G4DN Excel here. These are good environments. We should be able to get some work done here. Let's train a bigger notebook now. This is one of the examples from the Hugging Face course, which I cannot recommend enough, and I'll put the link in the video description. Here we're fine-tuning a distilled BERT model for binary classification on the IMDB movie review dataset, which includes positive and negative movie reviews. We have all the dependencies we need here and can just run everything. Run all cells. It will download the dataset, tokenize it, start from the pre-trained model on the hub, and we set up our training arguments. We're training for a single epoch and define our accuracy metric, and then the trainer object, everything together, and then we call and go train. It works out of the box. It's going to run for 18 minutes, so we won't wait for that. That's all there is to it. It works out of the box, and again, zero AWS stuff. Now, how would we work with AWS here? If I go to the terminal, the AWS command line is installed. You would just go and call AWS configure and enter your access key, secret key, default region name, and you could issue AWS command line commands. You could install the Boto3 SDK and do the same from Python. One thing we cannot do is install native dependencies. If we try to call apt-get update, it says permission denied. If I try to sudo, it says sudo command not found. So that could be a problem if you have Python libraries that need native dependencies. I'm not sure what the solution is, but I don't see how that would work. Hopefully, you don't need any fancy Python library with native stuff. Otherwise, you'll be kind of stuck. I guess that's an okay restriction for a totally free environment. If you need bigger things, you can move on to SageMaker Studio, which will let you install native dependencies. There is no seamless way to migrate those notebooks to Studio. Maybe that's coming later. If you need to do that, you could use a Git workflow, push your notebooks, and then clone them again in Studio or download the notebooks and upload them again. At some point, it'd be nice to have some kind of right-click, send to Studio kind of thing. I think it's missing for now. But okay, I'm always complaining. This is making progress. We see the logging and the checkpointing, etc. So, you know, that's going to work. I've run this a few times, and it does complete. There you go. That's StudioLab. You can sign up. Super easy. You don't need to know anything about AWS, nothing about IAM, S3, etc., which will probably come as a relief to some of you. You can create your CPU and GPU runtime with a fair amount of resources, as we saw. You should be able to get some work done. It's not a toy. It's definitely usable for your daily work. Go and try it. If you have questions, you can post them in the video. Keep learning. Bye-bye.