JavaDay UA 2017 Machine Learning and AI in Java on AWS Julien Simon

Transcript

So here's the agenda. A few words about AI at Amazon. Then we're going to start looking at some high-level services that are just API calls to perform AI services like text-to-speech, image recognition, face detection, etc. Thanks. As you will see, they're really, really simple to use. They're based on deep learning, but all you have to do is just call an API. Then we'll go down one level and we will look at a service called Amazon Machine Learning, which is machine learning as a service. You don't have to worry about infrastructure. You don't have to worry about starting a cluster or managing servers, etc. You just upload your data, build a model, and invoke it to get predictions. Then we'll keep going down. We'll look at another service called Amazon EMR, which stands for Elastic MapReduce. Specifically, we'll run a notebook based on Spark ML Lib for spam classification. Last but not least, we will take a quick look at Apache MXNet, which is an open-source library that allows you to build and train deep learning models. Of course, I'll share some links. Of course, you will get the slides. I usually put them on Twitter right after the talk. All the code that you're going to see is also on GitHub, and the link is in the slides. So just a few definitions before we start to make sure we actually all talk about the same thing. Artificial intelligence is the high-level discipline. It's been around for decades, literally since the 50s. The purpose of AI is to build systems that show human-like behavior, right? Systems that can understand natural language, systems that can talk, systems that can show reasoning, intuition, prediction, etc. It's a very wide field. Within AI, there's a subset called machine learning. Machine learning tries to teach machines to learn without being programmed. So basically, you take an algorithm, a standard algorithm, you run some data through it, and the algorithm learns how to predict correctly or to classify correctly based on the dataset that you used. Then there's the third thing called deep learning, which is a subset of machine learning. It's the same story. We try to teach machines to learn automatically, but this time we give them datasets that are usually very complex, very large, like images or sound or video, etc., and datasets where features are not visible. When you learn from, let's say, web logs or CSV files or structured data, it's quite obvious what the features are. When you learn from images and sound, it's not obvious at all. And actually, deep learning is able to find the features automatically. So we'll cover all three today. AI is a key thing for Amazon. We've been doing AI for a long time, especially on Amazon.com, the retail part of the company. Anytime you go to Amazon, you see personalization, recommendation, A-B tests, etc. It's AI all over the place on the website. It's also heavily used on the logistics side. I'm sure you've probably seen those videos on YouTube where you see the Amazon robots moving around in the fulfillment centers and moving the shelves to the people who actually prepare the orders, etc. And we built some new products like the Kindle and the Echo device. And of course, all of those use AI and machine learning. AWS is now bringing machine learning and AI services to all of you, to everyone. It's not just Amazon; we have tons of customers. It's mostly a technical talk today, but I will give you some customer examples on pretty interesting use cases for AI. If you're interested in those, you can go to our website and you will find all the use cases and all the stories for this. So here's the current stack. Like I said earlier, multiple levels. High-level services that are just API-level, SaaS-level services. So, call an API, get the job done. We'll cover a couple of those today. Going down a level, we have platforms like Amazon Machine Learning, Elastic MapReduce, and some other services where you're free of managing infrastructure. We manage the infrastructure for you, but you still have the flexibility to build your app, write your code, etc. Then going down even deeper, we have the open-source libraries like MXNet and TensorFlow and all the others that you can run, of course, on our instances to build custom models and train them. At the bottom, of course, we have the infrastructure, we have CPU instances, we have GPU instances. We recently announced support for the NVIDIA Volta GPU, so it's available now. I guess we're the first cloud to have them. And you can do IoT deployments, mobile deployments, etc. So we're going to look at all of these things. Well, not all of them, a few of those. First, we're going to start from the top level and we're going to look at two services, Polly and Rekognition. Polly is a very simple service; it's text-to-speech. You give a text string, you select a voice, we have 50 voices in 24 languages, you call an API, and in real-time, you get a sound file with a voice. It's lifelike, so high-quality sound. As simple as that. That's really literally the only thing you could say about Polly. So let's look at an example now. Oh, sure, before we do that, who uses Polly? Well, we have customers who use it, but I guess this is probably the most famous example. So if you've seen that Echo device before, the voice that comes out, the Alexa voice, and the voices that are used by all the Echo devices, they are generated by Polly. It's definitely production quality. Let's switch to code. Can you see okay in the back? Yeah? Alright. If it's too small, you just yell. Here's a very simple example. And what we're doing here is, first of all, we're going to connect to the Amazon Polly client. You have to do that for most AWS services. Just grab a client. And here I'm using the EU West region, which is Ireland. And the only thing I'm going to do really is this. I'm going to call that one API with a voice. We have 50 voices and they all have names. Here I'm going to use Brian. It's a UK English voice. The text string and the output format that I want. Then I'm going to play it. With Polly, you can do plain text, but you can also use SSML, which, if you're not familiar with it, is a markup language to provide extra semantics to the text. Here I'm building a second message. It's SSML, but when you listen to it, it will make more sense. I'm using a different voice and playing it again. Let's try this. Let's have some sound. Hopefully, it's going to work. Do we have sound coming up? No, I don't think I have sound. Let me try it like this. So, it's working. But only the front row can hear it. You heard it, right? Yeah, all right. I've got witnesses. Should I try again? Yeah? Okay, all right. Okay, so pretty cool. It's much better to have those, for example, the phone numbers. If you have triple A, triple five, it's easy to say it like that, it's easy to remember. You could also say 888555 if you wanted to, but it's probably easier to remember. And the times and dates, you can make sure they're pronounced exactly the way you want them to be pronounced. So that's Polly in 24 languages. And there are some more features, of course, you can change the speed and you can change the pitch. You can do lots of different things. But literally, it's one API call and you can have real-time speech in your application. Or you could save the sound files and play them in the application as well, of course. Okay, so here's Polly. Super simple. Alright, let's move to the next one. The next one is called Rekognition. As the name implies, it's going to be used for object detection, scene detection, face detection, face comparison, and some alternative use cases like celebrity detection. So you should try it. You should try the Ukrainian football team and see if we actually can detect those guys. I didn't try it. And there's content moderation to detect explicit or suggestive content. And no, I will not show you a demo of this. You can try it yourself. All right. So here's an example of a customer using Rekognition. It's a US TV channel called C-SPAN. They broadcast all the political debates in the US. The problem that C-SPAN has is they want to be able to find a given speaker in their archives. They have years and years of video archives and they want to be able to say, okay, Senator X did talk about this specific issue on this specific day and then this other day and then a week later, etc. They want to basically find speakers in their archives. Until then, they did that manually, if you can imagine that. So they have almost 100,000 people to keep track of. So they have to literally write down, I guess they have a huge Excel sheet or something, hopefully. And they write down all of that stuff. It's insanely time-consuming. So instead of doing that, they tried Rekognition. So they built a database with 100,000 faces, 100,000 people. And they run their video archives through that, and automatically Rekognition can find that, okay, this guy here is Senator McConnell, and you can look at the agenda for that day and then you know what he's talking about, right? So it's now fully automated, and they're actually processing all their archives, right? Years and years and years of archives going back to, you know, probably decades, and being able to now very quickly find their speakers in their terabytes or petabytes of archives. Let's look at a few examples. The first thing I want to do is object detection. Here is my picture. It's Oktoberfest, but I'm sure you have a Kievfest or something. There's something like this everywhere. I want to know what's in this picture. This is how I'm going to do it. I have to copy my picture to an S3 bucket. I could upload the picture directly, but here it's in a bucket. Again, I've got a client. I'm connecting to the Rekognition client in Ireland again. The only thing I have to do is this. Create a request saying, here's the image, give me 10 labels and give me 10 labels that have at least 75% confidence. Okay? And then I'm calling detect labels and then getting the result and just iterating through the labels. Right. Want to try it? And if we have the console, it's going to be better. This is what Rekognition found: people, person, human, alcohol, drink, crowd, female, girl. Sounds like a good definition. Again, as you can see, it's real-time. Here, I've got just 10 labels, but I can literally have 100 labels if I want. Scores go a little lower. Again, one API call and you get deep learning-based object detection. It's very fast, it's very easy, you don't have to be a deep learning expert to do this. Now let's look at a second example. Let's try to do face detection. We're going to use that same picture and we want to find faces in there and get some information about the faces. What does the code look like now? Same story, get a client, build a request. It's almost the same. Then I'm calling detect faces. One API call. Then I'm iterating through those details. I'm going to print out how many faces I found and for each face, I'm going to print out quite a lot of information. So let's try it out. Yeah, quite a lot. Okay, so we found 15 faces, right? Which is the maximum that you can find in a picture. It's a limit that we set, right? It's not really a technical limit, I guess we don't want to look for 100 faces. 15 was found to be a reasonable number. For each face, I'm going to get some information. Of course, I get an age range, but what I really get is a JSON document with what we call the bounding box. So the positioning of the face in the picture, right? So these are relative values, not pixel values, of course. So the age range, smile, eyeglasses, sunglasses, gender, does that guy have a beard or a mustache, etc., etc. Emotions. And then the landmarks, you know, where are the eyes, where is the nose, etc., etc. It's very funny if you want to add fake sunglasses or if you want to do Snapchat-like stuff. It's useful. Lots of information here. Then we have the same thing for the second face and all the faces. That's Rekognition for faces. The last example I want to show you is comparing faces. Let's look at the images. Image one, that's my reference image. Image two, am I in this picture or not? Here, same thing. I'm going to grab a client, and this time I'm going to actually upload the images. I'm creating an input stream and sending the byte stream directly to Rekognition, not using an S3 bucket. Creating a request for this, calling compare faces, and getting results. That's it. Am I in this picture or not? In the second picture, again these are relative values, so it's kind of in the middle of the picture, right? 60% and 51%. Okay, that's me. I can confirm that's me. So, as you can see, not a lot of code, not a lot of complexity, just grab a client, build your request, send it, and you're done. And you can get extra information, of course, I'm just showing the match here. Just what I'm interested in. Alright? Okay, so let's move on. So there's another service called Lex for chatbots, but in the interest of time, I can't show all of them to you. But feel free to go and check out Lex if you have chatbot use cases. So now going down one level, let's look at Amazon Machine Learning. So the high-level services like you saw, they're just API services. Now let's say you want to build your own machine learning model. How do you do that in the easiest way possible? So this service was really designed for non-experts. It's really designed for developers who don't know much or possibly don't know anything about machine learning, but they still have data and they want to build models. Okay? So it's super easy to use, really. First of all, there's no infrastructure to manage, so that's good. And second, you pick from a small set of algorithms, and they work, they do the work for you. So you don't have to worry too much about the machine learning part. It's based on actually a project that has been running in production at Amazon for years and years and years. And it became an AWS service in 2015, actually. So it's pretty robust. It's still being used by Amazon today. You can grab data that is already living, of course, in AWS. So you can upload data that lives in S3, RDS, or Redshift. The really cool thing is once you've trained the model, you can deploy it to an API in just a few seconds. It's basically clicking a button in the console or calling an API to deploy the model. If you've tried to do this before, you know it's the complicated part. I always say the machine learning part is easy because usually, you do it on your own machine, your own laptop, right? And then you take that model and you try to put it in production on 100 websites, and it needs to be fast, it needs to be highly available, etc., etc. And then the real problem starts, right? Here, this is completely solved in a few seconds. So here's an example of a customer who uses that, Fraud.net. So basically what they do is their customers are e-commerce websites all over the world, and they want to help these customers detect fraudulent transactions. So if you have a stolen credit card and you order from them and you have a delivery address in the middle of nowhere, that's probably not a good transaction. So this is what they do. They build machine learning models to detect bad transactions on e-commerce websites. There's no single model that can find all bad transactions. You need to look at 20, 30, 50 different things and probably you need to build a model for each of those problems. Amazon Machine Learning lets you do that very easily because, again, it's all API-driven. It's very easy to create a model automatically, train it automatically, deploy it automatically. So if you want to work at scale, it's very, very comfortable. So let's look at an example. So here... Oops. I have a dataset. It's a banking dataset. It's a public one. And it's got information on whether all these people here, based on these features, are likely to subscribe to a new financial product, right? Are they going to be interested or not in that new offer? Okay? And so there are tons of features, age and work type, marital status, level of studies, etc., etc., plus more features. And well, we have about 40,000 something. Yeah, about 40,000 samples. So how can we build a model from this and predict with new samples? That's the thing we're trying to do here. Again, in the interest of time, I did prepare a few things. The steps that you would go through is first, you would create a data source. You would upload that data. Again, here it's a CSV file, so you can upload it directly. But that data could be in S3, it could be in one of the different backends. Upload that data, and it's going to perform some quality checks, give you some basic statistics information, etc., etc., on what the dataset is. You can explore it. Then you're going to build a model from it. Here what I want to do is a binary classifier. I want to know if these people are going to be buying that new product or not. I need to look for a binary classification problem. I'm going to say, well, in that training set, the attribute that I want to predict is yes or no, will they buy that product? That's the target value, the target attribute that my model needs to learn. Then I'm going to basically use that data source that I created and let the model train. It's really as easy as I say it is. I'm just going a little fast here, but there's nothing happening besides, okay, build a binary classifier, here's the target attribute you want to learn, click, go and do it. It's going to run for a few minutes, depending on how much data you send, and then you get an evaluation. Amazon Machine Learning is going to keep some of the samples, to exclude some samples from training, and it's going to use them for evaluating the model. I get different quality scores. For example, for classifiers, the AUC indicator area under curve is a good one. The closer it is to one, the better. And I can look at this evaluation and say, yeah, okay, actually this is an interesting model. So I can get some stats and some graphs and I can adjust the thresholds for the model. Come on, Wi-Fi. All right, let's try that again. Oh, here it goes, no? Come on. Okay, well, not today. The curse of Wi-Fi. Okay, no worries. So, you see a graph and you see the area on the curve and you see the false positives and the false negatives, etc. You can adjust for that. Now, basically, once I've done this, I've got a model, I know how good it is, and I can deploy it. I could upload another CSV file missing that target attribute and predict all of those values. Or I could just click here and deploy my model to that endpoint. And again, this only takes a few seconds. And now I can do predictions. You don't have to worry about deploying that to a website or anything. It's done automatically. So what does it look like? So this is what it looks like. So here my model has been deployed behind that API, and I want to use it. So I'm listing all my available models, printing out some information for the model I'm interested in. We'll see some information there. Then the interesting part is this. I build a prediction request. I assign it to that endpoint. Then I build my sample. You have to love or hate the Java API here. I have to say in Python, it's a little easier to do, but the builder pattern is there to stay, I suppose. You have to call add record entry a million times to add all the attributes for that new sample. Then you basically just call predict with the request. That's how you invoke the model. Let's try it. I see some information about my model. I see it's a binary classifier. I get my endpoint here. And I get my prediction results. The actual score was very low, 0.02 something percent, which I decided to set as a zero because it's so low. It has to be 0 or 1. So my threshold here says, no, that's a zero. And that's all there is to it. And once again, if I had time, I could show you the whole thing from scratch in probably 15 minutes. Create everything from uploading the CSV to doing this in 15 minutes, maybe less. So no infrastructure to start, no infrastructure to manage, no expert machine learning knowledge required, just go and do it. And you can build regression models and you can build classification models. So if your use case is exactly this, there is no faster and simpler way to do it. But sometimes your use case is a little more different, a little more complicated than classifying or predicting numerical values. And then you have to go a little deeper and build your own model and your own code, and you cannot rely on a managed service. And so when you have that kind of need, then probably the right solution would be to look at Amazon EMR. So Amazon EMR has been around for a number of years now and it's basically a managed service for all the Hadoop apps. So it started with MapReduce and then Spark and Hive and Presto and Impala and Flink and God knows what else, you know. We keep running after all those Apache Hadoop apps. But they're all in there. It's the cloud, so it's elastic. You can launch a cluster in a few minutes. I'll show you how to do that, maybe less than a minute. You can choose between our open-source distribution, which is pretty much the Apache distribution, or the MapR distribution if you're interested in that. Then you get all the good stuff, you know, pay as you go. Now you're paying by the hour. Or can you pay? No, you can pay already. I think you can pay by the second now. I think EMR is part of the second, pay by the second thing. You can use spot instances to save a lot of money. So you get all the good stuff that you already know from EC2 and so on. But it's not just that. You can also benefit from multiple integrations. So if you have data that lives in, for example, DynamoDB, there's a connector to upload it. If you have data in RDS, there's a connector. If you have data in Amazon Elasticsearch, there's a connector. If you have data in S3, you can directly load, well, read and write data from S3. You don't have to load it or read it in HDFS. You can connect to Kinesis if you have streaming use cases. And you can also read it from Redshift. So that's the interest also of EMR, is that it's going to find your data in AWS, in all those backends, and it's going to make it very easy to load it. You don't have to worry too much about writing scripts and doing the loading yourself. All these connectors and all the training, all the tutorial codes, etc., will be there. When it comes to Spark specifically, which we're going to look at in a second. There are multiple ways you can submit a job, a Spark job on EMR. You can use the EMR API where we have what we call the step API, which is a way to start a cluster and get it to run a job or a sequence of jobs automatically. And you could say, well, once the step is complete, just terminate the cluster. So it's really on-demand clusters. Start a cluster, run that job, terminate it when the job is over. You can automate quite a lot of things here. You could use AWS Lambda to trigger jobs. And then you could use other services like Glue or Data Pipeline to build, to integrate Spark into your ETL workflows. And of course, you could be running your own schedulers on EC2 instances. If you want to run Airflow, you can do that and it's going to connect to the Spark cluster. And if you want to experiment, you can also use Zeppelin, which is included in the EMR distribution. So when you start an EMR cluster, actually Zeppelin is pre-installed and you can connect to it. Okay. Let me show you how you start a Hadoop cluster. I'll do it in the console. But of course, you can do this with APIs. So let's count the clicks. One. I want Spark. Two. Three nodes. You can choose different node sizes here. Let's have something nice here, M4. Let's select a key pair, SSH key pair to connect. So how many clicks is that for? Five. Done. Wait for a few minutes and your cluster is ready to work. Of course, it's five clicks but it's one API call. If you call the create EMR cluster, it's one API call with those five parameters or something. Wait a few minutes and you have a cluster. I don't want to wait for a few minutes. I already have a cluster but I did the exact same day. I launched it the other day and it's got Spark running. How do you connect to that? First, you need to open an SSH tunnel to the cluster. I could SSH directly to the cluster, but here I want to see Zeppelin. I need to have the web app. I have a tunnel to that cluster. I need to set my proxy here to use that tunnel. Hopefully, I see that. I see my notebook. Yeah, alright. Okay. So let's clear the output. Okay, so it's Zeppelin running on my Hadoop cluster. Can you read it? Yeah, in the back? Yeah? Oh, you have screens on the side. Okay, that's good. So Zeppelin notebook running on my cluster. So here I'm going to build a Let's run some imports. I'm going to load my dataset. I have two text files. One with correct content. It's one line per message. and one with spam and crappy content. I'm going to load those. As you can see, they're hosted in S3. We pull them directly from S3. I'm going to map the features. I'm going to extract a thousand features from those datasets, so a thousand words, the most meaningful thousand words that help me understand if it's spam or if it's correct. I'm going to build my dataset here. So the positive examples, so all the features that are found in the spam messages are going to be labeled with a one, meaning it's spam. All the features in the correct messages are labeled with a zero to say no, it's not spam. Then I'm splitting my dataset. I want to keep 20% for evaluation, so I'm going to train on 80%. And then I'm going to train a number of models. Okay, so that's the Spark ML API. So I can try an SVM model and see how well it does. Accuracy is not bad, but area under curve is too low, so that's probably not a good model. I can try another algorithm, logistic regression. Good accuracy, slightly better AUC. Let's try a decision tree. Yeah, not great. Let's try a variant of trees. And as you can see, here I've got a small cluster with three nodes. I could have a cluster with 100 nodes and crunch huge datasets and I would still be working in my notebook and I literally don't worry about infrastructure. That cluster could be managed by your ops team, and you would just connect to it. As a data scientist or as a developer, you don't need to know what's behind this. Again, it's all pre-installed. This was a little longer. Hopefully, it's a little better too. Yeah? Okay. But then, NaiveBayes is the answer for spam. Best accuracy, best AUC. It's like the hello world for classification. I'm sure you've done this one since school. NaiveBayes for spam classification still works the best. We can try new samples. Here's a message. Let's say that's an email I got this morning. I want to know if it's good or not. Let me be very clear. I have nothing against the good people of Nigeria. It's just an example. Here's another message. Let's take those messages and run them through that NaiveBayes classifier. The first one is definitely spam and the other one is not. Here's an example of using Amazon EMR with Spark and actually Spark ML and Zeppelin to get the job done. You can see how easy that is. Again, if I could do this from scratch, the cluster is probably ready now. We should check. Let's just refresh the page. The new cluster has been ready for five minutes. Again, in the interest of time, I did create a cluster, but it really takes a few minutes. Then you connect and you start working. When you're done, you just terminate it. I would just say, I don't need this guy anymore, just terminate it. It stops paying. Pretty cool, right? And if I need 100 nodes, I just ask for 100 nodes. Alright. Let's keep going. Okay, so that's where we are now. So let's go down one level. Okay, the last level. Now, you want to go into deep learning. The examples that I showed you before in Amazon Machine Learning, the features were very clear. In that Spark example, the features were very clear too, even though they were computed automatically. But now, let's say I want to do image recognition, that's more difficult. What are the features? I give you a thousand pixels by a thousand pixels image, what are the features? Are all the pixels a feature? Do you have one million features? No, probably not. So if you try to use traditional machine learning algorithms on that kind of content, it's not going to work. And this is where deep learning is going to help. So here I'm going to show you MXNet, which is our preferred library. It's our preferred library for a number of reasons, because it has the widest language support. You can do Python, C++, Scala, R, Julia, and probably more. It works pretty well in embedded environments. I've got a very nice demo with a Raspberry Pi where I have a deep learning model running on a Raspberry Pi and it works fine. So it doesn't have to be a huge machine to run MXNet. It scales very well as well. If you need to run very large trainings, you can scale linearly. If you have twice the number of GPUs, it's going to be twice faster. If you have 16 times the number of GPUs, it's going to be 16 times faster. It scales almost linearly to 256 GPUs. If you have huge datasets, images, videos, etc., etc., and you train on a small number of GPUs and it takes days and weeks and it drives you crazy, you can use MXNet, add more GPUs, and save some time. And last but not least, it's part of the Apache project, which we love, because it means it's very independent. So it doesn't belong to anybody. We've got committers from Apple, and AWS of course, and from the research community and from universities. So a lot of people contribute to MXNet, but it's managed by Apache, so it's independent. So again, I don't have much time, so I'll show you a simple example. It's probably the Hello World for Deep Learning. It's the MNIST dataset, 60,000 handwritten digits, and of course the game is to show digits to the model and have the model tell you that it's a zero, two, four, or six, or something else. Let's take a look. I won't go into all the details, but I'll point you to more content if you want. Now I'm not working on my Mac anymore. I'm working on a GPU instance. And this one is the bigger one. It has 16 GPUs. It's a nice toy. I'm glad I don't pay my bills. And so this is part of the MXNet distribution actually. I just want to show you quickly just a bit of code. No, that's the other one, sorry. Just to show you, to give you a sense of the MXNet API. Here I'm building a model. It's a simple multi-layer perceptron. I can define each of those lines here. It's basically defining one layer at a time. Programmatically, I'm able to define my network. Here's another different type of network I could define. Here I'm loading my data. Actually, there's a pretty object for that dataset. I can load my training dataset, I can load my validation dataset, etc. It's not a ton of code. Let's do the training. You will find some Scala tutorials on the MXNet website. I pre-built all that thing and here's how I'm going to launch it. I've got my dataset set up here. That's my dataset here, and I'm going to use that simple multi-layer perceptron network and I'm going to run that training on CPU because it's not a very large dataset and let's see what happens. Okay. Alright, so here I'm running on CPU. It's still quite fast. You can start an EMR cluster with GPU instances, use Spark for ETL, and then immediately start training. This eliminates the need for separate infrastructure. It's free to try, and the basic use cases should work. If you want to contribute, you're more than welcome. This is a quick overview of the Amazon AI stack, from high-level AI services to platforms and deep learning with open-source libraries, running on CPU and GPU instances. Our annual conference, re:Invent, is coming up soon, where we make many announcements. Keep an eye on the re:Invent announcements for more updates. Good resources include the AI, machine learning, and EMR web pages, as well as dedicated blogs for big data and AI. We have SDKs for Java and Scala on GitHub, and the code for all of this is available there. Thank you. If you want to follow me on Twitter for more AI and machine learning content, please do. I'm happy to answer any questions you have. **Q: For a support use case with many questions, should I choose machine learning or deep learning?** A: I would choose a combination of both. Use Amazon Lex to define the conversation flow. Lex simplifies the process with no infrastructure to manage. During the Lex conversation, you can call Lambda functions to invoke machine learning models. For example, you can classify user questions (network problem, email problem, etc.) and point users to relevant answers. Build a dataset from user sentences, train a model to classify them, and call this model from a Lambda function during the Lex conversation. This approach should work well. **Q: What does intuition mean in the context of artificial intelligence?** A: In AI, intuition can refer to unsupervised learning, where the model learns patterns without labeled data. For example, clustering algorithms group similar data points without prior training. When you query the model with a new sample, it predicts the cluster. This is akin to intuition because it's not based on learning from human-defined labels but on mathematical and statistical methods. The model provides answers that seem intuitive to us, even though the process is algorithmic. No more questions? If you have any later, feel free to ping me on Twitter. Thank you.