AWS re:Invent 2019 — AI/ML recap — Part 2: Amazon SageMaker
In a previous post, I introduced you to our new high-level services. Now let’s go down one layer and talk about the new capabilities added to Amazon SageMaker: SageMaker Processing, SageMaker Experiments, SageMaker AutoPilot, SageMaker Debugger, SageMaker Model Monitor, SageMaker Notebooks, SageMaker Studio, and SageMaker Operators for Kubernetes.
If you’re just looking for an overview of the new capabilities, this is the one.
If you want the full enchilada, keep reading. I’ll share learning resources along the way.
As always, happy to answer questions here or on Twitter.
Here we go!
Amazon SageMaker Processing
Amazon SageMaker Processing lets you easily run your preprocessing, postprocessing and model evaluation workloads on fully managed infrastructure. At launch, you can use either Scikit-learn or Spark.
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/processing-job.html
Examples: https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker_processing
Amazon SageMaker Experiments
Amazon SageMaker Experiments is a capability of Amazon SageMaker that lets you organize, track, compare, and evaluate your machine learning experiments. It’s nicely integrated with hyperparameter tuning and SageMaker Autopilot: no code needed!
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/experiments.html
Examples:
- AIM361 workshop at re:Invent2019: https://gitlab.com/juliensimon/aim361/
Amazon SageMaker AutoPilot
Amazon SageMaker AutoPilot Amazon lets you automatically create the best classification and regression machine learning models, while allowing full control and visibility.
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-automate-model-development.html
Examples:
- https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker_processing
- AIM361 workshop at re:Invent2019: https://gitlab.com/juliensimon/aim361/blob/master/Lab2.ipynb
Here’s a gentle introduction.
I also recorded a complete 4-part demo using Amazon SageMaker Studio.
Amazon SageMaker Debugger
Amazon SageMaker Debugger provides full visibility into the training of machine learning models by monitoring, recording, and analyzing the tensor data that captures the state of a machine learning training job.
Blog: https://aws.amazon.com/blogs/aws/amazon-sagemaker-debugger-debug-your-machine-learning-models/
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/train-debugger.html
Examples: https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker-debugger
Here’s an example with Keras.
Amazon SageMaker Model Monitor
Amazon SageMaker Model Monitor automatically monitors ML models in production and notifies you when data quality issues arise.
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/model-monitor.html
Examples: https://github.com/awslabs/amazon-sagemaker-examples/tree/master/sagemaker_model_monitor
Amazon SageMaker Studio and Amazon SageMaker Notebooks (preview)
Amazon SageMaker Studio is the first fully integrated development environment (IDE) for machine learning (ML). It includes Amazon SageMaker Notebooks, one-click Jupyter notebooks that you can start working with in seconds
Documentation: https://docs.aws.amazon.com/sagemaker/latest/dg/notebooks.html
Amazon SageMaker Operators for Kubernetes
Amazon SageMaker Operators for Kubernetes makes it easier for developers and data scientists using Kubernetes to train, tune, and deploy ML models in Amazon SageMaker.
Blog: https://aws.amazon.com/blogs/machine-learning/introducing-amazon-sagemaker-operators-for-kubernetes/
Documentation: https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_operators_for_kubernetes.html
Examples: https://github.com/aws/amazon-sagemaker-operator-for-k8s