.. image:: https://github.com/aws/sagemaker-python-sdk/raw/master/branding/icon/sagemaker-banner.png
:height: 100px
:alt: SageMaker
====================
SageMaker Python SDK
====================
.. image:: https://img.shields.io/pypi/v/sagemaker.svg
:target: https://pypi.python.org/pypi/sagemaker
:alt: Latest Version
.. image:: https://img.shields.io/pypi/pyversions/sagemaker.svg
:target: https://pypi.python.org/pypi/sagemaker
:alt: Supported Python Versions
.. image:: https://img.shields.io/badge/code_style-black-000000.svg
:target: https://github.com/python/black
:alt: Code style: black
.. image:: https://readthedocs.org/projects/sagemaker/badge/?version=stable
:target: https://sagemaker.readthedocs.io/en/stable/
:alt: Documentation Status
.. image:: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml/badge.svg
:target: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml
:alt: CI Health
SageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker.
With the SDK, you can train and deploy models using popular deep learning frameworks **Apache MXNet** and **TensorFlow**.
You can also train and deploy models with **Amazon algorithms**,
which are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training.
If you have **your own algorithms** built into SageMaker compatible Docker containers, you can train and host models using these as well.
For detailed documentation, including the API reference, see `Read the Docs <https://sagemaker.readthedocs.io>`_.
Table of Contents
-----------------
#. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__
#. `Using the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html>`__
#. `Using MXNet <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html>`__
#. `Using TensorFlow <https://sagemaker.readthedocs.io/en/stable/using_tf.html>`__
#. `Using Chainer <https://sagemaker.readthedocs.io/en/stable/using_chainer.html>`__
#. `Using PyTorch <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html>`__
#. `Using Scikit-learn <https://sagemaker.readthedocs.io/en/stable/using_sklearn.html>`__
#. `Using XGBoost <https://sagemaker.readthedocs.io/en/stable/using_xgboost.html>`__
#. `SageMaker Reinforcement Learning Estimators <https://sagemaker.readthedocs.io/en/stable/using_rl.html>`__
#. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__
#. `Amazon SageMaker Built-in Algorithm Estimators <src/sagemaker/amazon/README.rst>`__
#. `Using SageMaker AlgorithmEstimators <https://sagemaker.readthedocs.io/en/stable/overview.html#using-sagemaker-algorithmestimators>`__
#. `Consuming SageMaker Model Packages <https://sagemaker.readthedocs.io/en/stable/overview.html#consuming-sagemaker-model-packages>`__
#. `BYO Docker Containers with SageMaker Estimators <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-docker-containers-with-sagemaker-estimators>`__
#. `SageMaker Automatic Model Tuning <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-automatic-model-tuning>`__
#. `SageMaker Batch Transform <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-batch-transform>`__
#. `Secure Training and Inference with VPC <https://sagemaker.readthedocs.io/en/stable/overview.html#secure-training-and-inference-with-vpc>`__
#. `BYO Model <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-model>`__
#. `Inference Pipelines <https://sagemaker.readthedocs.io/en/stable/overview.html#inference-pipelines>`__
#. `Amazon SageMaker Operators in Apache Airflow <https://sagemaker.readthedocs.io/en/stable/using_workflow.html>`__
#. `SageMaker Autopilot <src/sagemaker/automl/README.rst>`__
#. `Model Monitoring <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html>`__
#. `SageMaker Debugger <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html>`__
#. `SageMaker Processing <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html>`__
Installing the SageMaker Python SDK
-----------------------------------
The SageMaker Python SDK is built to PyPI and the latest version of the SageMaker Python SDK can be installed with pip as follows
::
pip install sagemaker==<Latest version from pyPI from https://pypi.org/project/sagemaker/>
You can install from source by cloning this repository and running a pip install command in the root directory of the repository:
::
git clone https://github.com/aws/sagemaker-python-sdk.git
cd sagemaker-python-sdk
pip install .
Supported Operating Systems
~~~~~~~~~~~~~~~~~~~~~~~~~~~
SageMaker Python SDK supports Unix/Linux and Mac.
Supported Python Versions
~~~~~~~~~~~~~~~~~~~~~~~~~
SageMaker Python SDK is tested on:
- Python 3.8
- Python 3.9
- Python 3.10
- Python 3.11
AWS Permissions
~~~~~~~~~~~~~~~
As a managed service, Amazon SageMaker performs operations on your behalf on the AWS hardware that is managed by Amazon SageMaker.
Amazon SageMaker can perform only operations that the user permits.
You can read more about which permissions are necessary in the `AWS Documentation <https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html>`__.
The SageMaker Python SDK should not require any additional permissions aside from what is required for using SageMaker.
However, if you are using an IAM role with a path in it, you should grant permission for ``iam:GetRole``.
Licensing
~~~~~~~~~
SageMaker Python SDK is licensed under the Apache 2.0 License. It is copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at:
http://aws.amazon.com/apache2.0/
Running tests
~~~~~~~~~~~~~
SageMaker Python SDK has unit tests and integration tests.
You can install the libraries needed to run the tests by running :code:`pip install --upgrade .[test]` or, for Zsh users: :code:`pip install --upgrade .\[test\]`
**Unit tests**
We run unit tests with tox, which is a program that lets you run unit tests for multiple Python versions, and also make sure the
code fits our style guidelines. We run tox with `all of our supported Python versions <#supported-python-versions>`_, so to run unit tests
with the same configuration we do, you need to have interpreters for those Python versions installed.
To run the unit tests with tox, run:
::
tox tests/unit
**Integration tests**
To run the integration tests, the following prerequisites must be met
1. AWS account credentials are available in the environment for the boto3 client to use.
2. The AWS account has an IAM role named :code:`SageMakerRole`.
It should have the AmazonSageMakerFullAccess policy attached as well as a policy with `the necessary permissions to use Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei-setup.html>`__.
3. To run remote_function tests, dummy ecr repo should be created. It can be created by running -
:code:`aws ecr create-repository --repository-name remote-function-dummy-container`
We recommend selectively running just those integration tests you'd like to run. You can filter by individual test function names with:
::
tox -- -k 'test_i_care_about'
You can also run all of the integration tests by running the following command, which runs them in sequence, which may take a while:
::
tox -- tests/integ
You can also run them in parallel:
::
tox -- -n auto tests/integ
Git Hooks
~~~~~~~~~
to enable all git hooks in the .githooks directory, run these commands in the repository directory:
::
find .git/hooks -type l -exec rm {} \;
find .githooks -type f -exec ln -sf ../../{} .git/hooks/ \;
To enable an individual git hook, simply move it from the .githooks/ directory to the .git/hooks/ directory.
Building Sphinx docs
~~~~~~~~~~~~~~~~~~~~
Setup a Python environment, and install the dependencies listed in ``doc/requirements.txt``:
::
# conda
conda create -n sagemaker python=3.7
conda activate sagemaker
conda install sphinx=3.1.1 sphinx_rtd_theme=0.5.0
# pip
pip install -r doc/requirements.txt
Clone/fork the repo, and install your local version:
::
pip install --upgrade .
Then ``cd`` into the ``sagemaker-python-sdk/doc`` directory and run:
::
make html
You can edit the templates for any of the pages in the docs by editing the .rst files in the ``doc`` directory and then running ``make html`` again.
Preview the site with a Python web server:
::
cd _build/html
python -m http.server 8000
View the website by visiting http://localhost:8000
SageMaker SparkML Serving
-------------------------
With SageMaker SparkML Serving, you can now perform predictions against a SparkML Model in SageMaker.
In order to host a SparkML model in SageMaker, it should be serialized with ``MLeap`` library.
For more information on MLeap, see https://github.com/combust/mleap .
Supported major version of Spark: 3.3 (MLeap version - 0.20.0)
Here is an example on how to create an instance of ``SparkMLModel`` class and use ``deploy()`` method to create an
endpoint which can be used to perform prediction against your trained SparkML Model.
.. code:: python
sparkml_model = SparkMLModel(model_data='s3://path/to/model.tar.gz', env={'SAGEMAKER_SPARKML_SCHEMA': schema})
model_name = 'sparkml-model'
endpoint_name = 'sparkml-endpoint'
predictor = sparkml_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', endpoint_name=endpoint_name)
Once the model is deployed, we can invoke the endpoint with a ``CSV`` payload like this:
.. code:: python
payload = 'field_1,field_2,field_3,field_4,field_5'
predictor.predict(payload)
For more information about the different ``content-type`` and ``Accept`` formats as well as the structure of the
``schema`` that SageMaker SparkML Serving recognizes, please see `SageMaker SparkML Serving Container`_.
.. _SageMaker SparkML Serving Container: https://github.com/aws/sagemaker-sparkml-serving-container
Raw data
{
"_id": null,
"home_page": "https://github.com/aws/sagemaker-python-sdk/",
"name": "sagemaker",
"maintainer": null,
"docs_url": null,
"requires_python": ">=3.8",
"maintainer_email": null,
"keywords": "ML Amazon AWS AI Tensorflow MXNet",
"author": "Amazon Web Services",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/a9/25/645df8b617585bc1981705ef45b1018b7ff27b1ceacb283756f1a152429c/sagemaker-2.217.0.tar.gz",
"platform": null,
"description": ".. image:: https://github.com/aws/sagemaker-python-sdk/raw/master/branding/icon/sagemaker-banner.png\n :height: 100px\n :alt: SageMaker\n\n====================\nSageMaker Python SDK\n====================\n\n.. image:: https://img.shields.io/pypi/v/sagemaker.svg\n :target: https://pypi.python.org/pypi/sagemaker\n :alt: Latest Version\n\n.. image:: https://img.shields.io/pypi/pyversions/sagemaker.svg\n :target: https://pypi.python.org/pypi/sagemaker\n :alt: Supported Python Versions\n\n.. image:: https://img.shields.io/badge/code_style-black-000000.svg\n :target: https://github.com/python/black\n :alt: Code style: black\n\n.. image:: https://readthedocs.org/projects/sagemaker/badge/?version=stable\n :target: https://sagemaker.readthedocs.io/en/stable/\n :alt: Documentation Status\n\n.. image:: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml/badge.svg\n :target: https://github.com/aws/sagemaker-python-sdk/actions/workflows/codebuild-ci-health.yml\n :alt: CI Health\n\nSageMaker Python SDK is an open source library for training and deploying machine learning models on Amazon SageMaker.\n\nWith the SDK, you can train and deploy models using popular deep learning frameworks **Apache MXNet** and **TensorFlow**.\nYou can also train and deploy models with **Amazon algorithms**,\nwhich are scalable implementations of core machine learning algorithms that are optimized for SageMaker and GPU training.\nIf you have **your own algorithms** built into SageMaker compatible Docker containers, you can train and host models using these as well.\n\nFor detailed documentation, including the API reference, see `Read the Docs <https://sagemaker.readthedocs.io>`_.\n\nTable of Contents\n-----------------\n\n#. `Installing SageMaker Python SDK <#installing-the-sagemaker-python-sdk>`__\n#. `Using the SageMaker Python SDK <https://sagemaker.readthedocs.io/en/stable/overview.html>`__\n#. `Using MXNet <https://sagemaker.readthedocs.io/en/stable/using_mxnet.html>`__\n#. `Using TensorFlow <https://sagemaker.readthedocs.io/en/stable/using_tf.html>`__\n#. `Using Chainer <https://sagemaker.readthedocs.io/en/stable/using_chainer.html>`__\n#. `Using PyTorch <https://sagemaker.readthedocs.io/en/stable/using_pytorch.html>`__\n#. `Using Scikit-learn <https://sagemaker.readthedocs.io/en/stable/using_sklearn.html>`__\n#. `Using XGBoost <https://sagemaker.readthedocs.io/en/stable/using_xgboost.html>`__\n#. `SageMaker Reinforcement Learning Estimators <https://sagemaker.readthedocs.io/en/stable/using_rl.html>`__\n#. `SageMaker SparkML Serving <#sagemaker-sparkml-serving>`__\n#. `Amazon SageMaker Built-in Algorithm Estimators <src/sagemaker/amazon/README.rst>`__\n#. `Using SageMaker AlgorithmEstimators <https://sagemaker.readthedocs.io/en/stable/overview.html#using-sagemaker-algorithmestimators>`__\n#. `Consuming SageMaker Model Packages <https://sagemaker.readthedocs.io/en/stable/overview.html#consuming-sagemaker-model-packages>`__\n#. `BYO Docker Containers with SageMaker Estimators <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-docker-containers-with-sagemaker-estimators>`__\n#. `SageMaker Automatic Model Tuning <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-automatic-model-tuning>`__\n#. `SageMaker Batch Transform <https://sagemaker.readthedocs.io/en/stable/overview.html#sagemaker-batch-transform>`__\n#. `Secure Training and Inference with VPC <https://sagemaker.readthedocs.io/en/stable/overview.html#secure-training-and-inference-with-vpc>`__\n#. `BYO Model <https://sagemaker.readthedocs.io/en/stable/overview.html#byo-model>`__\n#. `Inference Pipelines <https://sagemaker.readthedocs.io/en/stable/overview.html#inference-pipelines>`__\n#. `Amazon SageMaker Operators in Apache Airflow <https://sagemaker.readthedocs.io/en/stable/using_workflow.html>`__\n#. `SageMaker Autopilot <src/sagemaker/automl/README.rst>`__\n#. `Model Monitoring <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_model_monitoring.html>`__\n#. `SageMaker Debugger <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_debugger.html>`__\n#. `SageMaker Processing <https://sagemaker.readthedocs.io/en/stable/amazon_sagemaker_processing.html>`__\n\n\nInstalling the SageMaker Python SDK\n-----------------------------------\n\nThe SageMaker Python SDK is built to PyPI and the latest version of the SageMaker Python SDK can be installed with pip as follows\n::\n\n pip install sagemaker==<Latest version from pyPI from https://pypi.org/project/sagemaker/>\n\nYou can install from source by cloning this repository and running a pip install command in the root directory of the repository:\n\n::\n\n git clone https://github.com/aws/sagemaker-python-sdk.git\n cd sagemaker-python-sdk\n pip install .\n\nSupported Operating Systems\n~~~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSageMaker Python SDK supports Unix/Linux and Mac.\n\nSupported Python Versions\n~~~~~~~~~~~~~~~~~~~~~~~~~\n\nSageMaker Python SDK is tested on:\n\n- Python 3.8\n- Python 3.9\n- Python 3.10\n- Python 3.11\n\nAWS Permissions\n~~~~~~~~~~~~~~~\n\nAs a managed service, Amazon SageMaker performs operations on your behalf on the AWS hardware that is managed by Amazon SageMaker.\nAmazon SageMaker can perform only operations that the user permits.\nYou can read more about which permissions are necessary in the `AWS Documentation <https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-roles.html>`__.\n\nThe SageMaker Python SDK should not require any additional permissions aside from what is required for using SageMaker.\nHowever, if you are using an IAM role with a path in it, you should grant permission for ``iam:GetRole``.\n\nLicensing\n~~~~~~~~~\nSageMaker Python SDK is licensed under the Apache 2.0 License. It is copyright Amazon.com, Inc. or its affiliates. All Rights Reserved. The license is available at:\nhttp://aws.amazon.com/apache2.0/\n\nRunning tests\n~~~~~~~~~~~~~\n\nSageMaker Python SDK has unit tests and integration tests.\n\nYou can install the libraries needed to run the tests by running :code:`pip install --upgrade .[test]` or, for Zsh users: :code:`pip install --upgrade .\\[test\\]`\n\n**Unit tests**\n\nWe run unit tests with tox, which is a program that lets you run unit tests for multiple Python versions, and also make sure the\ncode fits our style guidelines. We run tox with `all of our supported Python versions <#supported-python-versions>`_, so to run unit tests\nwith the same configuration we do, you need to have interpreters for those Python versions installed.\n\nTo run the unit tests with tox, run:\n\n::\n\n tox tests/unit\n\n**Integration tests**\n\nTo run the integration tests, the following prerequisites must be met\n\n1. AWS account credentials are available in the environment for the boto3 client to use.\n2. The AWS account has an IAM role named :code:`SageMakerRole`.\n It should have the AmazonSageMakerFullAccess policy attached as well as a policy with `the necessary permissions to use Elastic Inference <https://docs.aws.amazon.com/sagemaker/latest/dg/ei-setup.html>`__.\n3. To run remote_function tests, dummy ecr repo should be created. It can be created by running -\n :code:`aws ecr create-repository --repository-name remote-function-dummy-container`\n\nWe recommend selectively running just those integration tests you'd like to run. You can filter by individual test function names with:\n\n::\n\n tox -- -k 'test_i_care_about'\n\n\nYou can also run all of the integration tests by running the following command, which runs them in sequence, which may take a while:\n\n::\n\n tox -- tests/integ\n\n\nYou can also run them in parallel:\n\n::\n\n tox -- -n auto tests/integ\n\n\nGit Hooks\n~~~~~~~~~\n\nto enable all git hooks in the .githooks directory, run these commands in the repository directory:\n\n::\n\n find .git/hooks -type l -exec rm {} \\;\n find .githooks -type f -exec ln -sf ../../{} .git/hooks/ \\;\n\nTo enable an individual git hook, simply move it from the .githooks/ directory to the .git/hooks/ directory.\n\nBuilding Sphinx docs\n~~~~~~~~~~~~~~~~~~~~\n\nSetup a Python environment, and install the dependencies listed in ``doc/requirements.txt``:\n\n::\n\n # conda\n conda create -n sagemaker python=3.7\n conda activate sagemaker\n conda install sphinx=3.1.1 sphinx_rtd_theme=0.5.0\n\n # pip\n pip install -r doc/requirements.txt\n\n\nClone/fork the repo, and install your local version:\n\n::\n\n pip install --upgrade .\n\nThen ``cd`` into the ``sagemaker-python-sdk/doc`` directory and run:\n\n::\n\n make html\n\nYou can edit the templates for any of the pages in the docs by editing the .rst files in the ``doc`` directory and then running ``make html`` again.\n\nPreview the site with a Python web server:\n\n::\n\n cd _build/html\n python -m http.server 8000\n\nView the website by visiting http://localhost:8000\n\nSageMaker SparkML Serving\n-------------------------\n\nWith SageMaker SparkML Serving, you can now perform predictions against a SparkML Model in SageMaker.\nIn order to host a SparkML model in SageMaker, it should be serialized with ``MLeap`` library.\n\nFor more information on MLeap, see https://github.com/combust/mleap .\n\nSupported major version of Spark: 3.3 (MLeap version - 0.20.0)\n\nHere is an example on how to create an instance of ``SparkMLModel`` class and use ``deploy()`` method to create an\nendpoint which can be used to perform prediction against your trained SparkML Model.\n\n.. code:: python\n\n sparkml_model = SparkMLModel(model_data='s3://path/to/model.tar.gz', env={'SAGEMAKER_SPARKML_SCHEMA': schema})\n model_name = 'sparkml-model'\n endpoint_name = 'sparkml-endpoint'\n predictor = sparkml_model.deploy(initial_instance_count=1, instance_type='ml.c4.xlarge', endpoint_name=endpoint_name)\n\nOnce the model is deployed, we can invoke the endpoint with a ``CSV`` payload like this:\n\n.. code:: python\n\n payload = 'field_1,field_2,field_3,field_4,field_5'\n predictor.predict(payload)\n\n\nFor more information about the different ``content-type`` and ``Accept`` formats as well as the structure of the\n``schema`` that SageMaker SparkML Serving recognizes, please see `SageMaker SparkML Serving Container`_.\n\n.. _SageMaker SparkML Serving Container: https://github.com/aws/sagemaker-sparkml-serving-container\n\n\n",
"bugtrack_url": null,
"license": "Apache License 2.0",
"summary": "Open source library for training and deploying models on Amazon SageMaker.",
"version": "2.217.0",
"project_urls": {
"Homepage": "https://github.com/aws/sagemaker-python-sdk/"
},
"split_keywords": [
"ml",
"amazon",
"aws",
"ai",
"tensorflow",
"mxnet"
],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "08d7f25c289ac5300f4fdf95861a0711b4d31dcf3176e84fd9a2ca203032e1cf",
"md5": "415047c41cdcc6722353de875d59c30d",
"sha256": "2a6e8bafda1b3756f1a2f852e45ac14091d3737f1ea2626aecef81b61c241b03"
},
"downloads": -1,
"filename": "sagemaker-2.217.0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "415047c41cdcc6722353de875d59c30d",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": ">=3.8",
"size": 1458498,
"upload_time": "2024-04-24T21:35:10",
"upload_time_iso_8601": "2024-04-24T21:35:10.519397Z",
"url": "https://files.pythonhosted.org/packages/08/d7/f25c289ac5300f4fdf95861a0711b4d31dcf3176e84fd9a2ca203032e1cf/sagemaker-2.217.0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "a925645df8b617585bc1981705ef45b1018b7ff27b1ceacb283756f1a152429c",
"md5": "2eec488368a2e883cd0ca294db236c36",
"sha256": "d80736ed1c64551082b148fd421e010120baefa8780c5389cc7a132b0666e5aa"
},
"downloads": -1,
"filename": "sagemaker-2.217.0.tar.gz",
"has_sig": false,
"md5_digest": "2eec488368a2e883cd0ca294db236c36",
"packagetype": "sdist",
"python_version": "source",
"requires_python": ">=3.8",
"size": 1096528,
"upload_time": "2024-04-24T21:35:17",
"upload_time_iso_8601": "2024-04-24T21:35:17.990574Z",
"url": "https://files.pythonhosted.org/packages/a9/25/645df8b617585bc1981705ef45b1018b7ff27b1ceacb283756f1a152429c/sagemaker-2.217.0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-04-24 21:35:17",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aws",
"github_project": "sagemaker-python-sdk",
"travis_ci": false,
"coveralls": true,
"github_actions": true,
"tox": true,
"lcname": "sagemaker"
}
JSON
