# Amazon SageMaker Construct Library
<!--BEGIN STABILITY BANNER-->---
> The APIs of higher level constructs in this module are experimental and under active development.
> They are subject to non-backward compatible changes or removal in any future version. These are
> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be
> announced in the release notes. This means that while you may use them, you may need to update
> your source code when upgrading to a newer version of this package.
---
<!--END STABILITY BANNER-->
Amazon SageMaker provides every developer and data scientist with the ability to build, train, and
deploy machine learning models quickly. Amazon SageMaker is a fully-managed service that covers the
entire machine learning workflow to label and prepare your data, choose an algorithm, train the
model, tune and optimize it for deployment, make predictions, and take action. Your models get to
production faster with much less effort and lower cost.
## Model
To create a machine learning model with Amazon Sagemaker, use the `Model` construct. This construct
includes properties that can be configured to define model components, including the model inference
code as a Docker image and an optional set of separate model data artifacts. See the [AWS
documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-marketplace-develop.html)
to learn more about SageMaker models.
### Single Container Model
In the event that a single container is sufficient for your inference use-case, you can define a
single-container model:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
import path as path
image = sagemaker.ContainerImage.from_asset(path.join("path", "to", "Dockerfile", "directory"))
model_data = sagemaker.ModelData.from_asset(path.join("path", "to", "artifact", "file.tar.gz"))
model = sagemaker.Model(self, "PrimaryContainerModel",
containers=[sagemaker.ContainerDefinition(
image=image,
model_data=model_data
)
]
)
```
### Inference Pipeline Model
An inference pipeline is an Amazon SageMaker model that is composed of a linear sequence of multiple
containers that process requests for inferences on data. See the [AWS
documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-pipelines.html) to learn
more about SageMaker inference pipelines. To define an inference pipeline, you can provide
additional containers for your model:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# image1: sagemaker.ContainerImage
# model_data1: sagemaker.ModelData
# image2: sagemaker.ContainerImage
# model_data2: sagemaker.ModelData
# image3: sagemaker.ContainerImage
# model_data3: sagemaker.ModelData
model = sagemaker.Model(self, "InferencePipelineModel",
containers=[sagemaker.ContainerDefinition(image=image1, model_data=model_data1), sagemaker.ContainerDefinition(image=image2, model_data=model_data2), sagemaker.ContainerDefinition(image=image3, model_data=model_data3)
]
)
```
### Model Properties
#### Network Isolation
If you enable [network isolation](https://docs.aws.amazon.com/sagemaker/latest/dg/mkt-algo-model-internet-free.html), the containers can't make any outbound network calls, even to other AWS services such as Amazon S3. Additionally, no AWS credentials are made available to the container runtime environment.
To enable network isolation, set the `networkIsolation` property to `true`:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# image: sagemaker.ContainerImage
# model_data: sagemaker.ModelData
model = sagemaker.Model(self, "ContainerModel",
containers=[sagemaker.ContainerDefinition(
image=image,
model_data=model_data
)
],
network_isolation=True
)
```
### Container Images
Inference code can be stored in the Amazon EC2 Container Registry (Amazon ECR), which is specified
via `ContainerDefinition`'s `image` property which accepts a class that extends the `ContainerImage`
abstract base class.
#### Asset Image
Reference a local directory containing a Dockerfile:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
import path as path
image = sagemaker.ContainerImage.from_asset(path.join("path", "to", "Dockerfile", "directory"))
```
#### ECR Image
Reference an image available within ECR:
```python
import aws_cdk.aws_ecr as ecr
import aws_cdk.aws_sagemaker_alpha as sagemaker
repository = ecr.Repository.from_repository_name(self, "Repository", "repo")
image = sagemaker.ContainerImage.from_ecr_repository(repository, "tag")
```
#### DLC Image
Reference a deep learning container image:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
repository_name = "huggingface-pytorch-training"
tag = "1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04"
image = sagemaker.ContainerImage.from_dlc(repository_name, tag)
```
### Model Artifacts
If you choose to decouple your model artifacts from your inference code (as is natural given
different rates of change between inference code and model artifacts), the artifacts can be
specified via the `modelData` property which accepts a class that extends the `ModelData` abstract
base class. The default is to have no model artifacts associated with a model.
#### Asset Model Data
Reference local model data:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
import path as path
model_data = sagemaker.ModelData.from_asset(path.join("path", "to", "artifact", "file.tar.gz"))
```
#### S3 Model Data
Reference an S3 bucket and object key as the artifacts for a model:
```python
import aws_cdk.aws_s3 as s3
import aws_cdk.aws_sagemaker_alpha as sagemaker
bucket = s3.Bucket(self, "MyBucket")
model_data = sagemaker.ModelData.from_bucket(bucket, "path/to/artifact/file.tar.gz")
```
## Model Hosting
Amazon SageMaker provides model hosting services for model deployment. Amazon SageMaker provides an
HTTPS endpoint where your machine learning model is available to provide inferences.
### Endpoint Configuration
By using the `EndpointConfig` construct, you can define a set of endpoint configuration which can be
used to provision one or more endpoints. In this configuration, you identify one or more models to
deploy and the resources that you want Amazon SageMaker to provision. You define one or more
production variants, each of which identifies a model. Each production variant also describes the
resources that you want Amazon SageMaker to provision. If you are hosting multiple models, you also
assign a variant weight to specify how much traffic you want to allocate to each model. For example,
suppose that you want to host two models, A and B, and you assign traffic weight 2 for model A and 1
for model B. Amazon SageMaker distributes two-thirds of the traffic to Model A, and one-third to
model B:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# model_a: sagemaker.Model
# model_b: sagemaker.Model
endpoint_config = sagemaker.EndpointConfig(self, "EndpointConfig",
instance_production_variants=[sagemaker.InstanceProductionVariantProps(
model=model_a,
variant_name="modelA",
initial_variant_weight=2
), sagemaker.InstanceProductionVariantProps(
model=model_b,
variant_name="variantB",
initial_variant_weight=1
)
]
)
```
### Endpoint
When you create an endpoint from an `EndpointConfig`, Amazon SageMaker launches the ML compute
instances and deploys the model or models as specified in the configuration. To get inferences from
the model, client applications send requests to the Amazon SageMaker Runtime HTTPS endpoint. For
more information about the API, see the
[InvokeEndpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/API_runtime_InvokeEndpoint.html)
API. Defining an endpoint requires at minimum the associated endpoint configuration:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# endpoint_config: sagemaker.EndpointConfig
endpoint = sagemaker.Endpoint(self, "Endpoint", endpoint_config=endpoint_config)
```
### AutoScaling
To enable autoscaling on the production variant, use the `autoScaleInstanceCount` method:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# model: sagemaker.Model
variant_name = "my-variant"
endpoint_config = sagemaker.EndpointConfig(self, "EndpointConfig",
instance_production_variants=[sagemaker.InstanceProductionVariantProps(
model=model,
variant_name=variant_name
)
]
)
endpoint = sagemaker.Endpoint(self, "Endpoint", endpoint_config=endpoint_config)
production_variant = endpoint.find_instance_production_variant(variant_name)
instance_count = production_variant.auto_scale_instance_count(
max_capacity=3
)
instance_count.scale_on_invocations("LimitRPS",
max_requests_per_second=30
)
```
For load testing guidance on determining the maximum requests per second per instance, please see
this [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-scaling-loadtest.html).
### Metrics
To monitor CloudWatch metrics for a production variant, use one or more of the metric convenience
methods:
```python
import aws_cdk.aws_sagemaker_alpha as sagemaker
# endpoint_config: sagemaker.EndpointConfig
endpoint = sagemaker.Endpoint(self, "Endpoint", endpoint_config=endpoint_config)
production_variant = endpoint.find_instance_production_variant("my-variant")
production_variant.metric_model_latency().create_alarm(self, "ModelLatencyAlarm",
threshold=100000,
evaluation_periods=3
)
```
Raw data
{
"_id": null,
"home_page": "https://github.com/aws/aws-cdk",
"name": "aws-cdk.aws-sagemaker-alpha",
"maintainer": null,
"docs_url": null,
"requires_python": "~=3.8",
"maintainer_email": null,
"keywords": null,
"author": "Amazon Web Services",
"author_email": null,
"download_url": "https://files.pythonhosted.org/packages/07/22/6c7c0058de8e312aec58e541c154b7f2fcdbbc8b811b4f3d977f635779ef/aws_cdk_aws_sagemaker_alpha-2.170.0a0.tar.gz",
"platform": null,
"description": "# Amazon SageMaker Construct Library\n\n<!--BEGIN STABILITY BANNER-->---\n\n\n\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n<!--END STABILITY BANNER-->\n\nAmazon SageMaker provides every developer and data scientist with the ability to build, train, and\ndeploy machine learning models quickly. Amazon SageMaker is a fully-managed service that covers the\nentire machine learning workflow to label and prepare your data, choose an algorithm, train the\nmodel, tune and optimize it for deployment, make predictions, and take action. Your models get to\nproduction faster with much less effort and lower cost.\n\n## Model\n\nTo create a machine learning model with Amazon Sagemaker, use the `Model` construct. This construct\nincludes properties that can be configured to define model components, including the model inference\ncode as a Docker image and an optional set of separate model data artifacts. See the [AWS\ndocumentation](https://docs.aws.amazon.com/sagemaker/latest/dg/sagemaker-marketplace-develop.html)\nto learn more about SageMaker models.\n\n### Single Container Model\n\nIn the event that a single container is sufficient for your inference use-case, you can define a\nsingle-container model:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\nimport path as path\n\n\nimage = sagemaker.ContainerImage.from_asset(path.join(\"path\", \"to\", \"Dockerfile\", \"directory\"))\nmodel_data = sagemaker.ModelData.from_asset(path.join(\"path\", \"to\", \"artifact\", \"file.tar.gz\"))\n\nmodel = sagemaker.Model(self, \"PrimaryContainerModel\",\n containers=[sagemaker.ContainerDefinition(\n image=image,\n model_data=model_data\n )\n ]\n)\n```\n\n### Inference Pipeline Model\n\nAn inference pipeline is an Amazon SageMaker model that is composed of a linear sequence of multiple\ncontainers that process requests for inferences on data. See the [AWS\ndocumentation](https://docs.aws.amazon.com/sagemaker/latest/dg/inference-pipelines.html) to learn\nmore about SageMaker inference pipelines. To define an inference pipeline, you can provide\nadditional containers for your model:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# image1: sagemaker.ContainerImage\n# model_data1: sagemaker.ModelData\n# image2: sagemaker.ContainerImage\n# model_data2: sagemaker.ModelData\n# image3: sagemaker.ContainerImage\n# model_data3: sagemaker.ModelData\n\n\nmodel = sagemaker.Model(self, \"InferencePipelineModel\",\n containers=[sagemaker.ContainerDefinition(image=image1, model_data=model_data1), sagemaker.ContainerDefinition(image=image2, model_data=model_data2), sagemaker.ContainerDefinition(image=image3, model_data=model_data3)\n ]\n)\n```\n\n### Model Properties\n\n#### Network Isolation\n\nIf you enable [network isolation](https://docs.aws.amazon.com/sagemaker/latest/dg/mkt-algo-model-internet-free.html), the containers can't make any outbound network calls, even to other AWS services such as Amazon S3. Additionally, no AWS credentials are made available to the container runtime environment.\n\nTo enable network isolation, set the `networkIsolation` property to `true`:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# image: sagemaker.ContainerImage\n# model_data: sagemaker.ModelData\n\n\nmodel = sagemaker.Model(self, \"ContainerModel\",\n containers=[sagemaker.ContainerDefinition(\n image=image,\n model_data=model_data\n )\n ],\n network_isolation=True\n)\n```\n\n### Container Images\n\nInference code can be stored in the Amazon EC2 Container Registry (Amazon ECR), which is specified\nvia `ContainerDefinition`'s `image` property which accepts a class that extends the `ContainerImage`\nabstract base class.\n\n#### Asset Image\n\nReference a local directory containing a Dockerfile:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\nimport path as path\n\n\nimage = sagemaker.ContainerImage.from_asset(path.join(\"path\", \"to\", \"Dockerfile\", \"directory\"))\n```\n\n#### ECR Image\n\nReference an image available within ECR:\n\n```python\nimport aws_cdk.aws_ecr as ecr\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n\nrepository = ecr.Repository.from_repository_name(self, \"Repository\", \"repo\")\nimage = sagemaker.ContainerImage.from_ecr_repository(repository, \"tag\")\n```\n\n#### DLC Image\n\nReference a deep learning container image:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n\nrepository_name = \"huggingface-pytorch-training\"\ntag = \"1.13.1-transformers4.26.0-gpu-py39-cu117-ubuntu20.04\"\n\nimage = sagemaker.ContainerImage.from_dlc(repository_name, tag)\n```\n\n### Model Artifacts\n\nIf you choose to decouple your model artifacts from your inference code (as is natural given\ndifferent rates of change between inference code and model artifacts), the artifacts can be\nspecified via the `modelData` property which accepts a class that extends the `ModelData` abstract\nbase class. The default is to have no model artifacts associated with a model.\n\n#### Asset Model Data\n\nReference local model data:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\nimport path as path\n\n\nmodel_data = sagemaker.ModelData.from_asset(path.join(\"path\", \"to\", \"artifact\", \"file.tar.gz\"))\n```\n\n#### S3 Model Data\n\nReference an S3 bucket and object key as the artifacts for a model:\n\n```python\nimport aws_cdk.aws_s3 as s3\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n\nbucket = s3.Bucket(self, \"MyBucket\")\nmodel_data = sagemaker.ModelData.from_bucket(bucket, \"path/to/artifact/file.tar.gz\")\n```\n\n## Model Hosting\n\nAmazon SageMaker provides model hosting services for model deployment. Amazon SageMaker provides an\nHTTPS endpoint where your machine learning model is available to provide inferences.\n\n### Endpoint Configuration\n\nBy using the `EndpointConfig` construct, you can define a set of endpoint configuration which can be\nused to provision one or more endpoints. In this configuration, you identify one or more models to\ndeploy and the resources that you want Amazon SageMaker to provision. You define one or more\nproduction variants, each of which identifies a model. Each production variant also describes the\nresources that you want Amazon SageMaker to provision. If you are hosting multiple models, you also\nassign a variant weight to specify how much traffic you want to allocate to each model. For example,\nsuppose that you want to host two models, A and B, and you assign traffic weight 2 for model A and 1\nfor model B. Amazon SageMaker distributes two-thirds of the traffic to Model A, and one-third to\nmodel B:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# model_a: sagemaker.Model\n# model_b: sagemaker.Model\n\n\nendpoint_config = sagemaker.EndpointConfig(self, \"EndpointConfig\",\n instance_production_variants=[sagemaker.InstanceProductionVariantProps(\n model=model_a,\n variant_name=\"modelA\",\n initial_variant_weight=2\n ), sagemaker.InstanceProductionVariantProps(\n model=model_b,\n variant_name=\"variantB\",\n initial_variant_weight=1\n )\n ]\n)\n```\n\n### Endpoint\n\nWhen you create an endpoint from an `EndpointConfig`, Amazon SageMaker launches the ML compute\ninstances and deploys the model or models as specified in the configuration. To get inferences from\nthe model, client applications send requests to the Amazon SageMaker Runtime HTTPS endpoint. For\nmore information about the API, see the\n[InvokeEndpoint](https://docs.aws.amazon.com/sagemaker/latest/dg/API_runtime_InvokeEndpoint.html)\nAPI. Defining an endpoint requires at minimum the associated endpoint configuration:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# endpoint_config: sagemaker.EndpointConfig\n\n\nendpoint = sagemaker.Endpoint(self, \"Endpoint\", endpoint_config=endpoint_config)\n```\n\n### AutoScaling\n\nTo enable autoscaling on the production variant, use the `autoScaleInstanceCount` method:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# model: sagemaker.Model\n\n\nvariant_name = \"my-variant\"\nendpoint_config = sagemaker.EndpointConfig(self, \"EndpointConfig\",\n instance_production_variants=[sagemaker.InstanceProductionVariantProps(\n model=model,\n variant_name=variant_name\n )\n ]\n)\n\nendpoint = sagemaker.Endpoint(self, \"Endpoint\", endpoint_config=endpoint_config)\nproduction_variant = endpoint.find_instance_production_variant(variant_name)\ninstance_count = production_variant.auto_scale_instance_count(\n max_capacity=3\n)\ninstance_count.scale_on_invocations(\"LimitRPS\",\n max_requests_per_second=30\n)\n```\n\nFor load testing guidance on determining the maximum requests per second per instance, please see\nthis [documentation](https://docs.aws.amazon.com/sagemaker/latest/dg/endpoint-scaling-loadtest.html).\n\n### Metrics\n\nTo monitor CloudWatch metrics for a production variant, use one or more of the metric convenience\nmethods:\n\n```python\nimport aws_cdk.aws_sagemaker_alpha as sagemaker\n\n# endpoint_config: sagemaker.EndpointConfig\n\n\nendpoint = sagemaker.Endpoint(self, \"Endpoint\", endpoint_config=endpoint_config)\nproduction_variant = endpoint.find_instance_production_variant(\"my-variant\")\nproduction_variant.metric_model_latency().create_alarm(self, \"ModelLatencyAlarm\",\n threshold=100000,\n evaluation_periods=3\n)\n```\n",
"bugtrack_url": null,
"license": "Apache-2.0",
"summary": "The CDK Construct Library for AWS::SageMaker",
"version": "2.170.0a0",
"project_urls": {
"Homepage": "https://github.com/aws/aws-cdk",
"Source": "https://github.com/aws/aws-cdk.git"
},
"split_keywords": [],
"urls": [
{
"comment_text": "",
"digests": {
"blake2b_256": "906954ac1593cf972bdfd86ac47f202240cc667b5179857fa381d5e51fbb54bc",
"md5": "1b753881c94f7ebf98784e738419c77b",
"sha256": "405d4218b4db9de5e54738d62e504d43eeb96c9131cfe2812aa3aa05935c6780"
},
"downloads": -1,
"filename": "aws_cdk.aws_sagemaker_alpha-2.170.0a0-py3-none-any.whl",
"has_sig": false,
"md5_digest": "1b753881c94f7ebf98784e738419c77b",
"packagetype": "bdist_wheel",
"python_version": "py3",
"requires_python": "~=3.8",
"size": 135137,
"upload_time": "2024-11-22T04:41:58",
"upload_time_iso_8601": "2024-11-22T04:41:58.774155Z",
"url": "https://files.pythonhosted.org/packages/90/69/54ac1593cf972bdfd86ac47f202240cc667b5179857fa381d5e51fbb54bc/aws_cdk.aws_sagemaker_alpha-2.170.0a0-py3-none-any.whl",
"yanked": false,
"yanked_reason": null
},
{
"comment_text": "",
"digests": {
"blake2b_256": "07226c7c0058de8e312aec58e541c154b7f2fcdbbc8b811b4f3d977f635779ef",
"md5": "1d109d0a52ee9bf8a10d28850076896e",
"sha256": "2c2bcb9215ce46459843eb544b33a7a65d2433a841eeed240f97a60d2d3ca30e"
},
"downloads": -1,
"filename": "aws_cdk_aws_sagemaker_alpha-2.170.0a0.tar.gz",
"has_sig": false,
"md5_digest": "1d109d0a52ee9bf8a10d28850076896e",
"packagetype": "sdist",
"python_version": "source",
"requires_python": "~=3.8",
"size": 136427,
"upload_time": "2024-11-22T04:42:43",
"upload_time_iso_8601": "2024-11-22T04:42:43.013585Z",
"url": "https://files.pythonhosted.org/packages/07/22/6c7c0058de8e312aec58e541c154b7f2fcdbbc8b811b4f3d977f635779ef/aws_cdk_aws_sagemaker_alpha-2.170.0a0.tar.gz",
"yanked": false,
"yanked_reason": null
}
],
"upload_time": "2024-11-22 04:42:43",
"github": true,
"gitlab": false,
"bitbucket": false,
"codeberg": false,
"github_user": "aws",
"github_project": "aws-cdk",
"travis_ci": false,
"coveralls": false,
"github_actions": true,
"lcname": "aws-cdk.aws-sagemaker-alpha"
}
JSON
