Skip to content

Commit 26e19f2

Browse files
nargokulpintaoz-aws
nargokul
authored and
pintaoz-aws
committed
Notebooks update for Bugbash (#1595)
* Notebooks update for Bugbash * Testing and updates * Testing and updates * Addressed comments * Fix * Fix
1 parent 87c2563 commit 26e19f2

File tree

4 files changed

+268
-69
lines changed

4 files changed

+268
-69
lines changed

‎src/sagemaker/image_uris.py

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -609,7 +609,6 @@ def _validate_py_version_and_set_if_needed(py_version, version_config, framework
609609

610610
def _validate_arg(arg, available_options, arg_name):
611611
"""Checks if the arg is in the available options, and raises a ``ValueError`` if not."""
612-
print("VALIDATE")
613612
if arg not in available_options:
614613
raise ValueError(
615614
"Unsupported {arg_name}: {arg}. You may need to upgrade your SDK version "

‎src/sagemaker/modules/testing_notebooks/model_builder_handshake.ipynb

Lines changed: 262 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,74 +1,174 @@
11
{
22
"cells": [
33
{
4-
"cell_type": "code",
5-
"execution_count": null,
6-
"id": "eca12282",
74
"metadata": {},
8-
"outputs": [],
5+
"cell_type": "markdown",
96
"source": [
10-
"! pip install \"pydantic>=2.0.0\" sagemaker-core"
11-
]
7+
"# Model Builder Redesign\n",
8+
"## This notebook highlights the new changes made to ModelBuilder and related utilities\n",
9+
"\n",
10+
"- Latest Container Image Utility function\n",
11+
"- Handshake with ModelTrainer \n",
12+
"- Unified Deployment from ModelBuilder"
13+
],
14+
"id": "609dabdc97ce0a62"
1215
},
1316
{
14-
"cell_type": "code",
15-
"execution_count": null,
16-
"id": "756ebbf2",
1717
"metadata": {},
18+
"cell_type": "code",
19+
"source": "alias = \"user\"",
20+
"id": "54b0ea46e1886184",
1821
"outputs": [],
19-
"source": [
20-
"! pip install sagemaker-2.232.4.dev0.tar.gz"
21-
]
22+
"execution_count": null
2223
},
2324
{
25+
"metadata": {},
26+
"cell_type": "markdown",
27+
"source": "## Inital Setup",
28+
"id": "b421db5dfb9a7fa7"
29+
},
30+
{
31+
"metadata": {},
2432
"cell_type": "code",
25-
"execution_count": null,
26-
"id": "initial_id",
27-
"metadata": {
28-
"collapsed": true
29-
},
30-
"outputs": [],
3133
"source": [
32-
"from sagemaker_core.main.shapes import TrainingJob\n",
33-
"\n",
3434
"from sagemaker import Session, get_execution_role\n",
3535
"\n",
3636
"sagemaker_session = Session()\n",
3737
"role = get_execution_role()\n",
3838
"region = sagemaker_session.boto_region_name\n",
3939
"bucket = sagemaker_session.default_bucket()"
40-
]
40+
],
41+
"id": "30656ece22011af6",
42+
"outputs": [],
43+
"execution_count": null
4144
},
4245
{
43-
"cell_type": "code",
44-
"execution_count": null,
45-
"id": "4b3a4f7d1713685f",
4646
"metadata": {},
47+
"cell_type": "code",
48+
"source": [
49+
"from sklearn.datasets import load_iris\n",
50+
"from sklearn.model_selection import train_test_split\n",
51+
"\n",
52+
"import pandas as pd\n",
53+
"\n",
54+
"# Prepare Data\n",
55+
"\n",
56+
"iris = load_iris()\n",
57+
"iris_df = pd.DataFrame(iris.data, columns=iris.feature_names)\n",
58+
"iris_df['target'] = iris.target\n",
59+
"\n",
60+
"os.makedirs('./data', exist_ok=True)\n",
61+
"\n",
62+
"iris_df = iris_df[['target'] + [col for col in iris_df.columns if col != 'target']]\n",
63+
"\n",
64+
"train_data, test_data = train_test_split(iris_df, test_size=0.2, random_state=42)\n",
65+
"\n",
66+
"train_data.to_csv('./data/train.csv', index=False, header=False)\n",
67+
"test_data.to_csv('./data/test.csv', index=False, header=False)\n",
68+
"\n",
69+
"# Remove the target column from the testing data. We will use this to call invoke_endpoint later\n",
70+
"test_data_no_target = test_data.drop('target', axis=1)\n",
71+
"\n",
72+
"prefix = \"DEMO-scikit-iris\"\n",
73+
"TRAIN_DATA = \"train.csv\"\n",
74+
"TEST_DATA = \"test.csv\"\n",
75+
"DATA_DIRECTORY = \"data\"\n",
76+
"\n",
77+
"train_input = sagemaker_session.upload_data(\n",
78+
" DATA_DIRECTORY, bucket=bucket, key_prefix=\"{}/{}\".format(prefix, DATA_DIRECTORY)\n",
79+
")\n",
80+
"\n",
81+
"\n",
82+
"s3_input_path = \"s3://{}/{}/data/{}\".format(bucket, prefix, TRAIN_DATA)\n",
83+
"s3_output_path = \"s3://{}/{}/output\".format(bucket, prefix)\n",
84+
"\n",
85+
"s3_test_path = \"s3://{}/{}/data/{}\".format(bucket, prefix, TEST_DATA)\n",
86+
"\n",
87+
"print(s3_input_path)\n",
88+
"print(s3_output_path)"
89+
],
90+
"id": "d0ea8169b21eaa29",
4791
"outputs": [],
92+
"execution_count": null
93+
},
94+
{
95+
"metadata": {},
96+
"cell_type": "markdown",
97+
"source": [
98+
"\n",
99+
"# Integration with ModelTrainer\n",
100+
"\n",
101+
"The handshake between ModelTrainer and ModelBuilder is made seamlessly as in this example. The created model trainer object is directly fed into the model attribute of ModelBuilder through resource chaining . Fetching of the model artifacts is done internally within the ModelBuilder. \n",
102+
"\n",
103+
"Note: \n",
104+
"- Other than the ModelTrainer, the ModelBuilder also supports chaining of attributes such as Estimator or sagemaker-core's TrainingJob into the model attribute. \n",
105+
"\n",
106+
"Other than this there is an upgrade designed for retrieving images for a particular framework. The enhanced `image_uris.retrieve()` method will fetch the latest version of an image automatically if the version is not provided.\n"
107+
],
108+
"id": "1dc5055ba5ccb2bb"
109+
},
110+
{
111+
"metadata": {},
112+
"cell_type": "code",
48113
"source": [
49114
"\n",
50-
"from sagemaker.modules.configs import SourceCode\n",
115+
"from sagemaker import image_uris\n",
116+
"from sagemaker_core.main.shapes import Channel, DataSource, S3DataSource, OutputDataConfig, \\\n",
117+
" StoppingCondition\n",
51118
"from sagemaker.modules.train.model_trainer import ModelTrainer\n",
52119
"\n",
53-
"xgboost_image = \"433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:latest\"\n",
120+
"# xgboost_image=\"433757028032.dkr.ecr.us-west-2.amazonaws.com/xgboost:latest\"\n",
121+
"xgboost_image = image_uris.retrieve(framework=\"xgboost\", region=\"us-west-2\", image_scope=\"training\")\n",
122+
"print(xgboost_image)\n",
54123
"\n",
55-
"source_code = SourceCode(\n",
56-
" command=\"echo 'Hello World' && env\",\n",
57-
")\n",
58124
"model_trainer = ModelTrainer(\n",
125+
" base_job_name=f'{alias}-mb-handshake',\n",
126+
" hyperparameters={\n",
127+
" 'objective': 'multi:softmax',\n",
128+
" 'num_class': '3',\n",
129+
" 'num_round': '10',\n",
130+
" 'eval_metric': 'merror'\n",
131+
" },\n",
59132
" training_image=xgboost_image,\n",
60-
" source_code=source_code,\n",
133+
" training_input_mode='File',\n",
134+
" role=role,\n",
135+
" output_data_config=OutputDataConfig(\n",
136+
" s3_output_path=s3_output_path\n",
137+
" ),\n",
138+
" stopping_condition=StoppingCondition(\n",
139+
" max_runtime_in_seconds=600\n",
140+
" )\n",
61141
")\n",
62142
"\n",
63-
"model_trainer.train()"
64-
]
143+
"model_trainer.train(input_data_config=[\n",
144+
" Channel(\n",
145+
" channel_name='train',\n",
146+
" content_type='csv',\n",
147+
" compression_type='None',\n",
148+
" record_wrapper_type='None',\n",
149+
" data_source=DataSource(\n",
150+
" s3_data_source=S3DataSource(\n",
151+
" s3_data_type='S3Prefix',\n",
152+
" s3_uri=s3_input_path,\n",
153+
" s3_data_distribution_type='FullyReplicated'\n",
154+
" )\n",
155+
" )\n",
156+
" )\n",
157+
"], )"
158+
],
159+
"id": "4b3a4f7d1713685f",
160+
"outputs": [],
161+
"execution_count": null
65162
},
66163
{
67-
"cell_type": "code",
68-
"execution_count": null,
69-
"id": "295a16ef277257a0",
70164
"metadata": {},
71-
"outputs": [],
165+
"cell_type": "markdown",
166+
"source": "",
167+
"id": "96be4afa22c64350"
168+
},
169+
{
170+
"metadata": {},
171+
"cell_type": "code",
72172
"source": [
73173
"import numpy as np\n",
74174
"from sagemaker.serve.builder.schema_builder import SchemaBuilder\n",
@@ -96,54 +196,151 @@
96196
" predictions = np.argmax(prediction_probabilities, axis=1)\n",
97197
" return predictions\n",
98198
"\n",
199+
"\n",
99200
"model_builder = ModelBuilder(\n",
100-
" model=model_trainer, # ModelTrainer object passed onto ModelBuilder directly \n",
201+
" model=model_trainer, # ModelTrainer object passed onto ModelBuilder directly \n",
101202
" role_arn=role,\n",
102203
" image_uri=xgboost_image,\n",
103204
" inference_spec=XGBoostSpec(),\n",
104205
" schema_builder=schema_builder,\n",
105206
" instance_type=\"ml.c6i.xlarge\"\n",
106207
")\n",
107-
"model=model_builder.build()\n",
108-
"predictor=model_builder.deploy()\n",
109-
"\n",
110-
"predictor\n",
111-
"assert model.model_data == model_trainer._latest_training_job.model_artifacts.s3_model_artifacts\n",
112-
"\n",
113-
"print(model.model_data)"
114-
]
208+
"model = model_builder.build()"
209+
],
210+
"id": "295a16ef277257a0",
211+
"outputs": [],
212+
"execution_count": null
213+
},
214+
{
215+
"metadata": {},
216+
"cell_type": "markdown",
217+
"source": "Once the model has been built , it can be deployed directly through the model_builder.deploy() method. This abstracts out information that was previously used commonly in workflows for different deployment modes. The deploy() method takes in an optional parameter `inference_config`. This determines attributes for modes such as serverless, async, batch and multi-model/multi-container endpoints. If the `inference_config` is not provided, the default real-time deployment is carried out.",
218+
"id": "99b403edaf616ef0"
219+
},
220+
{
221+
"metadata": {},
222+
"cell_type": "markdown",
223+
"source": "## ModelBuilder - Real-Time Deployment",
224+
"id": "44416566576e26df"
115225
},
116226
{
117-
"cell_type": "code",
118-
"execution_count": null,
119-
"id": "935ea8486278d7b1",
120227
"metadata": {},
228+
"cell_type": "code",
229+
"source": [
230+
"predictor = model_builder.deploy(endpoint_name=f\"{alias}-xgboost-deploy-realtime\")\n",
231+
"sklearn_input = np.array([1.0, 2.0, 3.0, 4.0])\n",
232+
"result = predictor.predict(sklearn_input)\n",
233+
"print(result)"
234+
],
235+
"id": "ab43000f6bd6018b",
121236
"outputs": [],
237+
"execution_count": null
238+
},
239+
{
240+
"metadata": {},
241+
"cell_type": "markdown",
242+
"source": "## ModelBuilder - Serverless Deployment\n",
243+
"id": "fbd0e6f6e92d0aeb"
244+
},
245+
{
246+
"metadata": {},
247+
"cell_type": "code",
122248
"source": [
123-
"training_job: TrainingJob = model_trainer._latest_training_job\n",
124-
"\n",
125-
"model_builder = ModelBuilder(\n",
126-
" model=training_job, # Sagemaker core's TrainingJob object passed onto ModelBuilder directly \n",
127-
" role_arn=role,\n",
128-
" image_uri=xgboost_image,\n",
129-
" schema_builder=schema_builder,\n",
130-
" inference_spec=XGBoostSpec(),\n",
131-
" instance_type=\"ml.c6i.xlarge\"\n",
132-
")\n",
133-
"model=model_builder.build()\n",
249+
"from sagemaker.serverless.serverless_inference_config import ServerlessInferenceConfig\n",
134250
"\n",
135-
"assert model.model_data == training_job.model_artifacts.s3_model_artifacts\n",
251+
"predictor = model_builder.deploy(endpoint_name=f\"{alias}-xgboost-deploy-serverless\",\n",
252+
" inference_config=ServerlessInferenceConfig(memory_size_in_mb=2048))\n",
253+
"sklearn_input = np.array([1.0, 2.0, 3.0, 4.0])\n",
254+
"result = predictor.predict(sklearn_input)\n",
255+
"print(result)"
256+
],
257+
"id": "40d3c9973d2d8934",
258+
"outputs": [],
259+
"execution_count": null
260+
},
261+
{
262+
"metadata": {},
263+
"cell_type": "markdown",
264+
"source": "## ModelBuilder - Async Deployment\n",
265+
"id": "93818038782f105d"
266+
},
267+
{
268+
"metadata": {},
269+
"cell_type": "code",
270+
"source": [
271+
"from sagemaker.async_inference.async_inference_config import AsyncInferenceConfig\n",
272+
"from sagemaker.s3_utils import s3_path_join\n",
136273
"\n",
137-
"print(model.model_data)"
138-
]
274+
"predictor = model_builder.deploy(endpoint_name=f\"{alias}-xgboost-deploy-async\", inference_config=AsyncInferenceConfig(\n",
275+
" output_path=s3_path_join(\"s3://\", bucket, \"async_inference/output\")))\n",
276+
"sklearn_input = np.array([1.0, 2.0, 3.0, 4.0])\n",
277+
"result = predictor.predict(sklearn_input)\n",
278+
"print(result)"
279+
],
280+
"id": "77e7104aaa9d6da2",
281+
"outputs": [],
282+
"execution_count": null
139283
},
140284
{
285+
"metadata": {},
286+
"cell_type": "markdown",
287+
"source": "## ModelBuilder - Batch Deployment\n",
288+
"id": "2ff3e043b5f5f8d7"
289+
},
290+
{
291+
"metadata": {},
141292
"cell_type": "code",
142-
"execution_count": null,
143-
"id": "757180da84407a1a",
293+
"source": [
294+
"from sagemaker.batch_inference.batch_transform_inference_config import BatchTransformInferenceConfig\n",
295+
"from sagemaker.s3_utils import s3_path_join\n",
296+
"\n",
297+
"transformer = model_builder.deploy(endpoint_name=f\"{alias}-xgboost-deploy-batch\",\n",
298+
" inference_config=BatchTransformInferenceConfig(\n",
299+
" instance_count=1,\n",
300+
" instance_type='ml.m5.large',\n",
301+
" output_path=s3_path_join(\"s3://\", bucket, \"batch_inference/output\"),\n",
302+
" test_data_s3_path = s3_test_path\n",
303+
" ))\n",
304+
"print(transformer)"
305+
],
306+
"id": "3ef3febc0f840133",
307+
"outputs": [],
308+
"execution_count": null
309+
},
310+
{
311+
"metadata": {},
312+
"cell_type": "markdown",
313+
"source": "## ModelBuilder - Multi-Model Endpoint Deployment\n",
314+
"id": "66e103b7292694eb"
315+
},
316+
{
317+
"metadata": {},
318+
"cell_type": "markdown",
319+
"source": "",
320+
"id": "a1c4d43bbd23efec"
321+
},
322+
{
144323
"metadata": {},
324+
"cell_type": "code",
325+
"source": [
326+
"from sagemaker.compute_resource_requirements.resource_requirements import ResourceRequirements\n",
327+
"\n",
328+
"predictor = model_builder.deploy(endpoint_name=f\"{alias}-xgboost-deploy-multi-model\",\n",
329+
" inference_config=ResourceRequirements(\n",
330+
" requests={\n",
331+
" \"num_cpus\": 0.5,\n",
332+
" \"memory\": 512,\n",
333+
" \"copies\": 2,\n",
334+
" },\n",
335+
" limits={},\n",
336+
" ))\n",
337+
"sklearn_input = np.array([1.0, 2.0, 3.0, 4.0])\n",
338+
"result = predictor.predict(sklearn_input)\n",
339+
"print(result)"
340+
],
341+
"id": "f3c622148377c964",
145342
"outputs": [],
146-
"source": []
343+
"execution_count": null
147344
}
148345
],
149346
"metadata": {

0 commit comments

Comments
 (0)