Merge pull request #36 from oracle/model-artifact

shail2612 · web-flow · commit 639b38ca1d5c · 2021-09-08T22:29:48.000+05:30
added notebooks
diff --git a/model_catalog_examples/notebook_examples/README.md b/model_catalog_examples/notebook_examples/README.md
@@ -1 +1,25 @@
+## Notebook Examples
 
+The following notebook examples serve as tutorials to prepare and save different model artifacts and then deploy them as well.
+
+1. [linear_regression_generic.ipynb](linear_regression_generic.ipynb)
+    * This notebook will provide an example to prepare and save an sklearn model artifact using ADS generic method and deploy the model as an HTTP endpoint
+    
+1. [pytorch_pretrained.ipynb](pytorch_pretrained.ipynb)
+    * This notebook will provide an example to prepare and save a pytorch model artifact using ADS, will publish a conda environment, and deploy the model as an HTTP endpoint.
+    
+1. [saving_model_oci_python_sdk.ipynb](saving_model_oci_python_sdk.ipynb)
+    * This example notebook demonstrates creating and uploading a XGBoost binary logisitic-based model, with metadata and schema, to the model catalog v2.0.
+    
+1. [simple-model-deployment.ipynb](simple-model-deployment.ipynb)
+    * This example notebook demonstrates to deploy a sklearn random forest classifier as an HTTP endpoint using Model Deployment.
+    
+1. [uploading_larger_artifact_oci_python_sdk.ipynb](uploading_larger_artifact_oci_python_sdk.ipynb)
+    * This example notebook demonstrates simple solution for OCI Python SDK which allows data scientists to upload larger model artifacts and eliminate the timeout error that is experienced by most folks when the artifact is large. It shows end-to-end steps from setting up the configuration till uploading the model artifact.
+    
+1. [uploading_larger_artifact_oracle_ads.ipynb](uploading_larger_artifact_oracle_ads.ipynb)
+    * This example notebook demonstrates simple solution for Oracle ADS Library which allows data scientists to upload larger model artifacts and eliminate the timeout error that is experienced by most folks when the artifact is large. It shows end-to-end steps from setting up the configuration till uploading the model artifact.
+    
+1. [xgboost_onnx.ipynb](xboost_onnx.ipynb)
+    * This example notebook demonstrates how to prepare and save an xgboost model artifact using the ADSModel `prepare()` method and deploy the model as an HTTP endpoint.
+    
diff --git a/model_catalog_examples/notebook_examples/uploading_larger_artifact_oci_python_sdk.ipynb b/model_catalog_examples/notebook_examples/uploading_larger_artifact_oci_python_sdk.ipynb
@@ -2,64 +2,52 @@
  "cells": [
   {
    "cell_type": "markdown",
-   "id": "5f77a6ca",
+   "id": "3a18e055",
    "metadata": {},
    "source": [
-    "### OCI Data Science - Useful Tips\n",
-    "<details>\n",
-    "<summary><font size=\"2\">Check for Public Internet Access</font></summary>\n",
-    "\n",
-    "```python\n",
-    "import requests\n",
-    "response = requests.get(\"https://oracle.com\")\n",
-    "assert response.status_code==200, \"Internet connection failed\"\n",
-    "```\n",
-    "</details>\n",
-    "<details>\n",
-    "<summary><font size=\"2\">Helpful Documentation </font></summary>\n",
-    "<ul><li><a href=\"https://docs.cloud.oracle.com/en-us/iaas/data-science/using/data-science.htm\">Data Science Service Documentation</a></li>\n",
-    "<li><a href=\"https://docs.cloud.oracle.com/iaas/tools/ads-sdk/latest/index.html\">ADS documentation</a></li>\n",
-    "</ul>\n",
-    "</details>\n",
-    "<details>\n",
-    "<summary><font size=\"2\">Typical Cell Imports and Settings for ADS</font></summary>\n",
-    "\n",
-    "```python\n",
-    "%load_ext autoreload\n",
-    "%autoreload 2\n",
-    "%matplotlib inline\n",
-    "\n",
-    "import warnings\n",
-    "warnings.filterwarnings('ignore')\n",
+    "<font color=gray>Oracle Cloud Infrastructure Data Science Sample Notebook\n",
     "\n",
-    "import logging\n",
-    "logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.ERROR)\n",
-    "\n",
-    "import ads\n",
-    "from ads.dataset.factory import DatasetFactory\n",
-    "from ads.automl.provider import OracleAutoMLProvider\n",
-    "from ads.automl.driver import AutoML\n",
-    "from ads.evaluations.evaluator import ADSEvaluator\n",
-    "from ads.common.data import ADSData\n",
-    "from ads.explanations.explainer import ADSExplainer\n",
-    "from ads.explanations.mlx_global_explainer import MLXGlobalExplainer\n",
-    "from ads.explanations.mlx_local_explainer import MLXLocalExplainer\n",
-    "from ads.catalog.model import ModelCatalog\n",
-    "from ads.common.model_artifact import ModelArtifact\n",
-    "```\n",
-    "</details>\n",
-    "<details>\n",
-    "<summary><font size=\"2\">Useful Environment Variables</font></summary>\n",
-    "\n",
-    "```python\n",
-    "import os\n",
-    "print(os.environ[\"NB_SESSION_COMPARTMENT_OCID\"])\n",
-    "print(os.environ[\"PROJECT_OCID\"])\n",
-    "print(os.environ[\"USER_OCID\"])\n",
-    "print(os.environ[\"TENANCY_OCID\"])\n",
-    "print(os.environ[\"NB_REGION\"])\n",
-    "```\n",
-    "</details>"
+    "Copyright (c) 2021 Oracle, Inc.  All rights reserved. <br>\n",
+    "Licensed under the Universal Permissive License v 1.0 as shown at https://oss.oracle.com/licenses/upl.\n",
+    "</font>"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "59909c92",
+   "metadata": {},
+   "source": [
+    "# Uploading Larger Size Model Artifact Using OCI Pyhton SDK \n",
+    "\n",
+    "This notebook demonstrates simple solution for OCI Python SDK which allows data scientists to upload larger model artifacts and eliminate the timeout error that is experienced by most folks when the artifact is large. It shows end-to-end steps from setting up the configuration till uploading the model artifact."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "96aa5b7b",
+   "metadata": {},
+   "source": [
+    "## Pre-requisites to Running this Notebook "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "483c5bed",
+   "metadata": {},
+   "source": [
+    "* We recommend that you run this notebook in a notebook session using the **Data Science Conda Environment \"Data Exploration and Manipulation for CPU Python 3.7 V2 conda environment\"** \n",
+    "* You need access to the public internet\n",
+    "* Upgrade the current version of the OCI Python SDK (`oci`): "
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "919b4389",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "!pip install --upgrade oci"
    ]
   },
   {
@@ -76,7 +64,7 @@
     "import os\n",
     "import logging\n",
     "\n",
-    "REGION = \"us-ashburn-1\"\n",
+    "REGION = f\"<replace-with-region>\"\n",
     "logger = logging.getLogger('upload_model_artifact')\n",
     "logger.setLevel(logging.DEBUG)\n",
     "ch = logging.StreamHandler()\n",
@@ -113,7 +101,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "SERVICE_ENDPOINT = \"https://datascience.us-phoenix-1.oci.oraclecloud.com/20190101\"\n",
+    "SERVICE_ENDPOINT = f\"<replace-with-service-endpoint>\"\n",
     "\n",
     "def create_data_science_client(config: dict) -> DataScienceClient:\n",
     "    \"\"\"\n",
@@ -147,8 +135,8 @@
    "source": [
     "# set Compartment Id\n",
     "COMPARTMENT_ID = os.environ['NB_SESSION_COMPARTMENT_OCID']\n",
-    "PROJECT_DESCRIPTION = \"Test Project\"\n",
-    "PROJECT_DISPLAY_NAME = \"ModelStore-ArtifactTest\"\n",
+    "PROJECT_DESCRIPTION = f\"<replace-with-your-project-description>\"\n",
+    "PROJECT_DISPLAY_NAME = f\"<replace-with-your-project-display-name>\"\n",
     "\n",
     "data_science_models = oci.data_science.models\n",
     "\n",
@@ -164,6 +152,7 @@
     "    \"\"\"\n",
     "\n",
     "    logger.info(\"Defining project details object...\")\n",
+    "    \n",
     "    # We need to create a project first. Get the create project details object.\n",
     "    create_project_details_object = data_science_models.CreateProjectDetails()\n",
     "\n",
@@ -249,7 +238,7 @@
     "    logger.info(\"Defining Model details object...\")\n",
     "    create_model_details_object = data_science_models.CreateModelDetails()\n",
     "    create_model_details_object.compartment_id = COMPARTMENT_ID\n",
-    "    create_model_details_object.display_name = \"MD-ModelArtifact-test\"\n",
+    "    create_model_details_object.display_name = f\"<replace-with-your-object-display-name>\"\n",
     "    create_model_details_object.project_id = project_id\n",
     "    return create_model_details_object\n",
     "\n",
@@ -303,14 +292,55 @@
    "id": "ec13aee5",
    "metadata": {},
    "outputs": [],
-   "source": []
+   "source": [
+    "# provide the artifact file path\n",
+    "ARTIFACT_FILE_NAME = f\"<replace-with-your-artifact-file-path>\"\n",
+    "\n",
+    "def upload_model_artifact(model_id: str):\n",
+    "    try:\n",
+    "        logger.info(\"uploading model artifact...\")\n",
+    "        # creates the model and model artifact\n",
+    "        # Make sure to provide the correct path of the zip file in ARTIFACT_FILE_NAME\n",
+    "        create_model_artifact(data_science_client, model_id)\n",
+    "    except Exception as e:\n",
+    "        return str(e)\n",
+    "    return \"True\"\n",
+    "\n",
+    "def create_model_artifact(data_science_client: DataScienceClient, model_id: str):\n",
+    "    \"\"\"\n",
+    "    Creates the model artifact.\n",
+    "\n",
+    "    Parameters:\n",
+    "    data_science_client (DataScienceClient): the data science client\n",
+    "    model_id (str): the model id to use in creating artifact\n",
+    "    \"\"\"\n",
+    "    logger.info(\"Create artifact\")\n",
+    "    f = open(ARTIFACT_FILE_NAME, \"rb\")\n",
+    "    logger.info(\"File open\")\n",
+    "    content_disposition = \"attachment;filename={}\".format(ARTIFACT_FILE_NAME)\n",
+    "    logger.debug(content_disposition)\n",
+    "    logger.debug(data_science_client)\n",
+    "    try:\n",
+    "        data_science_client.base_client.timeout = 30 * 60\n",
+    "        data_science_client.create_model_artifact(\n",
+    "            model_id, f, content_disposition=content_disposition)\n",
+    "        logger.info(\"Upload success\")\n",
+    "        print(\"Upload Success\")\n",
+    "        logger.info(\"Finished creating artifact\")\n",
+    "    except Exception as e:\n",
+    "        print(\"==================\", str(e))\n",
+    "        logger.error(\"Upload error\")\n",
+    "        logger.debug(str(e))\n",
+    "    f.close()\n",
+    "    return"
+   ]
   }
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python [conda env:dataexpl_p37_cpu_v2]",
+   "display_name": "Python 3",
    "language": "python",
-   "name": "conda-env-dataexpl_p37_cpu_v2-py"
+   "name": "python3"
   },
   "language_info": {
    "codemirror_mode": {
@@ -322,9 +352,9 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.7.10"
+   "version": "3.8.8"
   }
  },
  "nbformat": 4,
  "nbformat_minor": 5
-}
+}
diff --git a/model_catalog_examples/notebook_examples/uploading_larger_artifact_oracle_ads.ipynb b/model_catalog_examples/notebook_examples/uploading_larger_artifact_oracle_ads.ipynb
