Microscope

[christy]

Issue #, if available:

Description of changes:

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[shimomut]

It seems "notebooks/advanced/Compare_Multiple_Models/Compare_Multiple_Models.ipynb" was deleted unintentionally.

[shimomut]

Please delete output cells.

[shimomut]

# display ipython widgets in notebook
!jupyter nbextension enable --py --sys-prefix widgetsnbextension

In my environment, "jupyter" command cannot be found by environment PATH, and it causes following error.

'jupyter' is not recognized as an internal or external command,
operable program or batch file.

[shimomut]

text_widget_bucket = create_text_widget( "bucket_name", "input your S3 bucket name"
                                             , default_value="bike-demo" )
text_widget_region = create_text_widget( "region", "input region name."
                                             , default_value="us-west-2" )

In these 2 lines, missing "util." and it causes following error.

NameError: name 'create_text_widget' is not defined

[shimomut]

# arn:aws:forecast:us-west-2:XXXXXXXXXXX:predictor/bike_demo2_auto     
# s3://christy-forecast/bike_share_open_data/demo/bike_backtest_export_files2/

These 2 comment lines have to be deleted.

[shimomut]

* [Set up and install libraries](#setup)

Link text ("Set up and install libraries") is different from actual section name text ("Set up").

[shimomut]

os.system('touch {}'.format(temp_file))

This is Linux/Mac dependend. On Windows "touch" command doesn't exist.

Probably this line is not needed, because "with open(temp_file, 'wb') as outfile:" should create file anyways.

[shimomut]

[ Optional ]

s3_resource = boto3.resource('s3')

"s3_resource" is not used after this line.

forecastquery = session.client(service_name='forecastquery') #Amazon Forecast Query api session

"forecastquery" is not used after this line.

[shimomut]

delete_predictor_backtest_export_job() is missing in this notebook.

[shimomut]

# copy concatted local .csv file back to S3
!aws s3 cp $temp_file $to_file_accuracies

This code is copying "bike_backtest_forecasts.csv" as "bike_backtest_accuracies.csv".
"to_file_accuracies" should be "to_file_forecasts".
$to_file_accuracies

[shimomut]

I am not sure why, but in my environment, following line fails.

df = pd.read_csv(to_file_forecasts)

with following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-50-a199f0580ced> in <module>
      1 ## READ THE FORECASTS FILE
      2 
----> 3 df = pd.read_csv(to_file_forecasts)
      4 
      5 # correct data types

c:\python37-64\lib\site-packages\pandas\io\parsers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision)
    686     )
    687 
--> 688     return _read(filepath_or_buffer, kwds)
    689 
    690 

c:\python37-64\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds)
    435     # See https://github.com/python/mypy/issues/1297
    436     fp_or_buf, _, compression, should_close = get_filepath_or_buffer(
--> 437         filepath_or_buffer, encoding, compression
    438     )
    439     kwds["compression"] = compression

c:\python37-64\lib\site-packages\pandas\io\common.py in get_filepath_or_buffer(filepath_or_buffer, encoding, compression, mode, storage_options)
    220         try:
    221             file_obj = fsspec.open(
--> 222                 filepath_or_buffer, mode=mode or "rb", **(storage_options or {})
    223             ).open()
    224         # GH 34626 Reads from Public Buckets without Credentials needs anon=True

c:\python37-64\lib\site-packages\fsspec\core.py in open(self)
    132         been deleted; but a with-context is better style.
    133         """
--> 134         out = self.__enter__()
    135         closer = out.close
    136         fobjects = self.fobjects.copy()[:-1]

c:\python37-64\lib\site-packages\fsspec\core.py in __enter__(self)
    100         mode = self.mode.replace("t", "").replace("b", "") + "b"
    101 
--> 102         f = self.fs.open(self.path, mode=mode)
    103 
    104         self.fobjects = [f]

c:\python37-64\lib\site-packages\fsspec\spec.py in open(self, path, mode, block_size, cache_options, **kwargs)
    901                 autocommit=ac,
    902                 cache_options=cache_options,
--> 903                 **kwargs
    904             )
    905             if not ac:

c:\python37-64\lib\site-packages\s3fs\core.py in _open(self, path, mode, block_size, acl, version_id, fill_cache, cache_type, autocommit, requester_pays, **kwargs)
    313                       version_id=version_id, fill_cache=fill_cache,
    314                       s3_additional_kwargs=kw, cache_type=cache_type,
--> 315                       autocommit=autocommit, requester_pays=requester_pays)
    316 
    317     def _lsdir(self, path, refresh=False, max_items=None):

c:\python37-64\lib\site-packages\s3fs\core.py in __init__(self, s3, path, mode, block_size, acl, version_id, fill_cache, s3_additional_kwargs, autocommit, cache_type, requester_pays)
    955         self.req_kw = {'RequestPayer': 'requester'} if requester_pays else {}
    956         super().__init__(s3, path, mode, block_size, autocommit=autocommit,
--> 957                          cache_type=cache_type)
    958         self.s3 = self.fs  # compatibility
    959         if self.writable():

c:\python37-64\lib\site-packages\fsspec\spec.py in __init__(self, fs, path, mode, block_size, autocommit, cache_type, cache_options, **kwargs)
   1222         if mode == "rb":
   1223             if not hasattr(self, "details"):
-> 1224                 self.details = fs.info(path)
   1225             self.size = self.details["size"]
   1226             self.cache = caches[cache_type](

c:\python37-64\lib\site-packages\s3fs\core.py in info(self, path, version_id)
    484             except ParamValidationError as e:
    485                 raise ValueError('Failed to head path %r: %s' % (path, e))
--> 486         return super().info(path)
    487 
    488     def isdir(self, path):

c:\python37-64\lib\site-packages\fsspec\spec.py in info(self, path, **kwargs)
    589             return {"name": path, "size": 0, "type": "directory"}
    590         else:
--> 591             raise FileNotFoundError(path)
    592 
    593     def checksum(self, path):

FileNotFoundError: shimomut-forecast-notebook/bike_share_open_data/demo/bike_backtest_forecasts.csv

Not sure if loading pandas DataFrame from S3 is stable. Why don't we load the data directly from local files without uploading onto S3?

[shimomut]

On my environment, "axx = axs.ravel()" fails with following error:

AttributeError: 'AxesSubplot' object has no attribute 'ravel'

Does this API exist on your environment?
I checked the document of matplotlib, but I cannot find such a method:
https://matplotlib.org/3.1.0/api/axes_api.html#matplotlib.axes.Axes

[christy]

Fixed. I was missing “s3://” in the to_file name. From: Tomonori Shimomura <[email protected]> Reply-To: aws-samples/amazon-forecast-samples <[email protected]> Date: Tuesday, November 24, 2020 at 12:29 AM To: aws-samples/amazon-forecast-samples <[email protected]> Cc: "Bergman, Christy" <[email protected]>, Author <[email protected]> Subject: Re: [aws-samples/amazon-forecast-samples] Microscope (#96) I am not sure why, but in my environment, following line fails. df = pd.read_csv(to_file_forecasts) with following error:

…

--------------------------------------------------------------------------- FileNotFoundError Traceback (most recent call last) <ipython-input-50-a199f0580ced> in <module> 1 ## READ THE FORECASTS FILE 2

----> 3 df = pd.read_csv(to_file_forecasts) 4 5 # correct data types c:\python37-64\lib\site-packages\pandas\io\parsers.py in read_csv(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, skipfooter, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, cache_dates, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, doublequote, escapechar, comment, encoding, dialect, error_bad_lines, warn_bad_lines, delim_whitespace, low_memory, memory_map, float_precision) 686 ) 687

--> 688 return _read(filepath_or_buffer, kwds) 689 690 c:\python37-64\lib\site-packages\pandas\io\parsers.py in _read(filepath_or_buffer, kwds) 435 # See python/mypy#1297 436 fp_or_buf, _, compression, should_close = get_filepath_or_buffer(

--> 437 filepath_or_buffer, encoding, compression 438 ) 439 kwds["compression"] = compression c:\python37-64\lib\site-packages\pandas\io\common.py in get_filepath_or_buffer(filepath_or_buffer, encoding, compression, mode, storage_options) 220 try: 221 file_obj = fsspec.open(

--> 222 filepath_or_buffer, mode=mode or "rb", **(storage_options or {}) 223 ).open() 224 # GH 34626 Reads from Public Buckets without Credentials needs anon=True c:\python37-64\lib\site-packages\fsspec\core.py in open(self) 132 been deleted; but a with-context is better style. 133 """

--> 134 out = self.__enter__() 135 closer = out.close 136 fobjects = self.fobjects.copy()[:-1] c:\python37-64\lib\site-packages\fsspec\core.py in __enter__(self) 100 mode = self.mode.replace("t", "").replace("b", "") + "b" 101

--> 102 f = self.fs.open(self.path, mode=mode) 103 104 self.fobjects = [f] c:\python37-64\lib\site-packages\fsspec\spec.py in open(self, path, mode, block_size, cache_options, **kwargs) 901 autocommit=ac, 902 cache_options=cache_options,

--> 903 **kwargs 904 ) 905 if not ac: c:\python37-64\lib\site-packages\s3fs\core.py in _open(self, path, mode, block_size, acl, version_id, fill_cache, cache_type, autocommit, requester_pays, **kwargs) 313 version_id=version_id, fill_cache=fill_cache, 314 s3_additional_kwargs=kw, cache_type=cache_type,

--> 315 autocommit=autocommit, requester_pays=requester_pays) 316 317 def _lsdir(self, path, refresh=False, max_items=None): c:\python37-64\lib\site-packages\s3fs\core.py in __init__(self, s3, path, mode, block_size, acl, version_id, fill_cache, s3_additional_kwargs, autocommit, cache_type, requester_pays) 955 self.req_kw = {'RequestPayer': 'requester'} if requester_pays else {} 956 super().__init__(s3, path, mode, block_size, autocommit=autocommit,

--> 957 cache_type=cache_type) 958 self.s3 = self.fs # compatibility 959 if self.writable(): c:\python37-64\lib\site-packages\fsspec\spec.py in __init__(self, fs, path, mode, block_size, autocommit, cache_type, cache_options, **kwargs) 1222 if mode == "rb": 1223 if not hasattr(self, "details"): -> 1224 self.details = fs.info(path) 1225 self.size = self.details["size"] 1226 self.cache = caches[cache_type]( c:\python37-64\lib\site-packages\s3fs\core.py in info(self, path, version_id) 484 except ParamValidationError as e: 485 raise ValueError('Failed to head path %r: %s' % (path, e))

--> 486 return super().info(path) 487 488 def isdir(self, path): c:\python37-64\lib\site-packages\fsspec\spec.py in info(self, path, **kwargs) 589 return {"name": path, "size": 0, "type": "directory"} 590 else:

--> 591 raise FileNotFoundError(path) 592 593 def checksum(self, path): FileNotFoundError: shimomut-forecast-notebook/bike_share_open_data/demo/bike_backtest_forecasts.csv Not sure if loading pandas DataFrame from S3 is stable. Why don't we load the data directly from local files without uploading onto S3? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#96 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAAXFRUUZKQJYJ3TNASH2E3SRNVENANCNFSM4UANFW7Q>.

[christy]

Fixed From: Tomonori Shimomura <[email protected]> Reply-To: aws-samples/amazon-forecast-samples <[email protected]> Date: Monday, November 23, 2020 at 11:07 PM To: aws-samples/amazon-forecast-samples <[email protected]> Cc: "Bergman, Christy" <[email protected]>, Author <[email protected]> Subject: Re: [aws-samples/amazon-forecast-samples] Microscope (#96) os.system('touch {}'.format(temp_file)) This is Linux/Mac dependend. On Windows "touch" command doesn't exist. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#96 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAAXFRTA6REXNCI63NG7W4TSRNLNDANCNFSM4UANFW7Q>.