Skip to content

Installation Issue #527

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ginxyang opened this issue Apr 15, 2022 · 9 comments
Closed

Installation Issue #527

ginxyang opened this issue Apr 15, 2022 · 9 comments

Comments

@ginxyang
Copy link

ginxyang commented Apr 15, 2022

Many installation issues are resolved by running the following in a fresh R session (you can restart R in Rstudio with Ctrl+Shift+F10) :

# install the development version of packages, in case the
# issue is already fixed but not on CRAN yet.
install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

Test to see if installation was successful.

tensorflow::as_tensor("Hello World")

If the above snippet succeeded and you saw something like tf.Tensor(b'Hello World', shape=(), dtype=string), then 🎉, you've successfully installed Tensorflow.

If the above installation failed, please gather some diagnostic info:

reticulate::py_config()
tensorflow::tf_config()
reticulate::import("tensorflow")
reticulate::py_last_error()
sessionInfo()

Please copy and paste the FULL OUTPUT of running all three snippets, and be sure to enclose the output lines with three backticks (```) for monospace formatting.

output for installation:

> install.packages("remotes")
--- Please select a CRAN mirror for use in this session ---
trying URL 'https://mirror.las.iastate.edu/CRAN/bin/macosx/contrib/4.1/remotes_2.4.2.tgz'
Content type 'application/x-gzip' length 395393 bytes (386 KB)
==================================================
downloaded 386 KB


The downloaded binary packages are in
	/var/folders/ly/wcdn4jr97v38s1vsvcknvtlm0000gn/T//RtmplLom6Y/downloaded_packages

> remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
Downloading GitHub repo rstudio/reticulate@HEAD
   checking DESCRIPTION meta-information ...y/wcdn4jr97v38s1vsvcknvtlm0000gn/T/RtmplLom6Y/remotes2f4922488cf6/rstudio-reticulate-1a62650/DESCRIPTION’ ...
* installing *source* package ‘reticulate’ ...
** using staged installation
** libs
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c RcppExports.cpp -o RcppExports.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c event_loop.cpp -o event_loop.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c libpython.cpp -o libpython.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c output.cpp -o output.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c python.cpp -o python.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c readline.cpp -o readline.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -I"/Library/Frameworks/R.framework/Resources/include" -DNDEBUG  -I'/Library/Frameworks/R.framework/Versions/4.1/Resources/library/Rcpp/include' -I/usr/local/include   -fPIC  -Wall -g -O2  -c signals.cpp -o signals.o
clang++ -mmacosx-version-min=10.13 -std=gnu++14 -dynamiclib -Wl,-headerpad_max_install_names -undefined dynamic_lookup -single_module -multiply_defined suppress -L/Library/Frameworks/R.framework/Resources/lib -L/usr/local/lib -o reticulate.so RcppExports.o event_loop.o libpython.o output.o python.o readline.o signals.o -F/Library/Frameworks/R.framework/.. -framework R -Wl,-framework -Wl,CoreFoundation
installing to /Library/Frameworks/R.framework/Versions/4.1/Resources/library/00LOCK-reticulate/00new/reticulate/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
*** copying figures
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (reticulate)
Downloading GitHub repo rstudio/tensorflow@HEAD
   checking DESCRIPTION meta-information ...y/wcdn4jr97v38s1vsvcknvtlm0000gn/T/RtmplLom6Y/remotes2f4975076298/rstudio-tensorflow-6e3e00e/DESCRIPTION’ ...
* installing *source* package ‘tensorflow’ ...
** using staged installation
** R
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (tensorflow)
Downloading GitHub repo rstudio/keras@HEAD
✔  checking for file ‘/private/var/folders/ly/wcdn4jr97v38s1vsvcknvtlm0000gn/T/RtmplLom6Y/remotes2f4944f8c19f/rstudio-keras-0cc377f/DESCRIPTION’ ...
   checking DESCRIPTION meta-information ...
* installing *source* package ‘keras’ ...
** using staged installation
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (keras)

* Miniconda has been successfully installed at "~/Library/r-miniconda".
[1] "/Users/misosoup/Library/r-miniconda"

> keras::install_keras()
+ . /Users/misosoup/Library/r-miniconda/bin/activate
+ conda activate '/Users/misosoup/Library/r-miniconda/envs/r-reticulate'
+ '/Users/misosoup/Library/r-miniconda/envs/r-reticulate/bin/python' -m pip install --upgrade --ignore-installed 'tensorflow==2.8.*' 'tensorflow-hub' 'scipy' 'requests' 'Pillow' 'h5py' 'pandas' 'pydot'
Collecting tensorflow==2.8.*
  Using cached tensorflow-2.8.0-cp38-cp38-macosx_10_14_x86_64.whl (217.4 MB)
Collecting tensorflow-hub
  Using cached tensorflow_hub-0.12.0-py2.py3-none-any.whl (108 kB)
Collecting scipy
  Using cached scipy-1.8.0-cp38-cp38-macosx_12_0_universal2.macosx_10_9_x86_64.whl (55.3 MB)
Collecting requests
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Collecting Pillow
  Using cached Pillow-9.1.0-cp38-cp38-macosx_10_9_x86_64.whl (3.1 MB)
Collecting h5py
  Using cached h5py-3.6.0-cp38-cp38-macosx_10_9_x86_64.whl (3.1 MB)
Collecting pandas
  Using cached pandas-1.4.2-cp38-cp38-macosx_10_9_x86_64.whl (11.0 MB)
Collecting pydot
  Using cached pydot-1.4.2-py2.py3-none-any.whl (21 kB)
Collecting numpy>=1.20
  Using cached numpy-1.22.3-cp38-cp38-macosx_10_14_x86_64.whl (17.6 MB)
Collecting typing-extensions>=3.6.6
  Using cached typing_extensions-4.1.1-py3-none-any.whl (26 kB)
Collecting absl-py>=0.4.0
  Using cached absl_py-1.0.0-py3-none-any.whl (126 kB)
Collecting tensorboard<2.9,>=2.8
  Using cached tensorboard-2.8.0-py3-none-any.whl (5.8 MB)
Collecting setuptools
  Using cached setuptools-62.1.0-py3-none-any.whl (1.1 MB)
Collecting termcolor>=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting flatbuffers>=1.12
  Using cached flatbuffers-2.0-py2.py3-none-any.whl (26 kB)
Collecting opt-einsum>=2.3.2
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting keras<2.9,>=2.8.0rc0
  Using cached keras-2.8.0-py2.py3-none-any.whl (1.4 MB)
Collecting gast>=0.2.1
  Using cached gast-0.5.3-py3-none-any.whl (19 kB)
Collecting six>=1.12.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting astunparse>=1.6.0
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting libclang>=9.0.1
  Using cached libclang-13.0.0-py2.py3-none-macosx_10_9_x86_64.whl (13.0 MB)
Collecting protobuf>=3.9.2
  Using cached protobuf-3.20.0-cp38-cp38-macosx_10_9_x86_64.whl (962 kB)
Collecting tensorflow-io-gcs-filesystem>=0.23.1
  Using cached tensorflow_io_gcs_filesystem-0.24.0-cp38-cp38-macosx_10_14_x86_64.whl (1.6 MB)
Collecting tf-estimator-nightly==2.8.0.dev2021122109
  Using cached tf_estimator_nightly-2.8.0.dev2021122109-py2.py3-none-any.whl (462 kB)
Collecting wrapt>=1.11.0
  Using cached wrapt-1.14.0-cp38-cp38-macosx_10_9_x86_64.whl (35 kB)
Collecting google-pasta>=0.1.1
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting grpcio<2.0,>=1.24.3
  Using cached grpcio-1.44.0-cp38-cp38-macosx_10_10_x86_64.whl (4.3 MB)
Collecting keras-preprocessing>=1.1.1
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting pytz>=2020.1
  Using cached pytz-2022.1-py2.py3-none-any.whl (503 kB)
Collecting python-dateutil>=2.8.1
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting pyparsing>=2.1.4
  Using cached pyparsing-3.0.8-py3-none-any.whl (98 kB)
Collecting wheel<1.0,>=0.23.0
  Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
Collecting markdown>=2.6.8
  Using cached Markdown-3.3.6-py3-none-any.whl (97 kB)
Collecting werkzeug>=0.11.15
  Using cached Werkzeug-2.1.1-py3-none-any.whl (224 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Using cached tensorboard_data_server-0.6.1-py3-none-macosx_10_9_x86_64.whl (3.5 MB)
Collecting tensorboard-plugin-wit>=1.6.0
  Using cached tensorboard_plugin_wit-1.8.1-py3-none-any.whl (781 kB)
Collecting google-auth<3,>=1.6.3
  Using cached google_auth-2.6.5-py2.py3-none-any.whl (156 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting cachetools<6.0,>=2.0.0
  Using cached cachetools-5.0.0-py3-none-any.whl (9.1 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.8-py3-none-any.whl (39 kB)
Collecting requests-oauthlib>=0.7.0
  Using cached requests_oauthlib-1.3.1-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata>=4.4
  Using cached importlib_metadata-4.11.3-py3-none-any.whl (18 kB)
Collecting zipp>=0.5
  Using cached zipp-3.8.0-py3-none-any.whl (5.4 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting oauthlib>=3.0.0
  Using cached oauthlib-3.2.0-py3-none-any.whl (151 kB)
Installing collected packages: tf-estimator-nightly, termcolor, tensorboard-plugin-wit, pytz, pyasn1, libclang, keras, flatbuffers, certifi, zipp, wrapt, wheel, werkzeug, urllib3, typing-extensions, tensorflow-io-gcs-filesystem, tensorboard-data-server, six, setuptools, rsa, pyparsing, pyasn1-modules, protobuf, Pillow, oauthlib, numpy, idna, gast, charset-normalizer, cachetools, tensorflow-hub, scipy, requests, python-dateutil, pydot, opt-einsum, keras-preprocessing, importlib-metadata, h5py, grpcio, google-pasta, google-auth, astunparse, absl-py, requests-oauthlib, pandas, markdown, google-auth-oauthlib, tensorboard, tensorflow
Successfully installed Pillow-9.1.0 absl-py-1.0.0 astunparse-1.6.3 cachetools-5.0.0 certifi-2021.10.8 charset-normalizer-2.0.12 flatbuffers-2.0 gast-0.5.3 google-auth-2.6.5 google-auth-oauthlib-0.4.6 google-pasta-0.2.0 grpcio-1.44.0 h5py-3.6.0 idna-3.3 importlib-metadata-4.11.3 keras-2.8.0 keras-preprocessing-1.1.2 libclang-13.0.0 markdown-3.3.6 numpy-1.22.3 oauthlib-3.2.0 opt-einsum-3.3.0 pandas-1.4.2 protobuf-3.20.0 pyasn1-0.4.8 pyasn1-modules-0.2.8 pydot-1.4.2 pyparsing-3.0.8 python-dateutil-2.8.2 pytz-2022.1 requests-2.27.1 requests-oauthlib-1.3.1 rsa-4.8 scipy-1.8.0 setuptools-62.1.0 six-1.16.0 tensorboard-2.8.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.1 tensorflow-2.8.0 tensorflow-hub-0.12.0 tensorflow-io-gcs-filesystem-0.24.0 termcolor-1.1.0 tf-estimator-nightly-2.8.0.dev2021122109 typing-extensions-4.1.1 urllib3-1.26.9 werkzeug-2.1.1 wheel-0.37.1 wrapt-1.14.0 zipp-3.8.0

Installation complete.

for testing tf

 *** caught illegal operation ***
address 0x1238ba489, cause 'illegal opcode'

Traceback:
 1: py_module_import(module, convert = convert)
 2: import(module)
 3: doTryCatch(return(expr), name, parentenv, handler)
 4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 5: tryCatchList(expr, classes, parentenv, handlers)
 6: tryCatch(import(module), error = clear_error_handler())
 7: py_resolve_module_proxy(x)
 8: `$.python.builtin.module`(tf, "convert_to_tensor")
 9: tf$convert_to_tensor
10: as_tensor.default("Hello World")
11: tensorflow::as_tensor("Hello World")

for diagnose:

> reticulate::py_config()
python:         /Users/misosoup/Library/r-miniconda/envs/r-reticulate/bin/python
libpython:      /Users/misosoup/Library/r-miniconda/envs/r-reticulate/lib/libpython3.8.dylib
pythonhome:     /Users/misosoup/Library/r-miniconda/envs/r-reticulate:/Users/misosoup/Library/r-miniconda/envs/r-reticulate
version:        3.8.13 | packaged by conda-forge | (default, Mar 25 2022, 06:05:47)  [Clang 12.0.1 ]
numpy:          /Users/misosoup/Library/r-miniconda/envs/r-reticulate/lib/python3.8/site-packages/numpy
numpy_version:  1.22.3
> tensorflow::tf_config()

 *** caught illegal operation ***
address 0x114d2e489, cause 'illegal opcode'

Traceback:
 1: py_module_import(module, convert = convert)
 2: import("tensorflow", delay_load = list(priority = 5, environment = "r-reticulate",     on_load = function() {        register_suppress_warnings_handler(list(suppress = function() {            if (tf_v2()) {                tf_logger <- tf$get_logger()                logging <- reticulate::import("logging")                old_verbosity <- tf_logger$level                tf_logger$setLevel(logging$ERROR)                old_verbosity            } else {                old_verbosity <- tf$logging$get_verbosity()                tf$logging$set_verbosity(tf$logging$ERROR)                old_verbosity            }        }, restore = function(context) {            if (tf_v2()) {                tf_logger <- tf$get_logger()                tf_logger$setLevel(context)            } else {                tf$logging$set_verbosity(context)            }        }))        register_tf_help_handler()        tryCatch(tf$python$util$deprecation$silence()$`__enter__`(),             error = function(e) NULL)        emit <- get("packageStartupMessage")        emit("Loaded Tensorflow version ", tf$version$VERSION)    }, on_error = function(e) {        stop(tf_config_error_message(), call. = FALSE)    }))
 3: fun(libname, pkgname)
 4: doTryCatch(return(expr), name, parentenv, handler)
 5: tryCatchOne(expr, names, parentenv, handlers[[1L]])
 6: tryCatchList(expr, classes, parentenv, handlers)
 7: tryCatch(fun(libname, pkgname), error = identity)
 8: runHook(".onLoad", env, package.lib, package)
 9: loadNamespace(x)
 *** caught illegal operation ***
address 0x1222eb489, cause 'illegal opcode'

Traceback:
 1: py_module_import(module, convert = convert)
 2: reticulate::import("tensorflow")
> reticulate::py_last_error()
NULL
> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.0.1

Matrix products: default
BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] tensorflow_2.8.0.9000
@t-kalinowski
Copy link
Member

t-kalinowski commented Apr 15, 2022

Hi, can you please include the full output from running the installation commands?

install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

@ginxyang
Copy link
Author

Hi, can you please include the full output from running the installation commands?

install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

Sure thing, But I have already have all these packages downloaded, should I force a re-download?

@t-kalinowski
Copy link
Member

Yes please :)

@ginxyang
Copy link
Author

Yes please :)

any following up on this? I have updated the output on the original post

@t-kalinowski
Copy link
Member

Thanks, I didn't see that you had edited the original post.

It looks like you're running R under rosetta, which the pre-built Tensorflow wheels are not compatible with.

The easiest solution is to install and use the arm64 version of R. Link

@ginxyang
Copy link
Author

Thanks, I didn't see that you had edited the original post.

It looks like you're running R under rosetta, which the pre-built Tensorflow wheels are not compatible with.

The easiest solution is to install and use the arm64 version of R. Link

Awesome, issue resolved. Thank you so much for your help!

@jcutlerbiostats
Copy link

I'm having the same problem. I'm on a MacBook M1 I bought this year, so it's not and old computer. I tried fixing the problem by running this:

remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

Everything installed fine. I then tried running dataset_mnist() and got the following error (in R, not in RStudio):

library(keras)
mnist <- dataset_mnist()

*** caught illegal operation ***
address 0x12dcd06d9, cause 'illegal opcode'

Traceback:
1: py_module_import(module, convert = convert)
2: import(module)
3: doTryCatch(return(expr), name, parentenv, handler)
4: tryCatchOne(expr, names, parentenv, handlers[[1L]])
5: tryCatchList(expr, classes, parentenv, handlers)
6: tryCatch(import(module), error = clear_error_handler())
7: py_resolve_module_proxy(x)
8: $.python.builtin.module(keras, "datasets")
9: keras$datasets
10: dataset_mnist()

Possible actions:
1: abort (with core dump, if enabled)
2: normal R exit
3: exit R without saving workspace
4: exit R saving workspace

I can open a new issue. I am just wondering though if you think I should install the arm64 version of R. I think I already have it. This is what R says when I open it:

R version 4.1.2 (2021-11-01) -- "Bird Hippie"
Copyright (C) 2021 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)

Thanks in advance for any help! I really hope I can get this to work.

@t-kalinowski
Copy link
Member

Platform: x86_64-apple-darwin17.0 (64-bit)

You need to install the arm64 (m1) version of R, not x86 (intel). You can find more detailed instructions here: https://tensorflow.rstudio.com/install/local_gpu (macOS pane)

Once you're installed the correct version of R, the code you ran above to install Keras should work without issue.

@jcutlerbiostats
Copy link

Awesome! I just installed the arm64, and now my R version says:

R version 4.2.1 (2022-06-23) -- "Funny-Looking Kid"
Copyright (C) 2022 The R Foundation for Statistical Computing
Platform: aarch64-apple-darwin20 (64-bit)

I then re-ran this:

install.packages("remotes")
remotes::install_github(sprintf("rstudio/%s", c("reticulate", "tensorflow", "keras")))
reticulate::miniconda_uninstall() # start with a blank slate
reticulate::install_miniconda()
keras::install_keras()

library(keras)

mnist <- dataset_mnist()

And it worked!! Thank you so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants