Import error for pyarrow when Ray built from source

[devin-petersohn]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu
• Ray installed from (source or binary): source
• Ray version: source
• Python version: 3.5
• Exact command to reproduce: from ray.pyarrow._parquet import (ParquetReader, FileMetaData)

Describe the problem

Import error on importing parquet from pyarrow. See full error below.

Source code / logs

ImportError: /data/devin/ray/python/ray/pyarrow_files/pyarrow/libparquet.so.1: undefined symbol: _ZNK5arrow6Status8ToStringB5cxx11Ev

[pcmoritz]

This is because of 4c82ac7. The _GLIBCXX_USE_CXX11_ABI=0 flag is not applied to the parquet build currently.

This is starting to affect more and more dependencies which is worriesome, maybe we need to figure out a plan to keep the new ABI here.

[robertnishihara]

This was never resolved, right? It came up recently when trying to compile the master.

@devin-petersohn did you have a workaround or some way of fixing this? Is this easily reproducible? Somehow I've never seen it.

@pcmoritz now that parquet-cpp is part of Arrow, would updating Arrow potentially make this problem go away?

cc @vengroff

[pethor]

I have the same issue when I try to compile the current master branch.
@robertnishihara did you find a workaround for this issue?
@devin-petersohn does the problem remain for you in Ray release 0.5.3?

[devin-petersohn]

@robertnishihara @pethor I am getting a different error on 0.5.3:

In [1]: import ray

In [2]: ray.__version__
Out[2]: '0.5.3'

In [3]: from ray.pyarrow._parquet import ParquetReader
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-3-481ff70270a9> in <module>()
----> 1 from ray.pyarrow._parquet import ParquetReader

ImportError: No module named 'ray.pyarrow'

In [4]: import ray.pyarrow
---------------------------------------------------------------------------
ImportError                               Traceback (most recent call last)
<ipython-input-4-33b14d007198> in <module>()
----> 1 import ray.pyarrow

ImportError: No module named 'ray.pyarrow'

As a workaround, we just use the user's installed Arrow library. This leads to some issues with deprecation, etc., but we are able to manage it.

[robertnishihara]

@devin-petersohn I suspect that doesn't actually use the user's installed pyarrow, because import ray modifies the Python path to make Ray's copy of pyarrow first, see

ray/python/ray/__init__.py

Lines 8 to 17 in 8aa7365

	if "pyarrow" in sys.modules:
	raise ImportError("Ray must be imported before pyarrow because Ray "
	"requires a specific version of pyarrow (which is "
	"packaged along with Ray).")
	# Add the directory containing pyarrow to the Python path so that we find the
	# pyarrow version packaged with ray and not a pre-existing pyarrow.
	pyarrow_path = os.path.join(
	os.path.abspath(os.path.dirname(__file__)), "pyarrow_files")
	sys.path.insert(0, pyarrow_path)

e.g., you can do pip uninstall pyarrow and that shouldn't change anything.

[devin-petersohn]

On release 0.5.3, I do not see the issue. I am closing this, but will reopen in the case others are experiencing this on 0.5.3. It seems to have been resolved.

[robertnishihara]

@vengroff do you still see this issue?

[pethor]

@robertnishihara I still have a very similar issue, I will open a new issue

[rjurney]

I am seeing this error on the AWS Ubuntu Deep Learning AMI with modin-0.5.4-py3 on Anaconda python 3.6.7. I can't figure a way around this. Anyone?

[devin-petersohn]

Hi @rjurney, is the error identical to the message above? Would you be able to post the error message?

I will try to reproduce the error on the AMI you mentioned, but if you have access to the error message, that would also be helpful.

The worrisome issue here is that this has happened from installing via pip, which has not been happening in the past.

[devin-petersohn]

I spun the AMI you mentioned and everything seems to be working fine. If you have access to the error message @rjurney or if you have more information that would be very helpful.