Skip to content

Commit fa71498

Browse files
authored
dev: add pyarrow to Dockerfile to suppress error (#2174)
<!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Adding `pyiceberg["pyarrow"]` dep to the Dockefile resolves these warning log lines: ``` Could not initialize FileIO: pyiceberg.io.pyarrow.PyArrowFileIO Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/pyiceberg/io/__init__.py", line 317, in _import_file_io module = importlib.import_module(module_name) File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module return _bootstrap._gcd_import(name[level:], package, level) File "<frozen importlib._bootstrap>", line 1030, in _gcd_import File "<frozen importlib._bootstrap>", line 1007, in _find_and_load File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked File "<frozen importlib._bootstrap>", line 680, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 850, in exec_module File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed File "/usr/local/lib/python3.9/site-packages/pyiceberg/io/pyarrow.py", line 62, in <module> import pyarrow as pa ModuleNotFoundError: No module named 'pyarrow' ``` I also took the liberty to upgrade the Dockerfile python version from 3.9 to 3.12 # Are these changes tested? # Are there any user-facing changes? <!-- In the case of user-facing changes, please add the changelog label. -->
1 parent d48a08d commit fa71498

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

dev/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@
1313
# See the License for the specific language governing permissions and
1414
# limitations under the License.
1515

16-
FROM python:3.9-bullseye
16+
FROM python:3.12-bullseye
1717

1818
RUN apt-get -qq update && \
1919
apt-get -qq install -y --no-install-recommends \
@@ -63,7 +63,7 @@ RUN chmod u+x /opt/spark/sbin/* && \
6363

6464
RUN pip3 install -q ipython
6565

66-
RUN pip3 install "pyiceberg[s3fs,hive]==${PYICEBERG_VERSION}"
66+
RUN pip3 install "pyiceberg[s3fs,hive,pyarrow]==${PYICEBERG_VERSION}"
6767

6868
COPY entrypoint.sh .
6969
COPY provision.py .

0 commit comments

Comments
 (0)