[Feature] Use base image to simplify Dockerfiles & reduce image sizes

[joshuayao]

Priority

P1-Stopper

OS type

Ubuntu

Hardware type

Xeon-GNR

Running nodes

Single Node

Description

Use base image to simplify Dockerfiles & reduce image sizes

Owner: @eero-t @lkk12014402

[eero-t]

@joshuayao Shared base image usage is v1.3 target only for "GenAIExamples" repo application containers.

"GenAIComps" containers (or application UI containers) using shared base image is something for after v1.3 release. Please remove the milestone.

(While app images are almost about identical, that's not the case with GenAIComps images, they need much more careful analysis before it's even clear whether there's any point in shared base image, especially now that the different comps service variants were consolidated to same images.)

[joshuayao]

mages are almost about identical, that's not the case with GenAIComps images, they need much more careful analysis before it's even clear whether there's any point in shared base image, especially now that the different comps service variants were consolidated to same images.)

Hi @eero-t Thank you for your information. I moved this feature to v1.4, and changed the owner to you.

[xiguiw]

Microservice-test (dataprep_redis_multimodal, intel-cpu)

The log shows dataprep_redis_multimodal, intel-cpu, download and install NVida cuda packages.
There should no dependency on Cuda

#8 10.35 Downloading urllib3-2.3.0-py3-none-any.whl.metadata (6.5 kB)
#8 10.36 Collecting markdown-it-py>=2.2.0 (from rich>=13.1.0->docarray[full]->-r /home/user/comps/lvms/src/requirements.txt (line 3))
#8 10.36 Downloading markdown_it_py-3.0.0-py3-none-any.whl.metadata (6.9 kB)
#8 10.38 Collecting pygments<3.0.0,>=2.13.0 (from rich>=13.1.0->docarray[full]->-r /home/user/comps/lvms/src/requirements.txt (line 3))
#8 10.38 Downloading pygments-2.19.1-py3-none-any.whl.metadata (2.5 kB)
#8 10.40 Collecting networkx (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.41 Downloading networkx-3.4.2-py3-none-any.whl.metadata (6.3 kB)
#8 10.42 Collecting jinja2 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.42 Downloading jinja2-3.1.6-py3-none-any.whl.metadata (2.9 kB)
#8 10.43 Collecting nvidia-cuda-nvrtc-cu12==12.4.127 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.43 Downloading nvidia_cuda_nvrtc_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.44 Collecting nvidia-cuda-runtime-cu12==12.4.127 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.44 Downloading nvidia_cuda_runtime_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.45 Collecting nvidia-cuda-cupti-cu12==12.4.127 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.45 Downloading nvidia_cuda_cupti_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
#8 10.46 Collecting nvidia-cudnn-cu12==9.1.0.70 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.46 Downloading nvidia_cudnn_cu12-9.1.0.70-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
#8 10.47 Collecting nvidia-cublas-cu12==12.4.5.8 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.48 Downloading nvidia_cublas_cu12-12.4.5.8-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.48 Collecting nvidia-cufft-cu12==11.2.1.3 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.49 Downloading nvidia_cufft_cu12-11.2.1.3-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.49 Collecting nvidia-curand-cu12==10.3.5.147 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.50 Downloading nvidia_curand_cu12-10.3.5.147-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.50 Collecting nvidia-cusolver-cu12==11.6.1.9 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.51 Downloading nvidia_cusolver_cu12-11.6.1.9-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
#8 10.51 Collecting nvidia-cusparse-cu12==12.3.1.170 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.52 Downloading nvidia_cusparse_cu12-12.3.1.170-py3-none-manylinux2014_x86_64.whl.metadata (1.6 kB)
#8 10.52 Collecting nvidia-cusparselt-cu12==0.6.2 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.53 Downloading nvidia_cusparselt_cu12-0.6.2-py3-none-manylinux2014_x86_64.whl.metadata (6.8 kB)
#8 10.53 Collecting nvidia-nccl-cu12==2.21.5 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.54 Downloading nvidia_nccl_cu12-2.21.5-py3-none-manylinux2014_x86_64.whl.metadata (1.8 kB)
#8 10.54 Collecting nvidia-nvtx-cu12==12.4.127 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.55 Downloading nvidia_nvtx_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.7 kB)
#8 10.55 Collecting nvidia-nvjitlink-cu12==12.4.127 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.56 Downloading nvidia_nvjitlink_cu12-12.4.127-py3-none-manylinux2014_x86_64.whl.metadata (1.5 kB)
#8 10.56 Collecting triton==3.2.0 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.57 Downloading triton-3.2.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.4 kB)
#8 10.58 Collecting sympy==1.13.1 (from torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.58 Downloading sympy-1.13.1-py3-none-any.whl.metadata (12 kB)
#8 10.61 Collecting mpmath<1.4,>=1.1.0 (from sympy==1.13.1->torch>=1.11->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.61 Downloading mpmath-1.3.0-py3-none-any.whl.metadata (8.6 kB)
#8 10.82 Collecting regex!=2019.12.17 (from transformers>=4.29->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.82 Downloading regex-2024.11.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (40 kB)
#8 10.87 Collecting safetensors>=0.4.1 (from transformers>=4.29->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 10.88 Downloading safetensors-0.5.3-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.8 kB)
#8 11.02 Collecting tokenizers<0.21,>=0.20 (from transformers>=4.29->optimum[habana]->-r /home/user/comps/lvms/src/requirements.txt (line 11))
#8 11.02 Downloading tokenizers-0.20.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.7 kB)
#8 11.05 Collecting colorlog (from trimesh[easy]>=3.17.1; extra == "mesh" or extra == "full"->docarray[full]->-r /home/user/comps/lvms/src/requirements.txt (line 3))
#8 11.05 Downloading colorlog-6.9.0-py3-none-any.whl.metadata (10 kB)
#8 11.06 Collecting manifold3d>=2.3.0 (from trimesh[easy]>=3.17.1; extra == "mesh" or extra == "full"->docarray[full]->-r /home/user/comps/lvms/src/requirements.txt (line 3))
#8 23.89 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 37.6/37.6 MB 321.3 MB/s eta 0:00:00
#8 23.92 Downloading tokenizers-0.20.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.0 MB)
#8 23.93 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.0/3.0 MB 274.9 MB/s eta 0:00:00
#8 23.94 Downloading tzdata-2025.2-py2.py3-none-any.whl (347 kB)
#8 23.94 Downloading urllib3-2.3.0-py3-none-any.whl (128 kB)
#8 23.95 Downloading wrapt-1.17.2-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (83 kB)
#8 23.95 Downloading zipp-3.21.0-py3-none-any.whl (9.6 kB)
#8 23.96 Downloading zstandard-0.23.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (5.4 MB)
#8 23.97 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.4/5.4 MB 428.7 MB/s eta 0:00:00
#8 23.98 Downloading colorlog-6.9.0-py3-none-any.whl (11 kB)
#8 23.98 Downloading embreex-2.17.7.post6-cp311-cp311-manylinux_2_28_x86_64.whl (17.2 MB)
#8 24.03 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.2/17.2 MB 422.4 MB/s eta 0:00:00
#8 24.03 Downloading jinja2-3.1.6-py3-none-any.whl (134 kB)
#8 24.03 Downloading jsonschema-4.23.0-py3-none-any.whl (88 kB)
#8 24.04 Downloading lxml-5.3.1-cp311-cp311-manylinux_2_28_x86_64.whl (5.0 MB)
#8 24.06 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 5.0/5.0 MB 302.1 MB/s eta 0:00:00
#8 24.06 Downloading networkx-3.4.2-py3-none-any.whl (1.7 MB)
#8 24.07 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.7/1.7 MB 486.0 MB/s eta 0:00:00
#8 24.08 Downloading opt_einsum-3.4.0-py3-none-any.whl (71 kB)
#8 24.08 Downloading rtree-1.4.0-py3-none-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (541 kB)
#8 24.08 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 541.1/541.1 kB 454.2 MB/s eta 0:00:00
#8 24.09 Downloading shapely-2.0.7-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (2.5 MB)
#8 24.10 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.5/2.5 MB 501.5 MB/s eta 0:00:00
#8 24.10 Downloading svg.path-6.3-py2.py3-none-any.whl (16 kB)
#8 24.11 Downloading vhacdx-0.0.8.post2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (266 kB)
#8 24.11 Downloading jsonschema_specifications-2024.10.1-py3-none-any.whl (18 kB)
#8 24.12 Downloading MarkupSafe-3.0.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (23 kB)
#8 24.12 Downloading mdurl-0.1.2-py3-none-any.whl (10.0 kB)
#8 24.13 Downloading mpmath-1.3.0-py3-none-any.whl (536 kB)
#8 24.13 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 536.2/536.2 kB 447.9 MB/s eta 0:00:00
#8 24.14 Downloading referencing-0.36.2-py3-none-any.whl (26 kB)
#8 24.14 Downloading rpds_py-0.24.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (389 kB)
#8 24.15 Downloading six-1.17.0-py2.py3-none-any.whl (11 kB)
#8 24.15 Downloading psutil-7.0.0-cp36-abi3-manylinux_2_12_x86_64.manylinux2010_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (277 kB)
#8 24.16 Downloading scikit_learn-1.6.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (13.5 MB)
#8 24.19 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 13.5/13.5 MB 416.4 MB/s eta 0:00:00
#8 24.20 Downloading joblib-1.4.2-py3-none-any.whl (301 kB)
#8 24.21 Downloading threadpoolctl-3.6.0-py3-none-any.whl (18 kB)
#8 26.72 Building wheels for collected packages: pycollada
#8 26.72 Building wheel for pycollada (pyproject.toml): started
#8 27.03 Building wheel for pycollada (pyproject.toml): finished with status 'done'
#8 27.03 Created wheel for pycollada: filename=pycollada-0.9-py3-none-any.whl size=128523 sha256=74fac8511010a0181db08f88801e5112d2ab731542aebb979078c6f2d8e30c95
#8 27.03 Stored in directory: /tmp/pip-ephem-wheel-cache-86cw6t9v/wheels/d8/f5/25/96914138081f519b3061d82b1306fccf17ea00e00a4b3f8fb2
#8 27.04 Successfully built pycollada
#8 27.41 Installing collected packages: triton, pytz, pydub, nvidia-cusparselt-cu12, mpmath, zstandard, zipp, xxhash, wrapt, urllib3, tzdata, typing-extensions, types-pillow, tqdm, threadpoolctl, tenacity, sympy, svg.path, sniffio, six, shortuuid, safetensors, rtree, rpds-py, regex, pyyaml, pygments, pyarrow, psutil, protobuf, propcache, prometheus-client, pillow, packaging, orjson, opt_einsum, nvidia-nvtx-cu12, nvidia-nvjitlink-cu12, nvidia-nccl-cu12, nvidia-curand-cu12, nvidia-cufft-cu12, nvidia-cuda-runtime-cu12, nvidia-cuda-nvrtc-cu12, nvidia-cuda-cupti-cu12, nvidia-cublas-cu12, numpy, networkx, mypy-extensions, multidict, mdurl, MarkupSafe, lz4, lxml, jsonpointer, joblib, jiter, idna, h11, grpcio, fsspec, frozenlist, filelock, distro, dill, colorlog, click, charset-normalizer, certifi, av, attrs, annotated-types, aiohappyeyeballs, yarl, vhacdx, uvicorn, typing-inspect, types-requests, trimesh, shapely, scipy, requests, referencing, python-dateutil, pydantic-core, opentelemetry-proto, nvidia-cusparse-cu12

[eero-t]

The log shows dataprep_redis_multimodal, intel-cpu, download and install NVida cuda packages.
There should no dependency on Cuda

Packages:
- nvidia-cublas-cu12
- nvidia-cuda-cupti-cu12
- nvidia-cuda-nvrtc-cu12
- nvidia-cuda-runtime-cu12
- nvidia-cudnn-cu12
- nvidia-cufft-cu12
- nvidia-curand-cu12
- nvidia-cusolver-cu12
- nvidia-cusparse-cu12
- nvidia-cusparselt-cu12
- nvidia-nccl-cu12
- nvidia-nvjitlink-cu12
- nvidia-nvtx-cu12

=> Are required by:
- torch

=> Are required by:
- sentence-transformers

=> Are required by:
- langchain-huggingface

=> For apps, this is due to bug #1465 and goes away once fix to that is in comps-base base image in relevant registries, and latest app images have been rebuilt.

[eero-t]

Microservice-test (dataprep_redis_multimodal, intel-cpu)

The log shows dataprep_redis_multimodal, intel-cpu, download and install NVida cuda packages. There should no dependency on Cuda

Err... The linked log does not show any Nvidia packages?

Note:

If langchain-huggingface and its torch dependency are unavoidable[1], as is the case with dataprep, GPU libraries can be avoided by telling PIP to use CPU version with: PIP_EXTRA_INDEX_URL="https://download.pytorch.org/whl/cpu"

And this is already used by CPU version of the inferencing engines, GenAIComps dataprep, retrievers and OpenVINO vLLM builds, as can be see with git grep PIP_EXTRA_INDEX_URL...

---

[1] Only thing needed from langchain-huggingface would be few small Python files from langchain_huggingface.embeddings, but unfortunately latter is not available as separate module, only as part of langchain-huggingface, which brings in torch.