Skip to content

Commit 309c2da

Browse files
Add preload env in run.sh to fix xft-vllm perf issue (#204)
* add preload env in run.sh Signed-off-by: lvliang-intel <[email protected]> * update comments Signed-off-by: lvliang-intel <[email protected]> --------- Signed-off-by: lvliang-intel <[email protected]>
1 parent 7589142 commit 309c2da

File tree

1 file changed

+3
-1
lines changed
  • comps/llms/text-generation/vllm-xft

1 file changed

+3
-1
lines changed

comps/llms/text-generation/vllm-xft/run.sh

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,9 @@
33
# Copyright (C) 2024 Intel Corporation
44
# SPDX-License-Identifier: Apache-2.0
55

6-
# Preload libiomp5.so by following cmd or LD_PRELOAD=libiomp5.so manually
6+
# Preloading libiomp5.so is essential for optimal performance.
7+
# libiomp5.so is the Intel OpenMP runtime library, providing parallel computation support,
8+
# thread management, task scheduling, and performance optimization on Intel X86 platforms.
79
export $(python -c 'import xfastertransformer as xft; print(xft.get_env())')
810

911
# convert the model to fastertransformer format

0 commit comments

Comments
 (0)