Description
Description
Problem with using triton_python_backend_stub in new image. Prepared triton_python_backend_stub by command inside nvcr.io/nvidia/tritonserver:25.03-py3
with python3.10.
Copy it to git repo. When trying to start triton-server i get:
I0421 10:08:17.917892 1 python_be.cc:1811] "Using Python execution env /opt/ruclip-venv"
I0421 10:08:17.918016 1 pb_env.cc:267] "Returning canonical path since EXECUTION_ENV_PATH does not contain compressed path. Path: /opt/ruclip-venv"
I0421 10:08:17.918905 1 stub_launcher.cc:385] "Starting Python backend stub: source /opt/ruclip-venv/bin/activate && exec env LD_LIBRARY_PATH=/opt/ruclip-venv/lib:$LD_LIBRARY_PATH /tmp/folderIbyKYQ/triton_python_backend_stub /tmp/folderIbyKYQ/1/model.py triton_python_backend_shm_region_6bd0fcc4-6ea1-49b4-a9f4-041f37b06c1b 1048576 1048576 1 /opt/tritonserver/backends/python 336 ruclip_visual_preprocessing DEFAULT"
/bin/dash: 0: cannot open /tmp/folderIbyKYQ/triton_python_backend_stub: Permission denied
And process stopped.
Triton Information
What version of Triton are you using?
nvcr.io/nvidia/tritonserver:25.03-py3
Are you using the Triton container or did you build it yourself?
Using a container made based on this nvcr.io/nvidia/tritonserver:25.03-py3 with pre-installed apt-get libs and python libs.
To Reproduce
Steps to reproduce the behavior.
- Prepare python_backend_stub in
nvcr.io/nvidia/tritonserver:25.03-py3
.
apt-get update
apt-get install cmake rapidjson-dev libarchive-dev zlib1g-dev -y
apt install software-properties-common -y
add-apt-repository ppa:deadsnakes/ppa
apt install python3.10 -y
apt install python3.10-dev -y
rm /usr/bin/python3
ln -s /usr/bin/python3.10 /usr/bin/python3
git clone https://github.com/triton-inference-server/python_backend -b r25.03
cd python_backend
mkdir build && cd build
cmake -DTRITON_ENABLE_GPU=ON -DTRITON_BACKEND_REPO_TAG=r25.03 -DTRITON_COMMON_REPO_TAG=r25.03 -DTRITON_CORE_REPO_TAG=r25.03 -DCMAKE_INSTALL_PREFIX:PATH=`pwd`/install ..
make triton-python-backend-stub
- Copy binary triton-python-backend-stub to working git repo with models.
- Prepare working image of triton with python3.10-venv
...
RUN apt-get update \
&& apt-get install --no-install-recommends \
libsndfile1=1.2.2-1ubuntu5.24.04.1 \
ffmpeg=7:6.1.1-3ubuntu5 \
python3.10-venv=3.10.17-1+noble1 \
python3.10-dev=3.10.17-1+noble1 \
-y \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
# Создаем виртуальное окружение venv в указанной директории (например, /opt/venv1)
ENV VENV_DIR=/opt/special-venv
RUN python3.10 -m venv $VENV_DIR
RUN $VENV_DIR/bin/pip install --no-cache-dir --upgrade pip \
&& $VENV_DIR/bin/pip install --no-cache-dir wheel \
&& $VENV_DIR/bin/pip install --no-cache-dir cython==3.0.12 \
...
-
Starting model with config:
parameters: {
key: "EXECUTION_ENV_PATH",
value: {string_value: "/opt/special-venv"}
} -
Start tritonserver with this model with "--log-verbose=3", get an error:
I0421 10:08:17.917892 1 python_be.cc:1811] "Using Python execution env /opt/ruclip-venv"
I0421 10:08:17.918016 1 pb_env.cc:267] "Returning canonical path since EXECUTION_ENV_PATH does not contain compressed path. Path: /opt/ruclip-venv"
I0421 10:08:17.918905 1 stub_launcher.cc:385] "Starting Python backend stub: source /opt/ruclip-venv/bin/activate && exec env LD_LIBRARY_PATH=/opt/ruclip-venv/lib:$LD_LIBRARY_PATH /tmp/folderIbyKYQ/triton_python_backend_stub /tmp/folderIbyKYQ/1/model.py triton_python_backend_shm_region_6bd0fcc4-6ea1-49b4-a9f4-041f37b06c1b 1048576 1048576 1 /opt/tritonserver/backends/python 336 ruclip_visual_preprocessing DEFAULT"
/bin/dash: 0: cannot open /tmp/folderIbyKYQ/triton_python_backend_stub: Permission denied
Describe the models (framework, inputs, outputs), ideally include the model configuration file (if using an ensemble include the model configuration file for that as well).
Expected behavior
A clear and concise description of what you expected to happen.
Starting model with custom python_backend_stub