Skip to content

ERNIE-4.5-VL-424B-A47B-Paddle加载卡住不动 #2723

Open
@zjbshk

Description

@zjbshk
Image
$python -m fastdeploy.entrypoints.openai.api_server \
>     --model ERNIE-4.5-VL-424B-A47B-Paddle \
>     --port 8180 \
>     --tensor-parallel-size 8 \
>     --quantization wint4 \
>     --metrics-port 8181 \
>     --engine-worker-queue-port 8182 \
>     --max-model-len 32768 \
>     --max-num-seqs 32 \
>     --mm-processor-kwargs '{"video_max_frames": 30}' \
>     --limit-mm-per-prompt '{"image": 10, "video": 3}' \
>     --reasoning-parser ernie-45-vl \
>     --enable-mm

环境:
paddlepaddle-gpu==3.1.0
fastdeploy-gpu==2.0.0

ERNIE-4.5-VL-28B-A3B-Paddle模型可以起来,换成 ERNIE-4.5-VL-424B-A47B-Paddle模型总是中途卡住,在不同的进度位置

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions