Skip to content

Update docs for 3.0rc #7099

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Mar 14, 2025
Merged

Update docs for 3.0rc #7099

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
4fe19e6
Update docs
nepeplwu Mar 13, 2025
15dbb8c
Update docs
nepeplwu Mar 13, 2025
9a63907
Update docs
nepeplwu Mar 13, 2025
0a345c9
Update mlu doc
nepeplwu Mar 13, 2025
bd0c6b3
Update xpu docs
nepeplwu Mar 13, 2025
d09a4cd
Update p800 docs
nepeplwu Mar 13, 2025
a436267
Update docs
nepeplwu Mar 13, 2025
a38618d
Fix typo
nepeplwu Mar 13, 2025
2a785b5
Update npu docker image
nepeplwu Mar 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 58 additions & 31 deletions docs/hardware_support/dcu/install_cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,22 +14,50 @@

## 运行环境准备

推荐使用飞桨官方发布的海光 DCU 开发镜像，该镜像预装有海光 DCU 基础运行环境库（DTK）。
您可以基于 docker、pip、源码等不同方式准备飞桨开发环境

### 基于 Docker 的方式（推荐）

我们推荐使用飞桨官方发布的海光 DCU 开发镜像，该镜像预装有海光 DCU 基础运行环境库（DTK）和飞桨 3.0rc 版本的 SDK。

```bash
# 拉取镜像
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:dtk23.10.1-kylinv10-gcc73-py310
docker pull ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:3.0.0rc1-dtk24.04.1-kylinv10-gcc82-py310
```

# 参考如下命令，启动容器
docker run -it --name paddle-dcu-dev -v `pwd`:/work \
```bash
# 启动容器
docker run -it --name paddle-dcu-dev -v $(pwd):/work \
-w=/work --shm-size=128G --network=host --privileged \
--cap-add=SYS_PTRACE --security-opt seccomp=unconfined \
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:dtk23.10.1-kylinv10-gcc73-py310 /bin/bash
ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:3.0.0rc1-dtk24.04.1-kylinv10-gcc82-py310 /bin/bash
```

#### 选项说明及可调整参数

##### ① `--name paddle-dcu-dev`
- **作用**：指定容器名称。
- **可调整**：
- 用户可改为其他名称，例如 `paddle-dcu-test`，方便区分不同实验。

##### ② `-v $(pwd):/work`
- **作用**：挂载本地目录到容器内 `/work` 目录。
- **可调整**：
- 可以修改 `$(pwd)` 为实际路径，例如 `-v /data/projects:/work`，让容器访问宿主机的数据。

# 检查容器内是否可以正常识别海光 DCU 设备
##### ③ `--shm-size=128G`
- **作用**：设置共享内存大小，影响数据处理和计算效率。
- **可调整**：
- 若内存有限，可降低，如 `--shm-size=32G`，但可能影响大规模训练。
- 若训练任务需要更大共享内存，可提高，如 `--shm-size=256G`。

```bash
# 检查容器内是否正常识别海光 DCU 设备
rocm-smi
```

# 预期得到输出如下
```bash
# 预期输出
============System Management Interface ============
====================================================
DCU Temp AvgPwr Fan Perf PwrCap VRAM% DCU%
Expand All @@ -41,26 +69,18 @@ DCU Temp AvgPwr Fan Perf PwrCap VRAM% DCU%
===================End of SMI Log===================
```

## 安装飞桨框架

**注意**：飞桨框架 DCU 版仅支持海光 C86 架构。

### 安装方式一：wheel 包安装

在启动的 docker 容器中，下载并安装飞桨官网发布的 wheel 包。
### 基于 pip 安装的方式

```bash
# 下载并安装 wheel 包
pip install paddlepaddle-rocm -i https://www.paddlepaddle.org.cn/packages/nightly/dcu
python -m pip install paddlepaddle-dcu==3.0.0rc1 -i https://www.paddlepaddle.org.cn/packages/stable/dcu/
```

### 安装方式二：源代码编译安装

在启动的 docker 容器中，下载 Paddle 源码并编译，CMAKE 编译选项含义请参见[编译选项表](https://www.paddlepaddle.org.cn/documentation/docs/zh/develop/install/Tables.html#Compile)。
### 基于源码编译的方式

```bash
# 下载 Paddle 源码
git clone https://github.com/PaddlePaddle/Paddle.git -b develop
git clone https://github.com/PaddlePaddle/Paddle.git -b release/3.0-rc
cd Paddle

# 创建编译目录
Expand All @@ -76,35 +96,42 @@ cmake .. -DCMAKE_EXPORT_COMPILE_COMMANDS=ON -DCMAKE_CXX_FLAGS="-Wno-error -w" \
make -j16

# 编译产出在 build/python/dist/ 路径下，使用 pip 安装即可
pip install -U paddlepaddle_rocm-0.0.0-cp310-cp310-linux_x86_64.whl
python -m pip install -U paddlepaddle_dcu-*-linux_x86_64.whl
```

## 基础功能检查

安装完成后，在 docker 容器中输入如下命令进行飞桨基础健康功能的检查。
输入如下命令进行飞桨基础健康功能的检查。

```bash
# 检查当前安装版本
python -c "import paddle; paddle.version.show()"
```
```bash
# 预期得到输出如下
commit: d37bd8bcf75cf51f6c1117526f3f67d04946ebb9
full_version: 3.0.0-rc1
major: 3
minor: 0
patch: 0-rc1
rc: 0
cuda: False
cudnn: False
nccl: 0

xpu_xre: False
xpu_xccl: False
xpu_xhpc: False
cinn: False
tensorrt_version: None
cuda_archs: []
```
```bash
# 飞桨基础健康检查
python -c "import paddle; paddle.utils.run_check()"
```
```bash
# 预期得到输出如下
Running verify PaddlePaddle program ...
PaddlePaddle works well on 1 GPU.
PaddlePaddle works well on 8 GPUs.
PaddlePaddle is installed successfully! Let's start deep learning with PaddlePaddle now.
```

## 如何卸载

请使用以下命令卸载：

```bash
pip uninstall paddlepaddle-rocm
```
14 changes: 2 additions & 12 deletions docs/hardware_support/dcu/paddle_tutorial_cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,9 @@

* 考虑到环境差异性，我们推荐使用教程提供的标准镜像完成环境准备：

* 镜像链接：ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle-dcu:dtk24.04.1-kylinv10-gcc82
* 镜像链接：ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:3.0.0rc1-dtk24.04.1-kylinv10-gcc82-py310

### 环境安装

安装 PaddlePaddle

*该命令会自动安装飞桨主框架每日自动构建的 nightly-build 版本*

*由于 dcu 代码位于飞桨主框架中，因此我们不需要安装额外的 Custom Device 包*

```shell
python -m pip install --pre paddlepaddle-dcu -i https://www.paddlepaddle.org.cn/packages/nightly/dcu/
```
* 镜像中默认装有 3.0rc 版本的 PaddlePaddle

## 二、运行示例

Expand Down
28 changes: 10 additions & 18 deletions docs/hardware_support/dcu/paddlex_tutorial_cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,19 +8,11 @@

* 考虑到环境差异性，我们推荐使用教程提供的标准镜像完成环境准备：

* 镜像链接：ccr-2vdh3abv-pub.cnc.bj.baidubce.com/paddlepaddle/paddle-dcu:dtk24.04.1-kylinv10-gcc82
* 镜像链接：ccr-2vdh3abv-pub.cnc.bj.baidubce.com/device/paddle-dcu:3.0.0rc1-dtk24.04.1-kylinv10-gcc82-py310

### 环境安装

1. 安装 PaddlePaddle

*该命令会自动安装飞桨主框架每日自动构建的 nightly-build 版本*

*由于 dcu 代码位于飞桨主框架中，因此我们不需要安装额外的 Custom Device 包*

```shell
python -m pip install --pre paddlepaddle-dcu -i https://www.paddlepaddle.org.cn/packages/nightly/dcu/
```
1. 镜像中默认装有 3.0rc 版本的 PaddlePaddle，无需额外安装

2. 安装 PaddleX 代码库

Expand Down Expand Up @@ -71,7 +63,7 @@ tar -xf ./dataset/cls_flowers_examples.tar -C ./dataset/

```shell
# PaddleX 支持对数据集进行校验，确保数据集格式符合 PaddleX 的相关要求。同时在数据校验时，能够对数据集进行分析，统计数据集的基本信息。
python main.py -c paddlex/configs/image_classification/ResNet50.yaml \
python main.py -c paddlex/configs/modules/image_classification/ResNet50.yaml \
-o Global.mode=check_dataset \
-o Global.dataset_dir=./dataset/cls_flowers_examples

Expand All @@ -86,10 +78,10 @@ python main.py -c paddlex/configs/image_classification/ResNet50.yaml \

* 参数 `-o Global.device` 指定的是即将运行的设备，这里需要传入的是 `gpu:0,1,2,3` ，通过指定该参数，PaddleX 调用飞桨的设备指定接口 `paddle.set_device` 来指定运行设备为 `gpu` ，在进行模型训练时，飞桨将自动调用 dcu 算子用于执行模型计算。关于设备指定的更多细节，可以参考官方 api [paddle.set_device](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/device/set_device_cn.html#set-device)。

* 参数 `-c paddlex/configs/image_classification/ResNet50.yaml` 表示读取指定目录下的配置文件，配置文件中指定了模型结构，训练超参等所有训练模型需要用到的配置，该文件中指定的模型结构为 `ResNet50`
* 参数 `-c paddlex/configs/modules/image_classification/ResNet50.yaml` 表示读取指定目录下的配置文件，配置文件中指定了模型结构，训练超参等所有训练模型需要用到的配置，该文件中指定的模型结构为 `ResNet50`

```shell
python main.py -c paddlex/configs/image_classification/ResNet50.yaml \
python main.py -c paddlex/configs/modules/image_classification/ResNet50.yaml \
-o Global.mode=train \
-o Global.dataset_dir=./dataset/cls_flowers_examples \
-o Global.output=resnet50_output \
Expand All @@ -105,7 +97,7 @@ python main.py -c paddlex/configs/image_classification/ResNet50.yaml \
训练完成后，最优权重放在 `resnet50_output/best_model/` 目录下，其中 `inference/inference.pdiparams`、`inference/inference.pdiparams.info`、`inference/inference.pdmodel` 3 个文件为静态图文件，用于推理使用，使用如下命令进行推理

```shell
python main.py -c paddlex/configs/image_classification/ResNet50.yaml \
python main.py -c paddlex/configs/modules/image_classification/ResNet50.yaml \
-o Global.mode=predict \
-o Predict.model_dir="./resnet50_output/best_model/inference" \
-o Predict.input="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_image_classification_001.jpg" \
Expand Down Expand Up @@ -169,7 +161,7 @@ tar -xf ./dataset/seg_optic_examples.tar -C ./dataset/

```shell
# PaddleX 支持对数据集进行校验，确保数据集格式符合 PaddleX 的相关要求。同时在数据校验时，能够对数据集进行分析，统计数据集的基本信息。
python main.py -c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
python main.py -c paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
-o Global.mode=check_dataset \
-o Global.dataset_dir=./dataset/seg_optic_examples

Expand All @@ -185,10 +177,10 @@ python main.py -c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml

* 参数 `-o Global.device` 指定的是即将运行的设备，这里需要传入的是 `gpu:0,1,2,3` ，通过指定该参数，PaddleX 调用飞桨的设备指定接口 `paddle.set_device` 来指定运行设备为 `gpu` ，在进行模型训练时，飞桨将自动调用 dcu 算子用于执行模型计算。关于设备指定的更多细节，可以参考官方 api [paddle.set_device](https://www.paddlepaddle.org.cn/documentation/docs/zh/api/paddle/device/set_device_cn.html#set-device)。

* 参数 `-c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml` 表示读取指定目录下的配置文件，配置文件中指定了模型结构，训练超参等所有训练模型需要用到的配置，该文件中指定的模型结构为 `Deeplabv3_Plus-R50`
* 参数 `-c paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R50.yaml` 表示读取指定目录下的配置文件，配置文件中指定了模型结构，训练超参等所有训练模型需要用到的配置，该文件中指定的模型结构为 `Deeplabv3_Plus-R50`

```shell
python main.py -c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
python main.py -c paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
-o Global.mode=train \
-o Global.dataset_dir=./dataset/seg_optic_examples \
-o Global.output=deeplabv3p_output \
Expand All @@ -204,7 +196,7 @@ python main.py -c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml
训练完成后，最优权重放在 `deeplabv3p_output/best_model/` 目录下，其中 `inference/inference.pdiparams`、`inference/inference.pdiparams.info`、`inference/inference.pdmodel` 3 个文件为静态图文件，用于推理使用，使用如下命令进行推理

```shell
python main.py -c paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
python main.py -c paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R50.yaml \
-o Global.mode=predict \
-o Predict.model_dir="./deeplabv3p_output/best_model/inference" \
-o Predict.input="https://paddle-model-ecology.bj.bcebos.com/paddlex/imgs/demo_image/general_semantic_segmentation_001.jpg" \
Expand Down
38 changes: 30 additions & 8 deletions docs/hardware_support/dcu/support_cn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,33 @@

| 模型库 | 模型类型 | 模型名称 | 训练 | 推理 |
| - | - | - | - | - |
| PaddleX | 图像分类 | [ResNet18](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/image_classification/ResNet18.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet34](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/image_classification/ResNet34.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet50](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/image_classification/ResNet50.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet101](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/image_classification/ResNet101.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet152](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/image_classification/ResNet152.yaml) | √ | √ |
| PaddleX | 语义分割 | [Deeplabv3_Plus-R50](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R50.yaml) | √ | √ |
| PaddleX | 语义分割 | [Deeplabv3_Plus-R101](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/semantic_segmentation/Deeplabv3_Plus-R101.yaml) | √ | √ |
| PaddleNLP | 自然语言理解模型 | [BERT](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/legacy/model_zoo/bert) | √ | √ |
| PaddleX | 图像分类 | [ResNet18](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_classification/ResNet18.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet34](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_classification/ResNet34.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet50](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_classification/ResNet50.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet101](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_classification/ResNet101.yaml) | √ | √ |
| PaddleX | 图像分类 | [ResNet152](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_classification/ResNet152.yaml) | √ | √ |
| PaddleX | 图像多标签分类 | [CLIP_vit_base_patch16_448_ML](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/multilabel_classification/CLIP_vit_base_patch16_448_ML.yaml) | √ | √ |
| PaddleX | 图像多标签分类 | [PP-HGNetV2-B0_ML](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/multilabel_classification/PP-HGNetV2-B0_ML.yaml) | √ | √ |
| PaddleX | 图像多标签分类 | [PP-HGNetV2-B4_ML](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/multilabel_classification/PP-HGNetV2-B4_ML.yaml) | √ | √ |
| PaddleX | 图像多标签分类 | [PP-HGNetV2-B6_ML](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/multilabel_classification/PP-HGNetV2-B6_ML.yaml) | √ | √ |
| PaddleX | 图像特征 | [PP-ShiTuV2_rec_CLIP_vit_base](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/image_feature/PP-ShiTuV2_rec_CLIP_vit_base.yaml) | √ | √ |
| PaddleX | 目标检测 | [PicoDet-XS](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PicoDet-XS.yaml) | √ | √ |
| PaddleX | 目标检测 | [PicoDet-S](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PicoDet-S.yaml) | √ | √ |
| PaddleX | 目标检测 | [PicoDet-M](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PicoDet-M.yaml) | √ | √ |
| PaddleX | 目标检测 | [PicoDet-L](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PicoDet-L.yaml) | √ | √ |
| PaddleX | 目标检测 | [PP-YOLOE_plus-S](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PP-YOLOE_plus-S.yaml) | √ | √ |
| PaddleX | 目标检测 | [PP-YOLOE_plus-M](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PP-YOLOE_plus-M.yaml) | √ | √ |
| PaddleX | 目标检测 | [PP-YOLOE_plus-L](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PP-YOLOE_plus-L.yaml) | √ | √ |
| PaddleX | 目标检测 | [PP-YOLOE_plus-X](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/object_detection/PP-YOLOE_plus-X.yaml) | √ | √ |
| PaddleX | 小目标检测 | [PP-YOLOE_plus_SOD-S](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/small_object_detection/PP-YOLOE_plus_SOD-S.yaml) | √ | √ |
| PaddleX | 小目标检测 | [PP-YOLOE_plus_SOD-L](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/small_object_detection/PP-YOLOE_plus_SOD-L.yaml) | √ | √ |
| PaddleX | 小目标检测 | [PP-YOLOE_plus_SOD-largesize-L](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/small_object_detection/PP-YOLOE_plus_SOD-largesize-L.yaml) | √ | √ |
| PaddleX | 语义分割 | [Deeplabv3_Plus-R50](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R50.yaml) | √ | √ |
| PaddleX | 语义分割 | [Deeplabv3_Plus-R101](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/semantic_segmentation/Deeplabv3_Plus-R101.yaml) | √ | √ |
| PaddleX | 图像异常检测 | [STFPM](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/anomaly_detection/STFPM.yaml) | √ | √ |
| PaddleX | 人脸检测 | [PicoDet_LCNet_x2_5_face](https://github.com/PaddlePaddle/PaddleX/blob/release/3.0-beta2/paddlex/configs/modules/face_detection/PicoDet_LCNet_x2_5_face.yaml) | √ | √ |
| PaddleX | 文本检测 | [PP-OCRv4_server_det](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv4_server_det.yaml) | √ | √ |
| PaddleX | 文本检测 | [PP-OCRv4_mobile_det](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_detection/PP-OCRv4_mobile_det.yaml) | √ | √ |
| PaddleX | 文本识别 | [PP-OCRv4_server_rec](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv4_server_rec.yaml) | √ | √ |
| PaddleX | 文本识别 | [PP-OCRv4_mobile_rec](https://github.com/PaddlePaddle/PaddleX/blob/develop/paddlex/configs/modules/text_recognition/PP-OCRv4_mobile_rec.yaml) | √ | √ |
| PaddleNLP | 自然语言理解模型 | [BERT](https://github.com/PaddlePaddle/PaddleNLP/tree/develop/slm/model_zoo/bert) | √ | √ |
Loading