Skip to content

Commit f910caf

Browse files
MengzhangLIMengzhangLI
authored
[Feature] Support FastFCN (open-mmlab#885)
* FastFCN first commit * FastFCN first commit * Fixing lint error * Fixing lint error * use for loop on JPU * Use For Loop * Refactor FastFCN * FastFCN * FastFCN * temp * Uploading models & logs (4x4) * Fixing typos * fix typos * rename config * change README.md * use _delete_=True * change configs * change start_level to 0 * change start_level to 0 * jpu * add unittest for start_level!=0
1 parent d966f98 commit f910caf

15 files changed

+466
-1
lines changed

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -90,6 +90,7 @@ Supported methods:
9090
- [x] [DMNet (ICCV'2019)](configs/dmnet)
9191
- [x] [ANN (ICCV'2019)](configs/ann)
9292
- [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
93+
- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
9394
- [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
9495
- [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
9596
- [x] [OCRNet (ECCV'2020)](configs/ocrnet)

‎README_zh-CN.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,7 @@ MMSegmentation 是一个基于 PyTorch 的语义分割开源工具箱。它是 O
8989
- [x] [DMNet (ICCV'2019)](configs/dmnet)
9090
- [x] [ANN (ICCV'2019)](configs/ann)
9191
- [x] [GCNet (ICCVW'2019/TPAMI'2020)](configs/gcnet)
92+
- [x] [FastFCN (ArXiv'2019)](configs/fastfcn)
9293
- [x] [Fast-SCNN (ArXiv'2019)](configs/fastscnn)
9394
- [x] [ISANet (ArXiv'2019/IJCV'2021)](configs/isanet)
9495
- [x] [OCRNet (ECCV'2020)](configs/ocrnet)

‎configs/_base_/models/fastfcn_r50-d32_jpu_psp.py‎

Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
# model settings
2+
norm_cfg = dict(type='SyncBN', requires_grad=True)
3+
model = dict(
4+
type='EncoderDecoder',
5+
pretrained='open-mmlab://resnet50_v1c',
6+
backbone=dict(
7+
type='ResNetV1c',
8+
depth=50,
9+
num_stages=4,
10+
dilations=(1, 1, 2, 4),
11+
strides=(1, 2, 2, 2),
12+
out_indices=(1, 2, 3),
13+
norm_cfg=norm_cfg,
14+
norm_eval=False,
15+
style='pytorch',
16+
contract_dilation=True),
17+
neck=dict(
18+
type='JPU',
19+
in_channels=(512, 1024, 2048),
20+
mid_channels=512,
21+
start_level=0,
22+
end_level=-1,
23+
dilations=(1, 2, 4, 8),
24+
align_corners=False,
25+
norm_cfg=norm_cfg),
26+
decode_head=dict(
27+
type='PSPHead',
28+
in_channels=2048,
29+
in_index=2,
30+
channels=512,
31+
pool_scales=(1, 2, 3, 6),
32+
dropout_ratio=0.1,
33+
num_classes=19,
34+
norm_cfg=norm_cfg,
35+
align_corners=False,
36+
loss_decode=dict(
37+
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
38+
auxiliary_head=dict(
39+
type='FCNHead',
40+
in_channels=1024,
41+
in_index=1,
42+
channels=256,
43+
num_convs=1,
44+
concat_input=False,
45+
dropout_ratio=0.1,
46+
num_classes=19,
47+
norm_cfg=norm_cfg,
48+
align_corners=False,
49+
loss_decode=dict(
50+
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=0.4)),
51+
# model training and testing settings
52+
train_cfg=dict(),
53+
test_cfg=dict(mode='whole'))

‎configs/fastfcn/README.md‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation
2+
3+
## Introduction
4+
5+
<!-- [ALGORITHM] -->
6+
7+
<a href="https://github.com/wuhuikai/FastFCN">Official Repo</a>
8+
9+
<a href="https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12">Code Snippet</a>
10+
11+
<details>
12+
<summary align="right"><a href="https://arxiv.org/abs/1903.11816">FastFCN (ArXiv'2019) </a></summary>
13+
14+
```latex
15+
@article{wu2019fastfcn,
16+
title={Fastfcn: Rethinking dilated convolution in the backbone for semantic segmentation},
17+
author={Wu, Huikai and Zhang, Junge and Huang, Kaiqi and Liang, Kongming and Yu, Yizhou},
18+
journal={arXiv preprint arXiv:1903.11816},
19+
year={2019}
20+
}
21+
```
22+
23+
</details>
24+
25+
## Results and models
26+
27+
### Cityscapes
28+
29+
| Method | Backbone | Crop Size | Lr schd | Mem (GB) | Inf time (fps) | mIoU | mIoU(ms+flip) | config | download |
30+
| --------- | --------- | --------- | ------: | -------- | -------------- | ----: | ------------- | --------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
31+
| DeepLabV3 + JPU | R-50-D32 | 512x1024 | 80000 | 5.67 | 2.64 | 79.12 | 80.58 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722-5d1a2648.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722.log.json) |
32+
| DeepLabV3 + JPU (4x4) | R-50-D32 | 512x1024 | 80000 | 9.79 | - | 79.52 | 80.91 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357-72220849.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357.log.json) |
33+
| PSPNet + JPU | R-50-D32 | 512x1024 | 80000 | 5.67 | 4.40 | 79.26 | 80.86 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722-57749bed.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722.log.json) |
34+
| PSPNet + JPU (4x4) | R-50-D32 | 512x1024 | 80000 | 9.94 | - | 78.76 | 80.03 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841-77e87b0a.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841.log.json) |
35+
| EncNet + JPU | R-50-D32 | 512x1024 | 80000 | 8.15 | 4.77 | 77.97 |79.92 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036-78da5046.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036.log.json) |
36+
| EncNet + JPU (4x4)| R-50-D32 | 512x1024 | 80000 | 15.45 | - | 78.6 | 80.25 | [config](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes.py) | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217-e1eb6dbb.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217.log.json) |
37+
38+
Note:
39+
40+
- `4x4` means 4 GPUs with 4 samples per GPU in training, default setting is 4 GPUs with 2 samples per GPU in training.
41+
- Results of [DeepLabV3 (mIoU: 79.32)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/deeplabv3), [PSPNet (mIoU: 78.55)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/pspnet) and [ENCNet (mIoU: 77.94)](https://github.com/open-mmlab/mmsegmentation/blob/master/configs/encnet) can be found in each original repository.

‎configs/fastfcn/fastfcn.yml‎

Lines changed: 126 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,126 @@
1+
Collections:
2+
- Name: fastfcn
3+
Metadata:
4+
Training Data:
5+
- Cityscapes
6+
Paper:
7+
URL: https://arxiv.org/abs/1903.11816
8+
Title: 'FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation'
9+
README: configs/fastfcn/README.md
10+
Code:
11+
URL: https://github.com/open-mmlab/mmsegmentation/blob/v0.18.0/mmseg/models/necks/jpu.py#L12
12+
Version: v0.18.0
13+
Converted From:
14+
Code: https://github.com/wuhuikai/FastFCN
15+
Models:
16+
- Name: fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes
17+
In Collection: fastfcn
18+
Metadata:
19+
backbone: R-50-D32
20+
crop size: (512,1024)
21+
lr schd: 80000
22+
inference time (ms/im):
23+
- value: 378.79
24+
hardware: V100
25+
backend: PyTorch
26+
batch size: 1
27+
mode: FP32
28+
resolution: (512,1024)
29+
memory (GB): 5.67
30+
Results:
31+
- Task: Semantic Segmentation
32+
Dataset: Cityscapes
33+
Metrics:
34+
mIoU: 79.12
35+
mIoU(ms+flip): 80.58
36+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py
37+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes_20210928_053722-5d1a2648.pth
38+
- Name: fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes
39+
In Collection: fastfcn
40+
Metadata:
41+
backbone: R-50-D32
42+
crop size: (512,1024)
43+
lr schd: 80000
44+
memory (GB): 9.79
45+
Results:
46+
- Task: Semantic Segmentation
47+
Dataset: Cityscapes
48+
Metrics:
49+
mIoU: 79.52
50+
mIoU(ms+flip): 80.91
51+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes.py
52+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes_20210924_214357-72220849.pth
53+
- Name: fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes
54+
In Collection: fastfcn
55+
Metadata:
56+
backbone: R-50-D32
57+
crop size: (512,1024)
58+
lr schd: 80000
59+
inference time (ms/im):
60+
- value: 227.27
61+
hardware: V100
62+
backend: PyTorch
63+
batch size: 1
64+
mode: FP32
65+
resolution: (512,1024)
66+
memory (GB): 5.67
67+
Results:
68+
- Task: Semantic Segmentation
69+
Dataset: Cityscapes
70+
Metrics:
71+
mIoU: 79.26
72+
mIoU(ms+flip): 80.86
73+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py
74+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes_20210928_053722-57749bed.pth
75+
- Name: fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes
76+
In Collection: fastfcn
77+
Metadata:
78+
backbone: R-50-D32
79+
crop size: (512,1024)
80+
lr schd: 80000
81+
memory (GB): 9.94
82+
Results:
83+
- Task: Semantic Segmentation
84+
Dataset: Cityscapes
85+
Metrics:
86+
mIoU: 78.76
87+
mIoU(ms+flip): 80.03
88+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes.py
89+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes_20210925_061841-77e87b0a.pth
90+
- Name: fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes
91+
In Collection: fastfcn
92+
Metadata:
93+
backbone: R-50-D32
94+
crop size: (512,1024)
95+
lr schd: 80000
96+
inference time (ms/im):
97+
- value: 209.64
98+
hardware: V100
99+
backend: PyTorch
100+
batch size: 1
101+
mode: FP32
102+
resolution: (512,1024)
103+
memory (GB): 8.15
104+
Results:
105+
- Task: Semantic Segmentation
106+
Dataset: Cityscapes
107+
Metrics:
108+
mIoU: 77.97
109+
mIoU(ms+flip): 79.92
110+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py
111+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes_20210928_030036-78da5046.pth
112+
- Name: fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes
113+
In Collection: fastfcn
114+
Metadata:
115+
backbone: R-50-D32
116+
crop size: (512,1024)
117+
lr schd: 80000
118+
memory (GB): 15.45
119+
Results:
120+
- Task: Semantic Segmentation
121+
Dataset: Cityscapes
122+
Metrics:
123+
mIoU: 78.6
124+
mIoU(ms+flip): 80.25
125+
Config: configs/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes.py
126+
Weights: https://download.openmmlab.com/mmsegmentation/v0.5/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes_20210926_093217-e1eb6dbb.pth

‎configs/fastfcn/fastfcn_r50-d32_jpu_aspp_4x4_512x1024_80k_cityscapes.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# model settings
2+
_base_ = './fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py'
3+
data = dict(
4+
samples_per_gpu=4,
5+
workers_per_gpu=4,
6+
)

‎configs/fastfcn/fastfcn_r50-d32_jpu_aspp_512x1024_80k_cityscapes.py‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# model settings
2+
_base_ = './fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py'
3+
norm_cfg = dict(type='SyncBN', requires_grad=True)
4+
model = dict(
5+
decode_head=dict(
6+
_delete_=True,
7+
type='ASPPHead',
8+
in_channels=2048,
9+
in_index=2,
10+
channels=512,
11+
dilations=(1, 12, 24, 36),
12+
dropout_ratio=0.1,
13+
num_classes=19,
14+
norm_cfg=norm_cfg,
15+
align_corners=False,
16+
loss_decode=dict(
17+
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0)),
18+
# model training and testing settings
19+
train_cfg=dict(),
20+
test_cfg=dict(mode='whole'))

‎configs/fastfcn/fastfcn_r50-d32_jpu_enc_4x4_512x1024_80k_cityscapes.py‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
# model settings
2+
_base_ = './fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py'
3+
data = dict(
4+
samples_per_gpu=4,
5+
workers_per_gpu=4,
6+
)

‎configs/fastfcn/fastfcn_r50-d32_jpu_enc_512x1024_80k_cityscapes.py‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,24 @@
1+
# model settings
2+
_base_ = './fastfcn_r50-d32_jpu_psp_512x1024_80k_cityscapes.py'
3+
norm_cfg = dict(type='SyncBN', requires_grad=True)
4+
model = dict(
5+
decode_head=dict(
6+
_delete_=True,
7+
type='EncHead',
8+
in_channels=[512, 1024, 2048],
9+
in_index=(0, 1, 2),
10+
channels=512,
11+
num_codes=32,
12+
use_se_loss=True,
13+
add_lateral=False,
14+
dropout_ratio=0.1,
15+
num_classes=19,
16+
norm_cfg=norm_cfg,
17+
align_corners=False,
18+
loss_decode=dict(
19+
type='CrossEntropyLoss', use_sigmoid=False, loss_weight=1.0),
20+
loss_se_decode=dict(
21+
type='CrossEntropyLoss', use_sigmoid=True, loss_weight=0.2)),
22+
# model training and testing settings
23+
train_cfg=dict(),
24+
test_cfg=dict(mode='whole'))

‎configs/fastfcn/fastfcn_r50-d32_jpu_psp_4x4_512x1024_80k_cityscapes.py‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
_base_ = [
2+
'../_base_/models/fastfcn_r50-d32_jpu_psp.py',
3+
'../_base_/datasets/cityscapes.py', '../_base_/default_runtime.py',
4+
'../_base_/schedules/schedule_80k.py'
5+
]
6+
data = dict(
7+
samples_per_gpu=4,
8+
workers_per_gpu=4,
9+
)

0 commit comments

Comments
 (0)