Support fcn dilate 6 (open-mmlab#400)

谢昕辰 · xvjiarui · web-flow · commit 0c31afe9eb62 · 2021-03-20T10:32:04.000-07:00
* Support fcn dilate 6

* Support dilate in FCNHead

* configs for cityscapse dataset

* add configs for pytorch pretrained model

* update README

* add fps test results

* add memory test results and links

* modify log names

* Update mmseg/models/decode_heads/fcn_head.py

Co-authored-by: Jerry Jiarui XU &lt;xvjiarui0826@gmail.com&gt;
diff --git a/configs/fcn/README.md b/configs/fcn/README.md
@@ -39,6 +39,18 @@
 | FCN    | R-18b-D8 | 769x769   |   80000 |      1.7 |           6.70 | 69.66 |         72.07 | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r18b-d8_769x769_80k_cityscapes/fcn_r18b-d8_769x769_80k_cityscapes_20201226_004430-32d504e5.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r18b-d8_769x769_80k_cityscapes/fcn_r18b-d8_769x769_80k_cityscapes-20201226_004430.log.json)         |
 | FCN    | R-50b-D8 | 769x769   |   80000 |      6.3 |           1.82 | 73.83 |         76.60 | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r50b-d8_769x769_80k_cityscapes/fcn_r50b-d8_769x769_80k_cityscapes_20201225_094223-94552d38.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r50b-d8_769x769_80k_cityscapes/fcn_r50b-d8_769x769_80k_cityscapes-20201225_094223.log.json)         |
 | FCN    | R-101b-D8| 769x769   |   80000 |     10.3 |           1.15 | 77.02 |         78.67 | [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r101b-d8_769x769_80k_cityscapes/fcn_r101b-d8_769x769_80k_cityscapes_20201226_170012-82be37e2.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_r101b-d8_769x769_80k_cityscapes/fcn_r101b-d8_769x769_80k_cityscapes-20201226_170012.log.json)     |
+|FCN-D6|R-50-D16|512x1024|40000|3.4|10.22|77.06|78.85| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_512x1024_40k_cityscapes/fcn_d6_r50-d16_512x1024_40k_cityscapes-98d5d1bc.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_512x1024_40k_cityscapes/fcn_d6_r50-d16_512x1024_40k_cityscapes-20210305_130133.log.json)|
+|FCN-D6|R-50-D16|512x1024|80000|-|10.35|77.27|78.88| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_512x1024_80k_cityscapes/fcn_d6_r50-d16_512x1024_40k_cityscapes-98d5d1bc.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_512x1024_80k_cityscapes/fcn_d6_r50-d16_512x1024_80k_cityscapes-20210306_115604.log.json) |
+|FCN-D6|R-50-D16|769x769|40000|3.7|4.17|76.82|78.22| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_769x769_40k_cityscapes/fcn_d6_r50-d16_769x769_40k_cityscapes-1aab18ed.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_769x769_40k_cityscapes/fcn_d6_r50-d16_769x769_40k_cityscapes-20210305_185744.log.json) |
+|FCN-D6|R-50-D16|769x769|80000|-|4.15|77.04|78.40| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_769x769_80k_cityscapes/fcn_d6_r50-d16_769x769_80k_cityscapes-109d88eb.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50-d16_769x769_80k_cityscapes/fcn_d6_r50-d16_769x769_80k_cityscapes-20210305_200413.log.json) |
+|FCN-D6|R-101-D16|512x1024|40000|4.5|8.04|77.36|79.18| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_512x1024_40k_cityscapes/fcn_d6_r101-d16_512x1024_40k_cityscapes-9cf2b450.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_512x1024_40k_cityscapes/fcn_d6_r101-d16_512x1024_40k_cityscapes-20210305_130337.log.json) |
+|FCN-D6|R-101-D16|512x1024|80000|-|8.26|78.46|80.42| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_512x1024_80k_cityscapes/fcn_d6_r101-d16_512x1024_80k_cityscapes-cb336445.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_512x1024_80k_cityscapes/fcn_d6_r101-d16_512x1024_80k_cityscapes-20210308_102747.log.json) |
+|FCN-D6|R-101-D16|769x769|40000|5.0|3.12|77.28|78.95| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_769x769_40k_cityscapes/fcn_d6_r101-d16_769x769_40k_cityscapes-60b114e9.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_769x769_40k_cityscapes/fcn_d6_r101-d16_769x769_40k_cityscapes-20210308_102453.log.json) |
+|FCN-D6|R-101-D16|769x769|80000|-|3.21|78.06|79.58| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_769x769_80k_cityscapes/fcn_d6_r101-d16_769x769_80k_cityscapes-e33adc4f.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101-d16_769x769_80k_cityscapes/fcn_d6_r101-d16_769x769_80k_cityscapes-20210306_120016.log.json) |
+|FCN-D6|R-50b-D16|512x1024|80000|3.2|10.16|76.99|79.03| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50b_d16_512x1024_80k_cityscapes/fcn_d6_r50b_d16_512x1024_80k_cityscapes-6a0b62e9.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50b_d16_512x1024_80k_cityscapes/fcn_d6_r50b_d16_512x1024_80k_cityscapes-20210311_125550.log.json) |
+|FCN-D6|R-50b-D16|769x769|80000|3.6|4.17|76.86|78.52| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50b_d16_769x769_80k_cityscapes/fcn_d6_r50b_d16_769x769_80k_cityscapes-d665f231.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r50b_d16_769x769_80k_cityscapes/fcn_d6_r50b_d16_769x769_80k_cityscapes-20210311_131012.log.json) |
+|FCN-D6|R-101b-D16|512x1024|80000|4.3|8.46|77.72|79.53| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101b_d16_512x1024_80k_cityscapes/fcn_d6_r101b_d16_512x1024_80k_cityscapes-3f2eb5b4.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101b_d16_512x1024_80k_cityscapes/fcn_d6_r101b_d16_512x1024_80k_cityscapes-20210311_144305.log.json) |
+|FCN-D6|R-101b-D16|769x769|80000|4.8|3.32|77.34|78.91| [model](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101b_d16_769x769_80k_cityscapes/fcn_d6_r101b_d16_769x769_80k_cityscapes-c4d8bfbc.pth) &#124; [log](https://download.openmmlab.com/mmsegmentation/v0.5/fcn/fcn_d6_r101b_d16_769x769_80k_cityscapes/fcn_d6_r101b_d16_769x769_80k_cityscapes-20210311_154527.log.json) |
 
 ### ADE20K
 
diff --git a/configs/fcn/fcn_d6_r101-d16_512x1024_40k_cityscapes.py b/configs/fcn/fcn_d6_r101-d16_512x1024_40k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_512x1024_40k_cityscapes.py'
+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))
diff --git a/configs/fcn/fcn_d6_r101-d16_512x1024_80k_cityscapes.py b/configs/fcn/fcn_d6_r101-d16_512x1024_80k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_512x1024_80k_cityscapes.py'
+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))
diff --git a/configs/fcn/fcn_d6_r101-d16_769x769_40k_cityscapes.py b/configs/fcn/fcn_d6_r101-d16_769x769_40k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_769x769_40k_cityscapes.py'
+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))
diff --git a/configs/fcn/fcn_d6_r101-d16_769x769_80k_cityscapes.py b/configs/fcn/fcn_d6_r101-d16_769x769_80k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_769x769_80k_cityscapes.py'
+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))
diff --git a/configs/fcn/fcn_d6_r101b-d16_512x1024_80k_cityscapes.py b/configs/fcn/fcn_d6_r101b-d16_512x1024_80k_cityscapes.py
@@ -0,0 +1,4 @@
+_base_ = './fcn_d6_r50b-d16_512x1024_80k_cityscapes.py'
+model = dict(
+    pretrained='torchvision://resnet101',
+    backbone=dict(type='ResNet', depth=101))
diff --git a/configs/fcn/fcn_d6_r101b-d16_769x769_80k_cityscapes.py b/configs/fcn/fcn_d6_r101b-d16_769x769_80k_cityscapes.py
@@ -0,0 +1,4 @@
+_base_ = './fcn_d6_r50b-d16_769x769_80k_cityscapes.py'
+model = dict(
+    pretrained='torchvision://resnet101',
+    backbone=dict(type='ResNet', depth=101))
diff --git a/configs/fcn/fcn_d6_r50-d16_512x1024_40k_cityscapes.py b/configs/fcn/fcn_d6_r50-d16_512x1024_40k_cityscapes.py
@@ -0,0 +1,8 @@
+_base_ = [
+    '../_base_/models/fcn_r50-d8.py', '../_base_/datasets/cityscapes.py',
+    '../_base_/default_runtime.py', '../_base_/schedules/schedule_40k.py'
+]
+model = dict(
+    backbone=dict(dilations=(1, 1, 1, 2), strides=(1, 2, 2, 1)),
+    decode_head=dict(dilation=6),
+    auxiliary_head=dict(dilation=6))
diff --git a/configs/fcn/fcn_d6_r50-d16_512x1024_80k_cityscapes.py b/configs/fcn/fcn_d6_r50-d16_512x1024_80k_cityscapes.py
@@ -0,0 +1,8 @@
+_base_ = [
+    '../_base_/models/fcn_r50-d8.py', '../_base_/datasets/cityscapes.py',
+    '../_base_/default_runtime.py', '../_base_/schedules/schedule_80k.py'
+]
+model = dict(
+    backbone=dict(dilations=(1, 1, 1, 2), strides=(1, 2, 2, 1)),
+    decode_head=dict(dilation=6),
+    auxiliary_head=dict(dilation=6))
diff --git a/configs/fcn/fcn_d6_r50-d16_769x769_40k_cityscapes.py b/configs/fcn/fcn_d6_r50-d16_769x769_40k_cityscapes.py
@@ -0,0 +1,10 @@
+_base_ = [
+    '../_base_/models/fcn_r50-d8.py',
+    '../_base_/datasets/cityscapes_769x769.py', '../_base_/default_runtime.py',
+    '../_base_/schedules/schedule_40k.py'
+]
+model = dict(
+    backbone=dict(dilations=(1, 1, 1, 2), strides=(1, 2, 2, 1)),
+    decode_head=dict(align_corners=True, dilation=6),
+    auxiliary_head=dict(align_corners=True, dilation=6),
+    test_cfg=dict(mode='slide', crop_size=(769, 769), stride=(513, 513)))
diff --git a/configs/fcn/fcn_d6_r50-d16_769x769_80k_cityscapes.py b/configs/fcn/fcn_d6_r50-d16_769x769_80k_cityscapes.py
@@ -0,0 +1,10 @@
+_base_ = [
+    '../_base_/models/fcn_r50-d8.py',
+    '../_base_/datasets/cityscapes_769x769.py', '../_base_/default_runtime.py',
+    '../_base_/schedules/schedule_80k.py'
+]
+model = dict(
+    backbone=dict(dilations=(1, 1, 1, 2), strides=(1, 2, 2, 1)),
+    decode_head=dict(align_corners=True, dilation=6),
+    auxiliary_head=dict(align_corners=True, dilation=6),
+    test_cfg=dict(mode='slide', crop_size=(769, 769), stride=(513, 513)))
diff --git a/configs/fcn/fcn_d6_r50b-d16_512x1024_80k_cityscapes.py b/configs/fcn/fcn_d6_r50b-d16_512x1024_80k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_512x1024_80k_cityscapes.py'
+model = dict(pretrained='torchvision://resnet50', backbone=dict(type='ResNet'))
diff --git a/configs/fcn/fcn_d6_r50b-d16_769x769_80k_cityscapes.py b/configs/fcn/fcn_d6_r50b-d16_769x769_80k_cityscapes.py
@@ -0,0 +1,2 @@
+_base_ = './fcn_d6_r50-d16_769x769_80k_cityscapes.py'
+model = dict(pretrained='torchvision://resnet50', backbone=dict(type='ResNet'))
diff --git a/mmseg/models/decode_heads/fcn_head.py b/mmseg/models/decode_heads/fcn_head.py
@@ -17,28 +17,32 @@ class FCNHead(BaseDecodeHead):
         kernel_size (int): The kernel size for convs in the head. Default: 3.
         concat_input (bool): Whether concat the input and output of convs
             before classification layer.
+        dilation (int): The dilation rate for convs in the head. Default: 1.
     """
 
     def __init__(self,
                  num_convs=2,
                  kernel_size=3,
                  concat_input=True,
+                 dilation=1,
                  **kwargs):
-        assert num_convs >= 0
+        assert num_convs >= 0 and dilation > 0 and isinstance(dilation, int)
         self.num_convs = num_convs
         self.concat_input = concat_input
         self.kernel_size = kernel_size
         super(FCNHead, self).__init__(**kwargs)
         if num_convs == 0:
             assert self.in_channels == self.channels
 
+        conv_padding = (kernel_size // 2) * dilation
         convs = []
         convs.append(
             ConvModule(
                 self.in_channels,
                 self.channels,
                 kernel_size=kernel_size,
-                padding=kernel_size // 2,
+                padding=conv_padding,
+                dilation=dilation,
                 conv_cfg=self.conv_cfg,
                 norm_cfg=self.norm_cfg,
                 act_cfg=self.act_cfg))
@@ -48,7 +52,8 @@ def __init__(self,
                     self.channels,
                     self.channels,
                     kernel_size=kernel_size,
-                    padding=kernel_size // 2,
+                    padding=conv_padding,
+                    dilation=dilation,
                     conv_cfg=self.conv_cfg,
                     norm_cfg=self.norm_cfg,
                     act_cfg=self.act_cfg))

Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+_base_ = './fcn_d6_r50-d16_512x1024_40k_cityscapes.py'`
	`2`	`+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+_base_ = './fcn_d6_r50-d16_512x1024_80k_cityscapes.py'`
	`2`	`+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+_base_ = './fcn_d6_r50-d16_769x769_40k_cityscapes.py'`
	`2`	`+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))`
Original file line number	Diff line number	Diff line change
`@@ -0,0 +1,2 @@`
	`1`	`+_base_ = './fcn_d6_r50-d16_769x769_80k_cityscapes.py'`
	`2`	`+model = dict(pretrained='open-mmlab://resnet101_v1c', backbone=dict(depth=101))`