You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
[From left to right: Test Image, Ground Truth, Predicted Result]
14
14
15
-
## Supported models:
15
+
## Highlights [NEW!]
16
+
17
+
### Syncronized Batch Normalization on PyTorch
18
+
This module differs from the built-in PyTorch BatchNorm as the mean and standard-deviation are reduced across all devices during training. The importance of synchronized batch normalization in object detection has been recently proved with a an extensive analysis in the paper [MegDet: A Large Mini-Batch Object Detector](https://arxiv.org/abs/1711.07240), and we empirically find that it is also important for segmentation.
19
+
20
+
The implementation is reasonable due to the following reasons:
21
+
- This implementation is in pure-python. No C++ extra extension libs.
22
+
- Easy to use.
23
+
- It is completely compatible with PyTorch's implementation. Specifically, it uses unbiased variance to update the moving average, and use sqrt(max(var, eps)) instead of sqrt(var + eps).
24
+
25
+
***To the best knowledge, it is the first pure-python implementation of sync bn on PyTorch, and also the first one completely compatible with PyTorch. It is also efficient, only 20% to 30% slower than un-sync bn.*** We especially thank [Jiayuan Mao](http://vccy.xyz/) for his kind contributions. For more details about the implementation and usage, refer to [Synchronized-BatchNorm-PyTorch](https://github.com/vacancy/Synchronized-BatchNorm-PyTorch).
26
+
27
+
### Dynamic scales of input for training with multiple GPUs
28
+
Different from image classification task, where the input images are resized to a fixed scale such as 224x224, it is better to keep original aspect ratios of input images for semantic segmentation and object detection networks.
29
+
30
+
So we re-implement the `DataParallel` module, and make it support distributing data to multiple GPUs in python dict. At the same time, the dataloader also operates differently. *Now the batch size of a dataloader always equals to the number of GPUs*, each element will be sent to a GPU. It is also compatible with multi-processing. Note that the file index for the multi-processing dataloader is stored on the master process, which is in contradict to our goal that each worker maintains its own file list. So we use a trick that although the master process still gives dataloader an index for `__getitem__` function, we just ignore such request and send a random batch dict. Also, *the multiple workers forked by the dataloader all have the same seed*, you will find that multiple workers will yield exactly the same data, if we use the above-mentioned trick directly. Therefore, we add one line of code which sets the defaut seed for `numpy.random` before activating multiple worker in dataloader.
31
+
32
+
33
+
## Supported models
16
34
We split our models into encoder and decoder, where encoders are usually modified directly from classification networks, and decoders consist of final convolutions and upsampling.
17
35
18
-
Encoder:
19
-
- vgg16_dilated8
20
-
- vgg19_dilated8
21
-
- resnet34_dilated16
22
-
- resnet34_dilated8
23
-
- resnet50_dilated16
24
-
- resnet50_dilated8
36
+
Encoder: (resnetXX_dilatedYY: customized resnetXX with dilated convolutions, output feature map is 1/YY of input size.)
37
+
- resnet34_dilated16, resnet34_dilated8
38
+
- resnet50_dilated16, resnet50_dilated8
25
39
26
-
(resnetXX_dilatedYY: customized resnetXX with dilated convolutions, output feature map is 1/YY of input size.)
40
+
***Coming soon***:
41
+
- resnet101_dilated16, resnet101_dilated8
27
42
28
43
Decoder:
29
44
- c1_bilinear (1 conv + bilinear upsample)
45
+
- c1_bilinear_deepsup (c1_blinear + deep supervision trick)
30
46
- psp_bilinear (pyramid pooling + bilinear upsample, see PSPNet paper for details)
47
+
- psp_bilinear_deepsup (psp_bilinear + deep supervision trick)
48
+
49
+
***Coming soon***:
50
+
- UPerNet based on Feature Pyramid Network (FPN) and Pyramid Pooling Module (PPM), with down-sampling rate of 4, 8 and 16. It doesn't need dilated convolution, a operator that is time-and-memory consuming. *Without bells and whistles*, it is comparable or even better compared with PSPNet, while requires much shorter training time and less GPU memory.
31
51
32
52
33
53
## Performance:
34
-
IMPORTANT: One obstacle to a good dilated ResNet model is that batch normalization layers are usually not well trained with a small batch size (<16). Ideally, batch size >64 will get you the best results. In this repo, we trained customized ResNet on Places365 (will be automatically downloaded when needed) as the initialization for scene parsing model, which partly solved the problem. You can simply set ```--fix_bn 1``` to freeze BN parameters during training.
54
+
IMPORTANT: We use our self-trained base model on ImageNet. The model takes the input in BGR form (consistent with opencv) instead of RGB form as used by default implementation of PyTorch. The base model will be automatically downloaded when needed.
- Software: Ubuntu 16.04.3 LTS, CUDA 8.0, ***Python3.5***, ***PyTorch 0.4.0***
102
+
103
+
*Warning:* We don't support the outdated Python 2 anymore. PyTorch 0.4.0 or higher is required to run the codes.
71
104
72
105
## Training
73
106
1. Download the ADE20K scene parsing dataset:
74
107
```bash
75
108
chmod +x download_ADE20K.sh
76
109
./download_ADE20K.sh
77
110
```
78
-
2. Train a network (default: resnet34_dilated8). During training, checkpoints will be saved in folder ```ckpt```, visual results will be saved in folder ```vis```.
111
+
2. Train a network (default: ResNet-50_dilated8 + psp_bilinear_deepsup). During training, checkpoints will be saved in folder ```ckpt```.
79
112
```bash
80
-
python train.py
113
+
python3 train.py --num_gpus NUM_GPUS
81
114
```
82
115
83
-
3. Input arguments: (see full input arguments via ```python train.py -h ```)
116
+
3. Input arguments: (see full input arguments via ```python3 train.py -h ```)
If you find the code or pre-trained models useful, please cite the following paper:
160
+
If you find the code or pre-trained models useful, please cite the following papers:
143
161
144
162
Scene Parsing through ADE20K Dataset. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba. Computer Vision and Pattern Recognition (CVPR), 2017. (http://people.csail.mit.edu/bzhou/publication/scene-parse-camera-ready.pdf)
145
163
@@ -150,6 +168,15 @@ Scene Parsing through ADE20K Dataset. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. B
150
168
year={2017}
151
169
}
152
170
171
+
Unified Perceptual Parsing for Scene Understanding. T. Xiao, Y. Liu, B. Zhou, Y. Jiang, and J. Sun. arXiv preprint
172
+
173
+
@article{xiao2018unified,
174
+
title={Unified Perceptual Parsing for Scene Understanding},
175
+
author={Xiao, Tete and Liu, Yingcheng and Zhou, Bolei and Jiang, Yuning and Sun, Jian},
176
+
journal={arXiv preprint},
177
+
year={2018}
178
+
}
179
+
153
180
Semantic Understanding of Scenes through ADE20K Dataset. B. Zhou, H. Zhao, X. Puig, S. Fidler, A. Barriuso and A. Torralba. arXiv:1608.05442. (https://arxiv.org/pdf/1608.05442.pdf)
0 commit comments