MS COCO dataset support

Author: rbgirshick

README.md

@@ -55,6 +55,8 @@ If you find Faster R-CNN useful in your research, please consider citing:
   ```make
   # In your Makefile.config, make sure to have this line uncommented
   WITH_PYTHON_LAYER := 1
+  # Unrelatedly, it's also recommended that you use CUDNN
+  USE_CUDNN := 1
   ```
 
   You can download my [Makefile.config](http://www.cs.berkeley.edu/~rbg/fast-rcnn-data/Makefile.config) for reference.
@@ -115,8 +117,6 @@ If you find Faster R-CNN useful in your research, please consider citing:
 
 *After successfully completing [basic installation](#installation-sufficient-for-the-demo)*, you'll be ready to run the demo.
 
-**Python**
-
 To run the demo
 ```Shell
 cd $FRCN_ROOT
@@ -158,7 +158,8 @@ The demo performs detection using a VGG16 network trained for detection on PASCA
     ```
     Using symlinks is a good idea because you will likely want to share the same PASCAL dataset installation between multiple projects.
 5. [Optional] follow similar steps to get PASCAL VOC 2010 and 2012
-6. Follow the next sections to download pre-trained ImageNet models
+6. [Optional] If you want to use COCO, please see some notes under `data/README.md`
+7. Follow the next sections to download pre-trained ImageNet models
 
 ### Download pre-trained ImageNet models
 
@@ -200,3 +201,17 @@ cd $FRCN_ROOT
 ```
 
 This method trains the RPN module jointly with the Fast R-CNN network, rather than alternating between training the two. It results in faster (~ 1.5x speedup) training times and similar detection accuracy. See these [slides](https://www.dropbox.com/s/xtr4yd4i5e0vw8g/iccv15_tutorial_training_rbg.pdf?dl=0) for more details.
+
+Artifacts generated by the scripts in `tools` are written in this directory.
+
+Trained Fast R-CNN networks are saved under:
+
+```
+output/<experiment directory>/<dataset name>/
+```
+
+Test outputs are saved under:
+
+```
+output/<experiment directory>/<dataset name>/<network snapshot name>/
+```

data/README.md

@@ -30,6 +30,20 @@ ln -s /your/path/to/VOC2007/VOCdevkit VOCdevkit2007
 ln -s /your/path/to/VOC2012/VOCdevkit VOCdevkit2012
 ```
 
+Install the MS COCO dataset at /path/to/coco
+
+```
+ln -s /path/to/coco coco
+```
+
+For COCO with Fast R-CNN, place object proposals under `coco_proposals` (inside
+the `data` directory). You can obtain proposals on COCO from Jan Hosang at
+https://www.mpi-inf.mpg.de/departments/computer-vision-and-multimodal-computing/research/object-recognition-and-scene-understanding/how-good-are-detection-proposals-really/.
+For COCO, using MCG is recommended over selective search. MCG boxes can be downloaded
+from http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/mcg/.
+Use the tool `lib/datasets/tools/mcg_munge.py` to convert the downloaded MCG data
+into the same file layout as those from Jan Hosang.
+
 Since you'll likely be experimenting with multiple installs of Fast/er R-CNN in
 parallel, you'll probably want to keep all of this data in a shared place and
 use symlinks. On my system I create the following symlinks inside `data`:

experiments/cfgs/faster_rcnn_alt_opt.yml

@@ -1,3 +1,5 @@
 EXP_DIR: faster_rcnn_alt_opt
+TRAIN:
+  BG_THRESH_LO: 0.0
 TEST:
   HAS_RPN: True

experiments/cfgs/faster_rcnn_end2end.yml

@@ -6,5 +6,6 @@ TRAIN:
   RPN_POSITIVE_OVERLAP: 0.7
   RPN_BATCHSIZE: 256
   PROPOSAL_METHOD: gt
+  BG_THRESH_LO: 0.0
 TEST:
   HAS_RPN: True

experiments/scripts/fast_rcnn.sh

@@ -1,8 +1,10 @@
 #!/bin/bash
 # Usage:
-# ./experiments/scripts/default.sh GPU NET [options args to {train,test}_net.py]
+# ./experiments/scripts/fast_rcnn.sh GPU NET DATASET [options args to {train,test}_net.py]
+# DATASET is either pascal_voc or coco.
+#
 # Example:
-# ./experiments/scripts/default.sh 0 CaffeNet \
+# ./experiments/scripts/fast_rcnn.sh 0 VGG_CNN_M_1024 pascal_voc \
 #   --set EXP_DIR foobar RNG_SEED 42 TRAIN.SCALES "[400, 500, 600, 700]"
 
 set -x
@@ -13,28 +15,49 @@ export PYTHONUNBUFFERED="True"
 GPU_ID=$1
 NET=$2
 NET_lc=${NET,,}
+DATASET=$3
 
 array=( $@ )
 len=${#array[@]}
-EXTRA_ARGS=${array[@]:2:$len}
+EXTRA_ARGS=${array[@]:3:$len}
 EXTRA_ARGS_SLUG=${EXTRA_ARGS// /_}
 
-LOG="experiments/logs/default_${NET}_${EXTRA_ARGS_SLUG}.txt.`date +'%Y-%m-%d_%H-%M-%S'`"
+case $DATASET in
+  pascal_voc)
+    TRAIN_IMDB="voc_2007_trainval"
+    TEST_IMDB="voc_2007_test"
+    PT_DIR="pascal_voc"
+    ITERS=40000
+    ;;
+  coco)
+    TRAIN_IMDB="coco_2014_train"
+    TEST_IMDB="coco_2014_minival"
+    PT_DIR="coco"
+    ITERS=280000
+    ;;
+  *)
+    echo "No dataset given"
+    exit
+    ;;
+esac
+
+LOG="experiments/logs/fast_rcnn_${NET}_${EXTRA_ARGS_SLUG}.txt.`date +'%Y-%m-%d_%H-%M-%S'`"
 exec &> >(tee -a "$LOG")
 echo Logging output to "$LOG"
 
 time ./tools/train_net.py --gpu ${GPU_ID} \
-  --solver models/${NET}/fast_rcnn/solver.prototxt \
+  --solver models/${PT_DIR}/${NET}/fast_rcnn/solver.prototxt \
   --weights data/imagenet_models/${NET}.v2.caffemodel \
-  --imdb voc_2007_trainval \
+  --imdb ${TRAIN_IMDB} \
+  --iters ${ITERS} \
   ${EXTRA_ARGS}
 
 set +x
 NET_FINAL=`grep -B 1 "done solving" ${LOG} | grep "Wrote snapshot" | awk '{print $4}'`
 set -x
 
 time ./tools/test_net.py --gpu ${GPU_ID} \
-  --def models/${NET}/fast_rcnn/test.prototxt \
+  --def models/${PT_DIR}/${NET}/fast_rcnn/test.prototxt \
   --net ${NET_FINAL} \
-  --imdb voc_2007_test \
+  --imdb ${TEST_IMDB} \
   ${EXTRA_ARGS}

experiments/scripts/faster_rcnn_alt_opt.sh

@@ -1,9 +1,11 @@
 #!/bin/bash
 # Usage:
-# ./experiments/scripts/default_faster_rcnn_alt_opt.sh GPU NET [--set ...]
+# ./experiments/scripts/faster_rcnn_alt_opt.sh GPU NET DATASET [options args to {train,test}_net.py]
+# DATASET is only pascal_voc for now
+#
 # Example:
-# ./experiments/scripts/default_faster_rcnn_alt_opt.sh 0 ZF \
-#   --set EXP_DIR foobar RNG_SEED 42 TRAIN.SCALES "[400,500,600,700]"
+# ./experiments/scripts/faster_rcnn_alt_opt.sh 0 VGG_CNN_M_1024 pascal_voc \
+#   --set EXP_DIR foobar RNG_SEED 42 TRAIN.SCALES "[400, 500, 600, 700]"
 
 set -x
 set -e
@@ -13,20 +15,38 @@ export PYTHONUNBUFFERED="True"
 GPU_ID=$1
 NET=$2
 NET_lc=${NET,,}
+DATASET=$3
 
 array=( $@ )
 len=${#array[@]}
-EXTRA_ARGS=${array[@]:2:$len}
+EXTRA_ARGS=${array[@]:3:$len}
 EXTRA_ARGS_SLUG=${EXTRA_ARGS// /_}
 
+case $DATASET in
+  pascal_voc)
+    TRAIN_IMDB="voc_2007_trainval"
+    TEST_IMDB="voc_2007_test"
+    PT_DIR="pascal_voc"
+    ITERS=40000
+    ;;
+  coco)
+    echo "Not implemented: use experiments/scripts/faster_rcnn_end2end.sh for coco"
+    exit
+    ;;
+  *)
+    echo "No dataset given"
+    exit
+    ;;
+esac
+
 LOG="experiments/logs/faster_rcnn_alt_opt_${NET}_${EXTRA_ARGS_SLUG}.txt.`date +'%Y-%m-%d_%H-%M-%S'`"
 exec &> >(tee -a "$LOG")
 echo Logging output to "$LOG"
 
 time ./tools/train_faster_rcnn_alt_opt.py --gpu ${GPU_ID} \
   --net_name ${NET} \
   --weights data/imagenet_models/${NET}.v2.caffemodel \
-  --imdb voc_2007_trainval \
+  --imdb ${TRAIN_IMDB} \
   --cfg experiments/cfgs/faster_rcnn_alt_opt.yml \
   ${EXTRA_ARGS}
 
@@ -35,8 +55,8 @@ NET_FINAL=`grep "Final model:" ${LOG} | awk '{print $3}'`
 set -x
 
 time ./tools/test_net.py --gpu ${GPU_ID} \
-  --def models/${NET}/faster_rcnn_alt_opt/faster_rcnn_test.pt \
+  --def models/${PT_DIR}/${NET}/faster_rcnn_alt_opt/faster_rcnn_test.pt \
   --net ${NET_FINAL} \
-  --imdb voc_2007_test \
+  --imdb ${TEST_IMDB} \
   --cfg experiments/cfgs/faster_rcnn_alt_opt.yml \
   ${EXTRA_ARGS}

experiments/scripts/faster_rcnn_end2end.sh

@@ -1,9 +1,11 @@
 #!/bin/bash
 # Usage:
-# ./experiments/scripts/default_faster_rcnn.sh GPU NET [--set ...]
+# ./experiments/scripts/faster_rcnn_end2end.sh GPU NET DATASET [options args to {train,test}_net.py]
+# DATASET is either pascal_voc or coco.
+#
 # Example:
-# ./experiments/scripts/default_faster_rcnn.sh 0 ZF \
-#   --set EXP_DIR foobar RNG_SEED 42 TRAIN.SCALES "[400,500,600,700]"
+# ./experiments/scripts/faster_rcnn_end2end.sh 0 VGG_CNN_M_1024 pascal_voc \
+#   --set EXP_DIR foobar RNG_SEED 42 TRAIN.SCALES "[400, 500, 600, 700]"
 
 set -x
 set -e
@@ -13,25 +15,43 @@ export PYTHONUNBUFFERED="True"
 GPU_ID=$1
 NET=$2
 NET_lc=${NET,,}
-ITERS=70000
-DATASET_TRAIN=voc_2007_trainval
-DATASET_TEST=voc_2007_test
+DATASET=$3
 
 array=( $@ )
 len=${#array[@]}
-EXTRA_ARGS=${array[@]:2:$len}
+EXTRA_ARGS=${array[@]:3:$len}
 EXTRA_ARGS_SLUG=${EXTRA_ARGS// /_}
 
-LOG="experiments/logs/faster_rcnn_${NET}_${EXTRA_ARGS_SLUG}.txt.`date +'%Y-%m-%d_%H-%M-%S'`"
+case $DATASET in
+  pascal_voc)
+    TRAIN_IMDB="voc_2007_trainval"
+    TEST_IMDB="voc_2007_test"
+    PT_DIR="pascal_voc"
+    ITERS=70000
+    ;;
+  coco)
+    # This is a very long and slow training schedule
+    # You can probably use fewer iterations and reduce the
+    # time to the LR drop (set in the solver to 350,000 iterations).
+    TRAIN_IMDB="coco_2014_train"
+    TEST_IMDB="coco_2014_minival"
+    PT_DIR="coco"
+    ITERS=490000
+    ;;
+  *)
+    echo "No dataset given"
+    exit
+    ;;
+esac
+
+LOG="experiments/logs/faster_rcnn_end2end_${NET}_${EXTRA_ARGS_SLUG}.txt.`date +'%Y-%m-%d_%H-%M-%S'`"
 exec &> >(tee -a "$LOG")
 echo Logging output to "$LOG"
 
-NET_INIT=data/imagenet_models/${NET}.v2.caffemodel
-
 time ./tools/train_net.py --gpu ${GPU_ID} \
-  --solver models/${NET}/faster_rcnn_end2end/solver.prototxt \
-  --weights ${NET_INIT} \
-  --imdb ${DATASET_TRAIN} \
+  --solver models/${PT_DIR}/${NET}/faster_rcnn_end2end/solver.prototxt \
+  --weights data/imagenet_models/${NET}.v2.caffemodel \
+  --imdb ${TRAIN_IMDB} \
   --iters ${ITERS} \
   --cfg experiments/cfgs/faster_rcnn_end2end.yml \
   ${EXTRA_ARGS}
@@ -41,8 +61,8 @@ NET_FINAL=`grep -B 1 "done solving" ${LOG} | grep "Wrote snapshot" | awk '{print
 set -x
 
 time ./tools/test_net.py --gpu ${GPU_ID} \
-  --def models/${NET}/faster_rcnn_end2end/test.prototxt \
+  --def models/${PT_DIR}/${NET}/faster_rcnn_end2end/test.prototxt \
   --net ${NET_FINAL} \
-  --imdb ${DATASET_TEST} \
+  --imdb ${TEST_IMDB} \
   --cfg experiments/cfgs/faster_rcnn_end2end.yml \
   ${EXTRA_ARGS}