FaPN: Feature-aligned Pyramid Network for Dense Image Prediction [arXiv] [Project Page]

@inproceedings{
  huang2021fapn,
  title={{FaPN}: Feature-aligned Pyramid Network for Dense Image Prediction},
  author={Shihua Huang and Zhichao Lu and Ran Cheng and Cheng He},
  booktitle={International Conference on Computer Vision (ICCV)},
  year={2021}
}

Overview

FaPN vs. FPN	Before vs. After Alignment

This project provides the official implementation for our ICCV2021 paper "FaPN: Feature-aligned Pyramid Network for Dense Image Prediction" based on Detectron2. FaPN is a simple yet effective top-down pyramidal architecture to generate multi-scale features for dense image prediction. Comprised of a feature alignment module (FAM) and a feature selection module (FSM), FaPN addresses the issue of feature alignment in the original FPN, leading to substaintial improvements on various dense prediction tasks, such as object detection, semantic, instance, panoptic segmentation, etc.

Installation

This project is based on Detectron2, which can be constructed as follows.

• Install Detectron2 following the instructions.
• Setup the dataset following the structure.
• Copy this project to /path/to/detectron2
• Install DCNv2 following Install DCNv2.md.

Training

To train a model with 8 GPUs, run:

cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8

For example, to launch Faster R-CNN training (1x schedule) with ResNet-50 backbone on 8 GPUs, one should execute:

cd /path/to/detectron2/tools
python3 train_net.py --config-file ../configs\COCO-Detection\faster_rcnn_R_50_FAN_1x.yaml --num-gpus 8

Evaluation

To evaluate a pre-trained model with 8 GPUs, run:

cd /path/to/detectron2/tools
python3 train_net.py --config-file <config.yaml> --num-gpus 8 --eval-only MODEL.WEIGHTS /path/to/model_checkpoint

Results

COCO Object Detection

Faster R-CNN + FaPN:

Name	lr sched	box AP	box APs	box APm	box APl	download
R50	1x	39.2	24.5	43.3	49.1	model |  log
R101	3x	42.8	27.0	46.2	54.9	model |  log

Cityscapes Semantic Segmentation

PointRend + FaPN:

Name	lr sched	mask mIoU	mask i_IoU	mask IoU_sup	mask iIoU_sup	download
R50	1x	80.0	61.3	90.6	78.5	model |  log
R101	1x	80.1	62.2	90.8	78.6	model |  log

COCO Instance Segmentation

Mask R-CNN + FaPN:

Name	lr sched	mask AP	mask APs	box AP	box APs	download
R50	1x	36.4	18.1	39.8	24.3	model |  log
R101	3x	39.4	20.9	43.8	27.4	model |  log

PointRend + FaPN:

Name	lr sched	mask AP	mask APs	box AP	box APs	download
R50	1x	37.6	18.6	39.4	24.2	model |  log

COCO Panoptic Segmentation

PanopticFPN + FaPN:

Name	lr sched	PQ	mask mIoU	St PQ	box AP	Th PQ	download
R50	1x	41.1	43.4	32.5	38.7	46.9	model |  log
R101	3x	44.2	45.7	35.0	43.0	53.3	model |  log