Official pytorch implementation for FEN (Feature Enhancement Network)
Official implementation for our paper,
“Self-supervised Feature Enhancement Networks for Small Object Detection in Noisy Images”.
This paper has been accepted at IEEE Signal Processing Letters 2021.
Our paper can be viewed here.
Authors: Geonsoo Lee, Sungeun Hong, and Donghyeon Cho
Keywords: Small Object Detection, Self-supervised Learning, Noisy Image
Note: This code cannot be executed in Windows or Multi-GPU environment.
Our code uses Detectron2 developed by FAIR (Facebook AI Research).
Therefore, please visit the repository and install the appropriate version that fits your environment.
We have tested our code in the following environment.
In this paper, we have used two datasets.
One is DOTA (for train and test) and the other is ISPRS Toronto (for only test).
You can download pre-processed DOTA dataset for our paper in this link directly.
Please note that you can also download the raw dataset and pre-process it by yourself.
The structure of the pre-processed data is as follows.
Make sure Label.json follows the COCO data format.
DOTA.zip|-- Train| |-- Label.json| `-- Image| |-- Image_00000.png| |-- Image_00001.png| |-- Image_00002.png| `-- ...|-- Test| |-- Label.json| `-- Image| |-- Image_00042.png| |-- Image_00055.png| |-- Image_00060.png| `-- ...|-- Val| |-- Label.json| `-- Image| |-- Image_00066.png| |-- Image_00125.png| |-- Image_00130.png| `-- ...`-- Mini|-- Label.json`-- Image|-- Image_00066.png|-- Image_00125.png|-- Image_00130.png`-- ...
Note: This data cannot be used immediately due to its large resolution,
and we will distribute the pre-processing code as soon as possible.
(1) Please complete the data request form here.
(2) Access the FTP link you received by email.
(3) Download all .tif image files in [FTP LINK]/ISPRS_BENCHMARK_DATASETS/Toronto/Images.
(4) Download the label files we made here. Like DOTA, these annotations also follow the coco data format.
You can run run_train_net.py directly using IDEs like Pycharm.
In this case, you have to manually fill in the required parameters in the code.
You can also run run_train.py from the terminal with the command below.
Without FEN
python run_train.py --arch [FILL] # Select one in ['R50-FPN', 'R101-FPN', 'X101-FPN'] (Default: 'X101-FPN')--data_root [FILL] # Directory which contains 'Train', 'Test', 'Val' folders--output_dir [FILL]--noise_type [FILL] # Select one in ['none', 'gaussian', 'snp'] (Default: 'none')--noise_params [FILL]--input_size [FILL] # Size of training data (Default: 800)
With FEN
python run_train.py --arch [FILL] # Select one in ['R50-FPN', 'R101-FPN', 'X101-FPN'] (Default: 'X101-FPN')--use_fen--data_root [FILL] # Directory which contains 'Train', 'Test', 'Val' folders--output_dir [FILL]--noise_type [FILL] # Select one in ['none', 'gaussian', 'snp'] (Default: 'none')--noise_params [FILL]--input_size [FILL] # Size of training data (Default: 800)--fen_levels [FILL] # Make combinations using ['p2', 'p3', 'p4', 'p5', 'p6']# For example, --fen_levels p2 p4 p5
You can run run_test_net.py directly using the IDE, or you can run run_test.py using the terminal.
When using run_test.py, the command is as follows.
python run_test.py --ckpt_root [FILL]--data_root [FILL] # Directory which contains 'Image' folder and 'Label.json'--noise_type [FILL] # Select one in ['none', 'gaussian', 'snp'] (Default: 'none')--noise_params [FILL]--input_size [FILL] # Size of inference data (Default: 800)
Column (a) is the base result when no method is applied, (b) and (c) are the results of applying Noise2Void and DnCNN at the pixel domain, respectively, and (d) is the result of applying our method at the feature domain.
We have used five out of the standard evaluation metrics of COCO.
@ARTICLE{9432743,author={Lee, Geonsoo and Hong, Sungeun and Cho, Donghyeon},journal={IEEE Signal Processing Letters},title={Self-Supervised Feature Enhancement Networks for Small Object Detection in Noisy Images},year={2021},volume={28},number={},pages={1026-1030},doi={10.1109/LSP.2021.3081041}}