Simple-SR

The repository includes MuCAN, LAPAR, Beby-GAN and etc. It is designed for simple training and evaluation.

Update

The training code of BebyGAN and LAPAR is released.

The well-trained models and all visual examples (including BebyGAN and other SOTAs) are available here.

Paper

MuCAN: Multi-Correspondence Aggregation Network for Video Super-Resolution

[ECCV] [arXiv]

LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-resolution and Beyond

[NeurIPS] [arXiv]

Best-Buddy GANs for Highly Detailed Image Super-Resolution

[arXiv]

Please find supplementary files of MuCAN and LAPAR here.

Usage

Clone the repository

git clone https://github.com/Jia-Research-Lab/Simple-SR.git

Install the dependencies
- Python >= 3.6
- PyTorch >= 1.2
- spatial-correlation-sampler
```
pip install spatial-correlation-sampler
```
- Other packages
```
pip install -r requirements.txt
```
Download pretrained models from Google Drive. We re-trained the LAPAR models and their results are slightly different from the ones reported in paper.
- MuCAN
  - MuCAN_REDS.pth: trained on REDS dataset, 5-frame input, x4 scale
  - MuCAN_Vimeo90K.pth: trained on Vimeo90K dataset, 7-frame input, x4 scale
- LAPAR: trained on DIV2K+Flickr2K datasets
  
  Scale x2 Scale x3 Scale x4
  
  LAPAR_A_x2.pth LAPAR_A_x3.pth LAPAR_A_x4.pth
  
  LAPAR_B_x2.pth LAPAR_B_x3.pth LAPAR_B_x4.pth
  
  LAPAR_C_x2.pth LAPAR_C_x3.pth LAPAR_C_x4.pth
- BebyGAN
  - RRDB_warmup.pth: provided for initialization
  - BebyGAN_x4.pth: trained on DIV2K+Flickr2K datasets, x4 scale

Quick test

You have to define the output_path or gt_path or both. If output_path is given, outputs will be saved. If gt_path is given, PSNR/SSIM will be calculated. If you want to calculate LPIPS results, please install lpips library.

For SISR,

python3 test_sample.py --sr_type SISR --model_path /model/path --input_path ./demo/SISR/LR_imgs --output_path ./demo/SISR/output --gt_path ./demo/SISR/HR_imgs

For VSR,

python3 test_sample.py --sr_type VSR --model_path /model/path --input_path ./demo/VSR/LR_imgs --output_path ./demo/VSR/output --gt_path ./demo/VSR/HR_imgs

Prepare Data

Training Datasets

Download DIV2K and Flickr2K. You may crop the HR and LR images to sub-images for fast reading referring to .utils/data_prep/extract_subimage.py.
Evaluation Datasets

Download Set5, Set14, Urban100, BSDS100 and Manga109 from Google Drive uploaded by BasicSR.
Update the dataset location in .dataset/__init__.py.
(Optional) You can convert images to lmdb files for fast loading referring to BasicSR. And you need to modify the data reading logics in .dataset/*dataset.py accordingly.

Train

Create a log folder as
```
mkdir logs
```
Create a new experiment folder in .exps/. You just need to prepare the config.py and network.py, while the train.py and validate.py are universal. For example, for LAPAR_A_x2, run
```
cd exps/LAPAR_A_x2/
bash train.sh $GPU_NUM $PORT
```
Please modify the path of initialization model of BebyGAN (found in Google Drive) in config.py before training.

Notice that you can find the checkpoints, log files and visualization images in either .exps/LAPAR_A_x2/log/ (a soft link) or .logs/LAPAR_A_x2/.

Test

Please refer to validate.py in each experiment folder or quick test above.

Acknowledgement

We refer to BasicSR for some details.

Bibtex

@inproceedings{li2020mucan,
  title={MuCAN: Multi-correspondence Aggregation Network for Video Super-Resolution},
  author={Li, Wenbo and Tao, Xin and Guo, Taian and Qi, Lu and Lu, Jiangbo and Jia, Jiaya},
  booktitle={European Conference on Computer Vision},
  pages={335--351},
  year={2020},
  organization={Springer}
}
@article{li2020lapar,
  title={LAPAR: Linearly-Assembled Pixel-Adaptive Regression Network for Single Image Super-resolution and Beyond},
  author={Li, Wenbo and Zhou, Kun and Qi, Lu and Jiang, Nianjuan and Lu, Jiangbo and Jia, Jiaya},
  journal={Advances in Neural Information Processing Systems},
  volume={33},
  year={2020}
}
@article{li2021best,
  title={Best-Buddy GANs for Highly Detailed Image Super-Resolution},
  author={Li, Wenbo and Zhou, Kun and Qi, Lu and Lu, Liying and Jiang, Nianjuan and Lu, Jiangbo and Jia, Jiaya},
  journal={arXiv preprint arXiv:2103.15295},
  year={2021}
}

License

linjie2023/Simple-SR

ErrorLooks like something went wrong!

About

Resources

License

Uh oh!

Stars

Watchers

Forks

Releases

Packages 0

Languages

Error
Looks like something went wrong!

Packages