Real-ESRGAN a generative adversarial network for image enanching
For a license plate recognition project I had the problem of having low quality source images and I wondered how to improve the image quality before carrying out the video analysis with a neural network.
Analyzing various gan neural networks for image enhancement I found this GAN for image enanching which gave impressive results with an average inference time of 3/4 seconds on NVIDIA Geforce RTX 2080 GPU which allowed me to build a video analysis and inference pipeline in which the source image passed to GAN for enhancement and later to CNN for license plate recognition and detection.
Real-ESRGAN is based on PyTorch framework and very easy to install both on linux/ubuntu and windows system using virtual enviromets like Anaconda https://github.com/xinntao/Real-ESRGAN
In the github repository there is a detailed guide for installing with CUDA both in windows and Linux / Ubuntu which is the system I used without encountering installation problems.
It is possible to download various weights of the pre-trained network from the repository depending on the scale factor to be used in the processing and the type of improvement processing required. However, it is possible to train the network with its own image dataset.
There are already parameters to select the gpu to be used in case of a multi gpu system, for the subdivision in analysis batches, the subdivision in tiles of the image very useful in case you have to work on very large images that can run in out of memory the gpu.
In my case I found myself very well using the pre-installed realesrnet-x2plus network which gave me an excellent compromise of processing time and improvement of the source image. Having a multi GPU system available, I was able to select the GPU to be used for image enhancement, reserving the other GPUs for the license plate recognition job.
Below is an example of a source image and then the result using Real-ESRGAN with pretrained weights realesrnet-x2plus. The license plate numbers are covered for privacy reasons but the improvement visible in the image is even better in the characters of the license plate.
