In few words: Face Hallucination describes Super-Resolution for facial images. It is a technique to improve the quality of noisy or low resultion facial images into high resolution images using knowledge about typical facial features. This technique can subsequently help to identify a person from an image faster and more effectively.
On the left image of figure 1 one can see the input from for example a surveillance camera in low resolution and upsampled via bicubic interpolation in order to make the size of the image larger. It is not easy to identify a unique person on the image. Image (d) shows a hallucinated face with a State-of-the-Art face cascaded bi-network (CBN). Image (b) and (c) of figure 1 are intermediate steps and not of interest in this example. The image on the right (e) show the ground truth image for comparison.
Figure 1: Visualization of the inputs outputs and intermediate results with an example image. (1)
It is obvious that a hallucinated face helps to improve detection performance and thus became a strong research area in the recent years.
One State-of-the-Art network shall be described in detail in this section to understand the functionalities. The Cascaded Bi-Network (CBN) was published in 2016 and has the following architecture.
Figure 2: Network architecture of CBN. Two branches are merged together with a gate branch. (1)
The network can be devided into three parts. First part is the common branch. This branch acts as a simple super resolution neural network. It predicts the residual image between bicubic interpolation and ground truth. It´s input are bicubic interpolated images. The second part is the high frequency branch. This part gets bicubic interpolated images and 10 contour maps, which are generated from all training examples prior the training. Each map contains the contour or a facial feature like, mouth, nose, etc. These two branches are merged with a gate which is the third part of the network. This network has as input all input from the branches before plus their output. All parts consist of convolutional neural networks and are trained independently.
Figure 3 illustrates some results from this network:
Figure 3: Results of CBN with three examples compared to 2 other approaches. (1)
The CBN approach outperforms in comparison to other state of the art techniques.
Common metrics to measure the performance of Face Hallucination are SSIM and PSNR. See Super-Resolution for more information about the metrics.
Current application possibilities are:
- Improving automatical face recognition or face identification
- Improving facial landmark detection
- Image enhancement
- Image compression
- http://personal.ie.cuhk.edu.hk/~ccloy/files/eccv_2016_hallucination.pdf (Deep Cascaded Bi-Network for Face Hallucination; Shizhan Zhu, Sifei Liu, Chen Change Loy, Xiaoou Tang)