Skip directly to content

Face Reconstruction with Variational Autoencoder and Face Masks

on Fri, 10/15/2021 - 15:54
TitleFace Reconstruction with Variational Autoencoder and Face Masks
Publication TypeConference Proceedings
Year of Conference2021
AuthorsToledo RS, Antonelo EA
Conference Name18th Encontro Nacional de InteligĂȘncia Artificial e Computacional (ENIAC 2021)

Variational AutoEncoders (VAE) employ deep learning models to learn a continuous latent z-space that is subjacent to a high-dimensional observed dataset. With that, many tasks are made possible, including face reconstruction and face synthesis. In this work, we investigated how face masks can help the training of VAEs for face reconstruction, by restricting the learning to the pixels selected by the face mask. An evaluation of the proposal using the celebA dataset shows that the reconstructed images are enhanced with the face masks, especially when SSIM loss is used either with l1 or l2 loss functions. We noticed that the inclusion of a decoder for face mask prediction in the architecture affected the performance for l1 or l2 loss functions, while this was not the case for the SSIM loss. Besides, SSIM perceptual loss yielded the crispest samples between all hypotheses tested, although it shifts the original color of the image, making the usage of the l1 or l2 losses together with SSIM helpful to solve this issue.