3D Face Reconstruction: Make a Realistic Avatar from a Photo

Source: Deep Learning on Medium

Comparison

I make some 3D faces form input photos. I choose some images from FFHQ dataset which has never be used as training dataset of both methods. So the samples are reasonable input photos for testing.

Both approaches generate beautiful results. Results of PRNet looks much more realistic, because the method provide high quality texture map which extract from an input photo directly. But sometime it is a cause of unexpected artifact especially on the invisible part of that like as side of her chin. Deep3DFace makes smoother results than PRNet. But it is really robust no large pose, so you can hard to find artifacts.

Both approaches can make good results for kids. We can easily guess the dataset include 3D shape for kids. For the texture map, PRNet well represents color of pupil, eyebrow and lip. And you can notice that Deep3DFace cannot generate teeth because the reconstruction is based on the 3DMM method. But Deep3DFace provide high resolution 3D shape, refer to details on philtrum.

Deep3DFace failed to reconstruct 3D shape from elders. It seems to be made from a photo when he was young. We can notice that it is a limitation of 3DMM method. Elders are unbounded the method. On the other hand PRNet shows features of elders likes white beard and wrinkles. But the method captured glasses as texture of skin. Glasses are attached on the skin. It Looks weird.

PRNet shows attributes of make-up based on high quality texture map. Deep3DFace failed to show for unbounded colors of skin. But the method can represent a detailed expression to a great extend. Look at her mouth up slightly!!

Did you notice the problem of PRNet? It is that the method extract unnecessary parts on face such as hair and glasses. It makes results more naturally on same pose, but if you change camera position, it would become artifacts. Since it attached on the skin like as tattoo. On the other hand Deep3DFace sometime failed to represent attribute of input photo, but it does not make unexpected artifacts. All results of Deep3DFace must be bounded on the 3DMM technique. Both methods have pros and cons.

  • Bad Lightning Condition(Shadow)

In the case of PRNet, the shadow and highlight are also extracted. We have to remove that, if we want to use the result for other applications. In the case of Deep3DFace, definitely proper teeth should be attached on the 3D shape.

In the case of profile, both method did not make a good result. In particular, PRNet could not extract textures from the invisible parts. You can see that the texture of the nose overlaps twice. If we turn the head more, we would see mouth ans eye overlaps several. I did not upload the image, because it’s terrible.

Conclusion

In above results, we saw that the quality of the results depends on conditions of input photo such as poses, ages, occlusions, lights and make-up. There are some conditions that work well for each method, but for getting good results you should use good input photos.

  • Frontal portrait
  • Reveal forehead, bind the hair
  • Take off glasses
  • Slight smile
  • Light makeup
  • Uniform illumination

Reference