Source: Deep Learning on Medium
First Large-scale Dataset with Geometric Meshes of Body and Clothes
This research summary is just one of many that are distributed weekly on the AI scholar newsletter. To start receiving the weekly newsletter, sign up here.
Deep learning has significantly improved the challenge that existed in predicting the geometry of the human body from single images. Now, CNN’s combined with large datasets have resulted in several developments that robustly predict the 3D position of the body joints.
However, existing techniques are not appropriate in representing clothing geometry.
3DPeople: Dataset for Modeling the Geometry of Dressed Humans
New research has introduced a new mechanism for modeling dressed humans and predicting their geometry from single images. Research contributions include 3DPeople, a new dataset, a new shape parameterization model and an end-to-end generative model to predict shape.
The 3DPeople dataset is a large-scale synthetic and comprises 2.5 million photo-realistic images of 80 characters performing 70 activities and clothed in different outfits. The dataset is annotated with segmentation masks, depth, skeletons, normal maps and optical flow which make it suitable for a myriad of tasks.
To generate the images, the researchers propose yet a new spherical area-preserving parameterization algorithm which is an improvement of the existing spherical maps which shrinks elongated body parts making the geometry images incomplete.
Lastly, the generative network is used to generate a geometry image of a dressed human in an end-to-end manner. The method achieves promising results in jointly capturing body pose and clothing shape, both for synthetic and wild images.
Potential Uses and Effects
The contributions in this research work have great potential to advance shapes in dressed human reconstruction that the deep learning community can leverage. Additionally, it presents further research that can extend to video, geometry images regularization schemes, segmentation, and 3D reconstruction integration as they all can significantly benefit from the 3DPeople dataset.
Read more: https://arxiv.org/abs/1904.04571