New Dataset for Modeling the Geometry of Dressed Humans from Single Images

Source: Deep Learning on Medium


First Large-scale Dataset with Geometric Meshes of Body and Clothes

This research summary is just one of many that are distributed weekly on the AI scholar newsletter. To start receiving the weekly newsletter, sign up here.

Deep learning has significantly improved the challenge that existed in predicting the geometry of the human body from single images. Now, CNN’s combined with large datasets have resulted in several developments that robustly predict the 3D position of the body joints.

However, existing techniques are not appropriate in representing clothing geometry.

3DPeople: Dataset for Modeling the Geometry of Dressed Humans

New research has introduced a new mechanism for modeling dressed humans and predicting their geometry from single images. Research contributions include 3DPeople, a new dataset, a new shape parameterization model and an end-to-end generative model to predict shape.

Annotations of the 3D People Dataset. The bottom of the figure shows 5 sample frames of the Running sequence. Every RGB frame is annotated with the information reported in the top of the figure. 3DPeople is the first large-scale dataset with geometric meshes of body and clothes

The 3DPeople dataset is a large-scale synthetic and comprises 2.5 million photo-realistic images of 80 characters performing 70 activities and clothed in different outfits. The dataset is annotated with segmentation masks, depth, skeletons, normal maps and optical flow which make it suitable for a myriad of tasks.

Qualitative results. For the synthetic images we plot our estimated results and the shape reconstructed directly from the ground truth geometry image. In all cases we show two different views. The color of the meshes encodes the xyz vertex position.

To generate the images, the researchers propose yet a new spherical area-preserving parameterization algorithm which is an improvement of the existing spherical maps which shrinks elongated body parts making the geometry images incomplete.

Lastly, the generative network is used to generate a geometry image of a dressed human in an end-to-end manner. The method achieves promising results in jointly capturing body pose and clothing shape, both for synthetic and wild images.

Potential Uses and Effects

The contributions in this research work have great potential to advance shapes in dressed human reconstruction that the deep learning community can leverage. Additionally, it presents further research that can extend to video, geometry images regularization schemes, segmentation, and 3D reconstruction integration as they all can significantly benefit from the 3DPeople dataset.

Read more: https://arxiv.org/abs/1904.04571

Thanks for reading. Please comment, share and remember to subscribe! Also, follow me on Twitter and LinkedIn. Remember to 👏 if you enjoyed this article. Cheers!