Home

Human4D: Acquisition, Analysis and Synthesis of Human Body Shape in Motion (2019-2024, ANR-19-CE23-0020) is a French national project funded by ANR (Agence National de la Recherche). It has been selected as a collaborative project (programme PRC: Projet de recherche collaborative) by AI scientific committee (CES 23 - Intelligence artificielle). The consortium is composed of 4 nationwide partners : CRIStAL, ICube, INRIA (Grenoble Rhône-Alpes), and LIRIS.

Human4D is concerned with the problem of 4D human shape modeling. Reconstructing, characterizing, and understanding the shape and motion of individuals or groups of people have many important applications, such as ergonomic design of products, rapid reconstruction of realistic human models for virtual worlds, and an early detection of abnormality in predictive clinical analysis. Naturally, the capture and analysis of people’s shape and motion have a long tradition in disciplines such as computer vision, computer graphics, and virtual reality. This is evidenced by the large amount of research done on shape reconstruction from images and 3D scans, on subspace construction with multiple shapes, on motion capture and action recognition from video inputs. However, most current techniques treat shape and motion independently, with devoted techniques for either shape or motion in isolation. This is largely due to the difficulty of acquiring proper observations on full moving shapes: Traditional systems have been devoted to capture either static shapes, e.g. 3D scanners, or motion only, e.g. motion capture.

Recent evolutions in the technology for capturing moving shapes have changed this paradigm with new multi-view acquisition systems that enable now full 4D models of human shapes including geometry, motion and appearance, as in commercial platforms deployed by Intel Studio or Microsoft Hololense, among others. Such data open new possibilities and challenges for the analysis and the synthesis of human shapes in motion that are yet largely unexplored but would be of benefit to a wide range of applications in virtual and augmented reality, or in the sport and medical domains. This is especially true with the rapidly growing VR/AR immersive applications based on head mounted displays, which require realistic and detailed models to improve the immersive experience. Magic leap, Microsoft Hololense, Facebook Oculus Rift, Sony PS4 HMD and the HTC Vive, among others, are clear examples of this recent and rapid evolution and the associated need to produce adapted realistic contents. In the future we will be able to make digital copies of moving persons using a handy imaging device, send them over the network, and make customized compositions of the retrieved 4D human data in our daily life. Human4D aims at contributing to this evolution with objectives that can profoundly improve the reconstruction, transmission, and reuse of digital human data, by unleashing the power of recent deep learning techniques and extending it to 4D human shape modeling.

The expected outcome of Human4D can be summarized as follows:

• A database of 4D human body shapes from multi-view sequences, composed of diverse individuals of different physical characteristics and exhibiting different actions.
• A full pipeline for the reconstruction and representation of highly precise 4D models of human.
• A compact, scalable, and readily available 4D human atlas based on the above database of 4D human models.
• An extension of Deep Learning techniques to 3D and 4D data.
• Representative applications demonstrating the efficiency of the above 4D human atlas with the recovery, the synthesis and the analysis of 4D human models.