2025 Theses Doctoral

Generative Models for 3D Content without Massive 3D Datasets

Wu, Rundi

The creation of high-quality 3D content remains a major bottleneck across a wide range of applications, from entertainment and virtual reality to robotics and industrial design. While traditional 3D modeling and reconstruction pipelines require extensive manual effort, recent advances in generative AI offer the promise of automating and accelerating this process. However, the development of generalizable 3D generative models has been severely hampered by the scarcity of large-scale high-quality 3D datasets, especially in contrast to the abundance of 2D image data that has fueled progress in image and video generation.

This thesis addresses this fundamental data challenge by presenting a collection of methods designed to circumvent the need for massive 3D datasets, by leveraging different forms of supervision and prior knowledge. We explore diverse strategies tailored to different applications, including creating 3D variations by learning the local patch distribution of a single exemplar; generating 3D shapes in the form of programs; enhancing 3D and 4D reconstruction by leveraging powerful 2D image generative priors.

Together, these works demonstrate viable pathways towards automating and democratizing high-quality 3D content creation by reducing the reliance on large, curated 3D datasets.