Farm3D

Farm3D Overview

Farm3D is an innovative tool developed by researchers at the University of Oxford that enables monocular 3D reconstruction. The tool learns an articulated object category entirely from "free" virtual supervision from a 2D diffusion-based image generator. The framework employs an image generator, such as Stable Diffusion, to produce training data for learning a reconstruction network from scratch. The diffusion model is also incorporated as a scoring mechanism to enhance learning. Farm3D is capable of generating controllable 3D assets from a single input image, whether real or generated, in a matter of seconds.

Farm3D Highlights

• Farm3D can reconstruct the shape of a wide range of categories to their fine details such as legs and ears despite not being trained on any real images.
• The tool enables the generation of controllable 3D assets from either a real image or an image synthesised using Stable Diffusion.
• Once generated, users have the ability to adjust lighting, swap textures between models of the same category, and even animate the shape.