Vision-based 3D Reconstruction for Large Scale Scenes using 360 Cameras
Vision-based 3D Reconstruction for Large Scale Scenes using 360 Cameras
We present a feed-forward neural network for visual geometry reconstruction that supports wide field-of-view camera models.
We visualize some in-the-wild videos, demonstrating the generalizability of our method.
💡 Please Note: To ensure a smooth interactive experience, the point clouds in this demo are downsampled. The quality of the full model is much higher.
Rotate
Hold & Drag Left Mouse
Pan
Hold & Drag Right Mouse
Zoom
Use Mouse Scroll Wheel
We deployed our reconstructed home environment into Unity and built a VR application that lets users explore the space with a Head-Mounted Display (HMD). Walking through the virtually reconstructed house in 6-DoF, users can experience an immersive tour that faithfully captures the layout, geometry, and appearance of the real-world scene.
We obtain dense 3D point clouds of the target scene using our feedforward depth estimation network, which recovers geometry from monocular video in a single pass.
The point cloud is converted into a watertight mesh via Poisson surface reconstruction, then high-fidelity textures are baked onto the mesh using MVS-Texturing.
The textured mesh is imported into Unity and rendered in a VR scene, enabling users to freely navigate the reconstructed home through an HMD.