Novel view synthesis and SfM from unposed and uncalibrated multiview images \(\mathcal{I}\) .
For a set of images, split it into two disjoint sets. Reconstruct scene using one set, train the model to render the second set.
Camera Estimation
- Take images, predict pose and intrinsics for each image, form Plücker ray maps \(\mathcal{R}\) .
Latent Scene Representation
- Take images and raymaps, learn scene tokens that encode the scene
Rendering
- take the scene tokens, raymaps and use a rendering decoder to predict images
The fact that this even works is interesting.