Given a set of images, output local and global pointmaps + confidences for each image. Can recover cameras later using PnP.
Architecture
Dust3r inspired (Croco ViT encoder)
Index embedding
Draw \(N \ll N', N' \approx 10^{3}, N \approx 20\) indices at random during training. This allows scaling to \(N'\) views during inference.
Performance
- Trained for 6.13 days on 128xA100-80GB
- Flash attention and DeepSpeed ZeRO stage-2 training
- Clever paralellization strategies for inference in thousands of images.