LoopVL: Introducing Recurrent Computation to Vision-Language Models
LoopVL is a new vision-language model architecture that leverages recurrent computation through Module-Loop and Model-Loop mechanisms, achieving superior performance and novel interpretability features compared to non-recurrent models.
LoopVL is a newly proposed model that extends Loop Transformers to the vision-language domain by employing both Module-Loop and Model-Loop computation. This design allows the model to iteratively update a unified vision-language state using shared modules, enabling deeper and more dynamic multimodal computation.
The researchers trained LoopVL from scratch on language, multimodal, and post-training phases. The model outperformed both similarly sized and larger non-recurrent baselines on a range of benchmarks focused on multimodal understanding and visual reasoning.
A notable interpretability feature observed in LoopVL is the emergence of 'Visual Aha Moments,' where the model demonstrates pronounced shifts in visual attention as it iterates, providing insights into its reasoning process.
This work provides practical evidence that recurrent architectures like Loop Transformers can be effective in vision-language modeling and highlights the benefits of parameter sharing for continuous state updates in multimodal settings.
- LoopVL employs recurrent updating of visual-language states via Module-Loop and Model-Loop computation.
- Outperforms non-recurrent models on multimodal reasoning and understanding tasks.
- Demonstrates distinct interpretability features ('Visual Aha Moments') during inference.
- Supports deeper, evolving multimodal computation through parameter sharing.
