References: . This new paper and study: .. Binocular encoders for RPE and image-goal navigation:.(ICLR 2024) .. 5/5.

We show that nav and RPE performance can be partially transferred from simple to realistic sim, but cannot be fully recovered. Binocular ViTs with cross-attention + pre-training for rel pose estimation (RPE) remain the most efficient solution for image goal navigation. 4/5

We show that nav training only works when the sim setting is easy (sliding across walls is ok). While the impact of this on sim2real transfer was known, its impact on image goal nav is surprising: the setting is related to physics, whereas RPE is a computer vision problem. 3/5

Recent work trains simple low-capacity architectures e2e with RL on image goal nav and achieves good results. Given that rel pose estimation (RPE) is a necessary skill, this raises the question whether a fundamental problem of computer vision (RPE) could be trained w. RL. 2/5.

In a new paper led by Gianluca Monaci, with @WeinzaepfelP and myself, we explore the relationship between rel pose estimation and image goal navigation and study diff. architectures: late fusion, channel cat, space2depth and cross-attention. 🧵1/5

We have a new blog post on how we optimized e2e training of navigation in simulation with physical models, allowing fast and precise motion. The post is simplified, animated, and should be accessible. Great work by the Spatial AI team, writing by Steeven, myself and NLE Coms.

We have an open internship position on socially aware navigation, human aware world models etc. At @naverlabseurope in Grenoble (Meylan), France.

You can find me over there. Posting real science instead of things like these 😁.

We have made some post-conference improvements of our CVPR'25 paper on end-to-end trained navigation. The agent has similar success rate but is more efficient, faster, less hesitant. Will be presented at CVPR in June. . Work by @JannySteeven et al.

We have written a blog post on our research in end-to-end trained models for navigation:. - Fast and precise motion by training w realistic motion models.- Geometric foundation models.- Efficient training losses. (The @naverlabseurope Spatial AI team)

A big thanks to the @naverlabseurope Spatial AI team for this, but in particular to @JannySteeven, not only for the great work linking e2e training to a Kalman filter but also the D3.js magic making the real-time analytics work, and the interactive website!.

We find evidence for a plan structured on the level of paths; that its estimate of success goes beyond the effect of the next action. Abandoning a navigation option for a more promising one increases the value estimate, as the agent now expects a higher future return. #CVPR2025

Interactive website: . Large-scale study w 262 real episodes finds evidence for presence of realistic dynamics used for open-loop forecasting, interplay with sensing; usage of latent memory (emerging maps); value relates to long-term plans. #CVPR2025

Our e2d trained navigation agent has been accepted to #CVPR2025: fast-moving, efficient: correct modelling of motion during training in simulation. . By S Janny, H Poirier, L Antsfeld, G Bono, G Monaci, B Chidlovskii, F Giuliari, A Del Bue and myself.

I didn't post on Twitter for 2 months, but I will make an exception for this.

RT @_austrian: Dear guests, you can stay connected with us on Instagram, TikTok, Facebook & LinkedIn. Feel free to reach out anytime via ou….

There are now several benchmarks testing spatial reasoning and agent capabilities of LLMs and VLMs: . - (does spatial cognition . ) .- (MMBench). - (BALROG) - additional points for the LOTR ref.

We have a beautiful setting in the French Alpes. The two Figures above are from: .ICLR'24 CVPR'24

At @naverlabseurope in Grenoble, France, we are searching for talented PhD interns for work on Spatial AI, geometric and robotic foundation models for navigation and manipulation. If you have experience in Embodied AI and are interested, DM me.

There are 20 more characters per . tweet . on the other side 😃.