Scaling 4D Representations – new preprint and models now available

3rd edition of the challenge with new exciting tasks and guest tracks; back during covid when we had the first workshop about the perception test ( some of us were afraid the benchmark was too difficult; now we just made it harder.

Most human knowledge was derived from our senses, part of it is passed around via language. Current AI excels on, but also depends on language. A next frontier of AI is developing the ability to create new knowledge from vision – large vision models may have a role to play there.

We found that larger models eventually surpass those trained with more sophisticated objectives, such as VJEPA, on non-semantic tasks (e.g. depth and camera pose estimation) while just matching them on higher-level tasks such as action recognition

There's been little work on video models beyond 1B parameters with self-supervision (e.g. no text or labels). In this work we used the simplest, lightest masked auto-encoding learning method and explored what happens when we scale models all the way to 22B parameters

RT @yanahasson: Thrilled to share our latest work on SciVid, to appear at #ICCV2025! 🎉.SciVid offers cross-domain evaluation of video model….

RT @sangwoomo: Can scaling data and models alone solve computer vision? 🤔.Join us at the SP4V Workshop at #ICCV2025 in Hawaii to explore th….

Individual frames out of generative video models tend to look reasonable; capturing actions happening over time realistically . that is way harder. TRAJAN is a new evaluation procedure to better guide progress in this (hot) area.

RT @TengdaHan: Check out our CVPR 2025 paper: Work with Dilara Gokay, Joseph Heyward, @ChuhanZhang5 , @DanielZoran….

RT @TengdaHan: We are looking for a student researcher to work on video understanding plus 3D, in Google DeepMind London. DM/Email me or pa….

RT @EEMLcommunity: Apply here: Confirmed speakers: @AaronCourville @AldenHung @dianaborsa @09Emmar @joaocarreira @….

RT @vansteenkiste_s: Excited to announce MooG for learning video representations. MooG allows tokens to move “off-the-grid” enabling better….

RT @dimadamen: Time to challenge VLMs?.Fed up of benchmarks claiming long-video reasoning but only need few seconds?. Try out Hour-Long VQA….

RT @skandakoppula: We're excited to release TAPVid-3D: an evaluation benchmark of 4,000+ real world videos and 2.1 million metric 3D point….

RT @shiryginosar: Join us next week at our second (high-level) intelligence workshop @SimonsInstitute!. Schedule: ..

The 2nd Perception Test Challenge is now on -- with a workshop happening in ECCV Milano later in the year. See all about it here and try out your top general perception models on it. Besides the original 6 tasks we'll have a new hour-long videoQA track.

RT @CarlDoersch: We present a new SOTA on point tracking, via self-supervised training on real, unlabeled videos! BootsTAPIR achieves 67.4%….

RT @shawshank_v: Delighted to host the 1st edition of our tutorial "Time is precious: Self-Supervised Learning Beyond Images" at @eccvconf….

RT @ShaneLegg: Our research project SIMA is creating a general, natural language instructable, multi 3D game-playing AI agent. The agent c….

Videos have a wealth of learning signal that is still underappreciated -- in fact, looks like a single long video can be as valuable as a large curated internet image dataset. Cool work from @shawshank_v et al with a new self-sup formulation where multi-object tracking emerges.