Human trajectory prediction often assumes consistent behavior trends over time. But human behavior is transient: party-goers act differently at night than when going to work. To address this, our ECCV ’24 paper introduces “latent corridors” for rapid deployment scene adaptation.

RT @mjagadeesan25: I'm so excited to be joining @Penn as an Assistant Professor in CS (@CIS_Penn) in Fall 2026!. I’ll be working on machin….

Join millions who have switched to Grok.

I’m giving an invited spotlight talk at 4:15 today at the agents-in-interactions workshop @CVPR (Room 213). Hope to see you there! . Workshop schedule/info: #CVPR2025.

I’ll be presenting this paper at CVPR! Check out my invited talk on this work at the Agents in Interaction workshop at 4:15 on Thursday June 12, and come chat at my poster at 10:30-12:30 on Saturday June 14.

A huge thanks to my co-authors @TaraSadjadpour @brjathu @shiryginosar @JitendraMalikCV . project: arxiv: code:

Overall, poly-autoregressive prediction is a strong paradigm and a great starting baseline for real-world interactive problems! We have released the code which is hopefully helpful to other researchers working on multi-agent predictive problems :).

All our results used the same small 4M parameter transformer without any modifications to the base framework, architecture, or hyperparameters (aside from LR/EMA).

Car trajectory prediction: modeling multiple cars together using PAR results in better prediction of the ego agent — here the PAR model predicts safer behavior than the AR model

Object pose forecasting: when predicting the pose of an object during hand-object interaction, also modeling the hand using PAR helps make better rotation and translation predictions

And across almost all of the multi-person AVA classes, we see a significant mAP gain using our PAR model over the AR baseline

Social action forecasting: taking into account an interacting agent via PAR results in more accurate action prediction. Here, the PAR model predicts accurate conversational turn-taking (the man stops talking and listens when the woman starts speaking) the AR model doesn't

A few simple components—like training for same-agent next-timestep prediction and adding agent ID embeddings—makes our framework effective at multi-agent modeling across diverse scenarios, including:

Compared to autoregressive models, our poly-autoregressive models take other agents’ tokens as input when making next timestep predictions.

Can we systematically generalize AR "word models" into "world models”? Our CVPR 2025 paper introduces a unified, general framework designed to model real-world, multi-agent interactions by disentangling task-specific modeling from behavior prediction.

RT @CarlDoersch: We're very excited to introduce TAPNext: a model that sets a new state-of-art for Tracking Any Point in videos, by formula….

RT @iclr_conf: For #ICLR2025, we piloted an LLM that provided optional feedback to some reviewers. Results are promising: over 12K suggesti….

Super exciting progress in representation learning from @brjathu !.

RT @brjathu: An Empirical Study of Autoregressive Pre-training from Videos. paper: website: .

RT @ml_angelopoulos: 🚨 New Textbook on Conformal Prediction 🚨. “The goal of this book is to teach the reader about….

RT @r_ahulravi: 🚨 New Preprint!. We're excited to announce our new paper, "Scaling Properties of Diffusion Models For Perceptual Tasks.". P….

RT @shiryginosar: I am recruiting exceptional PhD students & postdocs with an adventurous soul for my💫new TTIC AI lab💫! We aim to understan….