Happy to present E.T. the Exceptional Trajectories: Text-to-Camera-Trajectory Generation with Character Awareness. ECCV2024 with @nico_dufour, @xiwang92, @MarcChristie4 and @VickyKalogeiton Paper: https://t.co/eNdGW9EOSz Webpage: https://t.co/UqrE6FlXyp

Text-to-Image models don't need 3 training stages anymore! 🤯 Our new MIRO method integrates human alignment directly into pretraining. 19x faster convergence ⚡ 370x less compute than FLUX-dev 📉 Train once, align to many rewards. The era of multi-stage training is over!

Introducing Chapter-Llama [#CVPR2025], a framework for 𝐯𝐢𝐝𝐞𝐨 𝐜𝐡𝐚𝐩𝐭𝐞𝐫𝐢𝐧𝐠 using Large Language Models! 🎬🦙 Check it out: 📄 Paper: https://t.co/1KhPsgZYUN 🔗 Project: https://t.co/68GevYyznx 💻 Code: https://t.co/MysWVlewRm 🤗 Demo: https://t.co/zKmL6v3PKU

Masked Diffusion Models (MDMs) are a hot topic in generative AI 🔥 — powerful but slow due to multiple sampling steps. We @Polytechnique and @Inria introduce Di[M]O — a novel approach to distill MDMs into a one-step generator without sacrificing quality.

1/13 🐊 Introducing our latest work on improving relative camera pose regression with a novel pre-training approach Alligat0R ( https://t.co/Mi6iy5rQ1A)! @GBourmaud @VincentLepetit2

🚨 News! 🚨 We have released the models from our latest paper "How far can we go with ImageNet for text-to-image generation?" Check out the models on HuggingFace: 🤗 https://t.co/jaNyoNDN6u... 📜 https://t.co/gH6gct7lUA

🎥 AKiRa provides control over camera motion and optics (focal length, distortion, aperture) in video diffusion, enabling cinematic effects like fisheye, focus shifts, and dolly zoom. 📄 Paper: https://t.co/0ajalZXZ3a 👉 Project Page: https://t.co/uGqwhFPWLK 🧵👇

🌍 Guessing where an image was taken is a hard, and often ambiguous problem. Introducing diffusion-based geolocation—we predict global locations by refining random guesses into trajectories across the Earth's surface! 🗺️ Paper, code, and demo: https://t.co/pNRFZk9NYP

I am presenting our paper MusicGen-Style “Audio Conditioning for Music Generation via Discrete Bottleneck Features” at @ISMIRConf this afternoon. The code as well as the weights of the model are available on https://t.co/tSvrr446v3. You can now play with it!

We are in Milan 🇮🇹 to present 🎥 E.T. the Exceptional Trajectories: Text-to-camera-trajectory generation with character awareness. 📍 Come see our poster #227 this afternoon at #ECCV2024! 🚀 Introducing new dataset, diffusion model, and evaluation metric for camera generation!

#ECCV2024 #Blender If you're interested in high-quality Gaussian Splatting representations that can be edited and animated in Blender without a single line of code, Please come to our Oral Presentation of our ECCV2024 paper Gaussian Frosting on Oct2 at 1:40PM! Thread (1/n) 🧵

#ECCV2024 Oct 2 (PM) E.T. the Exceptional Trajectories: Text-to-Camera-Trajectory Generation with Character Awareness. @robin_courant, @nico_dufour, @xiwang92, @MarcChristie4 and @VickyKalogeiton pdf: https://t.co/ehKsij3e2V webpage: https://t.co/CbimUjiGhR

(1/8) 🎬 Introducing the Short Film Dataset (SFD), a long video QA benchmark with 1k short films and 5k questions. Why another videoQA dataset? 📖 Story-level QAs 🎥 Publicly available videos 🔒 Minimal data leakage ⏳ Long temporal context questions https://t.co/FJQzIRgDxV

We now have a first version of the 512x512 model in the demo! Still training but we will release weights soon!

Very happy to announce that my paper “Audio Conditioning for Music Generation via Discrete Bottleneck Features“ done with @honualx @adiyossLC @jadecopet and Axel Roebel has been accepted at ISMIR24. Paper: https://t.co/2KwG6Bk1jH Sample: https://t.co/Dkom70Eoie Code: soon

Paper: https://t.co/eNdGW9EOSz Webpage: https://t.co/UqrE6FlXyp Code: https://t.co/h2PxDUXEYn Dataset: https://t.co/1AFQWtQoU5 Demo:

Additionally, we train CLaTr, a Contrastive Language-Trajectory embedding, to facilitate the evaluation of camera trajectory generation models.

DIRECTOR exhibits high controllability and diversity, is character-aware, and handles complex input conditions.

We demonstrate the potential of our dataset with DIRECTOR, a camera trajectory diffusion model that leverages both character trajectories and captions.

We introduce a camera trajectory dataset called Exceptional Trajectories (E.T.), extracted from real movies. E.T. includes camera and character trajectories along with textual captions.