Does 3D generation always have to be either slow or complex and data-hungry?🤔 We don’t think so! With Geometry Image Diffusion, we’re all about reusing (and recycling ♻️) what already works — making it faster and easier by reducing complexity and data needs 🚀(1/10)

Thrilled to share that Geometry Image Diffusion has been accepted to #ICLR2025! 🚀 Paper:

I’m currently exploring new job opportunities🧑‍🔬 My work revolves around text-to-3D with Geometry Images, generative UV mapping, multi-view models for texturing, and other genAI applications in graphics. I’d love to discuss how I can contribute to your research efforts!

P.P.S. We recommend you check out Omages ( https://t.co/HUwFh95dA1) by @yan_xg , an awesome concurrent work that also explores geometry images (called "Omages") for 3D generation. We believe GIMs have a bright future in deep learning — let’s bring it forward together 🚀

P.S. Thanks to @CiaraRowles1, Simon Donné, @esx2ve, @danteCIM, and @bostadynamics for being such an awesome team! Additional thanks to @xdralex and Dr. Lev Melnikovsky from the Weizmann Institute for all the insightful discussions we had during this project

So whether you’re looking for speed, flexibility, or eco-friendly workflows, Geometry Image Diffusion has you covered. Got curious? Dive into our paper to learn more! Paper: https://t.co/48UQp1qY3O Site: https://t.co/PaUwPcDmEM (10/10)

And we’re not just saving forests. The assets you generate with Geometry Image Diffusion are free from baked-in lighting. Re-light them in any environment to fit your scene and save some energy while you’re at it! 💡 (9/10)

(But I must admit that it’s hard to resist generating thousand barrels because they’re all so different)

Why produce a thousand barrels? Let’s save the forest! 🌳 Just edit the one you’ve already generated (8/10)

Want an unexpected twist? The generated 3D objects come with meaningful, separable parts, making them easy to edit and manipulate (7/10)

Our assets can be easily triangulated by connecting neighboring pixels, and come unwrapped with textures included—no waste here ♻️ (6/10)

(Prompts: Lovecraftian teacup with a tentacle instead of the handle; A steampunk airplane; An avocado-shaped chair)

Our model is trained on a 100k subset of Objaverse — smaller than what’s typically used for 3D generation. Yet, it generalizes well across a wide range of prompts (5/10)

With a frozen Stable Diffusion model for textures and its trainable copy for geometry, the geometry model can tap into SD’s powerful natural image prior (4/10)

At the heart of our method is Collaborative Control. It allows two models to work together — one for generating the geometry image and another for creating textures — all while sharing information to ensure everything lines up perfectly 🤝(3/10)

The secret? We use geometry images, which are essentially 2D representations of 3D surfaces 🖼️ (think of GIMs as UV maps’ close cousins) This lets us recycle existing Text-to-Image models like Stable Diffusion, instead of building complex 3D architectures from scratch (2/10)

We're excited to release our new research paper: IP Adapter Instruct: Resolving Ambiguity in Image-based Conditioning using Instruct Prompts arxiv: https://t.co/UrzZJn8RE0 project page (with live demo!): https://t.co/qSs9WRARhA

IPAdapter-Instruct, a new release led by @CiaraRowles1 @unitygames Feed the IPAdapter image and instruct the model what to use from it: style, color, composition, pose, face! SD1.5, XL and 3 support 🔥 ✴️ Weights: https://t.co/JaGhW7LXiO ▶️ Demo: https://t.co/GeCqSGO98N 👩‍💻

Unity presents IPAdapter-Instruct Resolving Ambiguity in Image-based Conditioning using Instruct Prompts discuss: https://t.co/SraVmlra4z Diffusion models continuously push the boundary of state-of-the-art image generation, but the process is hard to control with any nuance:

We’re excited to introduce our group's new research paper, “Collaborative Control for Normal-Conditioned PBR Image Generation”, in which we tackle high quality single-view PBR materials! 🧵 arxiv: https://t.co/f2wp9p6ywz project page (with live demo!): https://t.co/5x4jZiAByb