Defended my PhD today!!! 🥳. Thank you to everyone who supported me along the way, especially my advisor @jhhays for taking a chance on my clueless little undergrad self and putting up with me ever since. I'll be joining Adobe next month as a research scientist under @aseemaa!.

RT @gcduncombe: I've had so much fun experimenting with our latest @Adobe Firefly Video model - coming from a background in film, I've love….

RT @AdobeVideo: We’re excited to share progress on the all-new Firefly Video Model with Text to Video & Image to Video capabilities across….

RT @aseemaa: After a nearly 20 month journey we are finally going public with our work on video generative models at Adobe! This work start….

I'll be presenting our work tomorrow (6/19) 5-630pm in Arch 4A-E (poster #329)!. #CVPR2024.

In collaboration w/ Matt, @jhhays, Nick, Yuchen, @rzhang88, Tobias. @mlatgt @AdobeResearch.

In our latest work, we introduce personalized residuals for efficient personalization of T2I diffusion models and localized attention-guided sampling that dynamically blends the learned concept into new contexts. #CVPR2024.

RT @_akhaliq: Personalized Residuals for Concept-Driven Text-to-Image Generation. We present personalized residuals and localized attention….

PS I'm looking to hire a research intern for summer 2024 to work on image/video generation or multimodal learning. If you're interested, please apply online and send me an email :)

MCM will part of the "Diffusion for Geometry" session tomorrow (Wed 8/9) at 9am in room 502 AB. Hope to see you there! :). #SIGGRAPH2023.

Thank you to my collaborators (@jhhays, Cynthia, Krishna, Zhifei, and Tobias)!. @mlatgt @AdobeResearch.

I'll be presenting our work on multimodal conditioning modules (MCM) at #SIGGRAPH2023 this week! MCM is a small network that enables multimodal image synthesis using pretrained diffusion models without any direct updates to the diffusion model parameters.

RT @_akhaliq: Modulating Pretrained Diffusion Models for Multimodal Image Synthesis. abs: .project page: https://t.….

Stop by poster #85 in the afternoon session tomorrow and say hi! #ECCV2022.

CoGS also offers an optional refinement step via unified embeddings for search and synthesis. We train a VAE that maps the codebook representations into a metric latent space, thus enabling retrieval and interpolation using the transformer's output.

To enable training & evalation, we create a large-scale paired dataset of images and "pseudosketches," which are derived from the images via an automated process, and filtered via AMT crowdsourcing. A subset corresponds to the Sketchy Database, which contains free-hand sketches.

CoGS enables decoupled control over the structure and style of images across a diverse set of classes. We train a transformer to encode an input sketch, exemplar style image, and class label into a composite codebook representation, which is decoded by a pre-trained VQGAN.

Excited to share "CoGS: Controllable Generation and Search from Sketch and Style" at #ECCV2022!. (w/ Gemma Canet Tarrés, @tuvbui, @jhhays, @JCollomosse, Zhe Lin). Project page:

So excited to be attending #CVPR2022 n person! Looking forward to seeing familiar faces (especially some of my virtual collaborators) and meeting new ones 😁.

RT @ak92501: CoGS: Controllable Generation and Search from.Sketch and Style.abs: