Excited to share our new work!.DOVE 🕊️: a dynamic vision encoder that adapts token count to image complexity. Fewer tokens, same fidelity—outperforming fixed-length AEs tokenizer on classification & VLM tasks!.Arxiv: Web: #AI #CV

RT @ZinengTang: Excited to share our new work!.DOVE 🕊️: a dynamic vision encoder that adapts token count to image complexity. Fewer tokens,….

Big thanks to my undergrad intern Lingjun for delivering such impressive work, and to Rudy for the thoughtful co-advising!.

🔥 DOVE uses 68 % fewer tokens but has better FID than VQGAN/TiTok and +10–12 pts on VQA/ImageNet/CIFAR. It achieves significantly stronger performance on classification, probing, and VLM tasks. DOVE also brings emerging properties—PCA heatmaps reveal sharper segmentation.

Also thanks to, @LongTonyLian, Seun Eisape (, @XDWang101, @roeiherzig, @Yalatweets, @alsuhr, and @trevordarrell for their great efforts!.

TULIP achieves state-of-the-art performance across multiple vision and vision-language benchmarks. It significantly improves zero-shot classification on ImageNet-1K, enhances fine-grained object recognition, and boosts multimodal reasoning scores. Compared to existing methods,

Despite the success of contrastive image-text models like CLIP, they struggle with vision-centric tasks requiring high-fidelity understanding. We introduce TULIP, a novel model integrating generative data augmentation, enhanced contrastive learning, and reconstruction.

We are thrilled to announce TULIP!. 🌷 A state of the vision language encoders coupled with generative model for stronger representation learning.

RT @ChengleiSi: Automating AI research is exciting! But can LLMs actually produce novel, expert-level research ideas?. After a year-long st….

RT @yzy_ai: We announced Phi 3.5 series today! .1️⃣ Multilingual Mini 3.8B: 2️⃣ MoE 16x3.8B (active 6.6B): https://….

CoDi-2 is selected as #CVPR2024 Highlight. Come joint us in today’s poster session Arch 4A-E #314 in 5pm to 6:30 pm!. @yzy_ai @nlpyang @ChenguangZhu2 @mohitban47.

RT @mohitban47: Honored and grateful to be named as one of the @UNC permanent distinguished professorships 🙏 . 100% of the credit goes to m….

Excited to share that CoDi-2 is accepted to @CVPR .In this work, we show that alignment of multimodal inputs to language unlocks ICL and Few-shot prompting ability for multimodal generation. #CVPR2024. @yzy_ai @nlpyang @ChenguangZhu2 @mohitban47.

RT @mohitban47: Had a great time speaking at @indoml_sym & meeting students+faculty+researchers (overall, kudos to organizers+volunteers on….

RT @pan_jiayipan: Generative model that operates across versatile modalities is where future is, which makes this work super interesting.

RT @mohitban47: 🚨 CoDi-2: In-Context, Interleaved, Interactive Any-to-Any Generation🚨. Interactive MLLM that can follow "multimodal-interle….

RT @jmin__cho: CoDi-2 presents multimodal 'generation' with interleaved in-context learning!. Another exciting work by @ZinengTang @yzy_ai….

RT @yzy_ai: Recall CoDi any-to-any generation? Now CoDi-2🔥any-to-any MLLM is here! It is.(1) 1st ever vision-language-audio zero/few-shot g….

RT @DrJimFan: Codi-2: an interleaved, multimodal LLM that supports arbitrary audio, image, and video mixture in I/O. You can give instructi….