Yue Yang
@YueYangAI
Followers
578
Following
155
Media
31
Statuses
108
Research scientist @allen_ai | PhD @upennnlp | Vision and Language
Philadelphia, PA
Joined July 2018
RT @allen_ai: 🤖✨ What if models that take action in the physical world could think through your instructions? Meet MolmoAct, our new fully….
0
73
0
RT @jeffrey_ch0: 🤖💬 Herding instincts… in AIs? Yes, even LLMs can follow the crowd!. • 📉 Conformity ↑ when agents lack confidence but trust….
0
7
0
Successfully defended my PhD thesis and got hooded this week! Thanks to all the friends who supported me throughout this incredible journey! Excited to join PRIOR at @allen_ai next and continue exploring open vision-language research!
16
4
154
🎉CoSyn is accepted by ACL2025!.
We share Code-Guided Synthetic Data Generation: using LLM-generated code to create multimodal datasets for text-rich images, such as charts📊, documents📄, etc., to enhance Vision-Language Models. Website: Dataset: Paper:
0
0
6
RT @jeffrey_ch0: #NAACL2025.How to compare cultural differences with social media data in scale? Our work uses lexica to annotate X 🇺🇸 & We….
aclanthology.org
Young Min Cho, Dandan Pang, Stuti Thapa, Garrick Sherman, Lyle Ungar, Louis Tay, Sharath Chandra Guntuku. Findings of the Association for Computational Linguistics: NAACL 2025. 2025.
0
4
0
RT @AnnieFeng6: #ICLR2025 Oral. LLMs often struggle with reliable and consistent decisions under uncertainty 😵💫 — largely because they can….
0
39
0
RT @artemispng: Exciting news! 🎉 Our paper “ViUniT: Visual Unit Tests for More Robust Visual Programming” got accepted at #CVPR2025.
0
2
0
RT @codezakh: ✨ Introducing MutaGReP (Mutation-guided Grounded Repository Plan Search) - an approach that uses LLM-guided tree search to fi….
0
38
0
This work is done during my great summer internship at @allen_ai with my awesome collaborators: Ajay Patel, @mattdeitke, @tanmay2099, @LucaWeihs, @drewmikehead, @yatskar, Chris Callison-Burch, @RanjayKrishna, @anikembhavi, Christopher Clark.
0
0
4
We also show we can create synthetic pointing data to improve the click accuracy of VLMs in GUI agent tasks. On the ScreenSpot click prediction benchmark, our model trained on synthetic pointing data can outperform existing methods with much less training data.
1
0
5
We notice open VLMs struggle with novel out-of-domain tasks like interpreting nutrition labels. However, CoSyn’s controllable data generation can create targeted synthetic data for task-specific fine-tuning, achieving strong zero-shot performance with significantly less data.
1
0
4
On 7 text-rich benchmarks (e.g., ChartQA, DocVQA), our model trained on synthetic data outperforms competitive open and proprietary VLMs. Our zero-shot model, trained without benchmark examples, beats most baselines, proving the generalizability of training on synthetic data.
2
0
4
CoSyn uses code as the intermediate representation to build synthetic multimodal datasets. We prompt a text-only LLM to generate code that renders images, and then we use code as context to create instruction-tuning data, such as QA pairs, for fine-tuning VLMs.
1
0
5
Our CoSyn framework integrates 11 rendering tools for 20 robust generation pipelines, which support the creation of diverse text-rich images, including charts, documents, diagrams, tables, and even music sheets 🎼, and many more!
1
0
5
We share Code-Guided Synthetic Data Generation: using LLM-generated code to create multimodal datasets for text-rich images, such as charts📊, documents📄, etc., to enhance Vision-Language Models. Website: Dataset: Paper:
6
47
196
RT @LongLeRobot: Articulate Anything has just been accepted to @iclr_conf #ICLR2025 ! Looking forward to seeing everyone in Singapore 🇸🇬 🙀….
0
8
0
RT @cmalaviya11: Excited to share ✨ Contextualized Evaluations ✨!. Benchmarks like Chatbot Arena contain underspecified queries, which can….
0
31
0
RT @veronica3207: 🤔What model explanation method should you use? How to ensure it reflects the model’s true reasoning?. 🌟 In our CL survey,….
0
8
0
RT @cmalaviya11: ✨Updates✨:.• Dolomites was accepted to TACL: .- Our data is now also up on HuggingFace: https://t.….
0
11
0