kartik goyal
@kartik_goyal_
Followers
639
Following
809
Media
8
Statuses
118
assistant professor at Georgia Tech @ICatGT; past @TTIC_Connect; phd @LTIatCMU; ml, nlp, and beyond...
Pittsburgh, USA
Joined May 2009
Sounds unreal so we conducted a bunch of ablations to control for confounding factors. Although we instantiate this framework only for MCQAs and word problems, modular evaluation and training holds tremendous promise. led by Yunxiang Yan and @tsawada_ml.
arxiv.org
While question-answering~(QA) benchmark performance is an automatic and scalable method to compare LLMs, it is an indirect method of evaluating their underlying problem-solving capabilities....
0
0
1
i.e. decompose the question into abstract non-overlapping parts and present them in a cascaded manner withholding info. We find that this not only better estimates the underlying LM capabilities but also elicits better internal traces, i.e "Weak" models are stronger & vice-versa.
1
0
1
We predominantly evaluate LMs on QA datasets when really, we are interested in evaluating the quality of strategies they apply for QA. In this work, we provide an extensible framework to better estimate this by changing *how* we ask questions to LMs. .
Why do we evaluate LLMs using multiple-choice QA. when in practice, we ask them to generate open-ended answers?.Standard evaluation rewards models for choosing the right letter — not for reasoning their way to an answer. A better alternative: Cascaded Information Disclosure.
1
0
1
RT @humphrey_shi: We’re recruiting future faculty leaders @ICatGT at all tenure-track ranks in strategic areas like AI, Robotics, and Respo….
0
14
0
RT @mark_riedl: Hi all. I am co-organizing a Summit on Responsible Computing, AI, and Society at @GeorgiaTech Oct 28-30. We will actively d….
rcais.github.io
2025 Summit on Responsible Computing, AI, and Society
0
64
0
Yay!! @davis_yoshida @kevingimpel.
🎉SAC Awards:. 4) RomanSetu: Efficiently unlocking multilingual capabilities of Large Language Models via Romanization by Husain et al. 5) MAP’s not dead yet: Uncovering true language model modes by conditioning away degeneracy by Yoshida et al. #NLProc #ACL2024NLP.
0
0
6
If you're at #ACL2024NLP, say hi to the amazing @davis_yoshida who is going to talk about why *search* for maximally scoring strings under LMs yields degenerate outputs and how to fix it. Joint work with @kevingimpel @davis_yoshida
1
0
12
Apply to Georgia tech for a PhD by Dec 15 -- It's a fun place to do ML, NLP, and explore a variety of adjacent research areas! More info:
If you are applying to Ph.D. programs in NLP this year - consider Georgia Tech!. More info on our programs along with links to apply are listed below. CS: ML: Info on Atlanta:
0
17
62
RT @ChrisVVarren: When @print_and_prob collaborator @NikolaiVogler heard that ESTC was down due to the BL cyberattack, he jumped into acti….
0
5
0
RT @TTIC_Connect: TTIC is accepting Research Assistant Professor applications until Dec. 1! This position includes no teaching requirements….
0
10
0
New CHR paper with an amazing set of collaborators: we find that high-recall bitext mining and sentence alignment is actually kinda tricky for messy historical literary text. Multilingual embeddings like LaBSE and friends work surprisingly well for literary ancient Greek though!.
Caroline Craig, @kartik_goyal_ , @farnooshamsian , and @PhilologistGRC have a CHR paper on getting document-level sentence alignment to work for the ancient Greek and Latin corpus to track multiple translations into English, French, German, Persian, etc.
0
1
6
🚨 I started as an assistant professor at Georgia Tech @ICatGT this fall! I am looking to hire PhD students interested in NLP, ML, and cultural analytics (LLMs inlcuded 😉). Come work with me! Apply to a PhD Program listed here and mention me in your app:
Please welcome @kartik_goyal_, who starts this semester as an associate professor! He will explore the analytical and practical benefits of statistical NLP as it pertains to the study of human languages. Get to know Kartik below!.
20
22
195
RT @mark_riedl: Hi 👋 The @GeorgiaTech School of Interactive Computing is continuing to grow! . Last year we hired 6 new faculty. This year….
0
29
0
RT @TTIC_Connect: TTIC is a proud sponsor of #MMLS2023, which will be held on May 16-17 at @UICEngineering Dorin Forum, featuring TTIC Prof….
0
1
0
RT @samuellemley: We'll be sharing some of our research in computational bibliography at @GrolierClub as part of @BibSocAmer's Bibliography….
0
4
0
RT @limufar: I will be presenting 2 papers @emnlpmeeting tmw, both on #privacy and #memorization in LLMs:.1. Poster session 1 @ 11AM: Quant….
0
19
0
RT @leoduw: When I first came across Mask-Predict, I always thought the decoding could be formulated in a principled MCMC way. This paper….
0
2
0
RT @limufar: Join us at 11am PT today, during *virtual* poster session 2 at @aclmeeting to discuss how you can condition text generation u….
0
4
0
Having observed certain energy parametrizations yielding good samples from masked LMs, we were curious if we could do controlled text generation by stacking appropriate blackboxes with a product-of-experts energy formulation in w/ @limufar @BergKirkpatrick.
35k trained models on @huggingface, yet whenever we want to generate text w/ given attributes, we train new models/clsfrs. In our #ACL2022 paper, we enable using ANY arbitrary (even non-differentiable) expert for controllable generation, w/o ANY TRAINING!.
1
1
8