Daniel Brown
@daniel_s_brown
Followers
794
Following
270
Media
22
Statuses
173
Assistant professor at University of Utah. Researching robot learning, reward/imitation learning, HRI, and AI safety.
Joined April 2019
Excited to announce that our research was recently highlighted in an AI Magazine article titled, "Toward robust, interactive, and human-aligned AI systems"
0
0
7
We hope this work can help inspire the development of better AI alignment tests and evaluations for LLM reward models. Check out the workshop paper here: https://t.co/MWTPYSSTGS 8/8
1
0
1
We applied this approach to RewardBench and found evidence that much of the data in safety and reasoning datasets may be redundant (44% for safety and 24% for reasoning) and that this can lead to inflated alignment scores. 7/8
1
0
1
By scaling up these ideas to LLMs, we can now estimate the set of reward model weights (weights that map the last decoder hidden state to a scalar output) that are consistent with a preference alignment dataset and also identify redundant examples in the preference dataset. 6/8
1
0
0
Once you find these core demonstrations or comparisons you can use them to craft efficient alignment tests. But empirically, we were only able to test these ideas on simple toy domains. 5/8
1
0
0
The main idea was that for linear rewards, we can determine, via an intersection of half-spaces, the set of reward functions that make a policy optimal and that this set of rewards is defined by a small number of "non-redundant" demonstrations or comparisons. 4/8
1
0
0
It was a fun paper and has some interesting nuggets, like the fact that there exist sufficient conditions under which we can verify exact and approximate AI alignment across an infinite set of deployment environments via a constant-query-complexity test. 3/8
1
0
0
As some background, a couple of years ago I worked with @j_j_schneider, @scottniekum, and @ancadianadragan on what we called "Value Alignment Verification" with the goal of efficiently testing whether an AI system is aligned with human values. https://t.co/Jve0pc8Hn5 2/8
arxiv.org
As humans interact with autonomous agents to perform increasingly complicated, potentially risky tasks, it is important to be able to efficiently evaluate an agent's performance and correctness....
1
0
0
Can you trust your reward model alignment scores? New work presented today at the COLM Workshop on Socially Responsible Language Modelling Research by Purbid Bambroo in collaboration with @anmarasovic that probes LLM preference test sets for redundancy and inflated scores. 1/8
1
1
6
This was a really fun collaboration with Jordan Thompson, Britton Jordan, and Alan Kuntz @URoboticsCenter Check out our paper here: https://t.co/spsRTkFYoH 5/5
openreview.net
Autonomous surgical robots are a promising solution to the increasing demand for surgery amid a shortage of surgeons. Recent work has proposed learning-based approaches for the autonomous...
0
0
1
Our approach also enables uncertainty attribution! We can backpropagate uncertainty estimates into an input point cloud to visualize and interpret the robot's uncertainty. If you're at #CoRL25, check out Jordan Thompson's talk and poster (Spotlight 6 & Poster 3). 4/5
1
0
1
We apply our approach to surgically-inspired deformable tissue manipulation and find it achieves a 10% lower reliance on human interventions compared to prior work that leverages variance-based uncertainty estimates. 3/5
1
0
0
Inspired by prior work on uncertainty-aware human-robot hand-offs like @ryan_hoque and @Ken_Goldberg's ThriftyDAgger ( https://t.co/8cll6XbdjI), we show that agreement volatility enables robots to know when they need help so they can request appropriate human interventions. 2/5
1
0
1
Check out our new paper being presented today at #CoRL2025 on uncertainty quantification: https://t.co/spsRTkFYoH. We propose a new second-order metric for uncertainty quantification in robot learning that we call "Agreement Volatility." 1/5
1
0
2
Super proud of my students and their great work that was showcased last week at AAMAS!
Last week at #AAMAS2025, I presented our full paper on discovering deployable emergent behaviors for robot swarms! 🤔Given known capabilities, what emergent behaviors are our swarms capable of? @AAMASconf @URoboticsCenter @daniel_s_brown 🧵1/12
0
0
5
📢Exciting news! Our workshop Human-in-the-Loop Robot Learning: Teaching, Correcting, and Adapting has been accepted to RSS 2025!🤖🎉Join us as we explore how robots can learn from and adapt to human interactions and feedback. 🔗Workshop website: https://t.co/SgkBCNaSD6 🧵👇
1
4
23
If you're in Melbourne, come check out Connor's talk in the Teleoperation and Shared Control session today! Paper: https://t.co/tYTW7Zbfow Website: https://t.co/SaziRcZe8r This is joint work with two of my other amazing PhD students @KarimiZohre and Atharv Belsare!
0
1
6
We study how to enable robots to use end-effector vision to estimate zero-shot human intents in conjunction with blended control to help humans accomplish manipulation tasks like grocery shelving with unknown and dynamically changing object locations.
1
0
4
Shared autonomy systems have been around for a long time but most approaches require a learned or specified set of possible human goals or intents. I'm excited for my student @connormat to present our work at #HRI2025 on a zero-shot, vision-only shared autonomy (VOSA) framework.
1
3
6
Excited to announce that I've been invited to give a talk at AAAI-25 on "Leveraging Human Input to Enable Robust, Interactive, and Aligned AI Systems" as part of their New Faculty Highlights program!
0
0
19