Jason Ma
@JasonMa2020
Followers
2,092
Following
1,015
Media
61
Statuses
481
PhD @Penn and @GoogleDeepMind . Apple Scholars in AI/ML. Prev: @NVIDIAAI , @MetaAI , @Harvard . Foundation Models for Robotics
Palo Alto, CA
Joined August 2018
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
ALL EYES ON RAFAH
• 696078 Tweets
DeNiro
• 337177 Tweets
Central
• 209491 Tweets
Sura
• 132286 Tweets
Hollywood
• 115092 Tweets
Fernando
• 71404 Tweets
Kylie
• 60178 Tweets
Julie
• 56574 Tweets
Thomsen
• 33105 Tweets
Coronado
• 32133 Tweets
Garro
• 26732 Tweets
Peñarol
• 21762 Tweets
Paulinho
• 21558 Tweets
Hulk
• 15722 Tweets
#WWENXT
• 13503 Tweets
#あさイチ
• 12324 Tweets
週の真ん中
• 11469 Tweets
Josh Gibson
• 11188 Tweets
Sudamericana
• 11088 Tweets
Copa Libertadores
• 10496 Tweets
Yuri Alberto
• 10037 Tweets
Pinned Tweet
Introducing DrEureka🎓, our latest effort pushing the frontier of robot learning using LLMs!
DrEureka uses LLMs to automatically design reward functions and tune physics parameters to enable sim-to-real robot learning. DrEureka can propose effective sim-to-real configurations
24
112
586
Super excited to share Eureka, our "spin" on how to use LLMs to teach low-level dexterity skills!
Eureka is an open-ended reward design agent that can write and evolve superhuman reward functions for a large suite of robots and tasks, including challenging pen spinning tricks!
Can GPT-4 teach a robot hand to do pen spinning tricks better than you do?
I'm excited to announce Eureka, an open-ended agent that designs reward functions for robot dexterity at super-human level. It’s like Voyager in the space of a physics simulator API!
Eureka bridges the
188
1K
6K
14
20
180
Excited to share VIP, a self-supervised visual reward and representation pre-trained on diverse human videos!
VIP’s frozen reward and rep. can solve diverse unseen robot tasks using TrajOpt, online RL, and enables real-world few-shot offline RL!
🧵:
2
40
170
Excited to share our
#ICML2023
paper ✨LIV✨!
Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control.
Project:
Code & Model:
🧵:
2
44
163
Humbled to share that I was selected as an Apple Scholar in AIML PhD Fellowship! Very grateful to Apple, my advisors
@dineshjayaraman
@obastani
as well as all my mentors and collaborators for their support!
22
5
157
Excited to share my first paper as an "advisor" :D
We show that pre-trained visual representations enable a simple, fast, no-training subgoal decomposition method for long-horizon robotic manipulation!
Paper:
Website:
(🧵1/n)
3
20
129
I am attending
#CORL2023
and presenting two new papers at various workshops!
Excited to make new friends and catch up! Please reach out if you are attending and would like to chat about anything robot learning :)
4
16
119
This is so impressive! I can't imagine the amount of progress we will unlock as a community with low-cost, highly capable robots. Congrats to the Unitree Team!
2
7
53
Learning policies in simulation and transferring to the real world (or Sim-To-Real in short) is a promising strategy for robots to learn complex skills. However, humans need to tune the simulator carefully so that the policies work robustly in the real world: this is difficult,
2
8
50
Due to popular requests, we have now uploaded our pre-trained LIV model on HuggingFace for easier downloads! This is my first time doing it, and the experience was quite smooth
@_akhaliq
Excited to share our
#ICML2023
paper ✨LIV✨!
Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control.
Project:
Code & Model:
🧵:
2
44
163
1
17
49
We are presenting LIV today at
#ICML2023
!
Exhibit Hall 1,
#827
2:00pm - 3:30pm HST
The future of robotics is multi-modal, and LIV demonstrates how multi-modal value pre-training from diverse human videos can bootstrap language-conditioned robot skill learning.
See you there!
Excited to share our
#ICML2023
paper ✨LIV✨!
Extending VIP, LIV is at once a pre-training, fine-tuning, and (zero-shot!) multi-modal reward method for (real-world!) language-conditioned robotic control.
Project:
Code & Model:
🧵:
2
44
163
1
6
41
Honored to see Eureka on this list along side many amazing works!
👀 Discover the top 10
#NVIDIAresearch
projects of the year.
✨ From Neuralangelo's high-fidelity neural surface reconstruction to Magic3D's text-to-3D content creation, these projects push the boundaries of innovation in
#AI
.
0
11
52
1
2
40
I am attending
#ICML2023
next week in Hawaii! Excited to make new friends and re-connect with old ones!
Please reach out if you are attending and would like to chat about anything related to research or ML! My particular interests include foundation models, RL, and robotics!
0
0
34
We are organizing Workshop on Goal-Conditioned Reinforcement Learning (GCRL) at
#NeurIPS
2023!
Submission Deadline: October 4th, 2023
Website:
1
4
32
Big congrats to my mentors and collaborators
@DrJimFan
and
@yukez
on the new group! Embodied AI and robotics research just kicked up a gear ;)
Career update: I am co-founding a new research group called "GEAR" at NVIDIA, with my long-time friend and collaborator Prof.
@yukez
. GEAR stands for Generalist Embodied Agent Research.
We believe in a future where every machine that moves will be autonomous, and robots and
241
471
4K
1
1
31
At a technical level, DrEureka, following our prior work Eureka (), uses LLM-guided evolutionary search to generate safety-aware reward functions in code that can be used to train policies in sim. Then, leveraging LLMs’ capability as hypothesis generators,
3
2
28
Excited to share some of my recent works on pre-training for robotics with the MILA community!
Hello! We have
@JasonMa2020
from UPenn giving a talk at this week's robot learning seminar (Thursday 11:30am EST online). Hope to see you all there!
Title: Foundation Reward Models for General Robot Skill Acquisition
#Robotics
#MachineLearning
0
1
14
0
3
27
Thanks
@_akhaliq
!
LIV is now on arXiv:
Check it out if you are interested in the space of (RL-based) vision-language pre-training for robotics! Happy to answer any questions about the paper :)
LIV: Language-Image Representations and Rewards for Robotic Control
paper page:
Language-Image Value (LIV) is a unified pre-training, fine-tuning, and reward learning algorithm for language-conditioned visual manipulation. LIV can perform zero-shot
0
41
184
0
7
25
Happy to announce that UVD is announced as the best paper at the CORL LEAP Workshop!
I am attending
#CORL2023
and presenting two new papers at various workshops!
Excited to make new friends and catch up! Please reach out if you are attending and would like to chat about anything robot learning :)
4
16
119
2
2
25
DrEureka is co-led by
@willjhliang
and me with collaborators from
@Penn
and
@NVIDIA
:
@johnnywang_16
,
@sam_wang23
, and our advisors
@yukez
@DrJimFan
@obastani
@dineshjayaraman
Check out our project website for the paper and more videos:
Code:
4
1
24
We have added example code for visualizing **animated** zero-shot VIP reward curve on robot videos!
Try it out (on your own robot videos) here:
Excited to share VIP, a self-supervised visual reward and representation pre-trained on diverse human videos!
VIP’s frozen reward and rep. can solve diverse unseen robot tasks using TrajOpt, online RL, and enables real-world few-shot offline RL!
🧵:
2
40
170
0
9
21
Check out our
#L4DC
paper on learning policy-aware dynamics model for reinforcement learning!
The idea is very simple: focus model learning on the current policy’s visitation distribution. We theoretically show why this is desirable and extend the dual RL paradigm to MBRL,
Excited to share our our
#L4DC2023
paper that introduces "Transition Occupancy Matching"(TOM)
TOM learns a dynamics model that keeps up with the improving policy, facilitating continued progress
Paper 📰:
Code 💻:
🧵
1
2
12
0
1
21
I am attending
#NeurIPS2022
next week to present several works on offline RL and pre-training for robotics! Would love to meet people and discuss anything, in particular, RL and robot learning topics! DM me if you are attending and want to chat or grab coffee 😀
1
0
21
We at
@GRASPlab
hosted
@ericjang11
last week for a thought-provoking talk on Humanoid robots and 1X! The full recording is up now, check it out!
Nice talk on the technical approach to intelligent humanoids at 1X!
As usual for
@ericjang11
’s spicy takes, I agree strongly with 60%, ambivalent on 20%, and disagree with 20%.
Highly recommend a watch! 💯
2
11
52
1
1
21
Jim has been a fantastic mentor for me! Apply if you are interested in foundation models for decision making and AI Agents!
1
1
20
This was a very fun project and we learned a lot about how to use LLMs to enable robot skill learning! There are many challenges and potential future directions. For example, how to combine DrEureka with real-world execution feedback and using vision to provide feedback on reward
2
0
19
Come to our workshop on Goal-Conditioned RL tomorrow at
#NeurIPS2023
!
Check out the
#NeurIPS2023
workshop on Goal Conditioned Reinforcement Learning:
* Tomorrow (Friday) 900 -- 1830 CST
* Great speakers:
@jeffclune
@r_mirsky
@olexandr
@ybisk
@SusanMurphylab1
* Program:
0
2
17
0
2
18
Reward learning is a fundamental challenge in RL. In VIP, we address this by pre-training a value function on action-free human videos, and the pre-trained VIP value function can zero-shot transfer to unseen robot tasks!
Find out more about VIP at the Deep RL workshop tomorrow!
Deep RL workshop
@NeurIPSConf
starts on Friday!
Hear invited talks from
@tobigerstenberg
on counterfactual simulation of causal judgments,
@j_foerst
on opponent-shaping in games,
@IMordatch
on sequence modeling,
@yayitsamyzhang
on learning generalist agents. Don't miss out! 🧵
1
20
81
0
1
18
sim2real for manipulation tasks is really hard! Great to see this work come out
Does your sim2real robot falter at critical moments 🤯? Want to help but unsure how, all you can do is reward tuning in sim 😮💨?
Introduce 𝐓𝐑𝐀𝐍𝐒𝐈𝐂 for manipulation sim2real. Robots learned in sim can accomplish complex tasks in real, such as furniture assembly.
🤿🧵
16
40
176
2
2
17
UVD code is open-source now:
Get your long-horizon demonstrations segmented in few seconds!
We support various SOTA pre-trained reps (VIP, R3M, LIV, VC-1, ...) as well as policy backbones (MLP, GPT).
Excited to share my first paper as an "advisor" :D
We show that pre-trained visual representations enable a simple, fast, no-training subgoal decomposition method for long-horizon robotic manipulation!
Paper:
Website:
(🧵1/n)
3
20
129
0
3
15
We highlight quadruped yoga ball walking. This task is particularly hard because (1) it is a novel task for which the LLM could not have seen human-generated reward functions or DR, and (2) simulation cannot model the deformable surface of an air-inflated ball, making good
3
0
16
@HaqueIshfaq
We are organizing a workshop on goal-conditioned RL! Please do consider submitting/participating if relevant :) more details to follow.
1
1
15
Giving contributed talks on VIP at the Offline RL, Foundation Model for Decision Making, and Deep RL workshops at
#NeurIPS2022
. Come check out how we can pre-train a value function on passive human data and zero-shot transfer to robotics manipulation!
#VIP
: Self-supervised pre-trained visual reward and representation for robotics
🎯 DeepRL, OfflineRL, SSL workshops
🔗
@shagunsodhani
@dineshjayaraman
@obastani
@Vikashplus
@yayitsamyzhang
@JasonMa2020
⏯️⏯️
1
0
3
0
0
15
The DrEureka policy, as you have seen in the uncut videos, is quite robust in the real world. It can successfully stay balanced over curbs, terrain changes, and even when the ball is being kicked!
1
0
14
This is my internship project at
@NVIDIAAI
! I had a blast working on it and learned a lot from the experience. I am really grateful to my mentors
@AnimaAnandkumar
@DrJimFan
@yukez
for their guidance on the project!
2
0
14
1
0
11
We even tried to deploy the policy while the yoga ball was being deflated. Here is a ~1-minute uncut video of the robot balancing well until it eventually loses control😅
1
0
13
We (
@yayitsamyzhang
) are presenting VIP at
#ICLR
tomorrow!
Talk: Oral 1 Track 5: Reinforcement Learning
Poster Session: MH1-2-3-4
#118
Looking forward to having discussions with you there!
Excited to share VIP, a self-supervised visual reward and representation pre-trained on diverse human videos!
VIP’s frozen reward and rep. can solve diverse unseen robot tasks using TrajOpt, online RL, and enables real-world few-shot offline RL!
🧵:
2
40
170
0
1
13
This looks fantastic!
Thrilled to announce the first annual Reinforcement Learning Conference
@RL_Conference
, which will be held at UMass Amherst August 9-12!
RLC is the first strongly peer-reviewed RL venue with proceedings, and our call for papers is now available: .
4
95
242
0
0
12
Check out our
#AISTATS2023
paper on fair exploration in RL! Led by
@wanqiao_xu
0
0
12
Beyond yoga ball walking, we also benchmarked DrEureka on Quadruped Locomotion and Dexterous Cube Rotation, two known tasks where there are pre-existing human-designed reward functions and DR configurations. We find DrEureka configurations to match or outperform human-designed
1
0
11
A very cool way of leveraging human videos for robotics manipulation!
How can robots learn manipulation *just* by watching videos of humans in different unstructured settings?
In our new paper, we develop a framework enabling zero-shot coarse robot manipulation from passive human videos (a 🧵)
w/ Abhinav Gupta,
@shubhtuls
,
@Vikashplus
1/N
5
37
198
1
1
11
One visual representation for a wide variety of manipulation and navigation tasks! Check out our recent work on building a Visual Cortex for Embodied AI.
Contemporary discussion (hype?) about LLMs and “pausing AGI development” seems oblivious of Moravec’s paradox.
We’ve hypothesized since the 80s — that the hardest problems in AI involve sensorimotor control, not abstract thought or reasoning.
It
22
153
811
0
2
10
Looking forward to this talk!
0
0
7
@MontrealRobots
@dineshjayaraman
Thanks for having me! I really enjoyed the conversation with everyone
0
0
7
Please check out the paper for more details and experimental results!
This is a fun collaboration with
@ZCCZHANG
@YunshuangL
@obastani
@abhishekunique7
@dineshjayaraman
@LucaWeihs
Also check out
@ZCCZHANG
's original thread on the project:
How can pre-trained visual representations help solve long-horizon manipulation? 🤔
Introducing Universal Visual Decomposer (UVD), an off-the-shelf method for identifying subgoals from videos - NO extra data, training, cost, or task knowledge required. (🧵1/n)
1
17
87
1
2
7
With UVD's recursive decomposition⏮️, it can seamlessly build upon any standard visuomotor policy training⏭️, in Sim & Real, across IL & RL!
1
2
6
2. Human interference during deployment
UVD can also enable agents to auto-skip sub-stages preemptively finished by humans and can reset to redo certain stages during deployment. (🧵5/n)
1
2
6
Check out some recent works from our lab on versatile goal-based task specification for robot learning!
Happening now!
@dineshjayaraman
from
@PennEngineers
is giving a talk at the Embodied Intelligence Seminar
@MIT_CSAIL
on Polyglot Robots: Versatile Goal-Based Task Specification for Robot Learning
Streaming at: , cohosted with
@du_yilun
0
1
8
0
0
6
@Joe__Black__
@willjhliang
@Penn
@nvidia
@johnnywang_16
@sam_wang23
@yukez
@DrJimFan
@obastani
@dineshjayaraman
Hi, we have an experiment in the paper using a robot hand to perform cube rotation!
0
0
2
Detailed Thread on VC-1, a first step towards a unified and single visual representation for all Embodied AI tasks
A visual cortex is the region of the brain that (together with the motor cortex) enables an organism to convert vision into movement.
We present an artificial visual cortex — the module in an AI system that enables an artificial agent to convert camera input into actions.
🧵👇
1
51
204
0
1
6
@EugeneVinitsky
You are too kind Eugene! I very much look forward to learning about your neat work soon ;)
0
0
5
Pre-trained reprs, e.g VIP (), can produce smooth embeddings on videos of atomic tasks. Given this, UVD discovers subgoals by recursively detecting phase shifts in the embedding space. This simple idea works well across human or robot videos! (🧵2/n)
1
2
5
Shagun is a great mentor! Highly recommend working with him :)
0
0
5
Cool work on RL based pre-training on observational data!
If you are interested in this line of work, also check out our work VIP (), which directly obtains a visual representation and universal value function from human videos!
"Ask not what representation learning can do for RL, ask what RL can do for representation learning" -- JFK?
In our new paper: a useful way to pre-train representations on video data for decision-making agents is basically to run RL on the video!
1
5
67
0
1
5
Cool work that explores how interaction can be used to verify task completion in the real world!
Really thrilled that our paper was picked a best paper finalist at CORL22! If you’re attending, come to our oral on Saturday at 11:20am!
We present Interactive Reward Function (IRF) policies, which interact with the world to provide rewards for training task policies. (1/3)
1
5
43
0
0
5
Importantly, UVD enables long-horizon compositional generalization across:
1. New task sequences
Plug-and-play UVD significantly boosts OoD results across various visual backbones and policies, while maintaining in-domain (InD) performance. (🧵4/n)
1
2
5
If you are interested in vision-language reward/representation pre-training for robotics/decision making, also check out concurrent work Voltron() from
@siddkaramcheti
and MineDojo() from
@DrJimFan
and co!
1
0
5
@natolambert
In my own VIP work (), we explore how fixed pre-trained models fare against increasing compute budget in downstream trajectory optimization:
0
0
4
@GMartius
@nico_guertler
@_sebastianblaes
@manuelwuethrich
@robo_challenge
Super cool work! This seems like a really useful and challenging benchmark for testing real-world offline RL.
We have also done some work on offline goal-conditioned RL for real-world dexterous manipulation that you may be interested!
0
2
4
Finally, big thanks to the organizing team:
@ben_eysenbach
@TongzhouWang
@IshanDurugkar
@TheAndiPenguin
@yayitsamyzhang
We are super excited for the workshop and look forward to all the great discussions that will take place!
0
0
3
@_k_sridhar
@willjhliang
@Penn
@nvidia
@johnnywang_16
@sam_wang23
@yukez
@DrJimFan
@obastani
@dineshjayaraman
Thanks Kaustubh!!
0
0
2
The code and pre-trained model are open-sourced:
It is also hosted on
@torchRL
as an encoder option!
It is super easy to load VIP for your new task:
from vip import load_vip
vip = load_vip()
vip.eval()
1
0
3
0
0
3
Our second paper is UVD:
UVD is a simple, fast, no-training subgoal decomposition method for long-horizon robotic manipulation using PVRs (e.g., VIP, R3M)!
@ZCCZHANG
@YunshuangL
will be giving the workshop spotlight!
How can pre-trained visual representations help solve long-horizon manipulation? 🤔
Introducing Universal Visual Decomposer (UVD), an off-the-shelf method for identifying subgoals from videos - NO extra data, training, cost, or task knowledge required. (🧵1/n)
1
17
87
0
0
3
@EthanJPerez
Nice work! We have some work from ICML 2023 on pre-training language-conditioned value functions that may be of interest!
2
0
3
Very cool paper with an awesome codebase! Kudos to the authors!
What if the form of visuomotor policy has been the bottleneck for robotic manipulation all along? Diffusion Policy achieves 46.9% improvement vs prior StoA on 11 tasks from 4 benchmarks + 4 real world tasks! (1/7)
website :
paper:
9
100
533
0
0
3
VIP enables a simple and practical real-world few-shot offline RL pipeline: just do reward-weighted regression (RWR) with VIP’s reward and the representation! With VIP, offline RL is as simple as BC but far more effective.
1
1
3
LIV was a fun and rewarding project with my advisors at
@MetaAI
and
@PennEngineers
:
@Vikashplus
@yayitsamyzhang
@obastani
@dineshjayaraman
Please check out the paper for more results and in-depth analysis!
1
0
2
0
0
2