John F. Wu
@jwuphysics
Followers
2K
Following
13K
Media
609
Statuses
3K
Associate Astronomer and Applied AI @SpaceTelescope. Secondary appointments in @JohnsHopkins Astro + CS. Opinions my own. He/him. 🦋 @jwuphysics.bsky.social
Baltimore, MD, USA
Joined June 2020
Sparse Autoencoders (SAEs) can be a powerful "discovery engine" for astrophysics, helping us find interpretable features in modern galaxy surveys. Check it out -- new @NeurIPSConf ML4PS workshop paper, jointly authored by me and Mike Walmsley!
2
3
35
Lol Gemini couldn't figure out which emoji to put in the heading so it chose 융 instead...
0
1
2
Goodfire is doing some really incredible stuff
📝 New paper! Two strategies have emerged for controlling LLM behavior at inference time: in-context learning (ICL; i.e. prompting) and activation steering. We propose that both can be understood as altering model beliefs, formally in the sense of Bayesian belief updating. 1/9
0
0
2
TensorFlow was all about making it easier for computers. PyTorch *won* because it was about making it easier for humans. It’s disappointing to see AI community focusing on what’s easiest for machines again (prioritizing AI agents & not centering humans). -- @jeremyphoward 1/
1
18
159
normalized flux units, interesting choice
0
0
2
Hey Google structured data team, I think your float_to_percentage is being applied twice...
0
0
4
🚨 New research! 🚨 Very cool results showing a method for locating + characterizing + intervening on memorized data in models (of all kinds, e.g. language & vision) Highly recommend reading this thread, then @jack_merullo_ 's thread, then the full paper for a deep dive
LLMs memorize a lot of training data, but memorization is poorly understood. Where does it live inside models? How is it stored? How much is it involved in different tasks? @jack_merullo_ & @srihita_raju's new paper examines all of these questions using loss curvature! (1/7)
0
2
12
So excited for the super cool workshop Language AI in the Space Sciences exploring the role of natural language processing and artificial intelligence in space science and astronomy ✨ . The workshop will take place at the Space Telescope Science Institute from March 9th to 12th
stsci.edu
1
2
12
Thank you! If you want to read more, I wrote about this journey on my blog: https://t.co/ZDvCjouqR0 Although now I'm nearly four years in, so I may need to write an update!
jwuphysics.github.io
Reflecting on my journey towards becoming a tenure-track astronomer
1
0
5
Super common pathway. Towards the end of my PhD in Physics/Astro, I trained my first deep neural net on a GeForce 780 after watching @jeremyphoward on the @fastdotai lectures. Now I'm tenure-track faculty in (astro)physics, but still maintain strong curiosity in AI/ML
Met a Meta AI researcher. He studied Physics in China, came to the US for a PhD in Physics, and then fell in love with AI, despite never having studied computer science. He watched Andrej Karpathy and Andrew Ng, bought a GPU, read every arXiv paper title daily, and dived into
9
20
337
Obscure question for @Ravens fans and journalists: I have this T-shirt that references "Splash '98 Funky Jungle" presented by the Baltimore Ravens. What event was this? Google search, ChatGPT turned up nothing...
1
0
0
Wow those are some serious gains over SFT
8/ The results are ridiculous. RGT hits 72-83% accuracy. Standard SFT needs 10x more data to reach the same performance. iSFT needs 1.5-1.7x more. We're teaching the model the causal structure of correctness, not hoping it reverse-engineers it from examples
1
1
6
We found a neat way to teach models the rules and not just the answers using strategy tokens, with implications for cleaner supervision and sample-efficiency gains. Fresh from @part_harry_ @charles0neill and us @parsedlabs!
We discovered that teaching models why answers are correct, not just what to output, dramatically improves training efficiency. By making latent strategies explicit during training (e.g., "don't infer diagnoses from medications"), we achieve the same performance with 10x fewer
0
1
6
Okay what now
New Anthropic research: Signs of introspection in LLMs. Can language models recognize their own internal thoughts? Or do they just make up plausible answers when asked about them? We found evidence for genuine—though limited—introspective capabilities in Claude.
0
0
2
Forgot to share all the public code! Github for SAEs: https://t.co/8RFEGXAuiS Euclid MAE + dataset: https://t.co/7sQekT1HFS Euclid MAE interactive demo:
huggingface.co
Sparse Autoencoders (SAEs) can be a powerful "discovery engine" for astrophysics, helping us find interpretable features in modern galaxy surveys. Check it out -- new @NeurIPSConf ML4PS workshop paper, jointly authored by me and Mike Walmsley!
0
0
3