I am recruiting Ph.D. students for my new lab at @nyuniversity! Please apply, if you want to work with me on reasoning, reinforcement learning, understanding generalization and AI for science. Details on my website: https://t.co/d8uId2LC47. Please spread the word!

Recently I've been playing around with a quarter-order-of-magnitude system for simple calculations. It gives better precision than single sig-fig calculations using only four, very intuitive, symbols. https://t.co/BO9mLi8pLF

If you miss the NYTimes needle, especially one that is statistically uniform ( https://t.co/uqLw9f69Sw), you can use this page: https://t.co/xQ5cFrtRSD I whipped together to reason about the correlations between the swing states tonight as results come in.

Why don't we measure probabilities in degrees? https://t.co/uqLw9f5C2Y

In which I try to make sense of most of machine learning:

Is Kevin onto something? We found that LLMs can struggle to understand compressed text, unless you do some specific tricks. Check out https://t.co/DRO2IbTFCg and help @hoonkp, @alemi, Jeffrey Pennington, @ada_rob, @jaschasd, @noahconst and I make Kevin’s dream a reality.

Ever wonder why we don’t train LLMs over highly compressed text? Turns out it’s hard to make it work. Check out our paper for some progress that we’re hoping others can build on. https://t.co/mceqpUfZQo With @blester125, @hoonkp, @alemi, Jeffrey Pennington, @ada_rob, @jaschasd

Each delivery service should use its own distinctive knock.

PaLM 540 Billion, Google's large language model used 4.2 moles of flops to train. 4.2 Moles!

Happy to announce DreamFusion, our new method for Text-to-3D! https://t.co/4xI2VHcoQW We optimize a NeRF from scratch using a pretrained text-to-image diffusion model. No 3D data needed! Joint work w/ the incredible team of @BenMildenhall @ajayj_ @jon_barron #dreamfusion

@dpkingma @poolio To accompany the colab, I've also written a blog post https://t.co/qvmp8pg1g6 attempting to make sense of the VDM Diffusion loss. In it, I try to motivate how the VDM diffusion loss is simply the joint KL between the forward and reverse process.

Want to understand and/or play with variational diffusion models? - See https://t.co/V1jP11fMmI for a simple stand-alone implementation and explanation. (Thanks @alemi and @poolio for making this)! - See https://t.co/kwlCncttBk for an even more basic implementation on 2D data.

A pretty cool paper (and I also hope useful) on using pre-training models to create highly informative priors for downstream tasks. Thanks to all the collaborators, it was a lot of fun!

1/ Super excited to introduce #Minerva 🦉( https://t.co/UI7zV0IXlS). Minerva was trained on math and science found on the web and can solve many multi-step quantitative reasoning problems.

We are thrilled to announce Imagen, a text-to-image model with unprecedented photorealism and deep language understanding. Explore https://t.co/mSplg4FlsM and Imagen! A large rusted ship stuck in a frozen lake. Snowy mountains and beautiful sunset in the background. #imagen

you can verify with `echo -n "answer" | md5sum`

here are the next few days wordle answers as md5 hashes 2022-01-11 = 0b18a3d7b9c43ff1750d2baa4606b8d0 2022-01-12 = 047fb90408a79f189d51cbcea168b1a5 2022-01-13 = ab3358313efb03210a1babfb372246f1 2022-01-14 = d821e448212defd91ac1e67f9653a34d

We are presenting our paper "Does Knowledge Distillation Really Work?" at #NeurIPS2021 poster session 2 today - come check it out! Joint work with @Pavel_Izmailov, @polkirichenko, @alemi, and @andrewgwils. Poster: https://t.co/N4PlsxnpZE Paper: https://t.co/UNSIizi2GG

Excited to kick-start focus #SciML series on #ML meets Info theory and statistical mechanics! Amazing speaker/session chair line-up: @alemi (@wellingmax), @pratikac (Karthik), @ShoYaida (@jaschasd), @yasamanbb (@SuryaGanguli) and Elena Agliari. Details at:

While most papers on knowledge distillation focus on student accuracy, we investigate the agreement between teacher and student networks. Turns out, it is very challenging to match the teacher (even on train data!), despite the student having enough capacity and lots of data.