📢 New paper on creativity & multi-token prediction! We design minimal open-ended tasks to argue:. → LLMs are limited in creativity since they learn to predict the next token. → creativity can be improved via multi-token learning & injecting noise ("seed-conditioning" 🌱) 1/ 🧵

we took a stab at designing such minimal tasks our ICML work but a *loooot* more needs to be done! . of course, careful large scale studies like the above are still very critical. just that the community should push on both fronts.

the debate around "can LLMs generate novel ideas" is why we also need *minimal* settings where we can.- evaluate novelty/diversity w little room for doubt, .- have cheaper iterations to test algorithmic/architectural ideas & .- most importantly, think clearly about LLM behavior!.

RT @_vaishnavh: 📢 New paper on creativity & multi-token prediction! We design minimal open-ended tasks to argue:. → LLMs are limited in cre….

RT @puneeshdeora: 🚨 New paper drop! 🚨. 🤔 When a transformer sees a sequence that could be explained by many rules, which rule does it pick?….

RT @_vaishnavh: Wrote my first blog post! I wanted to share a powerful yet under-recognized way to develop emotional maturity as a research….

RT @chrome1996: Have you noticed….🔍 Aligned LLM generations feel less diverse?.🎯 Base models are decoding-sensitive?.🤔 Generations get more….

RT @ZhengyangGeng: Excited to share our work with my amazing collaborators, @Goodeat258, @SimulatedAnneal, @zicokolter, and Kaiming. In a….

RT @EugeneVinitsky: We now know RL agents can zero-shot crush driving benchmarks. Can we put them on a car and replace the planning stack?….

Not just Galton, but also the Fisher (in Fisher information) and the Pearson (in Pearson's correlation) were all actively involved in eugenics. Important statistical tools were developed out of a desire to analyze human populations/inheritance and support eugenics 😬.

Also: Galton is a cousin of Darwin whose work greatly inspired him, leading Galton to develop (and also coin) the terms "eugenics" and "nature vs nurture" .

btw, there's a 2 page paper by Francis Galton -- the statistician behind concepts like standard deviation and regression toward the mean . and also, eugenics --- on doing arithmetic using your nose 👃

His offer from the business school at the University of Chicago was rescinded by the dean; relatedly, his theory about financial markets was in direction opposition to what was the prevailing school of thought there.

Mandelbrot joined IBM in the 1950s when they were pushing pure research (until 1987 when they said bye bye to supporting it 👋👋)

Mandelbrot once gave a lecture at Princeton, but the audience of famous people allegedly did not find it engaging enough, following which Oppenheimer and von Neumann gave their own talks explaining what Mandelbrot had delivered!

TIL that Mandelbrot (the mathematician who coined the term "fractal" in fractal geometry) had a "geometric" way of thinking, and that this possibly came from his childhood interest in chess and maps . [source: an essay from "When Einstein Walked with Gödel" by Jim Jolt]

RT @__tm__157: Yep what @JunhongShen1 said. I started working on ML for PDEs during my PhD. And the first three years was just reading book….

RT @__tm__157: This is the first step in a direction that I am very excited about! Using LLMs to solve scientific computing problems and po….

loved many of the points here! (loved the colors too).

Here's the current list, but I will keep it updated as I read more!

Reading is what helped me recover every time the system crushed me --- it helped me ✨contextualize ✨ my local experience in the grand timeline of science.