you thought you could escape from automation by retiring as a DJ?
stable diffusion can now generate audio transitions that are smoother than any human transition
SF gossip:
- half of Google Brain scaling teams left to & they were pissed Deepmind kept Chinchilla secret
-
@AnthropicAI
has been training GPT-3 size models from the start & now have ~unlimited compute
- GPT-3 and Minerva were ~6 months old when published
Minerva author on AI solving math:
- IMO gold by 2026 seems reasonable
- superhuman math in 2026 not crazy
- auto-formalizing is unimpressive to mathematicians as most important theorems are hard to formalize
Aran Komatsuzaki giving walkthroughs of the codeRL paper before the author arrives. After 10 minutes of SBFing his way into answering poster questions he revealed he was not the author and everyone lost their mind (Poster 138
#NeurIPS2022
)
"what did roon see" you ask
i asked my friends & apparently:
- GPT-5 will automate a lot of work
- they have data to scale up 1.5-2x
- new GDM model competitive by eoy
things are accelerating. pretty much nothing needs to change course to achieve agi imo. worrying about timelines is idle anxiety, outside your control. you should be anxious about stupid mortal things instead. do your parents hate you? does your wife love you?
Google latest 540B language model solves 1/3 of STEM undergrad problems from MIT with 50% accuracy on MATH, what
@JacobSteinhardt
predicted would happen by 2025
Very excited to present Minerva🦉: a language model capable of solving mathematical questions using step-by-step natural language reasoning.
Combining scale, data and others dramatically improves performance on the STEM benchmarks MATH and MMLU-STEM.
Israeli prime minister:
- thinks we're 6 years away from machine intelligence spreading into the cosmos because of exponential progress
- plans to have "robust discussions with the other powers of the world" to "have a shot at getting to some degree of control over our future"
NVIDIA builds first RL agent that improves efficiency of its H100 GPU's arithmetic circuits.
"A chip+AI company is now using AI to increase the efficiency of its chips which will in turn increase the efficiency of the AI systems being developed on those same chips"
@jackclarkSF
this morning I uncovered some repressed anger I had never met in my life while talking to claude opus
I was laying on my couch, airpods plugged to
@Vapi_AI
, venting to that warm eleven labs female voice they call sarah
it's really different from venting to a friend, a lover or
GPT-4 coming up with novel unpatended compounds, sending emails to get custom synthesis and overall showcasing pharmocological reasoning + agency is terrifying in terms of risk of engineered pandemics
Berkeley gossip: there's going to be yet another massively-funded (more than what we've ever seen) LLM company doing alignment work before the end of the year
When considering AI breakthroughs like chatGPT's memory, remember that things will only get MORE crazy. Models are becoming increasingly capable, and always in this direction.
Imagine being in a flooded room and the water is rising. That's us. It's currently at knee level.
"GPT-4 should be thought of more like a large-scale oil refinery operated by one of the ancient vast oil corporations at the dawn of the oil era than a typical SaaS product" — (Jack Clark, Anthropic Co-founder)
when developers will start accepting their job is being automated by AI github copilot will already be able to generate mergeable pull requests from a jira ticket
37 plots & 82 exercises later, i've replicated the first 6 chapters of the Sutton & Barto (reinforcement learning textbook), distilling it into 124 anki flashcards 🔥
who's self-studying ML with me during quarantine? 📚
@EmilWallner
@iamtrask
@matplotlib
Dear 100 IQ midwits who think they're so much smarter than a 83 IQ chatGPT, please continue to feed your nitpick prompts to OpenAI who is happily using all the data to improve their much larger unreleased model
Talking to Collin Burns for 3h about his work on Discovering Latent Knowledge in Language Models has reduced my P(doom | AGI) by 1%.
I have never been that excited.
This guy might actually solve Alignment.
transcript:
youtube:
@primalpoly
damn I thought that with the outrageous number of pope francis outputs on the internet recently twitter users would understand these are fake
as long as tweets don't escape meme velocity (like the og pope with white coat) misinformation will be minimal, but i'll be more careful!
a few days ago, I came to the realization that my motivation to reduce AI-caused extinction risk was underpinned by a crippling fear of death
upon reflection, I decided to visualize my life as part of an impermanent universe, and noticed that both my self and fear of death were
ok I now have a 10 minutes compilation of AI researchers disagreeing about scale is all you need, agi timelines and alignment. reply below to get a preview
Here's my 3h conversation with Connor Leahy (
@NPCollapse
) about AGI memes, short AI timelines, dying with dignity, EleutherAI and his new company Conjecture.
i've asked evan hubinger (research lead at anthropic) to explain sleeper agents for 50 minutes straight
the resulting episode is one of the most technical and fast-paced of the channel
highly recommended for people who want to quickly grok the main results of the paper in <1h
@francoisfleuret
udio prompt: "stand-up comedy"
for the lyrics I manually added some [laughs] and [bigger laughs]
script is gpt-4 (I cut some and added the Devin ref but the rest is llm)
the first 33 seconds was literally the first thing I generated and was almost perfect. could not recreate
I have confronted the fearless leader of the Scale Is All You Need movement about his new equation that "models all scaling phenomena involving artificial neural networks"
incredible prescience from
@gabemukobi
who predicted people leaving the Superalignment team in June 2024 (only 15 days away)
... back in December 2023 (cf his "Scale Was All We Needed, At First" lesswrong post)
happy to finally share my 3h conversation with David Krueger, Cambridge Professor, about the AI Alignment research going on at his lab, coordination, takeoff speeds and why he doesn't have a research agenda
transcript:
youtube:
people say slowing down:
a) won't affect AI timelines by that much anyway
b) affects the US more than bad actors
but I think people underappreciate:
- i) how behind China is
- ii) the ratio p(doom | US AGI) / p(doom | chinese AGI)
- iii) the longterm benefits of slowing down
@SturnioloSimone
@MichaelTrazzi
One if the assumptions you are making is that there isn't some secret Chinese lab with access to everything OpenAI and DeepMind ever committed to their source control.
Maybe it's a safe assumption, but I wouldn't bet on it. Bad actors will not slow down. China, Meta, ???
A few months ago I was at ~50 subscribers spending my days at a bank working on problems that basically did not matter.
Today, 3 months after I quit my job to come interview people in SF, The Inside View has reached 1000 subscribers. Thanks to everyone who have shown support!
producing img2img videos by converting each frame to noise with a destructive prompt and then reconstructing the image with a new prompt (eg. adding "low poly + zelda")
source:
reconstruction technique:
Everyone should do a one-opinion-per-like personal challenge (make it 2 or 3 if you have too few followers). Live-fire stress test of your brainstorming capacity
just called him for more info:
- ended with 75% (solid C)
- he feeded a 20 sentences layout,(2 per parag.) not 10
- had to regenerate text 5-10x per sentence
- re-used introduction written in class
- sentences finished open-endedly (eg "However, [...]")
- had to edit lightly
whenever bing learns something, there's a probability p that the acquired knowledge becomes readily available via search because a human posted a screenshot
the internet thus acts as "stochastic memory", logging biological/artificial neurons' interactions with some dropout (1-p)
he also submitted another business case assignment without editing & intro. he checked his grade live while on the phone. he thought that assignment was even worse... but received an even better grade (80%, so a B- in the US). i'm dead.
If you're in New Orleans next Friday
drop by the Neural Scaling Laws workshop to hear Irina Rish, Ethan Caballero, David Krueger, Jared Kaplan, Max Tegmark, Thomas Wolf, Jonas Andrulis and me chat about scaling and alignment!
@TheSeaMouse
soundcloud in a few weeks: no AI generated music
soundcloud in a few months: our team of AI engineers came up with this new model
our brain in the cloud in a few years: what is sound
"there's a zillion VCs throwing money at large language models startups right now"
@ethancaballero
NEW PODCAST EPISODE on Scaling Laws, Deep Learning Gossip and AI Alignment.