the primary criticism of AI you hear has nothing to do with water use or existential risk whatsoever: most people just think it’s fake and doesn’t work and is a tremendous bubble eating intellectual property while emitting useless slop along the way. when GPT-5 came out and

The year is 1965. The Soviet Union, a manufacturing powerhouse, will soon overtake U.S. GDP. The year is 1985. Japan, a manufacturing powerhouse, will soon overtake U.S. GDP. The year is 2015. China, a manufacturing powerhouse, will soon overtake U.S. GDP.

Starlink Mini offers fast, reliable internet on the go—great for traveling, camping, exploring, boating, RVing, and more. Stay connected without dead zones or slow speeds. Order online in under 2 minutes.

Intuitively, cross-run model merging (like Gemma 2) would break with MoE, but I don't entirely see why SWA/EMA should fail. DSv3 even mentions keeping an EMA copy of the weights on CPU, but they only use it for evaluation!

Llama 3 used SWA during pretraining and EMA during DPO. As far as I know, landmark open-weight releases since then haven't used SWA/EMA. Does anyone know why?

Many daily frustrations come down to credence goods, principle-agent problems, and ultimately information asymmetry. As information becomes very cheaply accessible they might go away.

I remember most restaurants being mediocre when I was a child. Then some guy made online restaurant reviews popular, and thanks to that incentive, most restaurants have fantastic food now. LLMs might do the same thing in other areas.

If you spend lots of time reading PDFs I recommend trying sioyek. Delightful software.

The good stuff is always in the appendix

2019 was so so long ago

Chinese recsys papers are always written by a company you've never heard of with three billion DAU

@filipviz Plato was the first Engineer of the Human Soul

The issue with today's "Deep Research" offerings is that they believe everything. Every research paper sells itself as being amazing! If you're not skeptical by default, the result is a 50-page survey of approaches which won't work in practice.

It's wild how good OpenAI's o3 model is at translating Ancient Greek poetry. It does a way better job (while preserving the poetic structure) than you find from academics who spend their whole life doing this. This would be incredibly depressing to me if I were studying Classics.

Many of history's greatest empires were established across two generations: Caesar and Augustus, Philip II and Alexander, Pepin the Short and Charlemagne, etc.

Every wealthy semi-retired banker I talk to is using the ChatGPT free tier and thinks it's incredible.

Caesar -> Augustus may be the best succession story ever. Imagine leaving your estate to your obscure 18-year-old nephew, and then having that nephew win over your army, trounce your political opponents, consolidate the empire, and rule excellently for 40 years. Insane run.

While I was asking about something else, Claude noticed a $30k writeoff my tax preparer missed last year. At this point Claude has saved me tens of thousands of dollars.

First you put a branch to fire. As a second step you begin farming. Then, you discover Haber-Bosch process.

This era of image gen was so good

I love the dreamy images GANs make the first epoch or two. Reminds me of 2021.