https://t.co/CJkgsryeqj

Yes. People begin realizing the real-world implications of brute-force scaling. Being "frontier" will soon be considered meaningless, unless it's the *Pareto frontier* of cost vs capability. And that needs new foundational improvements.

All the "ARC-AGI is toast" people forget that there are two axes. This is a benchmark for skill acquisition efficiency. @fchollet @GregKamradt @mikeknoop @arcprize

I think the best mental model for today's agents is Guy Pearce's character in one of Nolan's first films, Memento. He's got extreme amnesia, and needs to look up instructions for every single action from notes (on his body). Learning still happens, but there's no updating of

A few things to keep in mind amidst the market's downturn...

.@ylecun definitely did not just call @elonmusk a gros con (f-ing idiot in French). He only meant to encourage him to grok on.

The METR chart clearly needs a third axis for cost. The guys @arcprize are onto something.

While impressive as an absolute score, this doesn't improve the scaling curve. The team @noemon_ai is feeling cute -- might announce something while in stealth. Should we?

"From my perspective, one of the biggest shortcomings is at the moment that I can't specify a problem and tell it to come back when it's found a solution, even if that takes a week. At best, it will try for an hour, and then inform me it didn't manage to solve the problem." Such

the big world blog post by @kjaved_ is a must-read for anyone in robotics. against the complexity of real world dynamics, continual learning is a prerequisite, not an afterthought. so long as foundation models are a mere crystallization of a static model of the world,

Global structure emerging from local rules. A trillion-dollar recipe for continual learning, long-horizon agents, and post-Moore's law non-Von Neumann computing. https://t.co/vqwMr9wLJs

There’s a big difference between “we claim it works” and “we deployed it at frontier scale”.

Blessed are the poor in spirit.

Water polo goal of the century by Stelios Argyropoulos-Kanakakis.

updated neolabs list. what's the right categorization? llms vs other models? agent-first vs model first @swyx? shipping to users vs not yet shipping (hype to ship ratio)? enterprise vs consumer? large general models vs specialized use cases? - @arcee_ai - @PrimeIntellect -

7/ @DBahdanau's attention, Alex Graves' neural Turing machines, Parikh's attention model, Miller's Key-Value Memory Networks, residual connections, lstms, highway networks, layer norm and fast weight programmers, etc @SchmidhuberAI's pic above is not there just for meme value.

6/ Now, why doesn't Google enforce the transformer patent? Is Google doing it for the love of the game? The more probable answer is that they just couldn't enforce it if they tried. A competitor would argue in court that the patent is merely an obvious combination of prior art.

5/ Notably, that change was only possible without changing the priority date because it's *narrowing* the protection. BTW you may notice the patent was filled in 2018, after the June 2017 paper. That's possible because they first filed a provisional in May 2017.

4/ To pin the transformer to a physical implementation that is not just math, Google added this part to the end of claim 1 of the patent. So, in Google's European transformer the encoder layers must operate in parallel!

3/ The European patent office did raise this point ("the transformer is just math"). And Google had to narrow down the European transformer.