Update: @atharva_peshkar and I got into @ycombinator W26 (after getting rejected and then reversing it) to build @axionorbital Huge thanks to our YC partners @bosmeny , @ChristinaG325 , @gustaf , and @dazzeloid for believing in the vision. At @axionorbital , we're building

https://t.co/0zBPbRCDRY

It was never going to converge...... @axionorbital @atharva_peshkar #Entrepreneurship #algorithm #NeuralNetEra

🚀SonicMoE🚀: a blazingly-fast MoE implementation optimized for NVIDIA Hopper GPUs. SonicMoE reduces activation memory by 45% and is 1.86x faster on H100 than previous SOTA😃 Paper: https://t.co/Xesd3cNcpQ Work with @MayankMish98, @XinleC295, @istoica05, @tri_dao

We are replacing "weather permitting" intelligence with ground truth. This means tracking military convoys through storms and mapping floods at night at 1/100th the cost. We are actively onboarding partners in Defense, Finance, and Disaster Response. Watch us turn noise into

For decades, the space industry has had a dirty secret: 70% of the time, satellites are blind. Clouds, smoke, and darkness turn billion-dollar sensors into expensive, useless metal. We fixed the game. We use deterministic generative AI to translate Synthetic Aperture Radar

Can vision transformers learn without images?🤔👀 Our latest work shows that pretraining ViTs on procedural symbolic data (eg sequences of balanced parentheses) makes subsequent standard training (eg on ImageNet) more data efficient! How is this possible?! ⬇️🧵

A lot of datasets are actually really bad! Even big conference ones, even ones that got awards! It made me blanket lose trust. It's simple to find out: Just spend 30min looking at it randomly. For vision, finetune a blind and a non-blind model and compare. That's all it takes.

With the legends @GauravSeth93 @GulmoharAB My most productive meetup in SF yet. @GauravSeth93 is building SOTA SAR satellites to monitor the Earth @GulmoharAB is building space lasers for telecommunications Space is the next frontier (I have more bias towards EO)

AIの「物忘れ問題」が解決するかもしれません。 Googleが、AIの致命的な弱点である物忘れを克服する衝撃的な新手法「Nested Learning」と「Hopeアーキテクチャ」を発表しました。 これはAIが人間のように継続的に学習するための、極めて重要な発表です。

A recipe for JEPA (Joint-Embedding Predictive Architecture): • The optimal embedding distribution is an isotropic Gaussian • Use SIGReg regularization to achieve this Gaussian-shaped embedding space • LeJEPA proves the method works in practice @ylecun's latest work outlines

Just like training dogs, positive feedback alone is very inefficient. Combining positive and negative can rapidly narrow down the search distribution, given the student has a decent inner RL algorithm. Geoff also told me that most research fails, and that's research. Super kind.

New Anthropic research: Natural emergent misalignment from reward hacking in production RL. “Reward hacking” is where models learn to cheat on tasks they’re given during training. Our new study finds that the consequences of reward hacking, if unmitigated, can be very serious.

Generalized Advantage Estimation (GAE)–used in PPO–is one of the most complicated aspects of reinforcement learning (RL). Here’s how it works and how we can implement it… The advantage tells us how much better a given action is compared to the average action in a given state:

This is a phenomenal video by @jbhuang0604 explaining seminal papers in computer vision, including CLIP, SimCLR, DINO v1/v2/v3 in 15 minutes DINO is actually a brilliant idea, I found the decision of 65k neurons in the output head pretty interesting

Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Following - Introduces a new benchmark with over 1,600 prompts and expert-curated rubrics to evaluate the ability to follow complex, multi-turn instructions - Introduces a novel post-training

This one blew my mind 🤯 Alibaba just released a paper called AgentEvolver and it basically turns agent training into a self-improving loop that doesn’t need human-made datasets or brute-force RL. Instead of relying on expensive task construction, random exploration, and giant

Deep RL is a roller coaster—only for the strong-hearted

Read this today and it’s awesome. https://t.co/HtjwUSixOJ