What if von Neumann was wrong?. Compute and memory should not be separated. Definitely, for #DeepLearning workloads, an architecture that combines memory and compute in the same physical place would be faster and more energy efficient.

The ability to ask the right questions will be the most important skill of the future. The information is free; you just need to filter out the noise.

Flash-DMA. is a high-performance attention implementation that integrates Flash Attention's memory efficiency with Dynamic Mask Attention's sparse computation capabilities for processing extremely long sequences in transformer models.

West World is coming.

The Bitter Lesson is coming for Tokenization. a world of LLMs without tokenization is desirable and increasingly possible.

A curated list of graph-based fraud, anomaly, and outlier detection papers & resources.

HighNoon LLM . Hierarchical Spatial Neural Memory (HSMN) to process language like humans, organizing text into a tree for efficiency. It cuts computing needs by 78x, excelling in summarization, coding, and Q&A, while running locally for privacy.

Introduction to Decision Theory: .Part 1. The Challenge of Decision-Making.

Visual DB SQLite. is the most full-featured UI ever made for SQLite. Use it to build data entry forms, spreadsheet-like grids for viewing and updating data, and interactive reports.

Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents.

Welcome DuckLake!. An integrated data lake and catalog format. DuckLake delivers advanced data lake features without traditional lakehouse complexity by using Parquet files and your SQL database. It's an open, standalone format from the DuckDB team.

Working with Anomaly detection is challenging, nobody has a clear evaluation procedure. Even an anomaly itself is highly contextual. P.S. Find anomaly in the photo.

We cannot even sell perfumes without #AI hype.

This is a collection of simple #PyTorch implementations of neural networks and related algorithms. These implementations are documented with explanations.

Every day I use Google Search less and less. ChatGPT, Gemini, and Perplexity are immensely helpful for searching solutions during the ideation phase. It's like having a team of researchers working with you.

Buckaroo. The data table UI for Notebooks. Quickly explore dataframes, scroll through dataframes, search, sort, view summary stats and histograms. Works with Pandas, Polars, Jupyter, Marimo, VSCode Notebooks.

And additionally: what do you think about replacing softmax with sigmoid? It might make attention scores focused less on a single token.

When you create a classification head on top of a large pre-trained transformer, . what is your preferred approach for handling the sequence hidden states: . Attention pooling, multi-head attention, or gated attention pooling?