If you want to learn more, read the full article: https://t.co/7NFps9OvwA

But this is not all. NNs are also, in practice, extremely well-suited to take advantage of the immense increase in data and compute availability of the last couple of decades. But that's a story for another thread.

6) NNs adapt to (almost) any problem Almost any task can be framed as the problem of minimizing the error of some differentiable loss function: binary cross-entropy for classification, MSE for regression, contrastive loss for similarity, ...

5) NNs are (relatively) easy to train Regardless of the complexities of the architecture, there is a single, universal algorithm called Backpropagation that can compute how to modify the weights to decrease the prediction error. All NNs are, at least in principle, trainable.

4) NNs dissentangle data Or manifold learning, the idea that NNs are projecting input data into a very high-dimensional space where the thing you want to learn is easy (e.g., there is a linear separator that classifies images of different types).

3) NNs learn to abstract automatically Representation learning is the technical term. We can often interpret a deep neural network as a sequence of increasingly complex pattern detectors built from simpler patterns. Each layer learns a more abstract representation of the input.

2) Neural networks are very flexible We have a gazillion layer types and architectures to encode all sorts of domain knowledge: convolutions for images, recurrent networks for sequences, transformers for context awareness, siamese networks for similarity, ...

1) Neural networks are universal approximators This means that, theoretically, there is a big enough, super simple NN that can approximate any (sensible) mathematical function. (Now, that network is not the one you often want).

Why are (artificial) neural networks so powerful? No, it's not because they are "biologically-inspired." That has little to do with it. It boils down to: - Their theoretical properties. - Their practical capacity to scale. Let me explain 👇

https://t.co/M8tILKUATl

If you're the kind of nerd (like me) who prefers to chat with an LLM in the terminal instead of a website or app, check this project I'm working on. It's still pretty rough around the edges, but it knows Bash and it can mess with your system in fun and unexpected ways.

Here's a fun little LLM-based project I'm working on: https://t.co/gguHa8JFcF It's a bot that lives in your terminal and knows a bit of Bash, Python, and Linux. It can run commands and create files and maybe wipe out your hard drive if you're not looking. Take it for a spin?

@alepiad Hey Alejandro! We are building a machine learning community here on X. Would you mind help us share it so more enthusiasts can find it? I would appreciate it a lot! 🦾 https://t.co/8k9fANRJcy

@YiMaTweets @alepiad @OpenAI @AnthropicAI @sama @ai_ctrl (EXPERIMENTAL section of README) #RIPrompt now encouraging the generation of internal 'AoT Symbolic Plans' to later follow and guide critical thought and output sections 🤩 for Claude projects

https://t.co/rYHIAnvpCN

But all is not lost. We have a bunch of tricks to extend the reasoning capabilities of LLMs, as OpenAI o1 shows. I lay it out in this article if you want to read more.

Reason 3: Inability to Loop. LLMs are not Turing-complete simply because they can't loop indefinitely. This means some semi-decidable problems are forever outside the reach of (pure) language models.

Reason 2: Bounded Computation. Each token processed requires constant computation. This means the total computational budget is determined by the input size. But we know some problems (including logical reasoning) require an exponentially large computation.

Reason 1: Stochastic Sampling. LLMs rely on probabilities to pick the next token. Even when you fix the temperature, randomness is still built into the language modeling paradigm. But logic is anything but random.

LLMs cannot reason. Despite their impressive capabilities, all LLMs, including OpenAI o1, are still fundamentally limited by design constraints that make them incapable of true, open-ended reasoning. Let's break it down. 🧵 (1/5)