Ryan Greene
@rabrg
Followers
4K
Following
2K
Media
22
Statuses
196
i have great respect for Ilya, and trust in his intentions. but i can’t comprehend the lack of transparency in his decision.
27
19
670
Excited for my first public contribution at @OpenAI — Embeddings v2: it unifies text search, text similarity, and code search, outperforms our best v1 models at most tasks, while being priced 99.8% cheaper!.
Our new embedding model is significantly more capable at language processing and code tasks, cost effective, and simpler to use.
18
32
525
too bad the narrow domains the best reasoning models excel at — coding and mathematics — aren't useful for expediting the creation of AGI.
31
15
458
seems pretty obvious
openAI researcher said this yesterday, and today sam said we are near the singularity. what's going on?
11
22
311
is a black hole a little ominous? yes.is it the optimal emblem of a *singularity*? also yes
2
1
86
born too late to explore the world, born too early to explore the universe. born just in time to explore latent space.
5
7
59
Project Gutenberg, with a better UX for reading and discovering its contents, would rival Wikipedia in accessibility to humanity's knowledge.
3
4
45
drop the "large language". just "model". it's cleaner.
@karpathy even the "large" is suspect because what is large today will seem small in the future.
2
2
36
The most popular use case of our new embedding model is, by far, retrieval-augmented question-answering. Checkout @isafulf's new plugin that makes this a native experience with ChatGPT!.
We've written plugins for browsing & Python code execution (amazing for data science use-cases), launched with 11 partners, and have open-sourced a high-quality plugin for retrieval over any data you'd like to make accessible to ChatGPT:.
1
1
24
love the phrasing of "world simulators": imagine the scale of intelligence needed to accurately model relationships between complex entities in a physical environment. the beautiful aesthetics are just a cherry on top
4
1
32
one of the biggest lessons i've learned from my first year doing research is that, when someone comes up with an idea, it's almost certain that someone else has either already done it or is about to do it.
3
0
31
the best, for free. love that @OpenAI's values are aligned to do things like this
Not only is this the best model in the world, but it's available for free in ChatGPT, which has never before been the case for a frontier model.
0
2
30
I am hearing disturbing rumors that Ilya Sutskever has made the Boltzmann Machine computationally tractable.
4
1
29
i wish more recommendation algorithms gave users control of the diversity of their feeds, like the temperature slider of a language model. give me entropy. help me discover the unknown.
3
0
25
imagine you have a model that puts uniform probability on every possible answer to a question. it is, in other words, random and dumb. if you sample an infinite number times, youll get the correct answer though (pass@infinity = 1). as you give the model more intelligence, you can.
it’s over . turns out the rl victory lap was premature. new tsinghua paper quietly shows the fancy reward loops just squeeze the same tired reasoning paths the base model already knew. pass@1 goes up, sure, but the model’s world actually shrinks. feels like teaching a kid to ace
2
1
25
i mean this literally. given an infinite universe where self-replicating (sustaining) is possible; after enough time, it is *inevitable*, and once created, entropy will destroy all else: the universe becomes more and more selective for the self-replicating.
in a system where self-replication is possible, its optimization is inevitable.
3
1
24
i can’t wait for content recommendation algorithms to get augmented with natural language feedback, letting users granularly and iteratively critique and refine their feeds in real-time.
1
2
13
we live in a time where the Church is writing treatises on the relation between human and artificial intelligence
2
1
21
if the universe is deterministic and was instantiated from a near zero-entropy initial state, it in whole would have a lower Kolmogorov complexity than any particular human construct (e.g Wikipedia).
5
0
19
large language models are Strange Loops: every token ascends through the entire depth of the model, and each subsequent token loops back to the beginning, self-referencing itself through the generation of the previous step.
2
0
19
reading these lines in a metacognition text has me even more deeply appreciating the current state of LLMs
0
0
8
1
0
17
brain-computer interfaces will enable communication through embeddings instead of tokens, increasing the fidelity to thought by OOMs.
2
3
16
reading the writings of my favorite authors sometimes feels like reading an articulation of my subconscious.
1
0
14
in a system where self-replication is possible, its optimization is inevitable.
A digital "primordial soup" with no rules or direction can lead to the emergence of self-replicating artificial life forms, in an experiment that may hint at how biological life began on Earth. Learn more:
0
1
15
@OpenAI huge thanks to @arvind_io, @lilianweng, @sherwinwu, @sandersted, and many more people that i don't have the twitter handles of for all of the help with this release!.
0
0
14
the simplest of changes, like going to a park, or turning on some music, can make a piece of writing resonate so much more deeply with me. like my comprehension is limited by my environment. and it's empowering to remember how easy it is to shape it.
0
0
12
being in tune with your tastes compounds: knowing what you like makes it easier to find things you do; the more you discover them, the more opportunities you have to understand what draws you to them.
1
0
12
it is of no coincidence that "comprehension" and "compression" spring from the same root.
1
0
11
@pli_cachete the average output of a contemporary LLM is of higher quality than the average document from the internet.
1
0
13
@willdepue pls pls pls somebody pls
Project Gutenberg, with a better UX for reading and discovering its contents, would rival Wikipedia in accessibility to humanity's knowledge.
0
0
12
fun fact: the first formal theory on the relation between compression and general intelligence (Solomonoff induction) occurred simultaneously and independently with the coining of the term "singularity" (von Neumann) in the late 1950s. the Zeitgeist knows what's up.
1
0
12
writing is a noisy serialization of the mind. language models are trained to reproduce the noise (as opposed to directly reducing it).
0
1
10
friendly reminder that microchips are the most precise and complex non-biological arrangements of matter in the known universe.
2
0
11
if entropy dictates the arrow of time, intelligence is a kind of time reversal.
1
0
11
it is interesting that two of the most inspiring scientist-philosophers for AI researchers, Douglas Hofstadter (Gödel, Escher, Bach) and David Deutsch (The Beginning of Infinity), are themselves AI skeptics.
2
0
11
@bcjordan @OpenAI i'm excited to see how such a cheap ($0.40 per 1m tokens), easy-to-use model (versatile representational space, reliable API / don't need to self-host) enables more innovative retrieval-based apps like @omniscience42 to be created and scaled :).
0
1
10
my inner monologue just justified crossing the street diagonally by having "increased sample efficiency". im so fried.
0
0
11
passing copies of books between friends, seeing each other’s notes and highlights, is one of the cutest things ever.
0
0
5
i can’t help but have fomo of all the spirit going on in the arena though. looking forward to getting back :).
0
0
8
Borges' Funes is a prophetic meditation on large language models: the mapping of words to numbers, the idea of something that has a greater capacity of memory than all of mankind, and contrasting that capacity with that of "thinking".
0
0
8
mere mortals is an existential romance between humanity and knowledge.
1
0
8
o1's long chain-of-thoughts make it one of the loopiest models yet.
1
0
8
the amount you are able to take away from an experience is scaled by your interest in it; in this sense, being in tune with your tastes is a type of meta-learning.
1
1
7
@dwarkesh_sp books referenced in texts you enjoy reading in general. discovered my favorite book this year from a footnote in Schiller's aesthetic letters.
0
0
6
a beautiful implementation of a beautiful idea.
Life Universe Explore the infinitely recursive universe of Game of Life! Works in real-time and is perfectly consistent, never fails to remember where you are and where you came from. 無限に再帰するライフゲームの宇宙を探索できる作品を作りました #indiedev
0
0
6
language is the cartography of the mind: words mapping the infinite territory of ideas.
0
0
5
taste points to subjective liminal experiences: ideas that lay on both sides of the frontier of the mind. the part that lays beyond it sparks curiosity, while the part that lays within it provides the requisite understanding to expand towards it.
0
0
5
ilya words it a lot better: i had overestimated the role of new ideas, and underestimated the significance of their execution
Many believe that great AI advances must contain a new “idea”. But it is not so: many of AI’s greatest advances had the form “huh, turns out this familiar unimportant idea, when done right, is downright incredible”.
1
0
5
@_smileyball Higashiyama Jisho-ji (Kyoto), Maruyama Park (Kyoto), and Shinjuku Gyoen Garden (Tokyo)!.
2
0
5
@ilyas121_real @mattshumer_ @OpenAI some inspiration:.- @rileytomasek's - @ultrasoundchad's - @CedricMakes's - @itsandrewgao
0
1
5
@yoheinakajima memory is the outcome of learning. novelty is a prerequisite for learning (you can’t learn what you already know).
0
0
3
@Muennighoff @openais @hongjin_su @WeijiaShi2 @MSFTResearch Props to the authors of INSTRUCTOR and E5! They were some of my favorite embeddings papers from the past year, along with MRL, Promptagator, Relative Representations, and, of course, your MTEB :).
0
0
4