adammaj
@MajmudarAdam
Followers
15K
Following
10K
Media
51
Statuses
666
research @openai // ex: founding eng @thirdweb, residence @baincapvc // cs + neuro (on leave) @Penn
Joined October 2019
spent way too long making my book review page. ended up building a 3d interactive bookshelf
137
104
3K
my learning philosophy:.1. you can learn anything.2. you can do it way faster than you think.3. so choose an insanely aggressive goal.4. shorten your timeline by 10-100x what you expect.5. pre-requisites are a myth. try to do the thing first, figure out what you don't know yet,.
75
609
5K
I've spent the past ~2 weeks building a GPU from scratch with no prior experience. It was way harder than I expected. Progress tracker in thread (coolest stuff at the end)👇
93
552
5K
I've spent the past ~2 weeks trying to make a chip from scratch with no prior experience. It's been an incredible source of learning so far. Progress tracker in thread (coolest stuff at the end)👇
104
425
4K
I've spent the past ~3 weeks going through the entire history of deep learning and reimplementing all the core breakthroughs. It has completely changed my beliefs about deep learning progress and where we're headed. Progress tracker in thread (all resources at the end) 👇
55
365
3K
> be Vinod Khosla.> spawn in India, parents want you to join the army.> fall in love with entrepreneurship instead after reading about Andy Grove immigrating to the US and starting Intel.> start the first ever programming club at IIT (early signs of crackedness developing).> get
46
249
3K
After spending the past ~1 year building conviction in what I want to do next, I've joined the research team @OpenAI!. I've been here for 2 weeks, and am so energized / convinced there’s no better place to be in the world right now. I’m working as a resident under @BorisMPower
110
12
874
I've spent the past ~6 weeks going through the entire history of robotics and understanding all the core research breakthroughs. It has completely changed my beliefs about the future of humanoid robotics. My process + biggest takeaways in thread (full resource at the end) 👇
26
63
749
I’ve spent the past ~4 weeks understanding how the energy industry will define our future. I synthesized everything I learned into a single resource. My biggest takeaways in thread + complete resource at the end👇
38
75
688
I was challenged to make a chip from scratch in <2 weeks with no prior experience. Here’s how I’m planning to speed run the entire learning process for chip design & fabrication (suggestions welcome). 1. Prerequisites.> Review circuits physics - (semiconductors, p-n, junctions,
29
60
666
👀.
14
3
562
it’s crazy how well the concept of embeddings in deep learning translates directly to human communication and how clear it makes everything. every idea, feeling, concept in your head can be framed as a physical embedding in your brain represented by some set of neural activations.
36
32
481
Here's another link to the GitHub since the above link get's cut off for some reason:.
14
41
424
Step 1 ✅: Learning the fundamentals of chip architecture. I started by learning how a chip works all the way from binary to C. This part was critical. In order to design a chip, you need a strong understanding of all the architecture fundamentals as you constantly work with
13
26
421
this year I learned that the only thing standing between you and your destiny is your unwillingness to let go. you must to be willing to let your current identity shatter before your eyes into nothing. and then you have to stare into the void for who knows how long, not knowing.
20
28
393
the farther you go into neuro the more it becomes clear that deep learning is basically strictly superior to the brain. the brain still has some architectural advantages but these will be solved in due time. in the limit, there is nothing the brain can do that DL can’t do better.
45
11
385
Here's me playing with my design in the EDA tool.> I can zoom in and look at the individual cells and transistors.> I can selectively hide different metal layers to get a sense of how everything is connected.> I can view energy density, component density, etc. on my design
5
21
356
worth noting that this deep dive prepared me surprisingly well for @OpenAI (so far). everything you need to understand the entire private frontier of deep learning can be found in the papers / topics I covered here.
I've spent the past ~3 weeks going through the entire history of deep learning and reimplementing all the core breakthroughs. It has completely changed my beliefs about deep learning progress and where we're headed. Progress tracker in thread (all resources at the end) 👇
9
16
355
now you can make your own book review page with a 3d interactive bookshelf. having a nice review page is part of what motivates me to read consistently, so I wanted to make something for everyone to have their own. also finally launched this! details below
spent way too long making my book review page. ended up building a 3d interactive bookshelf
23
24
342
Step 7 ✅: Reverse-engineering and designing a GPU from scratch. My initial goal for my project was to build a minimal GPU. I didn't realize how hard that was going to be. My expectation was that building a GPU was going to be similar to building a CPU, where there are a ton of
17
19
317
Finally - checkout my tiny-gpu project for all the project details!. I built tiny-gpu to create a single resource for people to learn about how GPUs work from the ground up. Built it with <15 files of fully documented Verilog, complete documentation on architecture & ISA,.
20
11
278
The essential philosophy of life I want to teach my kids:.1. You can want things, set goals, visualize, make plans, act, and achieve anything. It’s good to want things and make them happen. 2. Observe everything around you and soak up as much context as possible. 3. Always try.
6
8
284
Step 5 ✅: Building my GPU in Verilog & running my kernels. After designing everything I needed to, I finally started building my GPU design in Verilog. This was by far the hardest part. I ran into so many issues, and learned the hard way. I rewrote my code several times.
7
13
270
It’s rarely discussed that both the brain and deep learning work by exploiting the exact same (very simple) principle of the structure of reality. When you understand this, it completely changes all your beliefs about why DL will out scale the brain in almost every respect.
the farther you go into neuro the more it becomes clear that deep learning is basically strictly superior to the brain. the brain still has some architectural advantages but these will be solved in due time. in the limit, there is nothing the brain can do that DL can’t do better.
27
13
273
Step 1 ✅: Learning the fundamentals of GPU architectures. I started by trying understand how modern GPUs function down to the architecture level. This was already harder than I anticipated - GPUs are proprietary tech, so there are few detailed learning resources online. I
7
22
258
If you ever feel like all the good opportunities have been arbitraged away, it's a sign that your information diet is too close to consensus. In reality, there's abundant opportunity everywhere, but you have to be looking in the places others aren't. This is especially relevant.
11
11
238
Here's me playing with a cool 3D visualization of my chip design - I can zoom into different parts of the chip, isolate different metal layers, and look at individual gates & structures in the design.
12
15
215
Step #1 ✅: Learning the fundamentals of deep learning from papers. I wanted to learn about the fundamentals of deep learning directly from the source of progress: the critical papers that have gotten us from simple feed-forward networks to models like GPT-4o. I suspected that
4
10
204
Working on a new web3/music project and want to give people who are early some cool free stuff + access. Drop your ENS if you want in 👇.
240
6
187
Step 2 ✅: Learning the fundamentals of chip fabrication. Next, I learned about how transistors are actually fabricated. Chip design tools are all built around specific fabrication processes (called process nodes), so I needed to understand this to fully grasp the chip design
1
7
170
Here’s the GitHub link again since the previous embed doesn’t show:.
3
16
168
Step 4 ✅: Creating my first full circuit in Verilog. This part was a cool unlock for me - my first experience with programming hardware using software. I made my first circuit with the hardware description language (HDL) Verilog. I made an RGB mixer circuit that converts
3
7
156
Step 6 ✅: Converting my design into a full chip layout. With my complete Verilog design, the last step was to pass my design through the EDA flow to create a finalized chip layout. I targeted the Skywater 130nm process node for my design (which I made a few designs for 2 weeks
3
8
157
Step 3 ✅: Starting electronic design automation by making a CMOS transistor, layer-by-layer. The CMOS transistor is the fundamental structure that enabled digital computation to take off because of it's unique energy efficiency. Drawing each layer of the CMOS manually made the
2
8
156
@karpathy Planning to make several more learning resources based on this:.> complete guide to the compute stack HDL > GDS > fabrication > transistors > CPU > C > OS.> open-source minimal GPU (called tiny-gpu) with full explanation. Maybe I'll take inspiration from you and do some videos.
7
3
154
Step 2 ✅: Creating my own GPU architecture. Next, I started to create my own GPU architecture based on what I learned. My goal was to create a minimal GPU to highlight the core concepts of GPUs and remove unnecessary complexities so other's could learn about GPUs more easily.
2
5
145
For people curious, here's the complete list/links to all the papers that I used to learn about the history of deep learning. I designed this list to include only the most critical breakthroughs & advancements that led us on the path we're on today (naturally, many great papers.
5
18
147
@GojalCappuccino it's mostly from repetitions - the more you learn/do deep-dives like these there are intuitions you pick up on over time that speed up your learning both b/c things are more familiar and your skills transfer + you know how to learn more effectively.
5
11
141
Step 3 ✅: Writing a custom assembly language for my GPU. One of the most critical elements was that my GPU could actually execute kernels written with the SIMD programming pattern. In order to make this possible, I had to design my own Instruction Set Architecture (ISA) for my
5
3
139
when someone shows you their art, they're inviting you into their soul. meet their openness with appreciation and care
4
7
142
Step 6 ✅: Designing my first full chip layout. This was the coolest part of the process so far. I used OpenLane (an open-source EDA tool) to perform the entire synthesis, optimization, and layout process on my design and come up with a complete chip design. Just seeing my
3
5
140
I posted my full learning plan earlier for anyone curious
I was challenged to make a chip from scratch in <2 weeks with no prior experience. Here’s how I’m planning to speed run the entire learning process for chip design & fabrication (suggestions welcome). 1. Prerequisites.> Review circuits physics - (semiconductors, p-n, junctions,
8
5
139
an artist can only hope, having poured his soul into his work to satisfy the taste of no one but himself, that the world may see in it the same divine beauty that he felt while producing it.
5
14
137
you can check it out here - and for anyone curious how its made -
6
7
139
some good clarifications from all the comments:.> point 5 means do top down learning first, not that fundamentals are useless. many people get stuck in the boredom of learning prerequisites without knowing why. maintaining motivation (why am I learning this) is critical to learn.
5
3
135
After tons of redesigns, finally running my matrix addition & multiplication kernels and seeing things work properly and my GPU output the correct results was an incredible feeling. Here's a video of me running the matrix addition kernel on my GPU, going through the execution
3
1
128
Step 5 ✅: Implementing simulation & formal verification for my circuit (disclaimer, this part might be boring, skip to the end if you want). Since the cost of bugs in hardware is far higher than in software (since you can't change stuff once you're design is fabricated), using
2
5
125
Step 4 ✅: Writing matrix math kernels using my ISA. Now that I had my own ISA, I created wrote 2 matrix math kernels to run on my GPU. Each kernel specifies the matrices to manipulate, the number of threads to launch, and code to execute in each thread. My matrix addition
4
5
118
the best life you can live is the one where you're always evolving toward your subjective ideal, sculpting yourself into art designed by your own taste, to be appreciated by yourself, those around you, and the world at large.
4
8
116
Step #4 ✅: Reframing the history of deep learning with the 7 constraints of deep learning progress (the most interesting part). The most interesting part of my deep-dive came from noticing a clear trend across all the key advancements, which has completely reframed how I
6
9
114
a person who thrives in uncertainty.> optimism in direction, confidence in agency.> high rate of directed action.> quick decision making with 60% confidence.> fast updates using learnings from past decisions. without these, you fall into fear, indecision, & paralysis.
4
5
113
never allow yourself to fully perceive reality by the boxes you create to understand it. randomness, chaos, unpredictability are core to reality, and they’re also what adds joy and beauty to life. predictability is boring - the unknown is the source of all thrill and adventure.
8
7
115
how to ask great questions to cracked people.> your question is about the frontier of human knowledge.> in their domain of expertise.> that they're specifically well positioned to answer.> and the answer can't be found online.
3
5
111
gradient descent works surprisingly well for personal direction too.> prediction - where am I at right now?.> cost function - where do I want to be?.> back prop - what do I need to do next to get there?.> update weights - do the next action: read the book, build the thing, etc.
5
9
98
"because I feel like it" becomes an increasingly good reason to do things as you get older and higher leverage. In an endless sea of options for whats possible, you return to the roots of your innate preferences to drive your actions.
2
6
97
as many in the comments have pointed out, important clarification that “invent Java” is referring to Sun - the individual credited with this at Sun is James Gosling, not VK himself.
1
4
96
Here's the full repository of my effort, with an explanation of conceptual & mathematical intuitions for each paper, and my implementations where relevant. Most importantly, the README has my complete reframing of the history of deep learning progress in terms of the 7.
2
7
94
College can be a very dangerous place for explorers. Exploration naturally involves uncertainty, and college exerts a strong pull toward paths of certainty. This is why it's so important to understand what you want to get out of college before going into it.
5
5
74
Step #2 ✅: Building core intuitions from each paper. I started by trying to understand the core intuitions and math for each paper. Going through the early CNN, optimization, and regularization papers, this process was straightforward. Each of these papers builds directly on
1
2
81
Just wrote this article on how to make money with @LiquityProtocol, one of the most innovative DeFi projects today: I provide an in depth explanation of the protocol, all of the areas with upside potential, and the potential risks to be aware of.
7
17
72
My final output (complete synthesis + predictions). Here's the repository of my full effort with:.> a complete in-depth synthesis of the research breakthroughs in robotics.> my perspective on the future of humanoids.
3
1
73
Devin demo is probably causing existential dread for lots of SWEs right now, which is justified. Many people have sunk years / decades into developing a software skillset. That skillset will likely be obsolete within the next decade. But here's another way to look at it:. No.
3
2
65
Step #3 ✅: Creating & training my own models. Going into this deep-dive, I was expecting that the implementation level would be one of the most interesting & important parts (similar to my chip dive, where I first learned the fundamentals, then build a GPU). But, as I continued
1
1
62
FINAL OUTPUT: Energy & Humanity ⚡️. A complete deep dive on how energy has defined our past, and how it will determine our future.
3
12
62
the one noteworthy asterisk is energy efficiency but that’s more of a hardware than a software problem. and also isn’t a blocker to achieving any sort of intelligence, just another lever to bring down cost.
6
0
64
in ML terms, good communication is like minimizing reconstruction error between the original embedding in your head and the reconstructed embedding in the recipients head where the bottleneck is the compression/signal in the words you select (resembles a VAE).
2
1
63
Get to a point where you can be genuinely happy for the successes of those around you. An inability to do this often comes from a place of insecurity - which stems from evaluating yourself based on comparison with others. Don't compete, win together :).
0
6
56
Step #1 ✅: Learning the fundamentals of robotics from papers. Over the past 2 years, $100Ms has been deployed into the robotics industry to fund the humanoid arms race. From twitter hype, public sentiment, and recent demos, it seemed to me that fully autonomous general-purpose
1
2
54
@OpenAI @BorisMPower Huge thanks to @tifafafafa (most amazing talent curator I've ever worked with) @BorisMPower @willdepue @GabrielPeterss4 for making this happen, and also thanks to @kevinzhang for his support!.
3
0
51
For those curious, here's the complete list of papers/links I used to learn about the history of robotics progress:. Perception.> SLAM - > SIFT - > ORB-SLAM - > DROID-SLAM - Planning.
1
6
48
Modern authentication systems are inherently restrictive, handing control over individuals' digital identities to large corporations. I wrote about how we used web3 wallets & security @thirdweb to build the first-ever truly self-custodied auth system 👇.
3
4
45
@MaxMynter I disagree with your disagreement, but reasonable points. these are all empirical from my deep dives (earlier on my twitter) where I learned sometimes several months/years of stuff in 2 weeks - you can see the quality on there. bad resources will waste your time more than the.
6
2
42
@amalsony_ that is exactly what made this possible
I've spent the past ~2 weeks trying to make a chip from scratch with no prior experience. It's been an incredible source of learning so far. Progress tracker in thread (coolest stuff at the end)👇
4
0
43
@SarvasvKulpati imo (from having the same thought and observing if it's acc true in my life) systematic inquiry is acc a myth in terms of having time to calculate the answers to everything (doesn't acc work, nor is it necessary). instead, it's actually better to have a bunch of open threads in.
1
2
38
This is a slightly inflammatory wording for a simpler point:. In many specific ways, the design of the brain is far better than current deep learning systems (energy efficiency, data efficiency, LTP, memory retrieval, etc.). But the deep learning paradigm is fundamentally capable.
3
0
39
Introducing zkMastermind 🧠. An on-chain code breaker game I built using zero-knowledge circuits. 1. Built a zkSNARK circuit and smart contract with Circom.2. Deployed the smart contract with @thirdweb 🚀.3. Made a frontend for it with Next + Chakra 😄
3
1
38
Finally, I want to thank @BainCapVC (especially @kevinzhang, @RonMiasnik) for supporting me on my deep dives!.
5
0
36
I'd recommend going through this repository and trying to build deep intuitions for every single paper:
2
2
35
If someone asks for help and it takes <15 minutes of your day, just do it. You never know how much of a difference you're going to make, and it's always great to be able to use your knowledge to help others.
1
1
34
launched on product hunt - upvotes/comments appreciated!. try it out yourself:
3
0
34
Part 1 ⚛: Understanding the fundamentals of energy. My goal was to understand the full picture of energy, which is rarely covered in one place because of how big the industry is. I thought this would be particularly interesting given the recent focus on energy as a primary
2
1
33
Step #3 ✅: Reframing the future of humanoid robotics (the most interesting part). With all this context, it became much more clear to me what we need to accomplish in order to get to the goal of fully-autonomous general-purpose robotics. Robotic perception and planning are
1
1
33
Finally, I want to thank @BainCapVC (especially @kevinzhang, @RonMiasnik) for supporting me on my deep dives!.
3
1
31
It's been great working out of SF w/ @thirdweb_ @_buildspace @Zeet_Co @trycovalent. The energy here is crazy.
0
0
29
Step #2 ✅: Building core intuitions from each paper. I initially focused on building the core intuitions for the innovations introduced by each paper. First, I tried to build an intuition for the fundamental goals and challenges of the robotics problem. I saw that at the
1
1
26
Part 2 🔥: Understanding humanities energy needs. Next, I focused on understanding how our energy usage has changed over the course of human history. This made it clear how critical energy has been to the development of modern civilization. It became clear that all of human
1
1
24
@anuatluru wrote about something similar here (wrt 48 laws of power). "choose your games wisely".
0
2
23
Our existence is a canvas for an unimaginable amount of novelty, beauty, and pain that we can't begin to conceptualize. Some of the most impactful experiences in life are those that remind us of this fact.
2
0
23
Depending on how this goes, I’ll also ship a dev react component + embed for everyone who wants to use this in their own websites 😄.
7
0
23
Part 4 ⚡️: Understanding energy distribution. After this, I focused on the energy distribution systems that actually deliver energy from the point of production to consumption. I specifically focused on:.> Oil & gas pipelines.> The electrical grid.> Electricity markets. Learning
1
0
22
feels like we’re watching the beginning of a scary polarization trend. the world feels everything accelerating. the group building the tech that’s going to change everyone’s lives is small . many outside that group will feel left out & polarize against it.
lol so much for the "decels wont threaten violence, that's just crazy talk" argument from connor leahy and others ¯\_(ツ)_/¯
2
0
22