I burned in🔥2000$ in finetuning so you don't have to.
I fine-tuned models with
@OpenAI
and
@anyscalecompute
API endpoints with 50million tokens. Here are the results I wish I knew before getting into finetuning.
If you just want a quick snapshot, look at the figure. A longer…
@karpathy
(one of my hero's along with Ilya) dropped a video after a long time. It's called : "Intro to Large Language Models". If you are in the field or generally curious, please go watch it! He is the best AI teacher that I know of. Simplifies concepts for simple folks like…
Had a very interesting thought. I call it Geographical GPU arbitrage.
Germany (where I live) electricity prices are high. I wondered how much it would cost to run a deep learning workflow at full load on my GPU (RTX 3090) here in Germany for one day? And extending that my…
1/ 🤯💥 Prepare to be amazed by the power of AI-generated art!
I used AI to create incredible portraits of women from different neighborhoods in Berlin, all 100% AI-generated. 🇩🇪👩🎨 EXPAND the thread to see these stunning AI portraits! ⬇️⬇️🔥
#AI
#GenerativeArt
#Berlin
Started fine-tuning newly released: 'codellama/CodeLlama-70b-Instruct-hf'
If it works, it will be state-of-the-art for a specific video application.
If it works, we will open-source the weights.
🚀 Summarized all "Huberman Lab" episodes for quick reading!
Free, no catch, no sales pitch, no paywall, no data collection, nothing of that sort. In turn, Looking for feedback to improve & make it valuable for everyone. Let's crowdsource this and make it a resource for the…
Woohoooooooooo 🎉! Wrote my own first AI model wrapper completely running locally on MAC.
It sits and nicely listens to a podcast in realtime. And then tells if any book was mentioned in it! See 10 sec clip below.
So coooooooooooooooool.
Whisper.cpp + llama.cpp + Grammar…
@karpathy
I already did something like this for your earlier "Intro to Large Language" video and put it in my blog :
The guides cites back to the original video and transcript.
But it misses just the "seeing" - the image part.
5 months ago. 0% technical idea about AI.
Today. My first contribution and merge in llama.cpp (A leading open source AI project).
It was a very tiny contribution (bug fix). But I am happy about it. One step at a time.
In future, I would like to grow to a point technically…
I will do the fine-tuning for you, or here's my DIY guide
👋 Hello Everyone, I'm Adi. About four months ago, I quit my job to focus solely on AI. Starting with zero technical knowledge, I've now ventured into the world of AI freelancing, with a specific interest in building LLMs…
@OpenAI
@anyscalecompute
Further reference :
-
-
Also
@anyscalecompute
pricing page is not clear like OpenAIs. I assume when they mean 1M$ pricing, it's the same pricing both for inference and finetuning. OpenAI charges different pricing for tuning and…
I created this table for myself now to see how
@anyscalecompute
compares now to
@OpenAI
. Especially after Dev Day. Sharing it to public in case others find its useful.
Some lessons :
1. Finetuning cost is getting close. Especially for ~1M token range. For large number of…
2024 Our Goal : Sustainable Revenue
✂️Cost : Ruthlessly cut down expense.
💹 Revenue : Work with
@dineshsai
to add insane amounts of value to customers. In exchange for high profit.
Chances of failure very high. But we will singularly focus and try to get to positive MRR…
Power of Opensource, community and internet!
@huggingface
&
@abhi1thakur
🙌🫡🙌
Oct 19 : I post long tweet on my finding and struggles on deploying my own AI. There was a short sidenote there on how it was slightly hard for a noobie like me to get my own AI model. I dint think…
Some AutoTrain LLM users were facing difficulties merging adapter models with base models. So, I've created this space that lets you merge adapters with base model easily. Just duplicate the space, add a GPU and merge in a few minutes. Check it out here:
If this is really true, it would be incredible!!!!
With
@OpenAI
I always was only concerned with pricing, never performance. This will totally change the game.
If it happens, I am stuck with
@OpenAI
for a long while I guess :D :P !
Of all the little AI tool that I have built over the last few weeks, this probably is my favourite. This little AI "watched" over 4000 hours of podcasts and extracted over 1000 books mentioned in all these podcasts.
Mainly my favourite because, I got into the weeds and…
For the past eight years, since graduating from uni, I've mostly lived in small spaces my home. Ranging from 13 (min) to 24 square meters (max).
It was mostly WG's in Germany (shared spaces) or tiny studios. It meant workspace was the same as living space (or shared with some…
Financial Independence and Entrepreneurship
When I was 23, my friends knew me as the guy who blogged extensively about financial independence and retire early (FIRE movement). I almost militantly believed in it back then. Wanted to own my time.
However, my perspective has…
Yesterday, someone asked on Twitter, so I created a simple, FREE app.
Enter your YouTube URL and you will receive the transcription file.
Transcription is a very simple one, but the start.
We will be slowly releasing all our internal models and tools…
@reach_vb
Hi Vaibhav,
Thank you for continued work on this for the OSS community!
Follow up question.
@sanchitgandhi99
published a blog post a week earlier : , about speculative decoding for Whisper.
Will speculative decoding work with insanely-fast-whisper? It…
Finally, I found the time to do this. I am moving away from cloud storage providers and setting up my own S3 bucket hosted with MinIO.
I had a couple of 4TB hard disks and connected them to my server PC. Now I have an 8TB S3 bucket locally.
Mainly because of the poor internet…
Today, I am officially an e-resident of Estonia.
I am taking an unconventional path here. Trying to be a location independent AI entrepreneur. By using the digital residency of Estonia.
I decided the best way to figure out if this is the right way is dive in and to try it…
@OpenAI
@anyscalecompute
Typo in original (and I am unable to edit):
a) Anyscale costs 40x cheaper to finetune.
b) Anyscale costs 56x cheaper to --->INFER<---
(a) comes up with a caveat mentioned here :
@OpenAI
@anyscalecompute
Further reference :
-
-
Also
@anyscalecompute
pricing page is not clear like OpenAIs. I assume when they mean 1M$ pricing, it's the same pricing both for inference and finetuning. OpenAI charges different pricing for tuning and…
Since the startup thing started. I have been burning the candles at both ends. I feel it all now.
All the yesteryears stored buffer of mental health and physical health is close to end.
Need to replenish and figure out work-life balance. If I want to play this game for…
PSA : If you own a MAC please do this!
Probably the best 1 hour I spent in recent memory.
I just hacked something together now. Likely I am going to use this for the rest of my life.
Pretty much anyone with MAC (+ OpenAI API key) can do this.
It just uses "Automator" for…
Excited to share our new venture, 🌟!
Co-founded with
@dineshsai
, we're on a mission to democratize AI for content creators. We simplify complex AI tech to empower creators, enhancing engagement and amplifying reach.
Our goal? Helping creators what they…
Building a State-of-the-Art Video Summarizer: Part 1 - Semantic Chunking and Building a Chunker
Warning ⚠️: This going to be a long deep dive technical blog for AI nerds.
Over the last four months, I've been developing a state-of-the-art video summarizer using Large Language…
Snapshot from life of POOR solopreneur.
AI model running overnight in my MAC and cooled by fan. Placed in kitchen because fan’s too loud in my bedroom for sleeping.
Things you gotta do to make it through 😂
@huggingface
@anyscalecompute
Oh wow!
@anyscalecompute
fine-tuning is fast! And it actually just works. Super compatible with current OpenAI workflow.
The best experience of fine-tuning and inferring!
Also, 1$ for 1M token.
@OpenAI
needs to step up its fine-tuning game now.
Interesting results!
Solopreneur Engineer Struggles
Solopreneur Me: "Let's create a simple MVP to validate our idea, build something useful for people and maybe earn enough for a sandwich🥪. Okay?"
Also the Engineer Me: "But nooooooooo... first, let's design and engineer a robust system 🛠️ that's…
I tried Claude Opus. It's very good.
My co-founder and I had been struggling with a problem for a while now. We tried with GPT-4, but the results were mediocre. So we wanted to fine-tune a model (all my earlier tweets).
Tried the same in claude and it got it right on the first…
@BenjaminDEKR
I know it's become fashionable now to pile on OpenAI. But for context and sake of transparency here is the complete quote :
“I am not sure.. not sure on the exact status of where that is right now. But I love Ilya, I think Ilya is an unbelievable person and researcher… it's…
Speech to text 😯🔥 in GPU.
20 minutes of speech to text in 20 seconds on a consumer GPU. That too High quality!
Man, just three years ago this was a pipe dream!
How fast things are moving?
AI developments are focused too much on chat use case. So much beyond that is…
🚀 Excited to unveil the best summaries of self-improvement videos on the internet at Wisdom In A Nutshell! After seven weeks of grind, it's now live. You can watch the video explain why I think its the best or read the official announcement :
Some…
Audio Cleaner: Remove Your "Ahhs," "Uhms," and Silences. Powered by Open Source (OSS).
We built an app out of necessity. One of our clients needed it to repurpose their podcast video for social media. The clip audio needed to be trimmed and cleaned. Hence, an internal app…
Now this is the cost for Power Parity.
What does this say?
I pay ~4 times more money than someone in USA to run a GPU AI inference workflow at my home. I knew it was high, but dint expect that to be this different.
You can infer for the rest of the countries directly from the…
@robertnishihara
@OpenAI
@anyscalecompute
Thanks for building this and making it compatible with OpenAI ! I was looking at many platforms but it was hard to switch from current OpenAI workflow. That ease of switch was the killer feature for me.
And also looking forward for other awesome models like mistral, zephyr.
Big Buck Bunny - Directors Cut - AI Version | From Yours Truly.
This is soooooooooooo awesome!!!! Probably the most mind blown I have been in a while.
So, as you might have heard, yesterday OpenAI dropped many features. I started playing with the API and wrote a script. You…
@init_malachi
@OpenAI
@anyscalecompute
GPT-4 with function calling and super long prompts and steering. And then further manual data pruning the data that I got. And that is where the other 1000$ went.
Every single one of photo mine below is AI generated. Bunch of my selfies go in and this comes out.
Of course, this is not new. But still this was my first real experience today with this side of the tech. Kinda surreal.
Interesting times ahead. Hope this is just used for good.
I was apathetic about life for a bit in the late 20s. Antidote to that apathy : work towards a future that I want to be in.
For me it’s not the end result; but the striving that is the antidote. I really don’t mind if I reach the end goal or not now.
I am happy and engaged.
I am going to track efficiency metric for my LLMs in a new way: Tokens/KWh.
If you are running batched AI workflows, that are not real time. And also you are hosting these LLM on your GPU servers. Your cost essentials boils down to electricity spent - kilowatt-hour (KWh).
I…
If you are poor solopreneur like me at early stages. And still trying to experiment. And figure the monetisation part out, money will be an issue. But you know AI 🤖is compute hungry and you need those 💸💸💸 to experiment.
Here are some programs I wish I had pre-applied and…
There are things that frustrate me about Germany, but what I love and think they do absolutely right are the social facilities for citizens.
I am in a public library right now. As a poor entrepreneur, if I feel too stuck at home, I come here. I always feel grateful when I'm…
Goal : Ruthless cost cutdown update.
Money saved : ~200$/month.
Three things :
1. Moved Database. MongoDB from managed Atlas to just simply running it in a simple PC at my home.
2. Got rid of plausible analytics. No analytics for now.
3. Got rid a social image generation SaaS…
2024 Our Goal : Sustainable Revenue
✂️Cost : Ruthlessly cut down expense.
💹 Revenue : Work with
@dineshsai
to add insane amounts of value to customers. In exchange for high profit.
Chances of failure very high. But we will singularly focus and try to get to positive MRR…
Probably my favourite memory in recent times.
Got this in my email today. I dont know how this person stumbled on our site. But they took their time to leave this message.
The greatest dopamine for a builder is that other people use what he has built. And find joy and utility…
I just realized, I messed up and there was a mistake in the previous table. While copying. I am sorry. I am unable to correct it anymore.
This is the correct value. Germany 6 times more expensive.
Finished Dev Day.
I was prepared but still that was very dense. Both rumours I posted earlier was true. But that not is important. There is something far more important.
This is an App Store in iPhone moment. Nobody predicted about Uber, Instagram and all other proliferation of…
Question for ML/AI experts: I'm stuck after doing a lot of research on this topic. Can anyone please help?
If you have expertise in this area and are open to a consultation, we're ready to compensate for your time.
Context : We need to choose a base model for a large-scale…
The FlanT5 set of models is often cited and used in most of the AI/ML research papers that I am reading, especially for a variety of specific and very useful tasks.
However, why don't we hear about these encoder-decoder architectures as much in the open-source software…
Documentation Paradigm for Large Language Models (LLMs) : Log Today, Train Tomorrow
I've been using tools like GPT-4, GPT-3.5 a lot for automating things. For my daily stuff.
Here's something important I learned: always "document" what you do with these models. My "document" I…
I have been using for months. But man did know about the "New AI project" feature.
Wow. I give a text description and it creates an entire project with all the code and the dependencies 😯😯. What! How did I not know this before? So cool!
Been doing…
Okay for posterity. I found the solution for my problem.
I setup a remote pycharm dev server in my linux pc.
I installed pycharm gateway in macOS.
Connected both via ssh.
It works now. I have the power of Nvidia GPU from my Mac!
And I can use all my favorite features from…
It is hard to describe how much you learn by actually doing — by carefully considering all factors, making a decision, and then taking responsibility for the outcome. Unlocks wisdom that cannot be arrived at any other way.
@davidtsong
@OpenAI
@anyscalecompute
There was no quantitive metric analysis done (mainly because I dont know how to).
It was just manual qualitative kind analysis. I just read 150 of the generated summaries. And checked is this at par or better than I would get with GPT-4? And is it in tone I want?
Details :…
Typical dev day. Building our new front end site.
Struggled 5 hours on a bug/mistake, a really stupid one in retrospect.
Kept running in circles and was just stuck. I gave up. Decided it's not going to get fixed today. Felt depressed (if you are/were a dev, you will know the…
Ah...🤦♂️ Same story again. Another one of my trained model has a "bug".
It will behave totally weird and unexpected in some cases. I will blame the open-source llm and its smaller parameters at first.
And then if I go back and look carefully at the training data, there it will…
Okay another mind blown moment. The following is AI generated in one shot. My first image from the first prompt with this model. No fiddling, no playing around 😲!
Text was an unsolved (or not well solved) problem so far in image-gen models. Now that's down too.
So much…
Running out of cash. Slowly migrating existing AI workflows from API to local MAC.
Running on llama.cpp. Open source llama.cpp is a gift. So many people contributing and doing things without much fanfare.
Will share learnings and data soon.
Okay I read the entire doc for new
@MistralAI
platform release. Here is the TLDR; of what you need to know.
Performance roughly the same.
Mistral `small` == GPT 3.5 `turbo`
At less the price.
And its opens source too.
Its happening🫨🤯⚡️.
Got access to GPTsssssss.
Dropping everything and running to home now to get to my laptop and start coding!!!!!
Updates and releases coming.
History does not repeat but it rhymes.
GPTs are like now when App Store opened up in iPhone.
When that happened, people created simple apps
- a calculator app
- a app to use and switch on camera light as flashlight
I remember those clearly.
(and actually those creators mad a…
2024 Our Goal : Sustainable Revenue
✂️Cost : Ruthlessly cut down expense.
💹 Revenue : Work with
@dineshsai
to add insane amounts of value to customers. In exchange for high profit.
Chances of failure very high. But we will singularly focus and try to get to positive MRR…
@SelfInfinity
@unslothai
Hope it works 🤞.
Highly recommend
@unslothai
. That has been my go-to for finetuning.
Now for example, I can do a 70b finetuning because of it in a single GPU. Otherwise, would be really hard or I need a multi GPU setup with axotol.
@gdb
What is OpenAIs secret sauce? You folks ship at such rapid pace. And every time they are jaw dropping features.
For such a small team as OpenAIs, what is the secret ingredient? Focus, Product-Engineering alignment?
Anycase kudos on all the release this week.
📍Neukölln: A vibrant immigrant man from Neukölln, dressed in a fashionable blend of traditional and modern clothing, with a bright smile on his face
#Neuk
ölln
@RamonDarioIT
@ClementDelangue
@OpenAI
@anyscalecompute
Super very much. For this simple model, I can confirm that. I cannot give a metric; but its more a qualitative thing that I just did manually and checked.
Details :
Maybe, I will later create a table and show the outputs side by side.
@davidtsong
@OpenAI
@anyscalecompute
There was no quantitive metric analysis done (mainly because I dont know how to).
It was just manual qualitative kind analysis. I just read 150 of the generated summaries. And checked is this at par or better than I would get with GPT-4? And is it in tone I want?
Details :…
Okay, too much FOMO with latest stable diffusion model. I had to try it myself. Here is one of the runs through the model. With one of our plush toy.
My first image-to-video model experience. The model is bit rough on the edge.
But still awesome! Was plug and play.
Here is real-war-experiment that I will be running over the next few days!
6-well-trained-smaller-llamas (vs) 1 GPT-3/4.
I want to test two hypothesis :
1. Performance beatdown at lower cost. Can the 6 agile smaller llama models beat the one giant model at lower cost for…
TIL : New AI feature in
@MongoDB
Compass!
You can type query in Natural Language and it will auto convert and run the query.
Pretty neat. I am new to DB world. Super useful.
@karpathy
Here is a detailed written summary of the entire talk, for anyone interested.
1. Four stages of training AI assistants.
The process of training AI assistants consists of four stages: pre-training, supervised fine tuning, reward modeling, and reinforcement learning. During…
My prediction is that in 2024, more people will begin dictating to computers.
It used to be broken and frustrating, but speech-to-text technology has matured to the point where we should expect to see it on ALL consumer edge devices and adopting from consumers soon.
As an…
Excited to be entering my Homeland. WIN-India launches today.
We will be exploring how to distill the wisdom from trusted video/podcast creators in India.
It's going to be an interesting experience. India has officially 22 languages. LLMs/AI also have to be regionalised and…
You can consume endless self-help content online. But if you don’t follow the right creators, you will not unlock actionable insights. This is the most valuable self-improvement asset in 2023:
5 𝗦𝗲𝗹𝗳-𝗵𝗲𝗹𝗽 𝗖𝗼𝗻𝘁𝗲𝗻𝘁 𝗖𝗿𝗲𝗮𝘁𝗼𝗿𝘀 𝘁𝗼 𝗙𝗼𝗹𝗹𝗼𝘄 𝗧𝗼𝗱𝗮𝘆
I thought I would just build something useful and figure out the money part when people themselves find it so useful that they offer money.
The day has come!
Time to implement stripe in backend.
Most of the times, you can get what you want from ChatGPT (or other LLMs) with just good clear prompts.
Now, did you know that you can use ChatGPT itself to create the good clear prompt for you?
Here is the prompt to generate a good prompt.
(Source : I had this bookmarked…
@anyscalecompute
@LangChainAI
@vercel
@pinecone
Thanks.
If possible, could you please also record the sessions and share them?
Every time see I such cool events happening in SF, I get FOMO.
Will be nice for us non-US and non-SF folks :)
Super fresh off the press!!
They actually did it!
@MistralAI
releases their own platform beating
@OpenAI
GPT-3.5.
They wrote their manifesto one year ago. And it is playing exactly as they wrote.
Mid year -> Release open source model.
End of year -> Build a platform and beat…
1. Spin up H100 in cloud.
2. Start Pycharm remote client + hook up with ssh into H100.
3. Boom 💥 you have an insanely powerful developer machine with IDE and GPU.
Tools and cool things I tried this week:
1. MacOS Automator
2. Pipedream
3. Arc Browser
I try new tools each week, mainly to automate repeated things and solve personal paint points. I thought I would share learnings, every week, in case you want to replicate the workflow. No…
Was just getting ready to focus back on work. Got two emails one after another.
- Got my hands on those A100s 🚀 from
@huggingface
- And also got into Anyscale tuning endpoints
@anyscalecompute
Going to start tuning.
Comparison and performance results coming soon.
Landscape to Portrait Video Resizer: Try It
Today, we spent the entire day searching for a decent tool that can convert landscape-sized (16:9) videos to mobile-sized portrait videos (9:16) to repurpose and generate social clips.
Frustrated, I quickly built a simple tool that…