Justus Mattern
@MatternJustus
Followers
902
Following
420
Media
49
Statuses
267
co-founder @redotvideo (YC S23)
Berlin / San Francisco
Joined March 2021
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
اليوم الوطني
• 1295060 Tweets
Lebanon
• 903443 Tweets
لبنان
• 652642 Tweets
REBECCA ICONIC GIRL PFW25
• 421958 Tweets
الاهلي
• 400120 Tweets
ايران
• 209228 Tweets
Zelensky
• 135067 Tweets
Bills
• 106011 Tweets
Kassia
• 36547 Tweets
Dolph
• 31192 Tweets
#grandefratello
• 31077 Tweets
連休明け
• 30913 Tweets
Jessica
• 30345 Tweets
Vince
• 27586 Tweets
كاس الملك
• 23831 Tweets
John Deere
• 21674 Tweets
Discovery
• 21366 Tweets
Sooj
• 18817 Tweets
Mallorca
• 14779 Tweets
Jaguars
• 12358 Tweets
Capriles
• 12182 Tweets
アップル
• 11701 Tweets
Pac-12
• 11635 Tweets
#طرد_ماتياس_الفاشل_مطلب
• 10106 Tweets
دوري يلو
• 10075 Tweets
Pinned Tweet
Took some time to make my first real Youtube video: An explainer / launch video about
@redotvideo
, created with
@redotvideo
:
8
6
40
1/n Introducing Mamba-Chat: the best non-transformer based chat model!
Mamba-Chat is mainly based on
@tri_dao
's and
@_albertgu
's awesome work on state-space models and Mamba. We've just added some fine-tuning on top.
Code:
4
32
246
Took a look at Google Scholar and realized that I’m a serious researcher now
6
1
139
2023 recap:
1. started a company with
@hkonsti_
2. got into
@ycombinator
, dropped out of university
3. spent 3 months in SF, met incredibly smart and ambitious people
4. moved to Berlin, turned 21, now live on my own and work full-time
This was clearly the highest-impact year
7
3
113
🚨 Exciting News:
@haven_run
is part of
@ycombinator
!
As of now, developers that want to train & deploy LLMs on their own infrastructure have to deal with ML code, CUDA, and manual resource scaling.
We want to make it as easy as calling the OpenAI API
5
14
100
Sad that
#EMNLP2022
is over, but super grateful to have met and exchanged ideas with so many brilliant people - see you at the next conference!
3
0
66
Introducing Haven's finetuning platform: You can now train LLMs with LoRA adapters, test them with <1s cold starts, and export them to Huggingface to run on your own terms.
Below you can find a demo video - when signing up to , you get $5 in credits :)
4
9
51
After four months in Berlin, we are back in the arena!
Feel free to DM if you want to grab a coffee. My goal is to speak to as many fellow devs and founders as possible :)
3
0
36
The pain of having to tell people that despite working on NLP, I do in fact not work on ChatGPT or LLM-powered chatbots is almost unbearable
1
1
28
We're live on Launch YC (
@ycombinator
) with Haven's managed offering!
Haven works like Replicate, but in your private GCP / AWS environment. You can select any LLM - Haven will deploy and scale it on a Kubernetes cluster running on your infrastructure.
2
7
26
Dealing with tax & accounting for a startup in Germany is hell. I just want to build stuff :(
6
1
25
I'm on my way to EMNLP! 🇦🇪
You can find me at our poster on "Differentially Private Language Models for Secure Data Sharing" - I'm also available for internship opportunities in 2023.
Also, feel free to DM if you'd like to explore Abu Dhabi or go to the beach on Tuesday!
1
2
23
Our work "Differentially Private Language Models for Secure Data Sharing", which proposes a simple, but highly effective method to generate textual datasets with DP guarantees, has been accepted at EMNLP 2022 🥳
w/
@ZhijingJin
, Benjamin Weggenmann,
@mrinmayasachan
,
@bschoelkopf
0
4
22
Being my grandmother's personal tech support via Whatsapp, I have access to the largest German instruction-tuning dataset known to mankind
0
0
21
Going to bed now, hoping that the Americans will figure out how to run Mistral
0
0
18
2/n When I saw the release of Mamba, I got super excited, so I basically took a day off to fine-tune it.
The original implementation was very easy to work with. I just had to change a few lines of code of
@huggingface
's Trainer class, and was immediately able to start training.
1
1
15
Just moved to SF - exciting times ahead! Can’t wait to share what I’m working on here.
PS: If you’re in the Bay Area and want to have a coffee, feel free to DM me :)
1
0
15
4/n In terms of results, I'm actually positively surprised about the model! Even though it just has 2.8B parameters and the base model was only trained on the Pile, it performs quite well. Chatting with it reminds me of chatting with Alpaca when it was released in March.
1
0
14
n/n Here are some relevant links:
Github:
Huggingface:
0
0
13
Girlfriend will have to watch the GPT-4 Developer Livestream instead of The Last of Us tonight
0
0
13
now lets you run inference on all of our base models, not just your fine-tunes! Also, we support Mixtral now :)
Feel free to give it a try:
1
0
12
🦙 Many of our users have asked us for help with fine-tuning Llama V2 on their own chat datasets. That's why we've built llamatune, which allows you to do so without writing any code!
Check it out on GitHub:
0
1
12
1/n With membership inference attacks (MIA), one can detect the training data of an LLM and compromise privacy! 🔍
While SOTA attacks need prior knowledge about the training distribution, our new paper shows that this is not necessary - we thus need to rethink our threat model!
3
0
12
Introducing Haven v0.2 🔮
Haven now lets you deploy almost any LLM in your own VPC! We've added support for custom models from
@huggingface
, and users are already deploying their own fine-tuned models!
Setting up a production ready LLM server takes just a few lines of code:
1
1
12
I don't get the excitement around GPTs, it's literally just adding a system prompt and uploading files
3
0
11
3/n We trained on 16k samples of the 200k filtered Ultrachat dataset, on a single A100 (40GB) GPU.
Please note that the implementation is definitely not optimized, this was just a fun hack. I'm sure you can get way faster and more efficient training with some optimizations.
2
0
10
@natfriedman
At Haven (YC S23), we enable startups to easily host LLMs (and Llama V2) on their own infrastructure. We‘re also open source!
0
0
9
6/n This work was done with amazing collaborators / mentors (
@limufar
@ZhijingJin
@bschoelkopf
@mrinmayasachan
@BergKirkpatrick
) and has been accepted to Findings of
#ACL2023
🎉
You can check it our on Arxiv:
0
0
8
Just added a Google Colab demo for Mamba-Chat, feel free to give it a try!
1/n Introducing Mamba-Chat: the best non-transformer based chat model!
Mamba-Chat is mainly based on
@tri_dao
's and
@_albertgu
's awesome work on state-space models and Mamba. We've just added some fine-tuning on top.
Code:
4
32
246
0
1
8
Still can’t wrap my head around the fact that there are actually ChatGPT detectors making a bunch of money by selling to schools. I’m really glad that I’m not a student anymore
0
1
6
I‘m 96.3% sure that you pulled that 8.5 number out of your ass
Mira Murati reached out to me in 2022 for a one-hour zoom call. Sam Altman never essayed any such contact. Also, I don't think Murati has made any jokes about how funny it would be if the world ended. I'm tentatively 8.5% more cheerful about OpenAI going forward.
216
140
2K
0
0
7
Interesting to see LLaMA 2 call itself Bard (and that it refuses to talk like a frat bro)
0
0
6
how are we going to tell them
Will releasing the weights of large language models grant widespread access to pandemic agents?
Turns out, yes, probably.
1/5
60
104
451
1
0
6
@tallinzen
On top of that, most exam dates are during the holidays, not within the actual semester. This makes it virtually impossible to participate in structured summer internship programs outside of the country
1
0
6
@giffmana
I might not get it, but where is the sarcasm here? It sounds similar to how ChatGPT would answer non-sarcastically.
1
0
4
@cloud11665
If you want to host fine-tuned models cheaply, we’ll have something for you in a few weeks :) we hotswap lora adapters, so pricing is usage-based, and cold start times are just as long as it takes to load an adapter onto the GPU (i.e. ~1-2s)
1
0
4
@theaiengineerco
I optimized for grad school applications, that’s why I was able to churn out a lot of papers that were good, but not great.
I don���t think it’s possible to publish that much when you actually work on high-impact, high-risk stuff.
0
0
3
@khoomeik
The expertise we’ve built over the last three years has been made obsolete by pip install openai
0
0
3
We've just released v0.1 - to get started, you can head to our repository and deploy Haven in a couple of minutes.
Alternatively, write me a DM if you have questions or want me to personally onboard you :)
1
0
4
While you spend time eating turkey, I (European founder) spend my time shipping new features. We are not the same
0
0
4
If reputational concerns really stop companies from building products with huge potential such as ChatGPT, then I feel very optimistic about the future impact that research in "trustworthy" ML/NLP (privacy, bias, adversarial robustness, etc.) can have
"The only real failure in life is the failure to try." - Unknown
174
225
3K
0
0
4
4/n Obviously, the assumption that an attacker has access to i.i.d data in order to train a reference model is not always realistic. Therefore, we decided to investigate whether attacks can be accurate without access to such data.
1
0
3
People say that open source LLMs can't compete with OpenAI and haven't even tried the best open models.
@huggingface
's zephyr-7b-beta is incredible, and also works great for finetuning. Try that, and then imagine what's possible when
@MistralAI
releases even better base models
0
0
3
2/n Membership inference attacks exploit that models tend to exhibit higher confidence on training samples.
Therefore, a common baseline attack simply classifies a data sample as training data if the loss under the model's distribution is below a certain threshold.
1
0
3
3/n This attack is not highly accurate in practice - therefore, existing SOTA attacks regularize this loss by comparing it to the loss of a reference model trained on data from the same distribution as the training set.
1
0
3
Happy to share that the projects I contributed to while interning at
@SAP
Security Research have been published!
1) DP-VAE: Human-Readable Text Anonymization for Online Reviews with Differentially Private Variational Autoencoders (
@TheWebConf
2022, )
/1
1
0
3
5/n We found that this the case! Concretely, we designed an alternative regularization function that compares the target model's confidence for a given sample to its confidence for perturbed samples that were generated through word replacements.
1
0
3
Do you consider it bad practice when investors book calls through the „Book a Call“ button on your landing page?
Our button doesn’t explicitly state that it’s for customer demos only, but I thought that this would be clear
1
0
2
We furthermore show that simply training GPT-2 to generate paraphrases and adjusting the softmax temperature to balance privacy and utility grants better protection against deanonymization attacks, reaps more fluent text and is in fact a differentially private mechanism
/end
0
0
2
Multi-Node Training 🤝 Haven LLM Fine-Tuning Platform
If your company wants to train LLMs with more than 8 A100s, feel to reach out! Also, do take a look at this wonderful graphic I made for Linkedin
0
0
2
2) The Limits of Word Level Differential Privacy (Findings of
@naaclmeeting
2022, )
Here, we examine word embedding perturbations for private text sharing and find strong limitations w.r.t. the mathematical privacy guarantee and language quality.
/3
1
0
2
1
0
1
If you want to get started, check out our repository:
Would love to hear your thoughts!
0
0
2
0
0
1
@HJCH0
@cloud11665
Having all adapters on the GPU limits us w.r.t how many adapters we can serve for a single base model (above a few hundred adapters you get CUDA OOM errors like this).
This way, we couldn't afford to charge usage-based since most users don't query their models very often.
0
0
1
0
0
1
❤️
0
0
1
@niloofar_mire
@hkonsti_
@ycombinator
Thanks Niloofar, it was great working with you! Research has taught me a lot :)
0
0
1