Edouard Grave
@EXGRV
Followers
3K
Following
236
Media
8
Statuses
128
large language models @kyutai_labs
paris, france
Joined October 2012
Today, we release our 🇫🇷 to 🇬🇧 simultaneous speech-to-speech translation system, called Hibiki. It runs on-device & the model, inference code and tech report are available. This is built using the same audio LLM as Moshi, showing its versatility. 🟢
Meet Hibiki, our simultaneous speech-to-speech translation model, currently supporting 🇫🇷➡️🇬🇧. Hibiki produces spoken and text translations of the input speech in real-time, while preserving the speaker’s voice and optimally adapting its pace based on the semantic content of the
0
1
20
Excited to release a preview of Helium-1, our 2B LLM targeting edge and mobile devices. 🚀 More to come in the future: training code, support for more languages, data pipeline, tech report & more… 🟢
Meet Helium-1 preview, our 2B multi-lingual LLM, targeting edge and mobile devices, released under a CC-BY license. Start building with it today! https://t.co/X4Dbx2T1cJ
0
4
44
Meet Helium-1 preview, our 2B multi-lingual LLM, targeting edge and mobile devices, released under a CC-BY license. Start building with it today! https://t.co/X4Dbx2T1cJ
huggingface.co
10
94
381
Local voice models FTW 🚀
Talking to Moshi locally on a Macbook M series (python 3.12) in 2 lines: pip install moshi_mlx python -m moshi_mlx.local_web -q 4
0
3
17
I am at ICML in Vienna! Let me know if you want to chat about (or to) Moshi, multimodal LLMs, Kyutai & more.
2
5
31
Looking forward to discuss open research at @kyutai_labs. If you want to work on large scale multimodal LLMs, come and talk to us, this is what we look like 👇☕️
Look for my @kyutai_labs colleagues at #NeurIPS2023 if you want to learn more about our mission. We are recruiting permanent staff, post-docs and interns!
4
9
102
✈️ I will be attending #NeurIPS2023: let me know if you want to chat about the future of LLMs, and how to democratize them. 🌐 We are also hiring members of technical staff and interns @kyutai_labs. Happy to talk about the lab and our mission.
1
6
58
/kyutai has landed! Super excited to build this new research lab. Pure focus on research. As open as it gets.
Announcing Kyutai: a non-profit AI lab dedicated to open science. Thanks to Xavier Niel (@GroupeIliad), Rodolphe Saadé (@cmacgm) and Eric Schmidt (@SchmidtFutures ), we are starting with almost 300M€ of philanthropic support. Meet the team ⬇️
13
6
152
Super excited by the release of LLaMA, a serie of large language models, from 7B to 65B parameters. 🎉 By training longer, LLaMA obtains GPT3 level performance with a 13B model, which can run on a single GPU. Excited to see what the research community will do with these models.
Today we release LLaMA, 4 foundation models ranging from 7B to 65B parameters. LLaMA-13B outperforms OPT and GPT-3 175B on most benchmarks. LLaMA-65B is competitive with Chinchilla 70B and PaLM 540B. The weights for all models are open and available at https://t.co/q51f2oPZlE 1/n
0
7
52
Introducing PEER, a new language model which makes text generation and editing more collaborative and controllable. It adds human in the loop, by following instructions and providing explanations. Work lead @timo_schick. Paper:
🎉 New paper 🎉 We introduce PEER, a language model trained to incrementally write texts & collaborate w/ humans in a more natural way. It can write drafts, add suggestions, follow instructions, perform edits, correct itself & provide explanations. Link: https://t.co/PCqgWJUo5k
1
1
11
Joint work with the great following team: @gizacard @PSH_Lewis @MariaLomeli_ @lucas_hosseini @Fabio_Petroni @timo_schick Jane Dwivedi-Yu @armandjoulin @riedelcastro
0
0
7
Our model, at 11B parameters, and significantly less training compute, outperforms LLMs on 64-shot question answering (+3 pts wrt SOTA) or 15-shot fact checking (+5 pts wrt SOTA).
1
0
5
Previous works showed that retrieval is helpful for knowledge intensive tasks, but mostly in settings with large training sets. Here, we show how to get the same benefits for few-shot learning.
1
0
4
Very excited to introduce Atlas, a new retrieval augmented language model which is competitive with larger models on few-shot tasks such as question answering or fact checking. Work lead by @gizacard and @PSH_Lewis. Paper:
🚨We’ve been working on better retrieval-augmented models & thrilled to present Atlas, led by @gizacard @EXGRV & myself🚨 Atlas is a end2end pretrained "RAG"-like model, beats models 50x its size on fewshot QA, sets numerous SotA on knowledge-intensive NLP https://t.co/6yBHlHJBdZ
2
11
73
Excited to present our work on single-sequence protein folding from a language model! By stacking a simple folding trunk and Alphafold2's structure module on top of the language model, we get accurate structure prediction in a fraction of the runtime.
We have trained ESMFold to predict full atomic protein structure directly from language model representations of a single sequence. Accuracy is competitive with AlphaFold on most proteins with order of magnitude faster inference. By @MetaAI Protein Team. https://t.co/APVoaawyOb
4
12
80
New release of our Contriever project! It includes multi-lingual models which can perform cross-lingual retrieval (eg, retrieve English documents to answer a question in Swahili), the code to (pre-)train your own retrievers, and an updated version of the paper with new results.
Code for Contriever is now available! Code: https://t.co/mCyDI99RAG Paper: https://t.co/CvruQO66FJ Additionally we trained mContriever, a state-of-the-art multilingual neural retriever, by applying a similar contrastive learning method.
0
2
12
1/2 Today, we’re announcing a long-term research initiative to better understand how the human brain learns and processes language. This project is in collaboration with @NeuroSpin_91 and @Inria. https://t.co/kh0w7DXgJh
22
52
276