Albert Jiang
@AlbertQJiang
Followers
3K
Following
2K
Media
149
Statuses
990
Mistral 7B paper is up on arxiv. The authorship order is alphabetical. Please cite with .author = {Mistral AI} 🙂
19
172
1K
Large language models can write informal proofs, translate them into formal ones, and achieve SoTA performance in proving competition-level maths problems!. LM-generated informal proofs are sometimes more useful than the human ground truth 🤯. Preprint: 🧵
8
146
641
If there's one thing you should NOT learn from the UK AI summit it's the curve fitting skill.
16
36
474
Officially submitted my thesis very recently. Extremely grateful to @Mateja_Jamnik @WendaLi8 for the three years of excellent and patient supervision. Just like how Cambridge terms work, each week in the PhD feels very long, but the entire thing feels like an incredibly short
28
35
348
Has been to one of the workshops on LLMs in Warsaw. Frankly it puts almost all the UK ones in shame. Open and deep exchange of bold ideas makes #IDEAS one of the best institutes to collaborate with. Do not destroy a powerhouse of European innovation.
Polska nauka tonie.
2
47
253
Many high-quality AI4Maths papers were submitted to ICLR and NeurIPS workshop this year!. My first research project was in summer 2018 with @Yuhu_ai_ @jimmybajimmyba and we saw reviews like "this is of limited novelty to a niche research field". So damn encouraging.
5
13
56
3 months, from 0 to SoTA. Incredibly proud.
magnet:?xt=urn:btih:208b101a0f51514ecf285885a8b0f6fb1a1e4d7d&dn=mistral-7B-v0.1&tr=udp%3A%2F%3A1337%2Fannounce&tr=https%3A%2F%3A443%2Fannounce. RELEASE ab979f50d7d406ab8d0b07d09806c72c.
7
14
150
Baldur: Whole-Proof Generation and Repair with Large Language Models. This is such amazing work. Congrats to Emily, Markus @MarkusNRabe, Talia @TaliaRinger, and Yuriy @YuriyBrun!.
6
26
141
Wrote a summary of my thoughts on the plane back from NeurIPS: That's the serious stuff. Will do a thread of silly things later.
5
18
138
Introducing Multilingual Mathematical Autoformalization (MMA): A dataset of 332K formal-informal parallel datapoints from Isabelle AFP + Lean4 Mathlib4, informalised by GPT-4. Dataset size and multilinguality help autoformalization a lot! See🧵:.
4
30
135
Join us to build with the best colleagues!.Offices in France, UK, and US west coast.
We are announcing €600M in Series B funding for our first anniversary. We are grateful to our new and existing investors for their continued confidence and support for our global expansion. This will accelerate our roadmap as we continue to bring frontier AI into everyone’s.
4
4
122
The Neurips Machine Learning for Theorem Proving tutorial is happening on Monday at 1:45 PM! Come hear Kaiyu, Emily, and I give intro talks if you're a machine learning practitioner interested in maths. We also have a fantastic panel moderated by Zhangir!
4
22
120
torrent, .pt, .html, now .pdf:. Kudos to everyone who put in the hard work.
6
8
113
Going to NeurIPS?.Interested in AI4Maths?. Come to the Machine Learning for Theorem Proving tutorial on 11 Dec!. Emily, @KaiyuYang4, and I will be presenting how machine learning can prove theorems (in Coq, Isabelle, and Lean!). Panel is stunning.
1
20
110
I love open-sourced models! Please add your favourites to the Mistral Convex Hull.
3
12
98
Was in a very bad spiritual and physical state for a bit where I could feel neither conviction nor relief. What got me out of it was. Hanging out with parents near a lake.Reading The Magic Mountain.Drinking this exceedingly good espresso. (in that order)
3
4
96
Incredibly proud of the amazing team!.Look no further, join us to create the best reasoning machines.
Six months ago, we launched Numina to lead open research in AI4Math. Today we are super excited to share that our Numina Math 7B model won the 1st progress prize of the AI Math Olympiad 🔥🔥🔥
1
8
94
Before NeurIPS, I write down some thoughts about AI4Math and why I am doing LLMs and informal reasoning now.
1
20
95
The moment finally came: . "But reviewer, I AM (Jiang et al. 2022).".
3
0
89
Super late to the party, but DSP was accepted to ICLR for an oral presentation (notable top 5%)! Let's chat when in Kigali!. We updated the paper according to reviews and released the code for reproduction:. Paper: Code:
Large language models can write informal proofs, translate them into formal ones, and achieve SoTA performance in proving competition-level maths problems!. LM-generated informal proofs are sometimes more useful than the human ground truth 🤯. Preprint: 🧵
6
16
89
Nice paper! Some surprising highlights:.1. Mixtral 8x22B is ~GPT4-turbo level on GSM8K and GSM1K. Mistral large is better on both. 2. On GSM1K, Mixtral-8x22B-Instruct (84.3%) > claude-2 (83.6%) >> claude-3-haiku (79.1%) >> claude-3-sonnet (72.4%) 🤔.Also worth highlighting how
Data contamination is a huge problem for LLM evals right now. At Scale, we created a new test set for GSM8k *from scratch* to measure overfitting and found evidence that some models (most notably Mistral and Phi) do substantially worse on this new test set compared to GSM8k.
2
21
80
Thor was accepted to Neurips. It's my first paper during my PhD. It's towards a direction I really want to push (conjecturing). I feel good.
Language models are bad at retrieving useful premises from large databases for theorem proving, mainly because they're limited by a small context window. We use symbolic tools to overcome this difficulty, boosting proof rates from 39% to 57%. Thor: 1/
4
6
79
[MATH-AI influencer mode on]. Decomposing the Enigma: Subgoal-based Demonstration Learning for Formal Theorem Proving. New miniF2F SoTA test: 45.5%. The paper seems to follow Draft, Sketch, and Prove but they found a better way to do formal sketches!
3
16
67
3/3 papers accepted at NeurIPS. Albert’s last batch of papers in PhD. A real Fibonacci soup of submissions because they have been rejected.0 time.1 time.2 times. Congratulations and gratitude to @andylolu24 @ZiarkoAlicja Bartosz Piotrowski @WendaLi8 @PiotrRMilos @Mateja_Jamnik!
6
2
70
Mistral Large 2 on It's amazing for difficult tasks, and free on Le Chat: The perfect tool for ACTUAL productivity: I've used it in many ways for my PhD. We've gathered unstoppable momentum in pre and post training. Join us.
4
11
63
If you're a mathematician interested in automatic formalization, or a machine learning practitioner interested in formal math, come to this workshop in April! I'm very honoured to organise it with Jarod, Dan, Kim and @wellecks!. Apply:
2
12
68
Exciting news: the article "Evaluating language models for mathematics through interactions" by @katie_m_collins and me is published in the Proceedings of the National Academy of Sciences!. Check out this original thread by Katie:
Evaluating large language models is hard, particularly for mathematics. To better understand LLMs, it makes sense to harness *interactivity* - beyond static benchmarks. Excited to share a new working paper 1/
1
5
60
Going to NeurIPS next Tuesday. We are hiring for lots of research positions: The new central London office (as well as Paris and Palo Alto) needs to be filled with passionate people!
2
6
61
It's funny to see people on twitter who have obviously never touched a job with more than 2^8 devices (be it gpu/tpus for training or cpus for data processing) comment on scaling laws slowing down.
4
1
59
Language models are bad at retrieving useful premises from large databases for theorem proving, mainly because they're limited by a small context window. We use symbolic tools to overcome this difficulty, boosting proof rates from 39% to 57%. Thor: 1/
3
13
55
What's better than science in the open? Science in motion. This report is a good read and really awakens the later-Wittgensteinian in me.
1
5
53
Want to tackle some of the challenges here? Apply for an AI for math grant: Web form proposal deadline 10 Jan!.
🚀 Excited to share our position paper: "Formal Mathematical Reasoning: A New Frontier in AI"!.🔗 LLMs like o1 & o3 have tackled hard math problems by scaling test-time compute. What's next for AI4Math?. We advocate for formal mathematical reasoning,.
1
11
47
If you are a theorem proving person at NeurIPS, come get together for a coffee/beer at PJ’s coffee at 5:30 on Wednesday :).
4
2
46
At #ICLR2023 in Kigali! Come to our oral session on Tuesday afternoon for the paper Draft, Sketch, and Prove: Guiding Formal Theorem Provers with Informal Proofs. DM for grabbing coffee/meal to chat about AI for maths, reasoning, large and small LMs!.
0
7
33
DeepSeek takes (assuming they are genuine ones not ironic), ranked:. - 💩: side project.- Bad: ccp project, 5.5m is all you need, bottom-up innovation.- Tolerable: cracked quant researchers.- Good: great engineering, architectural innovation, Sputnik (surprised haven’t seen this).
3
0
41
Life hack: several days of forced rest and exercise due to a jaw disorder can turn a monotonous zombie on caffeine and diet coke into an energetic being who feels everything intimately and is ready to leap for the most exciting work of my life.
2
0
40
Keep these posts to LinkedIn plz.
Nobel Prize is NOT about h-index or citations. It is about the emergence of big new fields. So many posts discuss Nobel awardees. And so many misunderstand the Nobel Prize. 📍 A bit of clarification from my side:. 1⃣ Nobel Prize is NOT about how useful your work is. It’s about
1
0
38
Announcing Codestral: our first-ever code model. - Open-weights under the new Mistral AI Non-Production License.- New endpoint via La Plateforme: - Try it now on Le Chat:
2
7
38
Super happy to release Mistral NeMo. 128K context length with top-notch performance, multilingual, FP8 awareness, and a more compressive tokenizer. Go fish!.
3
2
38
I dressed up as Jeff Dean for Halloween and got the same treatment of being kept outside Deepmind :(
3
0
37
Nice to see mathstral 7B out in the wild ^ ^
🔥o1-preview has shown incredible improvements in reasoning ability across complex disciplines on our OlympicArena (val + text-only) subset! .We’re also eagerly looking forward to the performance of the multimodal version of o1 in the future!
0
3
37
What Christian had | What I heard.in mind when he |.suggested the name |
Magnushammer - the mythical weapon of the supreme transformer - beats sledgehammer by a large margin for Isabelle proof automation, improves Thor!. Many thanks to @s_tworkowski @Yuhu_ai_, @PiotrRMilos et al for the great work:.
2
9
36
Why have there been so many AIxFormalMaths researchers who started their own companies🤔apart from the fact that they are extremely talented?. To list a few: Guillaume, Tim, Tony, Christian, Markus, Igor, Stan, Jesse. Does the LLM and logic combination attract startup founders?.
2
3
33
Giving a talk on evaluating large language models for mathematics through interactions (work co-lead with @katie_m_collins) on Thursday. In the same session is the one and only @ChrSzegedy!.
0
3
34
Two edge models out with impressive capabilities. High time to have a silicon intelligence on your laptop or your phone :)
1
4
34
Jeremy Avigad just dropped an essay "MATHEMATICS AND THE FORMAL TURN" This is a well-researched and well-balanced essay that people new to or already familiar with formal maths will find helpful, to stay informed given the great strides it's been making.
3
8
31
Big thanks to the amazing team @wellecks @JinPZhou @jiachengliu1123 @WendaLi8 @tlacroix6 @Mateja_Jamnik and @Yuhu_ai_ @GuillaumeLample!. We have a team photo during AITP this year which I saved until this very moment (with @ChrSzegedy sipping beer in the background).
2
3
32
AI for maths workshop at ICML with challenge tracks (incl. autoformalization) in Vienna!. Now what should my talk focus on 🤔.
Excited to announce the AI for Math Workshop at #ICML2024 @icmlconf! Join us for groundbreaking discussions on the intersection of AI and mathematics. 🤖🧮. 📅 Workshop details: 📜 Submit your pioneering work: 🏆 Take on our
1
5
31
Panel starting!
📢 Can't wait to see you at the 3rd #MathAI Workshop in the LLM Era at #NeurIPS2023!. ⏰ 8:55am - 5:00pm, Friday, Dec 15.📍 Room 217-219.🔗 📽️ Exciting Lineup:.⭐️ Six insightful talks by @KristinLauter, @BaraMoa, @noahdgoodman,
1
6
31
Huh? Why?
2
0
31
The new Le Chat on web is a beast. Some nice web search, image generation, writing, and Flash Answers give me a Melville-style story in seconds.
Introducing the all new Le Chat: your ultimate AI sidekick for life and work! Now live on web and mobile!
1
1
29
Every time someone says "This can be interpreted as X", where X usually = Bayesian blah, GNN, etc., they should be forced to add: What does the interpretation bring? Did it help you discover something? Does it add to the physical intuition?. So often it's nothing.
1
3
29
In two days, I translated ~150 theorems from Lean to Isabelle (all of the validation set from @KunhaoZ @jessemhan @spolu's except a huge bunch which I found too difficult and threw to @WendaLi8). I'm tired and happy. Some thoughts: 1/n.
2
4
28
Holding the model until Mistral ICML happy hour.
3
0
30
Super happy to contribute a very small part to Llemma. Let's move open model scaling perfs completely above closed ones!.
We release Llemma: open LMs for math trained on up to 200B tokens of mathematical text. The performance of Llemma 34B approaches Google's Minerva 62B despite having half the parameters. Models/data/code: Paper: More ⬇️
0
3
29
Reading is like fine-tuning for humans. It requires parsing and comprehension to only mimic the style of the author: Proust is 1M tokens, it cannot shift you deeply. It requires grinding, imagining, RAGing, and feedback to achieve anything profound. 学而不思则罔 思而不学则殆.
2
1
28
I had a dream last night in which my manager told me a rule in the charter of mistral “You’re explicitly forbidden from releasing anything that doesn’t kick ass.”.
2
0
29
Pre-trained 7B outperforming LLaMA2 13B on all metrics. Apache 2.0.
At @MistralAI we're releasing our very first model, the best 7B in town (outperforming Llama 13B on all metrics, and good at code), Apache 2.0. We believe in open models and we'll push them to the frontier . Very proud of the team !.
2
8
28
OK 30.3% is not SOTA on miniF2F test/Lean. In May 2022, we have 41% with HTPS. 35% with only supervised training:.
InternLM-Math: Open Math Large Language Models Toward Verifiable Reasoning. Obtains open-sourced SotA performance in various benchmarks including GSM8K, MATH, Hungary math exam, and MiniF2F. repo: abs:
2
1
29
Organising the MATH-AI workshop @NeurIPSConf on 15 Dec in New Orleans this year!. We have a fantastic line-up of speakers and panelists (updated). Please consider sharing your work in AI4Math and Math4AI!. Website: Paper submission ddl: 29 Sept
We're organizing the 3rd #MathAI workshop at @NeurIPSConf #NeurIPS. 🚀 Excited for our speakers on AI for mathematical reasoning, @guyvdb, @noahdgoodman, @wtgowers, @BaraMoa, @KristinLauter, @TaliaRinger, @paul_smolensky, Armando Solar-Lezama, @Yuhu_ai_, @ericxing, @denny_zhou.
0
9
27
Autoformalization work accepted to Neurips. I was surprised when @TaliaRinger listed autoformalization as "one of the things most AI for proofs people are doing" despite there's one or two published work on it with deep learning.
After showing a few examples, large language models can translate natural language mathematical statements into formal specifications. We autoformalize 4K theorems as new data to train our neural theorem prover, achieving SOTA on miniF2F!. 1/. Paper:
2
5
26
《Morning Cure at the International Sanatorium Thousand-island Lake》, 1911, colourised
1
0
25
My internship project #2 went into this 💻 that was a blast!.
Using HyperTree Proof Search we created a new neural theorem solver that was able to solve 5x more International Math Olympiad problems than any previous AI system & best previous state-of-the-art systems on miniF2F & Metamath. More in our new post ⬇️.
1
1
24
Legend has it Tony is driving to New Orleans because no airline allowed him to carry 10 poster rolls on the plane. Congratulations! Great to work together!.
Hello #NeurIPS2022! I'm at New Orleans and will be here until Thursday morning (Dec 1). Let's brainstorm AI for math, LLMs, Reasoning 🤯🤯!. We'll present 8 papers (1 oral and 7 posters) + 2 at workshops (MATHAI and DRL). Featuring recent breakthroughs in AI for math! See👇.
1
0
22
The Evaluating Language Models for Mathematics through Interactions preprint has been updated with annotated behaviour taxonomy, key findings, and more!. 🧵of exciting additions.
Evaluating large language models is hard, particularly for mathematics. To better understand LLMs, it makes sense to harness *interactivity* - beyond static benchmarks. Excited to share a new working paper 1/
1
4
22
Catch the based authors tmr and learn about how to .1. supercharge Sledgehammer with transformers!.2. use LMs to prove theorems!. My formal maths friends: read 1.My machine learning friends: read 2.
0
6
22
Missed opportunity to say "left as an exercise to the reader" here
1
0
21
We introduce Draft, Sketch, and Prove (DSP), a three-stage method to prove mathematical theorems formally. It uses informal proofs drafted by humans/machines, translates them into proof sketches, and then executes off-the-shelf provers to complete sketches into verified proofs.
1
0
21
I’ve found the spice of autoformalization and it’s available for £1.9 at Culinaris on Mill Road.
3
1
21
I'll be at ICML in Vienna from Sunday to Saturday. Let's chat about LLM and math.
MathΣtral is the first milestone of many from Mistral to create reasoning machines. Creating them is a good choice for a defining commitment!.
1
0
21
We previously ( showed that you can effectively automate the formalization process of mathematical statements by leveraging the few-shot learning ability of large language models. In this work we take it one step further: formalize mathematical proofs!.
1
0
21
*Having a meeting with my window open and bumping into the German couple renting an airbnb above my flat*. Them: are you a biologist or something?.Me: ?.Them: we keep hearing you talking about llamas and chinchillas earlier.
1
0
22
👁️👁️. M.
magnet:?xt=urn:btih:7278e625de2b1da598b23954c13933047126238a&dn=pixtral-12b-240910&tr=udp%3A%2F%3A1337%2Fannounce&tr=udp%3A%2F%3A1337%2Fannounce&tr=http%3A%2F%3A80%2Fannounce.
1
0
22
I had an extremely enjoyable first year of PhD. What a privilege to be supervised by @Mateja_Jamnik and mentored by @Yuhu_ai_ and @WendaLi8 to work on a topic I enjoy (machine learning x mathematics). Really cannot hope for a better support team.
2
3
21
Cat's out of the bag now!.
We’re announcing a new optimised model today! Mistral Large has top-tier reasoning capacities, is multi-lingual by design, has native function calling capacities and a 32k model. The pre-trained model has 81.2% accuracy on MMLU. Learn more on Mistral.
2
1
20
Formalizing proofs brings a new challenge: informal and formal proofs are less aligned. There are many more justifications one needs to include in formal proofs. To tackle this we introduce sketches as an intermediate stage. Sketches outline the high-level structures of proofs.
1
2
20
+1.Man do we have some absolute bangers chez Mistral, each with its own lore and etiquette.
You can gauge a company’s culture by how robust and unhinged the slack emoji repertoire is.
2
0
20
Formal theorem proving guarantees correctness, but machine learning methods for it are limited by the scarcity of formal data. The amount of informal mathematical data (e.g., in LaTeX) is huge and would be beneficial if it can be used to augment formal data. (photo attr. in ALT)
2
2
19
I can't believe this is what I want to say to the author(s) of a paper I'm reviewing: Don't be confined to the EA bubble because it's comfortable, for that is lower immediacy. There is a lot of interesting research to be done beyond mechinterp and passively probing transformers.
2
0
19
A preamble from the heart.
1
2
20
IMHO, this deserves attention from a much larger community: A thread of undergraduate-level courses taught with Lean3/4 from CMU, imperial, IISC, universite paris-saclay, and more. We are seeing the beginning of formal maths education and it's glorious.
0
5
19
It's a good model and you should look beyond the (impressive) benchmark scores. Don't be a reviewer 2.
We also released Pixtral Large, a new SOTA vision model.
0
1
20
@wtgowers And you have it!. The recent “scaling laws hitting a wall” is an empirical observation: under the classical pretrain-finetune-align regime of LLM training, after a certain threshold, multiplying the compute pumped into the model yields negligible benefits. This violates the.
2
1
20
MATH-AI workshop tomorrow!.
📢 Can't wait to see you at the 3rd #MathAI Workshop in the LLM Era at #NeurIPS2023!. ⏰ 8:55am - 5:00pm, Friday, Dec 15.📍 Room 217-219.🔗 📽️ Exciting Lineup:.⭐️ Six insightful talks by @KristinLauter, @BaraMoa, @noahdgoodman,
0
3
18
When we achieve AGI internally I won't say it but there'll be signs
0
0
19
I went to my first in-person conference (AITP 2022) and it was terrific! A list of highlights in chronological order:. 1. This view
1
5
17
I need the confidence of random people on linkedin messaging: "Use our dataset to make Mistral pretraining corpus 10x better!".
1
0
17
The NeurIPS reviews I received this year are actually good and thought-provoking for 2 out of 3 papers I was on, even if the scores are not particularly high. It's a shame that the most rubbish reviews went to the paper my master student wrote for their dissertation.
0
0
18
at this moment precisely it's nice to explore and create art before the technology gets over-optimised for business and becomes soulless.
0
3
16
Being able to. git diff my_commit glample_commit.git diff my_commit tlacroix_commit. is a great privilege and the fastest way to improve my software quality.
0
1
16
Theorems for free with autoformalization!.
After showing a few examples, large language models can translate natural language mathematical statements into formal specifications. We autoformalize 4K theorems as new data to train our neural theorem prover, achieving SOTA on miniF2F!. 1/. Paper:
0
3
16
LM-generated informal proofs also help solve one international mathematical olympiad problem (and it's the first problem from the first ever IMO)! The human proof draft does not help solve this one. See why the huamn proof fails in the paper 😉
1
1
16