LLM360
@llm360
Followers
1,827
Following
52
Media
11
Statuses
67
LLM360 is an open research lab enabling community-owned AGI through open-source large model research and development.
Joined November 2023
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Pakistan
• 323954 Tweets
#INDvsPAK
• 198611 Tweets
Alcaraz
• 153563 Tweets
Bardella
• 103632 Tweets
Rohit
• 86378 Tweets
Zverev
• 84280 Tweets
Russell
• 82719 Tweets
Doom
• 73610 Tweets
Ferrari
• 66567 Tweets
#XboxShowcase
• 64723 Tweets
Le RN
• 54928 Tweets
Pant
• 52070 Tweets
Black Ops 6
• 51710 Tweets
Verstappen
• 48989 Tweets
Grand Slam
• 45975 Tweets
Leclerc
• 36433 Tweets
Dube
• 35864 Tweets
Call of Duty
• 35574 Tweets
Lando
• 32547 Tweets
Nadal
• 25441 Tweets
The Dark Ages
• 25390 Tweets
Le Pen
• 24872 Tweets
Gears of War
• 23737 Tweets
Life is Strange
• 21725 Tweets
Haas
• 20828 Tweets
McLaren
• 20198 Tweets
Fable
• 18498 Tweets
Jadeja
• 18180 Tweets
Benny Gantz
• 17282 Tweets
Dragon Age
• 16555 Tweets
Rinku
• 16132 Tweets
Hardik
• 15298 Tweets
Sargeant
• 13067 Tweets
Perfect Dark
• 13030 Tweets
Ergebnis
• 12651 Tweets
Bumrah
• 12049 Tweets
Albon
• 11796 Tweets
Rizwan
• 11477 Tweets
Magnussen
• 11234 Tweets
Pinned Tweet
Please welcome K2-65B🏔️, the most performant fully-open LLM released to date.
As a blueprint for open-source AGI, we release all model checkpoints, code, logs, and data.
About K2:
🧠65 billion parameters
🪟Fully transparent & reproducible
🔓Apache 2.0
📈Outperforms Llama 2 70B
6
143
487
🚀 1/7 We are thrilled to launch LLM360 — pushing the frontier of open-source & transparent LLMs!
Starting with Amber (7B) & CrystalCoder (7B), we are releasing brand new pre-trained LLMs with all training code, data, and up to 360 model checkpoints.
🔗
19
191
1K
1/3 We are releasing CrystalChat 🔮 — a top-scoring 7B chat model, fully open source!
As always, CrystalChat is released under Apache 2.0, along with all training data, checkpoints, and implementation details.
Grab the model here:
4
34
160
🔍 2/7 By releasing all data, code, & checkpoints, LLM360 makes it easy to reproduce results and build off our models for research and industry purposes.
All models are released under Apache 2.0 license.
Learn more
Blog:
Paper:
1
6
91
1/3 We’re adding two new models to LLM360! Presenting AmberChat 💬 & AmberSafe 🦺.
AmberChat is instruction tuned on Amber (7B) using
@WizardLM_AI
&
@ShareGPT
data.
AmberSafe is DPO FT’d AmberChat using PKU-SafeRLHF data.
AC:
AS:
3
24
91
🙏 6/7 Huge thank you to the OSS ecosystem used in LLM360!
@huggingface
for hosting the models and data
@weights_biases
terrific metrics dashboards
@AiEleuther
set the precedent for OS LLMs with Pythia (and more)
@lmsysorg
great finetuning tools
@LightningAI
lit-llama was lit
1
5
63
🌟 3/7 Amber, a 7B English LLM, is pre-trained on 1.2T tokens. See Amber’s emergent capabilities over 360 checkpoints & dive into 6.8TB of data.
Model:
Metrics:
Data:
Code:
1
2
52
🔮 4/7 CrystalCoder, a 7B code & text model, combines the best of StarCoder & Llama. Trained w/ 1.4T tokens on
@CerebrasSystems
Condor Galaxy 1.
Model:
Metrics:
Data:
Code:
1
3
47
🤖 7/7 LLM360 was built on datasets curated by:
@AiEleuther
,
@togethercompute
,
@bigcodeproject
,
@WizardLM_AI
Thank you to all!
0
1
46
Great to see LLM360 in the top 5 on the ‘Openness of instruction-tuned LLMs’ scoreboard along with
@BigscienceW
,
@laion_ai
, and
@togethercompute
.
Check out the work from
@Radboud_Uni
here:
0
3
36
🤝 5/7 We are excited to continue contributing to the community through open releases. Join us through direct collaboration or by telling us how we can help.
LLM360 is proudly brought to you by
@PetuumInc
,
@MBZUAI
, and
@CerebrasSystems
.
1
1
39
Huge congratulations to the
@allen_ai
team!
The OLMo series is a substantial contribution to the OSS LLM community with:
- open training datasets
- 500 checkpoints
- analysis and evaluations
Couldn’t be happier to see more projects with similar goals (cc
@AiEleuther
)
OLMo is here! And it’s 100% open.
It’s a state-of-the-art LLM and we are releasing it with all pre-training data and code. Let’s get to work on understanding the science behind LLMs. Learn more about the framework and how to access it here:
29
352
1K
1
7
37
We evaluated K2-65B across 22 standard benchmarks to assess its broad knowledge on topics such as coding, medicine, and math, in addition to Open LLM Leaderboard metrics.
🔗Check out the model here:
K2 was generously sponsored by
@MBZUAI
and
@PetuumInc
.
1
4
34
🎉 Congratulations to an awesome fully open source model, by the m-a-p team!
Paper: 📎
Includes great info on:
-Data Curation
-Infra details
-Intermediate checkpoints
-Scaling law
LLM360 is happy to work with this thriving community on open source AI.
🚀 Excited to announce that the tech report of MAP-Neo (): a fully open-source and transparent bilingual LLM suite with superior performance to bridge the gap with closed-source models, is now available:
🔧MAP-Neo's workflow
2
9
29
1
2
31
Our technical report:
The model and data are available on
@huggingface
:
Analysis is available on
@weights_biases
:
Code on
@github
:
3
5
25
A huge thank you to the OSS ecosystem used in (and inspiring) LLM360!
@huggingface
@weights_biases
@github
@lmsysorg
@AiEleuther
@allen_ai
@SnowflakeDB
@togethercompute
@bigcodeproject
@nousresearch
@PyTorch
@nvidia
@CerebrasSystems
And many more.
3
2
24
We also released a fine-tuned chat model, K2-Chat.
K2-Chat outperforms Llama 2 70B-Chat in medicine and math metric groups, and outperforms Llama 3 70B-Instruct on coding tasks.
1
1
23
We open-source a variety of components in three suites 📕
1. *LLM360 Research Suite*: artifacts such as model ckpts, code, and data.
2. *LLM360 Pretraining Suite*: tutorials to reproduce and build on our models.
3 . *LLM360 Developer Suite*: tutorials on fine-tuning, running
1
3
18
❄️Congrats to
@SnowflakeDB
for openly releasing Arctic!❄️
Arctic is available to all with an Apache 2.0 license!
Great to see LLM360 member
@AurickQ
and the whole Snowflake AI Research’s team's amazing contribution to the open-source LLM community!
Introducing Snowflake Arctic. An efficiently intelligent and truly open LLM built by Snowflake.
22
150
569
2
3
16
🔥Congrats to
@MaitrixOrg
for releasing Pandora, a World Model that can predict and simulate the world’s states in visual space, controllable by language 🎮
Excited to see the MaitrixOrg is indeed building something like the Matrix, great work by
@szxiangjn
,
@guangyi_l
,
@YiGu025
🔥Introducing Pandora 🌏 🪐
a World Model that generates videos of world states with real-time language control 🎥🕹️
Simulate the world across domains in an _interactive_ way!
check out more
7
74
210
0
3
9
Join our AMA in the Mozilla AI discord this Thursday (5/30) at 3pm EST.
We will answer questions about our models, how we trained them, and anything in between.
Thanks to Mozilla for the invite!
Link in the thread below.
1
2
10
🔥Congrats to
@MaitrixOrg
for their new library: LLM Reasoners v1.0🔥
Great to see
@Ber18791531
and LLM360 members
@YiGu025
and
@ZhitingHu
’s continued contributions to open-source LLMs!
Check them out:
Releasing 🔥LLM Reasoners v1.0🔥
🥇Popular library for advanced LLM reasoning
- Reasoning-via-Planning (RAP)🎶
- Chain-of-Thoughts (CoT)⛓️
- Tree-of-Thoughts (ToT)🌴
- Grace decoding💄
- Beam search🔎
🥇Enhances
#Llama3
, GPT4, LLMs on
@huggingface
2
61
201
0
6
9
🔎The LLM360 Research Suite was developed to aid academic and industry researchers further the ability and understanding of LLMs
📈We release detailed artifacts so everyone has access to the same material, as if they are model trainers
Research Suite artifacts highlighted below
1
5
21
Awesome development in open-weight llms!
Introducing Meta Llama 3: the most capable openly available LLM to date.
Today we’re releasing 8B & 70B models that deliver on new capabilities such as improved reasoning and set a new state-of-the-art for models of their sizes.
Today's release includes the first two Llama 3
275
1K
6K
0
1
8
3/3 CrystalChat is on
@huggingface
It is fine-tuned from CrystalCoder-7B, originally trained on
@CerebrasSystems
Condor Galaxy 1.
Stay tuned as we peel back the onion on CrystalChat and show you everything under the hood.
As always, drop us a line on ✍️
2
1
8
3/3 Both models are available on
@huggingface
.
Special shout out to
@TheBlokeAI
for quantization!
We'd love to hear how LLM360 can do more to foster a transparent, trustworthy, and collaborative ecosystem.
Drop us a line in the feedback form on ✍️
0
0
6
2/3 Comparing AmberChat and AmberSafe side-by-side shows additional work is needed to ensure all LLMs are safe for human usage.
Transparent safety alignment will continue the progress of open-source LLMs.
Thanks to
@AnthropicAI
and others for opening their alignment data!
1
0
3
@WizardLM_AI
Thank you very much! We also use your evol-instruction sets, they work well with our models! We will be posting some results about the fine tuning too.
0
0
3
Loss spikes are a poorly understood phenomena.
We encountered two malignant loss spikes resulting in significant performance degradation while training K2-65B.
We saved the spike checkpoints for further evaluation.
Malignant spikes:
0
1
5
We release full details to produce K2-65B, Crystal-7B, and Amber-7B.
Artifacts include:
- 600 total intermediate checkpoints
- full data sequence and dataset
- training and data prep code
- evaluation code and results
Training dynamics:
1
0
6