Xuan-Son Nguyen
@ngxson
Followers
6K
Following
1K
Media
242
Statuses
897
Engineer @huggingface
Paris
Joined August 2020
Introducing: The most visually intuitive article about RoPE, 2D-RoPE, and M-RoPE that you can find on the internet 😆 Link in 🧵
7
40
316
That reminds me about the XZ utils attack where Jia Tan was actually not a chinese guy 🤷♂️
We disrupted a highly sophisticated AI-led espionage campaign. The attack targeted large tech companies, financial institutions, chemical manufacturing companies, and government agencies. We assess with high confidence that the threat actor was a Chinese state-sponsored group.
1
0
4
Got me thinking... why don't we train the LLM to output BSON or even protobuf directly 😂😂😂😂
If you’re still sending raw JSON into your LLMs, you’re burning tokens, latency, and budget! Try TOON (Token-Oriented Object Notation). Clear like YAML, compact like CSV: • 30–60% fewer tokens • Up to 50% lower costs • Shines for tabular data. Free and Open source 🧵↓
0
0
6
Imagine if it's <$400 and you can run llama.cpp on it 🤑🤑
<$400 Instant success with everyone buying it and selling millions >$400 Meh, not that great of a deal >$700 Lol
1
0
3
Initial M5 Neural Accelerators support in llama.cpp Enjoy faster TTFT in all ggml-based software (requires macOS Tahoe 26) https://t.co/HWbCQvFR2w
github.com
Rework matrix-matrix multiplication Use Tensor API when available TODOs Update mul_mm_id kernel Test on M5 (looking for volunteers to test as I won't have hardware anytime soon) How to...
10
38
369
When you run AI on your device, it is more efficient and less big brother and free! So it's very cool to see the new llama.cpp UI, a chatgpt-like app that fully runs on your laptop without needing wifi or sending any data external to any API. It supports: - 150,000+ GGUF models
52
179
2K
Don't miss the new WebUI of llama.cpp 👀 - SvelteKit-based web app - Sleek UI - Support multimodal (image, audio, PDF input) - Powered by llama.cpp 💪💪 All thanks to Aleksander Grygier for taking the lead!
1
1
22
A very lightweight OCR model with surprisingly good quality, check it out!
🦉LightOnOCR has landed in llama.cpp! thanks to @ngxson for the quick integration! https://t.co/XMEJeIfwWD
0
0
7
Super proud to have presented my work on open source and Reachy Mini at @AIatMeta during their Display Glass launch! 🤖✨ Our joint goal was to showcase France’s competitiveness in AI: 🎓 French universities are among the best for AI - Paris-Saclay is ranked 2nd in the world in
0
2
3
simple
Important info. The issue in that benchmark seems to be ollama. Native llama.cpp works much better. Not sure how ollama can fail so hard to wrap llama.cpp. The lesson: Don’t use ollama. Espacially not for benchmarks.
12
28
357
Amazing! GGML ecosystem works out-of-the-box on DGX Spark 🚀🚀
Setting up NVIDIA DGX Spark with ggml I got to play for a few days with this new device and wrote a short guide about how to configure it for various local AI use cases. https://t.co/O2gnWgZQvD
0
0
5
What I suspect is that this book's quality won't be that good. Just look at how dark parts have kinda "noise". Black text on white paper has been used for centuries, good for your eyes. Why change it then? "Dark mode" only exists because of screens.
0
0
2
I was in China last month talking with a couple of young parents. And yes, this is now a real concern.
A Chinese father's video of his daughter tearfully saying goodbye to her broken Al learning robot. People already get emotionally attached to AI. Now imagine Figure03 at home and it breaks, people will have a breakdown.
0
0
2
TIL that I can use my flipper zero as USB-UART adapter to de-cloud Tuya smart devices Custom firmware powered by esphome
1
0
5
HuggingFace just shipped in-browser GGUF editing It allows you to edit GGUF metadata in the comfort of your browser, without having to even download the full model. This feature is enabled via the Xet technology that makes partial file updates possible.
6
51
380