GitHub 上一款完全本地化运行的 AI 助手浏览器插件：NativeMind，. 支持 Ollama 和 WebLLM 两种本地模型调用，所有数据处理都在设备本地完成，无需担心隐私泄露。. GitHub： 主要功能：. - 智能对话助手，支持多标签页上下文理解.- 网页内容分析，可快速总结页面和文档内容

Running locally on iphone 13 mini without cellular, without wifi, without bluetooth. Real-time, not sped-up. Insightful in-depth response to a question involving 2 relatively obscure and complex topics. (I plan to make it even more effecient soon 😉) ty to WebLLM & MLCChat.

What if psql had an \ai command?.This is PGlite (WASM Postgres) and Llama3 (via WebLLM & WebGPU) running completely in the browser - no cloud, no server🤯.Local AI is coming sooner than you think!

WebLLM を使って Qwen2.5-1.5B のブラウザ上でのローカル実行をお試し中。

🚀✨Introducing XGrammar: a fast, flexible, and portable engine for structured generation!. 🤖Accurate JSON/grammar generation.⚡️3-10x speedup in latency.🤝Easy LLM engine integration.✅ Now in MLC-LLM, SGLang, WebLLM; vLLM & HuggingFace coming soon!.

we just added webllm support to @bashnota .now you can still vibe code in the woods

Running Phi-3.5-Mini in the browser at 67 tokens per second just feels right 🤗. 🔸 Powered by MLC WebLLM + WebGPU.🔸 Fully Private / Running on Device

手机上跑LLM大模型.继WebLLM，作者又发布了MLC-LLM.在任意支持 CUDA、Vulkan、Metal的平台上，当然包括手机，就可以运行大模型. Demo:.

Fully open source code interpreter running 100% local and in your browser! 🔥. Powered by Qwen Code 2.5 & WebLLM! ⚡

Qwen-2.5 on WebGPU 🏎️. • 42 tok/sec for Qwen2.5-Coder-1.5B on Mac ⚡.• Powered by MLC WebLLM and WebGPU 🔥. Watch Qwen2.5-Coder-1.5B build a website entirely in the browser!

LFG!! XGrammar: a lightning fast, flexible, and portable engine for structured generation! 🔥. > Accurate JSON/grammar generation.> 3-10x speedup in latency.> 14x faster JSON-schema generation and up to 80x CFG-guided generation.> Now in MLC-LLM, SGLang, WebLLM; vLLM &

Run Llama 3.2 LLM in Web Browser Locally. Llama 3.2 is the new series of LLMs in Llama family. Now you can run Llama 3.2 (1B and 3B) in your browser locally with the help of WebLLM and WebGPU. WebLLM is a high-performance in-browser LLM inference engine. WebGPU is a new,

What if we could use AI models like Llama 3.2 or Mistral 7B in the browser with JupyterLite? 🤯. Still at a very early stage of course, but making some good progress!. Thanks to WebLLM, which brings hardware accelerated language model inference onto web browsers, via WebGPU 🚀

Gemma 2 2B running in a browser, powered by WebLLM & WebGPU! 🔥. 100% local & on-device. In less than 24 hours, we've already got the model to the edge! ⚡. Try it out on an HF space below:

Run Llama 3.2 in the browser on WebGPU 🔥. • Llama 3.2 (1B + 3B)🤏 🦙.• Running 100% locally in the browser at 62 tok/sec 🏎️.• Powered by MLC WebLLM + WebGPU ⚡

Try out the new WebLLM Playground with the new model picker 🏎️. Try LLama 3.2, Qwen 2.5, Mistral, Gemma and more entirely in the browser.

Phi-3.5-miniが、WebLLMとして利用可能に. インストール、セットアップ不要でLLMをブラウザから自分のPCのGPUで。. ほかの人と共有しないので、いつでも基本的に同じ性能で利用できるところが良さ。. VRAM6GBくらいあれば動くと思うが、

Just adding real time streaming batched inference to WebLLM. No big deal.

wip: allowing local models via WebLLM for OpenBB Copilot 🤯🤯🤯. all data stays in your control without leaving the browser. small models are becoming powerful. WebLLM project from @charlie_ruan is just mind blowing, took me a few minutes to integrate. even supports streaming