One of the coolest parts about building open source software is the amazing stuff that the community creates.
This is a forever thread of the cool stuff built by the LiveKit community 👇
When AI is as smart as a human, we’ll interact with it like we do with each other.
Human interaction is real-time and multimodal:
We use → 👀👂👄
AI uses → 📹 🎙️🔈
Today we're launching a stack for building real-time multimodal AI apps.
Add spatial audio to your app (or game)!
Here the voice chat audio plays in stereo based on the relative position of the other player. Player positions are synced over WebRTC data channels.
(sound on, obviously)
decided to migrate off self hosted webrtc solution to
@livekit
. it's still open source if I ever need to self host again but it'll save me a lot of time in managing everything on my own for now. feels good!
Launch Week Day 5:
It's Community Day!
There's no way LiveKit could improve as quickly without the support and contributions of developers in the WebRTC ecosystem. A huge thank you from the core team! 🙏❤️
Element Call Beta 3 is here at last! 🚀🎥📺 This is a huge update, now with 100s of users per call & Brady Bunch layout thanks to
@LiveKitted
+ Element Web/Desktop Develop/Nightly native support too! One catch: E2EE is temporarily off while we hook it up.
🔌 Plugins
Integrations with
@elevenlabsio
,
@DeepgramAI
,
@fal_ai_data
, and
@openai
make it easy to compose together multimodal AI applications.
We’re excited to build more integrations with the LiveKit community!
🚨 Big drop!
v0.15.1 is out today:
🚦Downstream congestion control
📹 Publisher-only mode (v. useful in robotics applications)
📶 Improved connection-quality updates
🏷️
Meet our latest product beta, Audio and Video!🎉
Developers can now quickly add real-time video and audio to their applications in just a few lines of code 🛠️
Best of all, we provide the UI components and logic to make the hard parts easy 😎
👉:
My 14-hour twitch clone tutorial is OUT!
Here is everything we will learn:
📡 Streaming using RTMP / WHIP protocols
@livekit
🌐 Generating ingress
🔗 Connecting Next.js app to OBS / Your favorite streaming software
🔐 Authentication
@ClerkDev
📸
🖥 SCREEN SHARING
Internal testing of screen sharing in Portals (and a real meeting using screen sharing for planning).
- Share a presentation
- Livestream an IRL event
- Introduce your product/game/project
- ... and so much more
Last screen is a
@Wuki
live set from Youtube 🎵
🎢 Playground
To quickly prototype and test your agents, we have the Agents Playground. It’s a client application wired up for streaming voice, video or text to your agent and receiving streams back.
🎻 Orchestration
Conversational agents are stateful, foreground workers and require a different approach to deployment and scale. We built an orchestration system for running thousands of agents in the cloud.
We've designed a system where you can build many different types of real-time audio and video applications, and LiveKit automatically handles the network performance and scaling aspects for you.
Today is a BIG day - we're proud to unveil the InfraRed 100, a comprehensive list of the next 100 most promising private companies in Cloud Infrastructure.
The list includes a variety of early, mid, and late-stage companies. See the full list here!
📣 Sharing some updates about my upcoming FREE Twitch Clone tutorial using
@nextjs
Here is everything we will learn:
📡 Streaming using RTMP / WHIP protocols
@livekit
🌐 Generating ingress
🔗 Connecting Next.js app to OBS / Your favorite streaming software
🔐 Authentication
Three egress options cover a broad range of use cases:
1. Room composite
2. Track composite
3. Single track
Record live performances, publish to YouTube/Twitch or integrate real-time audio transcription.
We highlighted how amazing LiveKit is before, but their work is so incredible it bears repeating:
It’s now possible for anyone to build real-time server-side agents that leverage our AI alongside many other models.
A fantastic feat of engineering & we’re proud to be part of it
Developers are increasingly thinking multi-platform and React Native is one of the most popular frameworks offering this possibility.
Since LiveKit's launch in July 2021, it's been our most-requested SDK!
With
@livekit
and using our friends at
@DeepgramAI
,
@fal_ai_data
, and
@elevenlabsio
- I made a celebrity face swapping guessing game. Pretty cool how LLMs can be used to parse messy human input into structured game inputs
🛠️ Backend SDKs
You can now use LiveKit on the server (python, rust, go, 🔜 node) to process incoming media and stream synthetic voice or video to clients.
This is a huge milestone for the project and there's a lot lurking behind the number. A fully end-to-end WebRTC stack requires deep integration between server and clients.
KITT can:
- Answer questions like Siri, Alexa, or Google Assistant - Take notes on or summarize what was discussed in a meeting
- Speak multiple languages or act as a third-party translator
Try it yourself here: .
We wanted KITT to feel human, and the biggest hurdle was latency. We solved it by streaming all the things:
- Convert streaming audio to text in 20ms frames
- Prompt GPT to be concise in its responses
- Sentences are converted to speech in real-time. The results are magical✨
@momoeureka
One of the coolest things about KITT is that it works with multiple participants. So KITT could, for example, take notes and create a summary of a long meeting!
@Pwoseidonn
Decentralized in the sense that anyone can run their own instances of our media server and auxiliary services, yes. A decentralized version of LiveKit as a cloud-based platform is something we want to/will eventually do.
@porada_app
LiveKit can record videos using our Egress service which can then be uploaded to Instagram using whichever technique you’d like. LiveKit also can stream to Instagram Live using our Egress service via RTMP.