We’ve had a blast building this. Hope you like it!

Yesterday we hosted SF's most exclusive demo day. Only 10 teams, all of them with moonshot visions. Here's an inside look at the demos: (🧵)

I have a new video up on YouTube — https://t.co/p0mnfCZ9tS — about doing complex event processing with #apacheflink. This might sound like an obscure topic, but I have found this technique to be super-powerful, and broadly applicable.

Prequal is a must-read paper that makes YouTube +5-10% faster. They show that load balancing w/ requests-in-flight and latency outperforms weighted round robin on CPU load (and why). A rare paper showing distributed systems research working in production. Here’s how: 1/4

It's not too hard to build a stream processor, right? Operators, network, add checkpoints, done. Making state and snapshots work really well and efficiently has been a continuous effort since we introduced this in @ApacheFlink in 2015. Here is a nice article revisiting all that

Watermarks are at the heart of what makes stream processing with #FlinkSQL possible. Join @alpinegizmo as he explains the ins and outs of watermarking in in the latest video from his new course on #ApacheFlink SQL. Check it out on Confluent Developer ➡️ https://t.co/TaqXU3HeYk

You haven't seen complexity in art until you've seen this 600-year-old painting. It's so detailed that modern medicine has been able to diagnose this man's exact type of blindness. And that's where things get mind-bendingly strange — this whole thing is an illusion... 🧵

He’s right! @alpinegizmo and I can’t stop, won’t stop practicing for #current24.

Meet the new emerging role: The Data Streaming Engineer. During our day 2 #Current24 Keynote, learn how this role is bringing together the work of app developers, data engineers, data scientists and more—potentially redefining the work you already do.

Blogged: Predicting the Future of Distributed Systems https://t.co/qwEYZ2SMkd

Our pre-#current24 summer meetup series continues on July 25th with some #apacheFlink! 🐿️ Sign up now to catch @sharon_rxie and @alpinegizmo as they dive into timing in #flinkSQL and enrichment patterns. ⤵️ https://t.co/U5CA9t4OzD

Adding queue semantics as an API over logs provides the cooperative consumption advantages of traditional queues while fixing some of the queue structure's disadvantages: lack of replay and the need for write amplification. Read more in @vanlightly's blog:

1/ There is a mini open source drama, this time because RedPanda, a company with a source-available Apache Kafka clone, bought an open source connector framework and made licensing and trademark changes to thwart other startup competition. (a thread)

I've spent the past ~3 weeks going through the entire history of deep learning and reimplementing all the core breakthroughs. It has completely changed my beliefs about deep learning progress and where we're headed. Progress tracker in thread (all resources at the end) 👇

On July 25th, we're inviting #apacheFlink to our #apacheKafka party. Hear from @alpinegizmo and Sharon Xie. https://t.co/U5CA9t4gK5

It’s no secret that LLM training data is running out. How close are we to the limit? To answer that, here's an estimate of the total amount of text in the world from every major source:

👨‍🍳 Looking for practical #ApacheFlink use case examples? We prepared a cookbook with a collection of best practices, with real-life use cases including: 🌰 Joining and deduplicating data 🌰 Deserializing JSON from #Kafka Plus many more! Start cooking: https://t.co/oFD5QGZS8g

❌ Delete the duplicates! With exactly-once semantics, each message is delivered precisely once. But how does it work? Learn about the concept and how #ApacheFlink enables exactly-once processing for your real-time streaming data with our new video:

Looking for practical #ApacheFlink use case examples? Our experts, @alpinegizmo, @MartijnVisser82, and Chesnay Schepler, have gathered a collection of recipes 🧑‍🍳 to help you tackle a variety of common on-premise challenges. Explore the resources: https://t.co/oFD5QGZS8g

Thrilled to share that I'll be talking about using #ApacheFlink for data enrichment at #Current24 in Austin, Sept 17-18: https://t.co/1X5GL6LDxY Hope to see you there!