Apache Kafka + Apache Flink… name a better duo, we’ll wait.
Our heroes are back and this time for a new adventure to learn the basics of Flink, the challenges of streaming processing, & why Kafka and Flink are better together.
💥📖 Get your copy here ➡️
🔔 Ringing the bell at
@Nasdaq
!
We’re incredibly proud of our team, and thankful for our customers, partners,
#ApacheKafka
community, and everyone who made this day possible. This is only the beginning of our journey to set the world’s
#datainmotion
.
#ConfluentIPO
We’re excited to announce Tutorials for
@apachekafka
, a collection of common event streaming use cases, with each tutorial featuring an example scenario and several complete code solutions. Learn more in our latest blog post by
@MichaelDrogalis
:
🎉 We’re excited to share our intent to acquire
@Immerokcom
!
Together, we’ll build a cloud-native service for
@apacheflink
that delivers the same simplicity, security, & scalability that you expect from Confluent for Kafka.
Learn more →
Today we announced Project Metamorphosis, the next major step for event streaming. This effort to solve the most pressing issues in achieving pervasive event streaming is supported by Confluent’s new funding round. Learn more from Confluent CEO
@jaykreps
Introducing Confluent Developer: a single source of truth for learning about
@apachekafka
, event streaming, and the surrounding ecosystem. Here, you'll find tutorials, demos, videos, and more to help you get started:
Today we launched Confluent Developer - the ultimate
@apachekafka
learning destination. Check out the training courses, event streaming patterns, deep-dive articles, and language-specific client programming guides - all FREE and available here:
Last week, we kicked off Project Metamorphosis! This week, we're highlighting the removal of ZooKeeper dependency in
@apachekafka
. Learn about the progress being made by the community to solve Kafka's biggest bottleneck in scalability:
Today’s
@apachekafka
2.8 release brings a highly anticipated early access version of KIP-500 (ZooKeeper removal), plus other exciting new features and improvements! Read about the key updates in this blog post by John Roesler:
Six years ago we built an entirely new data infrastructure to set
#datainmotion
. Our journey is built on the passion, energy, and dedication of our team, community, customers, and partners. We’re here because of you, THANK YOU!
#ConfluentIPO
Tencent PCG uses
@apachekafka
to transfer 4 million messages/second for a single product. Learn how they scale their real-time data pipeline using federated Kafka clusters to handle this peak workload with low latency, a high SLA, and flexibility:
Today’s
@apachekafka
3.0 release introduces a variety of new features, API changes, and improvements to KRaft—Kafka’s built-in consensus mechanism that will replace ZooKeeper. Read about these and other key updates in this blog post by
@karantasis
:
By introducing the incremental cooperative rebalancing protocol, stop-the-world rebalances are a thing of the past as the
@apachekafka
consumer,
@kafkastreams
, and
@ksqlDB
apps are one step closer to high availability. Learn more in our latest blog post:
.
@apachekafka
2.5 is here! This release makes progress on KIP-500, adds new metrics and security features, and includes exactly once semantics and operational improvements, plus more. Learn about what’s new in the blog:
Confluent and
@elastic
are excited to work together to make it easier than ever to integrate
@apachekafka
and Elasticsearch with the fully managed Elasticsearch Sink Connector. Read more from Confluent CEO
@jaykreps
on our expanded partnership:
Confluent Platform 5.5 is out! This release adds support for Protobuf and JSON schemas, introduces exactly-once semantics for librdkafka-based clients and admin functions in preview, and includes an expansive new feature set for
@ksqlDB
. Learn more:
In light of global health concerns, we've decided to transform
#KafkaSummit
Austin into a virtual experience. We look forward to gathering the global
@apachekafka
community together for Kafka Summit 2020, Event Streaming Everywhere. Stay tuned for details:
Learn
@apachekafka
fundamentals with our new blog series Streams and Tables in Apache Kafka. In part 1,
@miguno
explains the basic elements of an event streaming platform: events, streams, tables, and the stream-table duality.
Yesterday was one big step on the path to setting the world’s
#datainmotion
. Huge thanks to everyone who helped us get to where we are today.
#ConfluentIPO
Event sourcing vs. derivative event sourcing:
@jbfletch_
introduces an observational approach to event sourcing that, together with
@apachekafka
and
@kafkastreams
, allows legacy applications to take full advantage of the event sourcing paradigm. Read more:
Traditional enterprise messaging systems are being eclipsed by
@apachekafka
. Kafka is not just another queue, it's a streaming platform: it scales company-wide, allows durable long-term message persistence, and rich stream processing.
👋Say hello to Stream Designer: The industry’s first visual interface for building streaming data pipelines natively on
@apachekafka
. 💥
Rapidly build, test, and deploy pipelines using a visual canvas extensible with SQL. Check it out 👇
#Current22
Recently,
@Twitter
built a machine learning logging pipeline supported by a customized
@apachekafka
streams join DSL. Results: reduced latency, better up-to-date data, and engineering time saved! Read about it in our latest blog post by
@PeilinYang
:
It's here!
@confluentinc
Platform 5.0 built on
@apachekafka
2.0 is available for download.
Introducing new functionalities and performance improvements, PLUS the new Confluent
#MQTT
Proxy for easier
#IoT
integration with
#Kafka
.
More details ➡️
.
@apachekafka
is the most popular event streaming platform, but is it the fastest?
@aloknnikhil
and
@byte_array
explore this question using the open messaging benchmark framework to compare Kafka, Pulsar, and RabbitMQ:
Ready to advance your
@apachekafka
knowledge to the next level? Take
@junrao
's free Kafka internals course—newly available on Confluent Developer—to learn about Kafka's architecture, design mechanics, producers, consumers, partitions, and more:
Today's
@apachekafka
3.3 release marks KRaft as production ready! 🎉 Read about this and other key updates, like exactly once support for source connectors, in the release blog post by
@jagsancio
:
Should you put several event types in the same
@apachekafka
topic?
@martinkl
explores this question and outlines a set of guidelines for addressing it in our latest blog post.
We're moving into our new Mountain View HQ and offices in San Francisco & India to keep up with demand for event streaming platforms. Learn more about the new spaces here:
Oh, and we're hiring!
The
@apachekafka
Connect S3 connector has been used to upload more than 75 PB of data from Kafka to
@awscloud
S3.
@karantasis
describes why we wrote the connector, the reasoning behind its design and how to use it in our latest blog post:
Take the first glimpse of what a ZooKeeper-less
@apachekafka
will look like!
@benstopford
and
@ijuma
explore the system’s event-driven design, scalability improvements, and key operational characteristics running up to 2 million partitions:
Watch the 6-part
@apachekafka
online talk series to understand the foundational concepts behind
#Kafka
, streaming data & real-time processing on streams.
Ever wonder how to use
#KSQL
for threat hunting? By taking data from Sysmon and streaming it into
@apachekafka
for processing in KSQL, you can rapidly detect suspicious behavior.
@Cyb3rWard0g
blogs about joining Sysmon events 1 and 3 in real time:
KIP-500 proposes replacing ZooKeeper with a self-managed metadata quorum. Colin McCabe & Jason Gustafson join
@tlberglund
on
#StreamingAudio
to discuss ZooKeeper’s role in
@apachekafka
and combatting security, stability & compatibility issues with KIP-500.
Today we announced our Q4 and full year 2021 financial results.
Our accelerating revenue growth throughout the year is powered by the rise of
#DataInMotion
and our customers’ shift to the cloud.
Learn more:
Consumer groups and rebalancing allow for the horizontal scaling of data consumption in
@apachekafka
.
@BdKozlovski
blogs about how a consumer works, how consistency is ensured and the rebalance protocol in action. Read about the internals here:
Today we announced Q3 earnings, marking our 1st quarter with $100M+ in total revenue. Our results are driven by customers around the world who are setting their
#DatainMotion
to thrive in the modern era. See here for more:
Planning to deploy
@apachekafka
in production this year? This reference architecture guide from
@gwenshap
is your go-to resource for successfully running a scalable streaming data platform in production.
Confluent Platform 5.4 has been released! And it’s packed with new features including RBAC, Structured Audit Logs, Multi-Region Clusters, Tiered Storage and pull queries in preview, and more.
@tlberglund
blogs about what’s new:
Ready to do some threat hunting?
@Cyb3rWard0g
, author of
@THE_HELK
, blogs about how with a few simple
#KSQL
commands, you can join in real time
#Sysmon
events 1 and 3 in order to spot lateral movement behavior.
What if you wanted to see every message inside of
@apachekafka
? What if you could see each message at top speed without ever having to slow them down? Good news: It's possible! And you can even do it in a web browser. Read our latest blog by
@chokeamancer
:
The May Preview release introduces powerful new capabilities for KSQL, streaming SQL for
@apachekafka
, and the Schema Registry UI. Read our latest blog to learn more.
Excited to announce our new Confluent Developer site is now live! With content for beginners and experts alike, it’s a one-stop-shop for all your
@ApacheKafka
learning.
Learn more from our CEO and
#ApacheKafka
co-creator
@Jaykreps
:
Today we announced our first earnings results as a public company: Total revenue of $88.3M (up 64% YoY), 617 customers with ≥ $100K in ARR (up 51% YoY) and Confluent Cloud revenue of $20M (up 200% YoY). See here for more info:
#DataInMotion
Sticky partitioning, introduced in
@apachekafka
2.4, is a new, more efficient strategy for assigning records to partitions with lower latency. Learn about this Kafka producer improvement in our latest blog post:
What is
@springcloud
Stream and how does it work with
@apachekafka
? Soby Chacko from
@pivotal
blogs about Spring Cloud Stream's programming model, Kafka integration in Spring Cloud Stream, stream processing with
@kafkastreams
, plus more!
KIP-360 and KIP-447 make exactly-once semantics in
@apachekafka
easier to use and more resilient. Learn about these stability and scalability improvements from Kafka Committer Boyang Chen and Confluent Engineer Bob Barrett:
Confluent Platform 6.0 is now available! This is one of our biggest releases yet. New capabilities make Apache Kafka more cloud-native w/greater elasticity, improved cost-effectiveness, infinite data retention, and global availability. See what’s new here:
Test
@kafkastreams
logic faster with TopologyTestDriver!
@apachekafka
2.4 introduces TestInputTopic and TestOutputTopic classes to simplify the test interface. Follow this tutorial by
@JukkaKarvanen
to write a test using the AssertJ framework: