Streaming data got a data problem? Don’t miss our workshop on schema-driven governance for Kafka. Our engineering and product teams will show what you can do when brokers understand the shape of your streaming data. Q&A throughout!. Thursday, July 10 9 AM PT / 12 PM ET / 5 PM BST.

"If you're aiming to build robust, modern data pipelines without Kafka’s complexity, schema-first is the way forward." Couldn't have said it better ourselves.

Don’t miss our interactive workshop on bringing schema-driven governance to Kafka. Our engineering and product teams will deep dive into how it’s done. Plus, we’ll answer your questions throughout. 📷 Thursday, July 10 📷 9 AM PDT/12 PM EDT/5 PM BST.

Another day, another great video on the data quality issues that plague the Kafka ecosystem. @AbhiVeeramalla goes in depth on schema validation, plus Kafka’s high storage and networking costs. Spoiler alert: Bufstream is the answer.

See how Clinician Nexus governs data quality with the Buf CLI + Buf Schema Registry and keeps costs low with Kafka-compatible Bufstream. Results: consistent validation, quick updates & complete audit trail. #DAIS #Kafka .

TL;DR: The first 15 field numbers are special: most runtimes will decode them much faster than the other field numbers. When designing a message type for decoding performance, it’s good to use these field numbers on fields that are almost always present.

Oneofs are a disaster. So we fixed them.

Big news from #DataAISummit! Bufstream now supports Databricks Managed Iceberg Tables in private preview, bringing together Buf's schema-first approach with Databricks' industry-leading data governance and optimization capabilities. Read all about it!.

TL;DR Don’t use required, no matter how tempting. You won’t be able to get rid of it later when you realize it was a bad idea.

TL;DR: buf convert is a powerful tool for examining wire format dumps, by converting them to JSON and using existing JSON analysis tooling. protoscope can be used for lower-level analysis, such debugging messages that have been corrupted.

Don’t miss our interactive workshop on bringing schema-driven governance to Kafka. Our engineering and product teams will deep dive into how it’s done. Plus, we’ll answer your questions throughout. 📷 Thursday, May 29 📷 9 AM PDT / 12 PM EDT / 5 PM BST.

TL;DR: Enum values can have aliases. This feature is poorly designed and shouldn’t be used. The ENUM_NO_ALLOW_ALIAS Buf lint rule prevents you from using them by default.

We’re excited to announce that Bufstream is now available on the AWS Marketplace. Enterprise customers can purchase through their AWS account and accelerate their deployment of Bufstream.

Learn about Bufstream, our Kafka-compatible #streamingdata platform. This workshop will show what’s possible when you understand the shape of your streaming data. Thursday, May 29  9 AM PDT/12 PM EDT/5 PM BST.

TL;DR: Avoid import public and import weak. The Buf lint rules IMPORT_NO_PUBLIC and IMPORT_NO_WEAK enforce this for you by default.

Engineers shouldn’t have to define their network APIs in OpenAPI or Protobuf, their streaming data types in Avro, and their data lake schemas in SQL. A unified schema approach dramatically reshapes our world, solving data quality problems at the source.

TL;DR: Protobuf’s distributed nature introduces evolution risks that make it hard to fix some types of mistakes. Sometimes the best thing to do is to just let it be.

TL;DR: enums inherit some unfortunate behaviors from C++. Use the Buf lint rules ENUM_VALUE_PREFIX and ENUM_ZERO_VALUE_SUFFIX  to avoid this problem (they’re part of the DEFAULT category).