I've written 18 posts (and counting) on table format internals. I've created a page that contains the list of my writings on the subject, including my formal verification work. Any suggestions on further table format analysis?.

In distributed systems, reliability isn’t just about retries and durability, it’s about knowing who owns recovery. My latest post, based on the Coordinated Progress model I posted previously, explores how reliable triggers create responsibility boundaries and how those boundaries.

Over the past few months, I’ve been thinking deeply about how systems make progress reliably in the face of partial failures, service boundaries, retries, and complex dependencies. Building reliable workflows across microservices, functions, and stream processors is one of the.

How to reliably distribute work across microservices, stream processors, durable execution, event-driven, orchestration and now AI agents?. Coordinated Progress is a 4-part series that explores the common structure behind reliable distributed systems.

Another Humans of the Data Sphere is out, with issue 10! In this issue people are talking fsyncs, tips for running ClickHouse at scale, the problems with MCP and more. Plus I dig up a classic paper from 1962.

The specs are in this repo for anyone interested.

Proud to have contributed formal verification (TLA+) for three key improvements in Kafka 4.0:. ✅ KIP-966: Strengthens the replication protocol. ✅ KIP-996: Introduces PreVote for more stable KRaft leadership. ✅ KIP-848: Delivers more efficient, predictable rebalancing.

Seems like I’m not alone. For what it’s worth, I’ve got a great fit at Confluent — but the more senior I get, the more I wonder how sustainable that is across future PE roles. Thinking of writing a blog post, maybe with interviews or perspectives from PEs who aren’t natural cat.

RT @ijuma: And the old group coordinator implementation is gone from Apache Kafka - love it when open-source projects can delete large chun….

Any Principal Engineers out there with ADHD or creative wiring — who don’t thrive in the tasks of project coordination, alignment meetings, and people management, but thrive on strategy, system design, writing, and shaping direction through ideas? Curious how you navigate the.

A new disaggregated log replication survey post is out. How does the combination of Apache Pulsar with Apache BookKeeper divide and conquer the responsibilities of log replication?

Another Humans of the Data Sphere is out, with issue #9! In this issue, we also look at whether software engineers can learn from mechanical engineering, and looking at table formats as a form of virtualization.

RT @ankushpd: If you are looking for formal models of a real-world distributed system, DeepSeek @deepseek_ai released P specifications for….

A new log replication disaggregation survey post is out! .The Kafka Replication Protocol:.🔹Separation of control plane from data plane. 🔹Role separation with minimal coupling. 🔹Kafka’s alignment with Paxos roles.

I may have to add Restate to my disaggregated log replication survey 😁.

Next in the survey is the Kafka Replication protocol.

The first post in the survey of disaggregated log replication systems is out! It looks at Neon's serverless Postgres write-path, which weaves consensus from heterogeneous components, based on MultiPaxos.

Part 1 of the survey of disaggregated log replication systems comes out tomorrow😀.

I updated my How to Disaggregate a Log Replication Protocol to include "separating ordering from IO". Basically I couldn't ignore CORFU as a way of separating responsibilities! So now we have A-F of ways of breaking apart the monolith.