Shutting down the cross-poster. If you'd like Jepsen updates, they'll be on Mastodon from now on:

multi-query optimization yield dramatic speedups in analyses: histories of hundreds of millions or even billions of operations are now tractable. Hopefully it makes your life better!. (2/2).

Phew! Jepsen 0.3.0 is ready, and it's a doozy. 1-2 order of magnitude throughput improvements across multiple benchmarks. Generators are way faster. Histories are now streamed incrementally to and from disk. Linear and concurrent folds over histories with stream fusion and (1/2).

If anyone out there works on Stripe eng and has a few minutes to spare, I could use a little help with something! DM me or email aphyr@jepsen.io, please. 😃.

Sneak preview of a new library for checking Jepsen histories. This will be shipping with the new disk-backed histories in the next release.

Also delighted to share that there's now a (limited) Jepsen test for local filesystems, which we've been using to find bugs in lazyfs:

Jepsen 0.2.7 is now available! Includes a (known-buggy) preview of lazyfs: a filesystem which can intentionally lose un-fsynced writes!.

Ayyyyyy, congratulations! 🎉.

Hey y'all! Doing an hour-long free webinar on May 25th with @redpandadata to talk about what we found in the last Jepsen analysis (. Come learn about streaming systems safety!.

Cheers to @redpandadata on a delightful collaboration, and congratulations on their new release. :-).

Redpanda has addressed most of these issues in the just-released 21.11.15, and the upcoming 22.1.1 fixes aborted reads and lost writes with transactions--lost/stale messages are still under investigation. A few more issues require only documentation to address.

A new #Jepsen report! We analyzed @redpandadata (a Kafka-compatible distributed queue) and discuss crashes, aborted reads, inconsistent offsets, and lost/stale messages, along with some potentially surprising aspects of the Kafka transaction protocol.

I am begging the cryptocurrency community to consider alternative ways of knowing, such as "emailing someone to ask them questions instead of speculating in chat" and "submitting a handful of transactions and seeing if they show up".

<sigh> No, Radix folks, Jepsen will not be accepting a follow-up engagement with any Radix-related entities. Y'all can stop suggesting there's going to be a follow-up analysis on Xi'an now. I take my ethical commitments seriously.

They also stressed the importance of end-to-end verification of safety properties, because APIs are how exchanges and users actually interact with DLTs. This is a challenge in traditional databases as well: composition of (e.g.) serializable transactional DBs is nontrivial!.

I'm not sure how widespread this understanding is in the DLT space (still looking for a citation for RDX Works's definition) but the researchers I've talked to were unanimous: losing committed transactions *is* a safety error, even if every validator agrees to throw away data.

Since the release I've had the chance to chat with a handful of analysts working specifically on verification of blockchain/cryptocurrency/DLT systems, and can confirm that they also use the usual distsys sense of "safety property"--namely: "something bad does not happen".

Some helpful and much-better-informed comments from @trianglesphere on tendermint/hotstuff latency, including a nicely drawn Lamport diagram.

RT @trianglesphere: @jepsen_io I'm not at all familiar with Radix DLT, but I've got a bunch of thoughts on consensus algorithms and improvi….

Thing is that none of this is even remotely close to saturating disk or network bandwidth. It's a fresh, empty cluster and request volumes are *tiny*, so like. page cache should be able to hold most if not all of this data. I dunno. Software is a ~rich tapestry~.