Nathan VanBenschoten
@natevanben
Followers
411
Following
322
Media
2
Statuses
92
search @turbopuffer, previously distributed system @CockroachDB
New York, NY
Joined April 2013
tpuf ANN v3 can search 100 billion vectors with a p99 of 200ms simplicity scales [v3 in beta, unfiltered search, 1024D, k=10, 92% recall]
7
7
132
Compare-and-swap has an infinite consensus number. It’s universal – powerful enough to solve any consensus problem. That’s why I’m excited about what we built here. It’s compare-and-swap all the way down.
S3’s conditional writes were the key ingredient that made our architecture possible now it's our time to pay it forward
0
0
12
we are looking for a p99 engineer to work on infra/autoscaling/k8s tooling for the 10s of clusters (soon 100s) we run across all 3 clouds. dm me
5
17
135
modern NVMe drives are blazing fast, but software needs to be architected from the ground up to take full advantage of them we recently rolled out a new I/O scheduler that uses Direct I/O to maximize performance of our SSD caches, avoiding page cache cpu and locking overhead
7
41
394
Week one hacking on search @turbopuffer and I’m shocked: the write:read ratio is way higher than OLTP. Ingesting and indexing a constant stream of massive batches is heavy on writes. Reads are less frequent—they just need to be fast.
0
0
19
last week team tpuf welcomed two more p99 database engineers @natevanben and @danhhz ❤️🐡💨
4
2
51
This is even more apparent in CASPaxos ( https://t.co/KikZPDFwpi) from @rystsov.
0
0
0
Sign you’ve spent too much time recently with weak iso levels: when trying to recall why consensus requires two durable writes, the prepare phase of paxos starts to look suspiciously like an optimistic, quorum-replicated SELECT FOR UPDATE lock to avoid lost updates (break safety)
2
1
4
Ever wanna just know if a line of code is being hit? Or how often? Or from where? In prod!? We've been hard at work making that just a click away with https://t.co/YIJsVGr0wd!
0
6
21
I couldn't be more proud of the many teams that contributed to making possible this incredible testament of @DoorDash's success migrating from Postgres to @CockroachDB. Incredible read that highlights so much database goodness and learnings!
careersatdoordash.com
To address the scalability issues in moving to new verticals our team migrated from PostgreSQL to CockroachDB as its new storage engine.
0
6
35
Long-awaited #golang soft memory limit: The core feature has landed, but I still need to land a few new metrics to help support visibility into this. by @mknyswe
https://t.co/7mDbRlIKeX
github.com
Proposal: Soft memory limit Author: Michael Knyszek Summary I propose a new option for tuning the behavior of the Go garbage collector by setting a soft memory limit on the total amount of memory t...
0
8
35
The next step is end-to-end low-latency serverless apps spread across the globe. One combination I'm excited about is @vercel + @prisma + @cockroachdb
0
0
6
We actually just found out that our paper describing the work was accepted to the SIGMOD'22 industry track!
2
0
7
Great thread. Geo latency necessitates a reimagining of infrastructure. Compromising on survivability/correctness of state isn't the answer. My team @CockroachDB spent 2 years addressing this head-on by letting apps express access locality in their schema
cockroachlabs.com
CockroachDB’s multi-region capabilities let you manage data across availability zones and regions, to optimize latency and availability. You can now control data latency and availability with a few...
Latency is increasingly important across variety of use cases. For example, in e-commerce, going beyond single digit millisecond latencies is believed to hurt revenue. But how is this need for speed driving infrastructure changes today? Here's one answer. 🧵 1/
1
2
18
hey cool @CockroachDB is on the top of @GitHub Trending today for some reason!⭐️ https://t.co/aqGCq7gDVj
2
8
46
I'm very excited to share what I've been working on at @datadoghq recently. ✨ Connecting Go Profiling With Tracing https://t.co/RyH6ass7XV Check out the blog post to learn how profiling can fill in the gaps for distributed tracing, and how it works under the hood!
3
28
143
so uh @Twitter banned @CockroachDB on its 7th birthday for being under 13 years old 😢 it's nice to know that they're protecting kids from the horrors of the internet. but even 7 year old businesses are actually made up of real adults it turns out! I mean, most of them anyway.
6
24
281
🪳 Today we're expanding the range of Prisma compatible databases with Preview support for @CockroachDB! Join our "What's new in Prisma" live stream to discuss this new release with @itsaydrian: 🗓 Thu, Feb 3rd, 2022 | 5pm CET 🎟 https://t.co/59fGVnWYCK
https://t.co/N4V49dTxkU
prisma.io
Learn about the Preview release of the Prisma CockroachDB connector and the benefits of using Prisma with CockroachDB.
2
10
42
@CockroachDB is reaching escape velocity! We’ve just raised $278M at a $5B valuation. How did we get here? What's next?🧵
6
24
243