"SQLite doesn't scale" is a common refrain but that's changed significantly in the last decade with the introduction of the write-ahead log, or WAL. I wrote on the internals of the WAL and how it's a huge leap over the rollback journal: Tweet added by Ben Johnson @benbjohnson

Ben Johnson

2 years

"SQLite doesn't scale" is a common refrain but that's changed significantly in the last decade with the introduction of the write-ahead log, or WAL. I wrote on the internals of the WAL and how it's a huge leap over the rollback journal:

How SQLite Scales Read Concurrency

Understanding how the write-ahead log turns your scaling up to 11.

fly.io

3

35

225

Trent

@trentbillington

2 years

@benbjohnson The magic number link is broken! Enjoying the read so far :)

1

0

Ben Johnson

@benbjohnson

2 years

@trentbillington Ah! I thought checked every link. Thanks for letting me know. I'm fixing it up now. 🙏

0

1

Sean Kelly

@StabbyCutyou

2 years

@benbjohnson I really need to read through some of the new material on SQLite you folks are putting out, the innovation is fascinating. Is it still single threaded? I remember that always being a big limiter to having multiple concurrent clients way back when.

1

0

2

Ben Johnson

@benbjohnson

2 years

@StabbyCutyou No, it works great with multiple threads and multiple processes. The SQLite FAQ says that "threads are evil" but they still support them. :)

2

0

5

Ben Johnson

@benbjohnson

2 years

@sandfoxthat Maybe this should have been the cover art... :)

1

0

2

Ben Johnson

@benbjohnson

2 years

@hyc_symas It seems like LMDB could benefit from a WAL in some workloads though. It would reduce write amplification and it gives you more flexibility with reducing fsync() calls without risking corruption. Right?

1

0

1

Replies