Spiral
@SpiralDB
Followers
670
Following
104
Media
5
Statuses
47
Multimodal warehousing that works with the tools you love. “Storage Packed In Recursive Arrays & Layers”
NYC & London
Joined March 2023
The talk on @SpiralDB at @CMUDB : https://t.co/6mRfsnDZiP is a great one. I think it would also be interesting to hear a counterpoint about @ApacheParquet that explains actual technical details of that format, the Cathedral vs Bizzaar management, options with Metadata, etc
1
15
112
Today's Future Data Systems Seminar Speaker: Will Manning (@_willmanning) will present @SpiralDB's Vortex file format (@vortexdotdev). Vortex is now a @LFAIDataFdn project. Zoom talk open to public at 4:30pm ET. YouTube video available after:
db.cs.cmu.edu
Apache Parquet revolutionized columnar storage after its initial release in 2013, but... Read More +
0
8
41
Their moat is building really awesome stuff.
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
1
3
21
Are you keeping your GPUs fed?
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
0
1
10
A day in early 2023: @Robert3005: We don't do this a lot, so this is, like, a really huge deal. @ngates_: We wanna invite you to have lunch with us every day. me: Oh, it's ok, I've kind of just left London. @_willmanning: Coolness. So we'll see you in a week.
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
1
3
14
So pumped to back @_willmanning @SpiralDB building data infrastructure for machine scale!
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
1
2
10
"What sets them [Spiral’s founders] apart: deep technical rigor paired with respect for academic research; relentless engineering execution balanced with creative problem solving, and a true obsession with customer needs." 💯 spiraldb dot com /careers
AI teams need a better data platform. Excited to support @SpiralDB's launch today and lead their seed: https://t.co/vvgniOhj9G
0
3
12
Congrats @_willmanning and team! We already took Vortex for a spin and had a great experience! There might more on this announced soon - sth sth dataloader! 👀
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
1
3
14
This is the craziest why I invested story you’ve ever read. So read it for the drama and to discover the database you need to saturate your GPUs, handle multimodal data without headaches, and build the genAI app you should be building. https://t.co/Xx2DtJDLWs
amplifypartners.com
Spiral is building the multimodal data platform AI teams really need.
4
4
30
8/ The gap between AI leaders and laggards is widening. The enterprises that get their data AI-ready today will have an insurmountable advantage tomorrow. https://t.co/CuOitUDggh
spiraldb.com
Data 3.0, backed by the best
0
0
22
7/ Backed by $22M from Amplify Partners & General Catalyst. Working with teams in computer vision, document intelligence, and multimodal AI. If you're spending >10% of your time on data infrastructure instead of model development, we should talk.
1
0
22
6/ Real talk: OpenAI and Anthropic aren't using traditional data warehouses. They built custom infrastructure because they had to. We're making that same capability accessible to everyone.
1
1
24
5/ We call it the "Third Age" of data: • First Age: Human inputs → Human outputs (Postgres era) • Second Age: Machine inputs → Human outputs (Big Data era) • Third Age: Machine inputs → Machine outputs (AI era) Your infrastructure needs to evolve.
1
1
21
4/ Spiral is our database built on Vortex: • Direct S3 → GPU data loading (skip the CPU bottleneck) • One API for 10KB embeddings AND 4GB videos • Time-bounded, audited permissions (no more credentials passed to AI agents) • Actually saturates your GPUs
1
0
19
3/ We built Vortex—a new columnar format that's 10-20x faster than Parquet for scans, 100-200x faster for random access. Microsoft, Snowflake, and Palantir are already backing it. We donated it to the Linux Foundation because infrastructure should be open.
1
1
40
2/ The problem: Your $40k/month H100 sits idle 70% of the time. Not because you lack data, but because your infrastructure can't feed it fast enough. Reading 4M images from S3? That's 55 HOURS of network overhead. For one second of GPU compute.
2
0
18
We're building the data infrastructure that AI actually needs. Current systems were built for humans reading dashboards. But an H100 can consume 4 million images per second. The future isn't human-scale. It's machine-scale. Introducing Spiral: Data 3.0 🌀 1/8
13
38
387
do checkout @vortexdotdev
@valyala @pauldix @andrewlamb1111 You'd also probably want to do some form of compaction. And check out @vortexdotdev
0
2
8
❤️
Go check out our latest post, sharing new developments from the past month 🗓️💻☕️ https://t.co/PmltbiDWXJ
0
0
8
Next, @maxikuschewski from the Technical University of Munich discusses why the cloud needs better building blocks. Services that are commoditized, leverage modern hardware, and work seamlessly across multiple providers.
1
3
13