Tired of writing the same content-loading code in every project? I've been working on FoldCMS: a type-safe static CMS built with @EffectTS_ with relations support. Comes with an SQLite content store. 👉🏽 Build the CMS, include the sqlite output in your sever, enjoy.

Query-Driven Sync is going to be transformational IMO. It's already changed my perspective on not just modern server state, but also app state. It's bred from so many great ideas and tools that have come before it, built on top of bulletproof patterns and even some of our own

PyPi: https://t.co/afzQyKk3r9 Crates: https://t.co/Sy99Feq8rL GitHub:

I just released Kiru: a text chunking lib, 1000x faster than LangChain splitters. Built in Rust for Python. If you're building RAG systems, you know the basic drill: load documents, split them into chunks, feed them to your vector database. Simple enough, except most chunking

PyPi: https://t.co/Vzj0Y04yQN Crates: https://t.co/090YrbOVFZ GitHub: https://t.co/aHXb1yfPC8 A really cool use case of Rust for Text Chunking 🥳🔥 Check out the repo for more details also couldn’t find the owner on X if anyone does know the account please tag him 😊

😅

@EffectTS_ If you’re building: • Static sites with complex content • Documentation with relations • Blogs that need type safety • Content-heavy apps Give FoldCMS a try. Would love your feedback. RT if you think this could help someone  https://t.co/TxnWxBVXR1

@EffectTS_ The MDX loader is just one of several: - JSON files - YAML files - JSON Lines - YAML streams Or build your own loader for: - Notion - Contentful - Your database - Anywhere Link:

@EffectTS_ Building this taught me: - Effect Streams > Promises for file processing - Type-safe errors are worth the learning curve - Composable pipelines beat monolithic functions Would I rebuild it? Yes. Worth it.

@EffectTS_ Why not just load all files at once? Streams give us: - Memory efficiency (process one at a time) - Backpressure handling (don't overwhelm the system) - Lazy evaluation (only process what's needed) - Easy composition (pipe operations together) For 10k+ files, this matters.

@EffectTS_ Putting it together. Each step is composable and testable.

@EffectTS_ Every step can fail: - File not found - Invalid YAML frontmatter - MDX compilation error - Schema validation error Effect's type system tracks all these.

@EffectTS_ Finally, validate against the user's schema. If validation fails, Effect gives us a detailed error with the exact field that's wrong. Type safety from file → database!

@EffectTS_ Merge frontmatter with compiled output. Now we have everything in one object!

@EffectTS_ If the user wants exports (like metadata, TOC) we extract them from the MDX content.

@EffectTS_ For each file, bundle the MDX with mdx-bundler. Effect.tryPromise converts Promise errors to typed errors.

@EffectTS_ First, stream all .mdx files from the folder. streamFiles is a helper that uses Effect's FileSystem service.

@EffectTS_ The loader returns an Effect Stream. Streams let us process files one-by-one without loading everything into memory.

@EffectTS_ MDX files need: - Frontmatter parsing (YAML) - MDX compilation (with bundler) - Export extraction (for metadata) - Schema validation - Error handling All while streaming files efficiently.

Few days ago I shared FoldCMS. Today I'm showing you how the MDX loader works under the hood. It's actually a great example of @EffectTS_ Streams in action. Let's dive in

@EffectTS_ If you’re building: • Static sites with complex content • Documentation with relations • Blogs that need type safety • Content-heavy apps Give FoldCMS a try. Would love your feedback. RT if you think this could help someone 🚀