This week’s episode of @drilltodetail features @Captaintobs from @TobikoData on @Airbnb, #DataOps and the engineering innovation behind @SQLMesh

Exciting new feature for @SQLMesh. Custom signals / triggers. SQLMesh's built-in scheduler controls which models are evaluated when the sqlmesh run command is executed. It determines whether to evaluate a model based on whether the model's cron has elapsed since the previous

SQLMesh concepts with plans that apply to different environments (prod, dev) are elegant. Even `fetchdf` is integrated into the CLI. Also, on the right, you see SQLMesh auto-detecting the new columns as non-breaking and simply applying the (virtual) changes `y`.

It surprises me how most poorly most data orchestrators support batch size for backfills Meanwhile, @SQLMesh open source supports out of the box

⚠️⚠️Read this before you start using dbt's microbatch models. There are three large gaps that could lead to serious data issues. Due to fundamental architectural design choices of dbt, the microbatch implementation is very limited. At its core, dbt is a stateless scripting tool

Tired of messy data pipelines? 🥹🫣 Check out the @SQLMesh + @dltHub integration for seamless metadata handovers, faster scaffolding, and incremental processing. 💻 Simplify your data workflows! 🔗 https://t.co/F1B1qSTRlp #DataEngineering #DataPipelines

If you're gonna be in Vegas next week hit me up. We can get coffee or hit the craps table... and talk data :)

The wait is over! You can now use Athena with @SQLMesh. Both Iceberg and Hive are supported but we heavily recommend Iceberg since it's a way better experience. And if you're still stuck on dbt, don't worry, we also have support for the dbt-athena adapter so you can have a

Featuring SQLMesh on this week's episode of Open-Source Spotlight, our series where we're discovering open-source tools. @TobikoData's Toby Mao, @Captaintobs, joined us in demonstrating how this tool can help data team workflows. Watch the demo here:

Join @Captaintobs and @Al_Grigor from @DataTalksClub on their Open-Source Spotlight series, where they talk through the benefits of @SQLMesh like: ♦ Column-level lineage ♦ Environment Management ♦ Instant Prod deployments and much more! https://t.co/SvUDjvygel

If you’re gonna be at Coalesce in Vegas, make sure to come to @TobikoData’s happy hour October 8 at 5pm! Also if you just wanna meet up with me and grab a coffee, I’d love to chat! DM me! https://t.co/UlgyCvPNWM

As a data engineer, you should consider how changes can be done in a non-breaking way. A non-breaking change to a data model is something that won't have any down stream impact, like adding a column or re-ordering columns. Adding columns only impacts down stream models when

Want to learn about running your #dbt project with @SQLMesh? Check out our latest YouTube video to run through a workflow using the classic Jaffle Shop example project! https://t.co/JIZ2sGywn4

Our latest #YouTube #video is up! #learn how to make changes to your #data #models with @SQLMesh plans! #opensource #community #tutorial https://t.co/UR6RhJWFR4

14 years since I started training #bjj, finally got to brown belt! It's been a long journey but I've never quit.

@Captaintobs I re-ran the DDL parsing experiments, comparing sqlglot 11.4.1 to 25.5.1. Parsing success rate improved by 12.8%, and also alignment with pglast improved substantially (up to 17.5%). More details in the charts. Published the experiment code in our repo https://t.co/MyQfFGnxTD

Ralph’s Pizza in Nutley, NJ

I’m gonna be in New Jersey tomorrow for a wedding by Florham Park. Looking to eat some great pizza for lunch but don’t have anyone to go with (lots of places only serve a whole pie). Anyone wanna meet up?

Shikoku 1889 is a robust and beautifully produced game that, well, might not be for everyone? That’s probably because of all the spreadsheets, but we promise there’s a devious GAME under the hood that’s oh-so-satisfying for ‘number enjoyers’. Link below!

Wow I relate to this comment so much! This is basically what happened to me and how I discovered I was allergic to bananas! I thought bananas were just like pineapples (mouth pain / tingling).