Easily configure shared memory size for CLI jobs with `--docker-shm-size`. Training PyTorch models on a GPU and need more memory? Ever run into "Error: No space left on device"?. Customize Docker shared memory size with `--docker-shm-size`.

🔨 Job setup option for Coiled Batch. Use `--host-setup-script` to configure your VM before your batch job starts. Easily:.✅ Install dependencies.✅ Mount cloud storage.✅ Handle authentication.or any other setup your jobs need.

RT @black_dot_box: @CoiledHQ ~5 years ago I worked at a startup where we had multiple engineers screwing around for months with Terraform,….

RT @mrocklin: Coiled 2024 in Review. It’s the time when companies issue year-end summaries, acclaiming success (or….

Calculating quantiles, a common application in.#geospatial workloads, used to be slow due to GIL contention in NumPy. The new implementation in @dask_dev + @xarray_dev is up to a hundred times faster and scales independently of the number of threads 🥳.

We're big fans of rich for a nice terminal experience, but have found sometimes folks log things even rich can't handle. In the latest coiled=1.67.0 release, coiled logs automatically falls back to non-rich printing in these situations. Release notes:

RT @mrocklin: New Post: Cloud Computing is Broken. Investor asks: "What's next for Data/Cloud Infrastructure?". My….

We're now on Bluesky!. Should be pretty easy to find us, since bluesky lets us use our domain as our handle ☀️.

RT @xarray_dev: Read about the latest improvement to with Dask: Thanks to Patrick Hoefler….

RT @mrocklin: New Post: SLURM-Style Job Arrays on the Cloud. HPC Job scripts were the first form of parallelism I….

RT @nazari_goudin: @CoiledHQ is amazing. If you want to have distributed compute and provisioned infrastructure from the code - its easy a….

RT @qlhoest: New blog post: Scale AI-based Data Processing EASY. The FineWeb-Edu dataset comes from processing 45TB (🤯) of FineWeb. And it….

RT @arpit__bansal: Implemented @CoiledHQ into our product to offload data syncing from BigQuery to Neo4j 🤯. Works like butter 🧈. Now I don’….

RT @mrocklin: We're to build a 100-TB scale geospatial benchmark suite. We've seen an uptick in geospatial users a….

RT @EarthmoverHQ: Arraylake and @CoiledHQ work great together! You can use Coiled to manage your cloud computing infrastructure with @dask_….

Run a Python script on a cloud GPU with one line of code. Training a @PyTorch model training takes ~10 minutes and cost ~$0.12 on the NVIDIA T4 GPU on AWS. Coiled handles provisioning hardware, setting up drivers, and installing CUDA-compiled PyTorch.

RT @dask_dev: Dask DataFrame is now 20x faster. Some of most prominent changes include:.- Apache Arrow support in @pandas_dev .- Better shu….

RT @mrocklin: TPC-H Cloud Benchmarks: Spark, Dask, DuckDB, Polars. Across scales: 10 GiB, 100 GiB, 1 TiB, 10 TiB.Hardware: MBP and AWS. It….

RT @anthonywu: Recommendation of the day: `coiled notebook start` to run a remote Jupyter Lab from big machines in cloud but with file sync….

RT @xhochy: On the 14th of May, @quantcotech Karlsruhe will host the next @PyDataSW Meetup. Florian Jetter will join us to talk about @dask….