Looking forward to @NeurIPSConf next week! Feel free to reach out to chat about physical dynamics, surrogate modeling, hybrid ML-numerical methods, or large-scale training! Also seeking burrito recs...

Partial observability is a key challenge in predicting physical systems, where only part of the state is observed. Check out our poster #2213 at #neurips2025 on Thu, Dec 4, 4:30pm! We propose a multiscale inference scheme for diffusion models to better predict these systems.

@PolymathicAI 11/ This was a massive collaborative effort that wouldn't have been possible without support from @FlatironInst , @schmidtsciences, and @nvidia for through the NAIRR program. We hope you all enjoy World Walrus Day (11/24) and are looking forward to seeing what Walrus unleashes!

@PolymathicAI 10/ We're super excited about these results, but also want to speak to the biggest current limitations. - Walrus is still trained with a grid-based encoder/decoder. - Even strong eye-norm results still require scientific validation.

@PolymathicAI 9/ This push for physical diversity matters. We show that even though specialization seems to help on pretraining metrics, heavy augmentation favoring representational diversity is key to strong downstream results.

@PolymathicAI 8/ These techniques let us train a single model on 19 physical settings, outperforming the landmark foundation models in the space both on 8 downstream transfer tasks and showing consistent performance across simulations drawn from multiple domains.

@PolymathicAI 7/ Co-designed Training strategy FSDP is an amazing tool, but the syncs at AllGather ops can result in huge deadweight for heterogeneous loads. We co-design our sampling and distribution strategies to limit the impact these differences have resulting in 262% higher throughput.

@PolymathicAI 6/ Joint 2D/3D Augmentation When we train, we want to learn general patterns, but the model wants to learn short-cuts. This makes it difficult to learn from 2D and 3D. We employ a new tensor-aware augmentation strategy that forces the model to share information across spaces.

@PolymathicAI 5/ Adaptive Compute Fixed resolution becomes a huge obstacle handling this level of diversity. Big patches help us handle big data, but hurt performance on coarse data. But what if we dynamically choose our internal resolution instead using convolutional stride modulation?

@PolymathicAI 4/ Patch jittering Physical models shouldn't blow up, right? One reason this happens is the loss of equivariance from resampling. We mitigated this by making randomizing resampling with a simple inference-time augmentation that improves long run metrics on 17/19 PT systems.

@PolymathicAI 3/ Scale is a big part, but not the only part. This is super heterogeneous data. We can't just throw it at a model and expect it to work well. We need to find new approaches for handling some of the biggest challenges in this space!

@PolymathicAI 2/ Walrus sets new standards for scale, accuracy, and pretraining diversity in foundation models for physical dynamics. It's a 1.3B parameter model trained on 19 scenarios across fluids, plasmas, acoustics, viscoelastics, and more utilizing 63 different fields in both 2D and 3D.

1/ Today with my colleagues @PolymathicAI, I'm excited to release our latest project, Walrus, a cross-domain foundation model for physical dynamics, into the world. https://t.co/ihv1MZGQM3 Paper: https://t.co/d6ah9LO4ud Git: https://t.co/s3p8qGhZQR HF: https://t.co/RufaBD9eJk

Awesome work by @FrancoisRozet from his time @PolymathicAI! Some great insight into the compression behavior of LDMs for physical dynamics plus some cool tricks for boosting performance!

CDS Research Scientist @cosmo_shirley, with @oharub & @mikemccabe210, led the creation of "The Well"—a 15TB dataset for physics-based AI. The Well offers simulations from biological systems to neutron stars, helping AI models learn physical principles. https://t.co/AmPIUElueJ

Heading off to #NeurIPS2024 today! Feel free to reach out if you're around to catch up! If you're interested, please visit our poster sessions tomorrow at 11 PST for MPP ( https://t.co/JbOmq9ruXs) and on Thursday at 11 for the Well ( https://t.co/L3rPzlFXoT).

This was a huge collaborative effort with too many contributors to list in one post, but it wouldn't have been possible without all of our teammates @PolymathicAI and elsewhere. We hope that access to more realistic problems pushes the development of #AI4Science forward!

and many more! All data in the Well is stored in a uniform format and whether its 2D or 3D it's accessible through the same API. The Well additionally includes benchmarking tools such as baseline models, tensor-aware augmentations, and metric implementations:

To complex newly understood non-Newtonian turbulent structures:

to orientation-dependent forcings in biological systems: