RT @MarkRuffalo: Thank you Javier Bardem for your decency, your courage, and love for all human beings. You are a beacon of light, amigo. h….

RT @tamaregev: Our paper is finally out at NatHumBeh! We find different time (info?) scales of integration of ling….

This work was a huge team effort. I had learned A LOT and I had so much fun working with the NeuralGCM core team: @shoyer , @dkochkov1 , @langmore, @pnorgaard314, @singledispatch.

I should stress that there are still A LOT of things to improve (instabilities, generalization) add (other earth system components, missing concepts) so we would love to see people harness the existing code base to further improve models. 20/.

Since NeuralGCM does not rely on existing parameterizations, it has the potential to reduce existing model biasses that are caused by current parameterizations. 19/

We ran NeuralGCM at 2.8 degree grid spacing for 40 years in an AMIP-like config (prescribed SST and Sea-ice extent) and find that NeuralGCM has lower bias compared to the 22 AMIP models we have compared against. 18/

NeuralGCM at 1.4 degree grid spacing can accurately simulate the seasonal cycle and many features of Earth's atmosphere, including emergent phenomena like tropical cyclones (at the correct locations/seasons/numbers). 17/

NeuralGCM models can be accurate even at low resolution (figure above is at 1.4 grid spacing). So they can be accurate but at the same time require orders of magnitude less compute than traditional models (with similar performance). 16/.

NeuralGCM has a lower bias compared to X-SHiELD (a global cloud resolving model) in a one year run. 15/

For longer (climate) time scales:.Although our models are trained on 3-5 day of weather forecast, it can generate much longer simulations which are accurate when compared against existing models. 14/.

For weather forecasting:.NeuralGCM is competitive with existing ML/physics-based models (for deterministic forecasts). NeuralGCM is the first published model that is competitive with ECMWF ensemble model (see paper on how we train deterministic/stochastic models).13/

A side comment for ML people: In the paper we have a discussion on what loss functions we use in order to avoid blurring (a problem which appears in ML models that use mean square error as a loss function and lead to blurry predictions).12/.

Furthermore, even if you don't think that a hybrid approach makes sense, you could still use the differentiable dycore to tune components of your favorite model (e.g., traditional parameterizations; keeping in mind that code needs to be differentiable). 11/.

In my opinion the second feature is especially interesting because it opens the possibility to learn from many types of observational datasets that so far were not used for improving atmospheric/climate models. 10/.

This approach solves the two problems I mentioned above at once:.(a) ML learns to interact with physics during training .(b) We do not need to diagnose (offline) what the ML needs to mimic. This allows us to learn from reanalysis (and in principle from observations!).9/.

This allows us to train NeuralGCM model in "online" (end-to-end) to predict 3-5 days of weather forecast. During the training process, the model runs for 100s of model steps before optimizing weights. 8/.

Instead of keep using offline training, we wrote a differentiable physics-based atmospheric model (a dycore), coupled it to an ML model (which replaces parameterizations). 7/

Another limitation of "offline" training: it is not clear what the ML model should predict in order to improve simulations. This is one of the reasons that hybrid models mostly learned from high-resolution models (and not reanalysis/observations). 6/.

The main limitation of offline training is that the physics-based model and the ML model never interact during training. When these two models are coupled when running the hybrid model, instabilities, inaccuracies and drift can emerge. 5/.

So far, hybrid models were developed in a way that they try to learn "offline" - which in the climate-ML community means that these models are optimized for a single time step and separately from the physics based model (. 4/.