These bounds enable us to quantify the interplay between optimization hyperparameters and the tail-index. Doing so, we contribute to the discussion on links between heavy tails and the generalization performance of neural networks. Find our paper here:

💻 We give explicit upper and lower bounds on the tail-index of the resulting parameter distribution and validate these bounds in numerical experiments. 5/6

🧮 In our paper, we analyze a continuous diffusion approximation of SGD and show mathematically (in a regularized linear regression framework) that it leads to an asymptotically heavy-tailed parameter distribution, even though local gradient noise is Gaussian. 4/6

📄 It has repeatedly been observed that loss minimization by stochastic gradient descent (SGD) leads to heavy-tailed distributions of neural network parameters. 3/6

👉 TL;DR: Heavy tailed parameter distributions can emerge from locally Gaussian gradient noise, as we show both theoretically and empirically. 2/6

On my way to represent @Sca_DS at #NeurIPS2024 and to present a poster on joint work with Zhe Jiao on "Emergence of Heavy Tails in Homogenized Stochastic Gradient Descent" 1/6

Please read this. It's literally been 8 years in the writing. The first wave of tech disruption of democracy 2016-2024 is over. What starts now is something much, much worse: the age of information chaos. 1/ https://t.co/bsnKnag51p

Of course

Isaac Newton has verified his email on Google Scholar. And has recently picked up a professorship at MIT. Good for him.

Will be disappointed if the Nobel Prize in literature doesn't go to ChatGPT for being the most prolific author.

nobody: nobel prize committee: the method of least squares was invented to do astrophysics so linear regression is technically physics

even if we agree that hopfield networks and boltzman machines are super-important in ML / DL / NN (I don't), calling them "inventions that *enable* machine learning with neural networks" is really a huge stretch. ML/DL/NN owes a ton to Hinton, but this ain't it.

The paper can be accessed for free until December 2024 at

It was awarded “for extremely interesting results in the framework of the 2-factor Vasicek model using the concept of envelopes of curves and their associated winding number, which help to address several questions of practical interest related to the shape of [...] yield curves”

😁 I am thrilled to announce that my paper "State Space Decomposition and Classification of Term Structure Shapes in the Two-Factor Vasicek Model" with Felix Sachse has won the 2023 Best Paper Award of the International Journal of Theoretical and Applied Finance @ws_ijtaf

To summarize, I can see why some people say that 'negative probabilities are used in financial math'. However, many people (like me) would disagree, because in 99% of cases you want an arbitrage-free model and in the other 1% you would not call Q' a `probability measure'

Another thing is that Q (and Q') are not physical/statistical probability meas in the sense that they tell you 'an event A will happen (or is estimated to happen) with a probability Q(A)' but instead are 'pricing tools' for assets, employed only through taking expectations. 6/7

There is a big problem with such models: If you have arbitrage *and* a linear pricing rule, crazy things happen: You can construct financial contracts, where you get paid $1 million to enter into the contract and get another guaranteed payout of $1 million at expiry. 5/7

Now if you allow for arbitrage (riskless profits) in your market model, then, instead of your probability measure Q you may have a signed measure Q' ('negative probabilities'), such that asset prices are discounted expectations under Q'. 4/7

This probability measure Q is distinct from the statistical measure P, which is used for prediction. The measure Q is a purely 'mathematical' construct for the consistent pricing of options.. 3/7