This is what I was looking for. I derived this independently and couldn’t work out why I hadn’t seen it before. I think this is a profoundly beautiful and important result. Learning is isomorphic to statistical mechanics!
4
1
22
Replies
From here:
1
0
9
@IAmDougLewis
To find the best weights for a neural network you can use an essentially identical algorithm to how we find the most likely positions a group of atoms will be in at a given temperature. This has already been discovered empirically, ie adding momentum to learning algorithms
0
0
3
@TimothyDuignan
I learned similar correspondence in the following lecture series, (lecture note 1-4)
1
0
1
@TimothyDuignan
Yes, actually the modern statistical machine learning was built up by physicists who studied statistical mechanics. This is the reason why many terms in machine learning are related to physics, e.g. Boltzmann machine.
0
0
1