RT @kilian_maciej: @mm_wojnar and I have been playing around with tensor alignments in neural networks. here’s a summary of our exploration….

Had a great time working on this with @kilian_maciej! We've been thinking about what the alignments are during training and and built a learning rate scheduler that adapts based on that. Hope this will open up new ways of thinking about training dynamics!.