One paper accepted at @NeurIPSConf with @apoorv2904 and @angeloskath on speeding up attention by clustering the queries. The nice thing is that this can be used for inference with standard pre-trained models. @Idiap_ch @unige_en @EPFL_en @snsf_ch Tweet added by François Fleuret @francoisfleuret

François Fleuret

4 years

One paper accepted at @NeurIPSConf with @apoorv2904 and @angeloskath on speeding up attention by clustering the queries. The nice thing is that this can be used for inference with standard pre-trained models. @Idiap_ch @unige_en @EPFL_en @snsf_ch

Fast Transformers with Clustered Attention

Transformers have been proven a successful model for a variety of tasks in sequence modeling. However, computing the attention matrix, which is their key component, has quadratic complexity with...

arxiv.org

110

Replies

Mac Abdul

@MacAbdul9

4 years

@francoisfleuret @NeurIPSConf @apoorv2904 @angeloskath @Idiap_ch @unige_en @EPFL_en @snsf_ch Sounds interesting and something similar to LSH, surely going to read.