Glitches in the Attention Matrix | Towards Data Science

A history of Transformer artifacts and the latest research on how to fix them