From sequences to attention
The 2017 paper Attention Is All You Need proposes the Transformer architecture, using attention mechanisms rather than recurrent and convolutional layers for the tasks described in the study.
Why relationships between words matter
Meaning depends on context. Attention allows a model to calculate relationships between different parts of an input sequence. This is a technical mechanism, not evidence of human understanding.
Read the original research
The paper describes the architecture and specific machine translation experiments. Its results should be considered in the context of those tasks, rather than as a promise that every modern model will be reliable.
This article is a brief editorial introduction to a historical research paper.
