Attention Is All You Need
(AIAYN) - Introducing multi-head self-attention neural networks with positional encoding to do sentence-level NLP without any RNN nor CNN - this paper is a must-read (also see this explanation and this visualization of the paper).
- from
- Artificial Intelligence, Education
- added
- 2026-10-10
- likes
- 0
similar
-
The Annotated Transformer nlp.seas.harvard.edu
Good for understanding the "Attention Is All You Need" (AIAYN) paper.
-
Attention and Augmented Recurrent Neural Networks distill.pub
Interesting for visual animations, it is a nice intro to attention mechanisms as an example.
-
Effective Approaches to Attention-based Neural Machine Translation arxiv.org
Exploring different approaches to attention mechanisms.
-
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention arxiv.org
LSTMs' attention mechanisms on CNNs feature maps does wonders.
-
Attention is not --not-- Explanation arxiv.org
-
Matching Networks for One Shot Learning arxiv.org
Interesting way of doing one-shot learning with low-data by using an attention mechanism and a query to compare an image to other images for classification.
Artificial Intelligence › Learn > Foundational papers: “Introduced the Transformer architecture.”
Education › Papers > Attention Mechanisms: “(AIAYN) - Introducing multi-head self-attention neural networks with positional encoding to do sentence-level NLP without any RNN nor CNN - this paper is a must-read (also see this explanation and this visualization of the paper).”