Effective Approaches to Attention-based Neural Machine Translation
Exploring different approaches to attention mechanisms.
- from
- Education
- added
- 2026-10-10
- likes
- 0
similar
-
Neural Machine Translation and Sequence-to-sequence Models: A Tutorial arxiv.org
Interesting overview of the subject of NMT, I mostly read part 8 about RNNs with attention as a refresher.
-
Neural Machine Translation by Jointly Learning to Align and Translate arxiv.org
Attention mechanism for LSTMs! Mostly, figures and formulas and their explanations revealed to be useful to me. I gave a talk on that paper here.
-
Show, Attend and Tell: Neural Image Caption Generation with Visual Attention arxiv.org
LSTMs' attention mechanisms on CNNs feature maps does wonders.
-
Attention and Augmented Recurrent Neural Networks distill.pub
Interesting for visual animations, it is a nice intro to attention mechanisms as an example.
-
Attention Is All You Need arxiv.org
(AIAYN) - Introducing multi-head self-attention neural networks with positional encoding to do sentence-level NLP without any RNN nor CNN - this paper is a must-read (also see this explanation and this visualization of the paper).
-
Massive Exploration of Neural Machine Translation Architectures arxiv.org
That yields intuition about the boundaries of what works for doing NMT within a framed seq2seq problem formulation.
Education › Papers > Attention Mechanisms: “Exploring different approaches to attention mechanisms.”