Do Llamas Work in English? On the Latent Language of Multilingual Transformers
ACL'24, 2024. [All Versions]. A preliminary work empirically showing that the intermediate embeddings of multilingual Transformers (1) start far away from output token embeddings; (2) already allow for decoding a semantically correct next token in the middle layers, but give…
- from
- AGI & CoCoSci
- added
- 2026-10-10
- likes
- 0
similar
-
Llama llama.com
Meta's open source large language model. #opensource
-
From task structures to world models: what do LLMs know? cell.com
Trends in Cognitive Sciences, 2024. [All Versions]. [Preprint]. In what sense does a large language model (LLM) have knowledge? The authors answer by granting LLMs ‘instrumental knowledge’: knowledge gained by using next-word generation as an instrument. The authors then ask…
-
LLaMA ai.meta.com
Llama LLM, a foundational, 65-billion-parameter large language model by Meta. Meta, February 23rd, 2023. #opensource
-
LXMERT: Learning Cross-Modality Encoder Representations from Transformers arxiv.org
, Hao Tan, et al., EMNLP-IJCNLP 2019, Dec 2019.
-
Compositionality and Generalization in Emergent Languages arxiv.org
ACL'20, 2020. [All Versions].
-
PAL: Program-aided Language Models proceedings.mlr.press
ICML'23, 2023. [All Versions]. Paper presenting an approach that uses the LLM to read natural language problems and generate programs as the intermediate reasoning steps, but offloads the solution step to a runtime such as a Python interpreter. With PAL, decomposing the natural…
AGI & CoCoSci › Papers > Concepts > AI Concept Representation: “ACL'24, 2024. [All Versions]. A preliminary work empirically showing that the intermediate embeddings of multilingual Transformers (1) start far away from output token embeddings; (2) already allow for decoding a semantically correct next token in the middle layers, but give…”