Training Compute-Optimal Large Language Models
Showed how model size and training data should scale together under a compute budget.
- from
- Artificial Intelligence
- added
- 2026-10-10
- likes
- 0
similar
-
Scaling Laws for Neural Language Models arxiv.org
Explored relationships between model performance, data, and compute.
-
How large language models need symbolism academic.oup.com
National Science Review, 2025. [All Versions]. Advances in artificial intelligence (AI), particularly large language models (LLMs), have achieved remarkable success. This progress stems from ‘scaling laws’---performance improves with greater computation, data and model size…
-
Program Synthesis with Large Language Models arxiv.org
2021. [All Versions]. This paper explores the limits of the current generation of large language models for program synthesis in general purpose programming languages.
-
Mathematical discoveries from program search with large language models nature.com
Nature, 2024. [All Versions]. Large language models (LLMs) have demonstrated tremendous capabilities in solving complex tasks, from quantitative reasoning to understanding natural language. However, LLMs sometimes suffer from confabulations (or hallucinations), which can result…
-
Scaling up Deep Learning videolectures.net
-
Exploring the Limits of Language Modeling arxiv.org
Nice recursive models using word-level LSTMs on top of a character-level CNN using an overkill amount of GPU power.
Artificial Intelligence › Learn > Foundational papers: “Showed how model size and training data should scale together under a compute budget.”