Progress Dashboard

Track the development of "Predicting the Next Word" - a comprehensive textbook on language modeling.

54%
Overall Progress
7
Chapters Complete
0
In Progress
6
Planned
196
Figures Created

Chapter Progress

1
Introduction: The Problem of Prediction
Foundations
Complete
28
Download
2
N-gram Language Models
Foundations
Complete
27
Download
3
Tokenization
Foundations
Complete
26
Download
4
Word Embeddings
Foundations
Complete
28
Download
5
RNNs and LSTMs
Neural Language Models
Complete
28
Download
6
Transformers
Neural Language Models
Complete
32
Download
7
Decoding Strategies
Neural Language Models
Complete
27
Download
8
Training Language Models
Neural Language Models
Planned
27
Coming soon
9
Large Language Models
Large Language Models
Planned
32
Coming soon
10
Scaling Laws
Large Language Models
Planned
28
Coming soon
11
Post-Training
Large Language Models
Planned
27
Coming soon
12
Efficient Language Models
Efficiency and Applications
Planned
27
Coming soon
13
Applications
Efficiency and Applications
Planned
27
Coming soon

Milestones

Part I: Foundations Complete
Chapters 1-4 complete with 109 figures
Part II: Neural LMs (75% Complete)
Chapters 5-7 complete (RNNs, Transformers, Decoding)
Chapter 8: Training Language Models
Next chapter - optimization at scale
Part III: Large Language Models
LLMs, Scaling Laws, Post-Training
Book Complete
All 13 chapters, 364 figures, ready for publication

Recent Updates

December 2024

Chapter 7 (Decoding Strategies) completed with 27 figures including 6 3D visualizations

December 2024

Chapter 6 (Transformers) completed with 32 figures covering attention mechanisms

December 2024

Chapters 3-5 completed: Tokenization, Word Embeddings, RNNs/LSTMs

November 2024

Chapters 1-2 (Introduction, N-grams) finalized with 55 figures total

(c) Joerg Osterrieder 2025

(c) Joerg Osterrieder 2025