Course Topics
12 comprehensive modules from statistical language models to modern transformers
Part 1: Foundations
Statistical models and neural representations of language
N-grams & Language Models
Statistical foundations of NLP: probability chains, Markov assumptions, smoothing techniques, and perplexity evaluation.
Word2Vec & Embeddings
Dense word representations, Skip-gram and CBOW architectures, semantic geometry in vector space.
RNN & LSTM Networks
Sequential processing, vanishing gradients, and memory mechanisms with gated architectures.
Part 2: Core Architectures
Attention mechanisms and transformer models
Seq2Seq & Attention
Encoder-decoder models, attention mechanisms, and alignment in neural machine translation.
Transformers
Self-attention, multi-head attention, positional encoding, and parallel processing.
BERT & Pre-training
Pre-trained language models, masked language modeling, and transfer learning pipelines.
Part 3: Advanced Topics
Scaling, tokenization, and text generation
Scaling Laws
Compute-optimal training, Chinchilla scaling, and emergent capabilities in large models.
BPE & Tokenization
Subword tokenization, Byte-Pair Encoding, and vocabulary optimization strategies.
Text Generation & Decoding
Greedy, beam search, sampling strategies, temperature, and nucleus sampling.
Part 4: Applications
Fine-tuning, efficiency, and responsible AI
Fine-tuning & LoRA
Parameter-efficient fine-tuning, adapters, LoRA, and prompt engineering techniques.
Efficiency & Quantization
Model compression, quantization, distillation, pruning, and deployment optimization.
Ethics & Bias
Bias detection, fairness metrics, WEAT tests, and responsible AI development.
Resources
Moodle HS25
Course schedule, lecture PDFs, Jupyter notebooks, and assignment information.
Chart Gallery
120+ professional visualizations covering all NLP concepts with filtering by topic.
Deep-Dive Modules
Specialized content on summarization, sentiment analysis, and embeddings.
GitHub Repository
Complete source code, slides, notebooks, and all course materials.