AI Foundations — Overview
This section builds modern AI from the ground up: the mathematics that every model rests on, the classical machine learning toolkit and how to evaluate it honestly, and then deep learning from a single neuron up to transformers, generative models, and reinforcement learning, ending with practical PyTorch. Lectures are university-grade but code-first: most contain runnable NumPy or PyTorch.
Roadmap
Section titled “Roadmap”- Linear Algebra for ML: vectors, matrices, norms, eigendecomposition, SVD, and the link to PCA.
- Calculus & Optimization: gradients, Jacobians, Hessians, convexity, GD, SGD, momentum, and Adam.
- Probability Essentials: distributions, Bayes’ rule, expectations, the CLT, the Gaussian, and first entropy.
- Statistics & Estimation: MLE, MAP, bias and variance of estimators, hypothesis tests, confidence intervals.
- Information Theory: entropy, cross-entropy, KL divergence, mutual information.
- Machine Learning Fundamentals: losses, generalization, bias-variance, regularization, cross-validation, metrics, leakage.
- Classical Models: linear and logistic regression, SVM, kNN, naive Bayes, trees, forests, boosting, clustering, PCA.
- Neural Networks from Scratch: perceptron, MLP, activations, universal approximation, NumPy implementation.
- Backpropagation & Autodiff: computational graphs, reverse mode, a micrograd-style engine.
- Training Deep Networks: initialization, normalization, dropout, schedules, mixed precision, debugging, scaling.
- CNNs, RNNs & Sequence Models: convolution, residual networks, LSTM/GRU, vanishing gradients.
- Attention & Transformers: self-attention, multi-head, positional encodings, the block, a PyTorch GPT.
- Generative Models: autoencoders, VAEs, GANs, diffusion, flow matching, energy views.
- Reinforcement Learning Basics: MDPs, Bellman, Q-learning, policy gradients, PPO, RLHF.
- PyTorch in Practice: tensors, autograd, modules, training loops, data, reproducibility, profiling.
Prerequisites
Section titled “Prerequisites”- Programming fluency in Python and comfort with NumPy-style array code.
- High-school to first-year calculus (derivatives, the chain rule, basic integrals). Everything else is built up in the lectures.
- Optional but helpful: any prior exposure to matrices and probability; the first five lectures cover what you need.
Suggested study plan
Section titled “Suggested study plan”Roughly 60 to 90 hours with exercises, which suits 8 to 10 weeks at 6 to 8 hours per week.
| Phase | Lectures | Time | Goal |
|---|---|---|---|
| Math foundations | 1-5 | 2-3 weeks | Fluency with matrices, gradients, distributions, likelihood, KL |
| Classical ML | 6-7 | 1-2 weeks | Evaluate models honestly; strong tabular baselines |
| Deep learning core | 8-10 | 2 weeks | Build and train an MLP and autodiff engine by hand; know the training toolbox |
| Architectures | 11-13 | 2 weeks | CNNs, RNNs, transformers (implement a GPT), generative families |
| RL and practice | 14-15 | 1 week | RL basics and RLHF connection; production-quality PyTorch |
Tips: do the hands-on exercises, especially the from-scratch ones in lectures 8, 9, and 12; skim lectures 1-5 if the math is fresh, returning when a later lecture needs a tool. If your goal is LLMs, the shortest path is 1, 2, 5, 8, 9, 10, 12, 15 and then the NLP and LLM section of this site. This is a fast-moving field: architectural defaults, recipes, and scaling numbers noted as of late 2026 will age, so prefer understanding principles over memorizing specifics.