Skip to content

AI Foundations — Overview

This section builds modern AI from the ground up: the mathematics that every model rests on, the classical machine learning toolkit and how to evaluate it honestly, and then deep learning from a single neuron up to transformers, generative models, and reinforcement learning, ending with practical PyTorch. Lectures are university-grade but code-first: most contain runnable NumPy or PyTorch.

  1. Linear Algebra for ML: vectors, matrices, norms, eigendecomposition, SVD, and the link to PCA.
  2. Calculus & Optimization: gradients, Jacobians, Hessians, convexity, GD, SGD, momentum, and Adam.
  3. Probability Essentials: distributions, Bayes’ rule, expectations, the CLT, the Gaussian, and first entropy.
  4. Statistics & Estimation: MLE, MAP, bias and variance of estimators, hypothesis tests, confidence intervals.
  5. Information Theory: entropy, cross-entropy, KL divergence, mutual information.
  6. Machine Learning Fundamentals: losses, generalization, bias-variance, regularization, cross-validation, metrics, leakage.
  7. Classical Models: linear and logistic regression, SVM, kNN, naive Bayes, trees, forests, boosting, clustering, PCA.
  8. Neural Networks from Scratch: perceptron, MLP, activations, universal approximation, NumPy implementation.
  9. Backpropagation & Autodiff: computational graphs, reverse mode, a micrograd-style engine.
  10. Training Deep Networks: initialization, normalization, dropout, schedules, mixed precision, debugging, scaling.
  11. CNNs, RNNs & Sequence Models: convolution, residual networks, LSTM/GRU, vanishing gradients.
  12. Attention & Transformers: self-attention, multi-head, positional encodings, the block, a PyTorch GPT.
  13. Generative Models: autoencoders, VAEs, GANs, diffusion, flow matching, energy views.
  14. Reinforcement Learning Basics: MDPs, Bellman, Q-learning, policy gradients, PPO, RLHF.
  15. PyTorch in Practice: tensors, autograd, modules, training loops, data, reproducibility, profiling.
  • Programming fluency in Python and comfort with NumPy-style array code.
  • High-school to first-year calculus (derivatives, the chain rule, basic integrals). Everything else is built up in the lectures.
  • Optional but helpful: any prior exposure to matrices and probability; the first five lectures cover what you need.

Roughly 60 to 90 hours with exercises, which suits 8 to 10 weeks at 6 to 8 hours per week.

Phase Lectures Time Goal
Math foundations 1-5 2-3 weeks Fluency with matrices, gradients, distributions, likelihood, KL
Classical ML 6-7 1-2 weeks Evaluate models honestly; strong tabular baselines
Deep learning core 8-10 2 weeks Build and train an MLP and autodiff engine by hand; know the training toolbox
Architectures 11-13 2 weeks CNNs, RNNs, transformers (implement a GPT), generative families
RL and practice 14-15 1 week RL basics and RLHF connection; production-quality PyTorch

Tips: do the hands-on exercises, especially the from-scratch ones in lectures 8, 9, and 12; skim lectures 1-5 if the math is fresh, returning when a later lecture needs a tool. If your goal is LLMs, the shortest path is 1, 2, 5, 8, 9, 10, 12, 15 and then the NLP and LLM section of this site. This is a fast-moving field: architectural defaults, recipes, and scaling numbers noted as of late 2026 will age, so prefer understanding principles over memorizing specifics.