Skip to content

Computer Vision — Overview

This section covers computer vision with an emphasis on techniques that matter in 2025-2026: transformer and self-supervised backbones, promptable segmentation, vision-language models, diffusion and flow-based generation, 3D reconstruction, and the engineering needed to ship systems. It opens with a short classical recap so you know what deep learning replaced and what it did not.

  1. Image Fundamentals & Classical CV Recap: pixels, color, filtering, edges, SIFT, camera geometry, and why deep learning won.
  2. CNN Architectures That Mattered: AlexNet to ResNet, EfficientNet and ConvNeXt, and the design lessons.
  3. Vision Transformers: ViT, DeiT, Swin and hierarchical designs, and ViT versus CNN inductive bias.
  4. Self-Supervised & Foundation Vision Models: SimCLR/MoCo, DINO family, MAE, JEPA, CLIP and SigLIP.
  5. Object Detection: R-CNN, YOLO, DETR and RT-DETR, open-vocabulary detection, mAP/IoU/NMS.
  6. Segmentation: semantic, instance and panoptic; U-Net, Mask R-CNN, Mask2Former, SAM and SAM 2.
  7. Vision-Language Models: LLaVA-style architectures, connectors, resolution, grounding, OCR and evaluation.
  8. Diffusion & Image Generation: DDPM math, latent diffusion, guidance, DiT, flow matching, ControlNet and LoRA.
  9. Video, 3D & Spatial Vision: video transformers, tracking, depth foundation models, NeRF, Gaussian splatting, point clouds, pose.
  10. Efficient & Edge Vision: quantization, distillation, pruning, TensorRT/ONNX, mobile architectures, latency measurement.
  11. Data, Augmentation & Evaluation: datasets, labeling, augmentation, synthetic data, distribution shift, metrics.
  12. Building CV Systems in Practice: workflow, library recipes, fine-tuning, deployment and failure analysis.
  • Python and PyTorch basics; comfortable with NumPy.
  • Linear algebra, calculus and probability at the level of a first ML course.
  • Neural-network fundamentals: CNNs and RNNs and Attention and Transformers. Lectures 2 and 3 assume these; lecture 8 additionally benefits from comfort with Gaussians and ELBO-style derivations.

About 35-50 hours at a steady pace, including exercises.

Week Lectures Focus
1 1, 2, 3 Foundations and backbones; reproduce a ResNet block and a small ViT
2 4, 5, 6 Representations, detection and segmentation; fine-tune a detector or segmenter
3 7, 8 Multimodal and generative models; run a VLM and a diffusion pipeline, implement the DDPM loop
4 9, 10 3D/video and efficiency; benchmark and export a model
5 11, 12 Data, evaluation and an end-to-end mini project

Shortcuts: if your goal is applied work, read 1 (skim), 2, 3, 5, 6, 11, 12 first, then 7, 4, 10 as needed. If your goal is generative models, read 3, 4, 8, then 7 and 9. A strong capstone is lecture 12’s workflow applied to a small dataset of your own: baseline with zero-shot VLM, fine-tune a specialist, quantize and export it, and write a failure analysis.