The lecture covers diffusion models and their noise prediction training approach, then explores foundation models emphasizing large-scale pre-training and adaptation techniques like linear probing, fine-tuning, and the hybrid LP-FT method for improved performance. It concludes with Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning strategy that enables scalable and resource-friendly customization of large models without extensive computational overhead.
The lecture begins with a detailed review of diffusion models, focusing on the forward noising process and the reverse denoising process parameterized by neural networks. The instructor revisits the mathematical formulation, explaining how the loss function is derived from the evidence lower bound (ELBO) and involves comparing Gaussian distributions at each timestep. A key insight is that the training objective can be reframed as a noise prediction problem, where the neural network learns to predict the noise added at each step, simplifying the implementation and improving training efficiency. The lecture also touches on practical considerations such as parameterization tricks and the iterative sampling procedure used to generate data from the trained diffusion model.
Transitioning from diffusion models, the lecture introduces the concept of foundation models, a paradigm shift in machine learning characterized by large-scale pre-training on massive, diverse, and often unlabelled datasets. The instructor highlights the two-phase process of foundation models: pre-training on broad data to build a general-purpose foundation, followed by adaptation to specific downstream tasks. This approach contrasts with traditional task-specific training and enables zero-shot or few-shot learning, where models can perform new tasks with little or no additional labeled data. The discussion emphasizes the evolution of foundation models and their growing capability to handle multimodal data, including text, images, and videos.
The lecture then delves into adaptation techniques for foundation models, focusing on representation learning. Here, a model is trained to map input data into a high-dimensional vector space (embedding), capturing semantic information useful for various tasks. A common adaptation method is linear probing, where a simple linear classifier is trained on top of fixed embeddings to solve downstream tasks efficiently. This approach leverages the rich representations learned during pre-training, enabling better performance than training linear models directly on raw input features. The instructor also contrasts linear probing with fine-tuning, where both the representation model and the classifier are jointly optimized, often leading to improved results due to better alignment between features and task objectives.
Further, the lecture discusses the importance of initialization in fine-tuning large models. Starting from a pre-trained foundation model provides a favorable initialization that guides optimization towards better-performing solutions, as opposed to training from scratch. The instructor introduces a hybrid approach called LP-FT (Linear Probe followed by Fine-Tuning), which first optimizes a linear head on fixed embeddings before jointly fine-tuning both the head and the representation. This staged training often yields better performance and stability compared to directly fine-tuning all parameters from random initialization. The discussion underscores the complexity of optimization landscapes in large models and the critical role of pre-training in navigating them effectively.
Finally, the lecture covers Low-Rank Adaptation (LoRA), a parameter-efficient fine-tuning technique designed to adapt large models with limited computational resources. Instead of updating all model parameters, LoRA restricts updates to low-rank matrices added to the original weights, significantly reducing the number of trainable parameters. While this approach does not drastically reduce inference computation or memory usage, it offers practical benefits such as enabling multiple users to share a base model while maintaining personalized adaptations with minimal overhead. LoRA facilitates scalable deployment of foundation models by allowing efficient storage, fast switching between user-specific adaptations, and reduced training costs, making it a popular choice in real-world applications.