The lecture reviews transformer attention mechanisms, introduces memory-efficient variants like group query and sliding window attention, and explores mixture of experts architectures for scalable model capacity. It also covers techniques for adapting large pretrained language models to downstream tasks, including in-context learning, zero-shot learning, and supervised fine-tuning to improve instruction-following capabilities.
The lecture begins with a review of the transformer architecture, focusing on the core components of attention: queries (Q), keys (K), and values (V). The attention mechanism computes inner products between queries and keys, applies a softmax with masking to ensure autoregressive behavior, and then multiplies by values to produce outputs. Multiple attention heads operate in parallel, each with its own set of Q, K, and V matrices, allowing the model to capture diverse relationships. The computational and memory costs of storing keys and values (KV cache) grow linearly with sequence length, limiting batch sizes during inference and causing inefficiencies in GPU utilization.
To address memory constraints, the lecture introduces group query attention, a variant where multiple query heads share a smaller number of key and value groups. This reduces the KV cache size by grouping queries to attend to shared keys and values, balancing memory savings with information retention. The mathematical formulation involves mapping query heads to key groups and normalizing inner products by the square root of the key dimension to maintain stable softmax inputs. This approach, along with sliding window attention—which restricts attention to a fixed-size recent context window—helps reduce computational complexity from quadratic to linear in sequence length, albeit with some trade-offs in long-range dependency modeling.
The discussion then shifts to the MLP (multi-layer perceptron) components within transformers and the concept of mixture of experts (MoE). Instead of a single large MLP, MoE architectures contain many expert subnetworks, with a routing mechanism selecting a subset of experts to activate per token and layer. This design allows models to scale to trillions of parameters while keeping computation manageable by activating only a fraction of experts at inference time. Experts may naturally specialize in different tasks or domains, though explicit specialization is not enforced during training. Routing decisions are made by a small module that scores experts and selects the top candidates, with regularization to ensure balanced expert usage and efficient hardware utilization.
The latter part of the lecture covers methods for adapting large pretrained language models to downstream tasks without retraining the entire model. In-context learning involves providing task examples as part of the input prompt, enabling the model to infer task structure and produce appropriate outputs without parameter updates. Zero-shot learning is an even simpler approach where only a task description is given, and the model generates answers based on its pretrained knowledge. These techniques greatly simplify deployment by eliminating the need for task-specific fine-tuning and allow a single model to serve multiple applications through prompt engineering.
Finally, the lecture discusses supervised fine-tuning (SFT), also known as instruction tuning, which strengthens a model’s ability to follow human instructions by training on datasets of input-output pairs formatted as instructions and responses. This phase involves updating model parameters to better align with desired behaviors, such as generating answers consistent with task descriptions or preferred output formats. While chain-of-thought reasoning can be included in training data, it is typically generated by the model itself rather than explicitly enforced. The lecture concludes with remarks on the complexity of routing in MoE models, the importance of balanced expert utilization, and the practical considerations in fine-tuning large models for specific tasks.