LLM Course | Eric Pasewark

Exact topics and schedule subject to change, based on student interests and course discussions. Feel free to let me know if there are some topics or papers you would like to cover.

Readings for a given week should be done prior to the class so we can discuss them. Extra readings are optional, but will help with your understanding.

Date	Topics	Readings	Extra Readings
9/19	Week 1: Course Introduction [course description] [nn intro] Course structure and outline Intro to ML, Neural Nets and Language Modeling
9/26	Week 2: Transformers 1 [discussion questions] Training Transformers	Attention Is All You Need The Illustrated GPT-2 Language Model Overview
10/3	Week 3: Transformers 2 [discussion questions] Transformers Transformer code	Attention Is All You Need The Illustrated GPT-2 Karpathy Transformer-decoder implementation
10/17	Week 4: Capabilities and Instruction Tuning [discussion questions] Language model capabilities Fine-tuning	Language Models are Few-Shot Learners Finetuned Language Models Are Zero-Shot Learners	BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
10/24	Week 5: Fine-tuning [discussion questions] Imitation Scratchpad fine-tuning LIMA Limitations	Alpaca: A Strong, Replicable Instruction-Following Model Show Your Work: Scratchpads for Intermediate Computation with Language Models LIMA: Less Is More for Alignment	Evaluating Large Language Models Trained on Code The False Promise of Imitating Proprietary LLMs
10/31	Week 6: Hardware, Implementation, and Parameter-efficient Methods [discussion questions] Optimizers, Gradient Accumulation, Batch Size, etc. Parallelism Strategies LoRA and other parameter-efficient methods	LoRA: Low-Rank Adaptation of Large Language Models Methods and tools for efficient training on a single GPU Efficient Training on Multiple GPUs	Fine-Tuning LLMs: LoRA or Full-Parameter? An in-depth Analysis with Llama 2 Prefix-Tuning: Optimizing Continuous Prompts for Generation
11/7	Week 7: RLHF and more Fine-tuning [discussion questions] RLHF Data Quality Do we need RL?	Training language models to follow instructions with human feedback Reinforcement Learning for Language Models Direct Preference Optimization: Your Language Model is Secretly a Reward Model	Reinforced Self-Training (ReST) for Language Modeling John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges
11/14	Week 8: Prompting and Agents [discussion questions] Few-shot learning review Chain of thought LLMs as agents	Large Language Models are Zero-Shot Reasoners Generative Agents: Interactive Simulacra of Human Behavior	Chain-of-Thought Prompting Elicits Reasoning in Large Language Models Tree of Thoughts: Deliberate Problem Solving with Large Language Models
11/21	Week 9: Multi-modal Models [discussion questions] Vision-Language Models Other Multi-modal Models	CogVLM: Visual Expert for Pretrained Language Models PaLM-E: An Embodied Multimodal Language Model	PaLI: A Jointly-Scaled Multilingual Language-Image Model Flamingo: a Visual Language Model for Few-Shot Learning
11/28	Week 10: Applications to RL Applications to Exploration Integration with Robotics	Guiding Pretraining in Reinforcement Learning with Large Language Models RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control	Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
12/5	Week 11: Ethics and Societal Impacts [discussion questions] Current Issues: bias, jobs, etc. Potential Future Issues	Ethical and social risks of harm from Language Models An Overview of Catastrophic AI Risks	On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? Identifying and Mitigating the Security Risks of Generative AI How do we know how smart AI systems are? How Rogue AIs may Arise
tbd	Week ?: Scaling Laws and Emergent Abilities Scaling Laws Emergent Abilities	Scaling Laws for Neural Language Models Training Compute-Optimal Large Language Models Emergent Abilities of Large Language Models	Are Emergent Abilities of Large Language Models a Mirage? Are Emergent Abilities in Large Language Models just In-Context Learning?
tbd	Week ?: Retrieval-Augmented Generation Retrieval-Augmented Generation	Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks Atlas: Few-shot Learning with Retrieval Augmented Language Models
tbd	Week ?: Tool Use Methods to Teach Tool Use	Toolformer: Language Models Can Teach Themselves to Use Tools Tool Documentation Enables Zero-Shot Tool-Usage with Large Language Models	TALM: Tool Augmented Language Models ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs
tbd	Week ?: LLMs as Agents Transformer code Training	Voyager: An Open-Ended Embodied Agent with Large Language Models Generative Agents: Interactive Simulacra of Human Behavior	AgentBench: Evaluating LLMs as Agents