Week 4: Planning and Reasoning
Reflexion, LATS, hierarchical planning, memory
Week 4 of 12
Learning Objectives
- Implement verbal reflection mechanisms
- Build self-improving agents
- Design effective memory systems
Topics Covered
- Reflexion framework
- Language Agent Tree Search (LATS)
- Plan-and-Solve prompting
- Episodic memory for agents
Resources
Jupyter Notebooks
Required Readings
| Paper | Authors | Year | Link |
|---|---|---|---|
| Reflexion: Language Agents with Verbal Reinforcement Learning | Shinn et al. | 2023 | arXiv |
| LATS: Language Agent Tree Search | Zhou et al. | 2024 | arXiv |
Reading Guide: Reflexion Paper
Deep dive into verbal reinforcement learning and self-reflection in agents
Primary Paper
Reflexion: Language Agents with Verbal Reinforcement Learning
Shinn, N., Cassano, F., et al. (2023)
NeurIPS 2023 arXiv
Shinn, N., Cassano, F., et al. (2023)
NeurIPS 2023 arXiv
Exercise: Planning Agent
Build an agent with planning and reflection capabilities
Learning Objectives
- Create: Implement verbal reflection mechanisms
- Apply: Build self-improving agents
- Analyze: Evaluate planning strategies
Tasks
| Task | Points | Description |
|---|---|---|
| Reflexion Implementation | 40 | Implement the Reflexion framework |
| Memory System | 30 | Build episodic memory for agent |
| Evaluation | 30 | Measure improvement over iterations |
Exercise
Implement a Reflexion agent that:
- Attempts coding problems
- Reflects on failures
- Improves based on self-generated feedback
- Tracks learning across attempts
Discussion Questions
- How does verbal reflection compare to gradient-based learning?
- What makes a good reflection prompt?
- When does Reflexion help vs. hurt performance?
Additional Resources
Discussion & Questions
Join the Conversation
Have questions about this week's material? Want to discuss concepts with fellow students?