Week 12: Research Frontiers
Open problems, future directions, and cutting-edge research
Week 12 of 12
Learning Objectives
- Define embodied agent, generative agent, and world model
- Explain key open research problems in agent AI
- Identify research opportunities in specific domains
- Compare different approaches to agent safety and alignment
- Assess feasibility and impact of proposed research directions
- Design a research proposal for advancing agent capabilities
Topics Covered
- Field evolution timeline (2022-2025)
- Open research problems (planning, world models, memory)
- Agent safety challenges (alignment, scalable oversight)
- World models and embodied agents
- Generative agents for simulated societies
Resources
Jupyter Notebooks
Required Readings
| Paper | Authors | Year | Link |
|---|---|---|---|
| Voyager: An Open-Ended Embodied Agent | Wang et al. | 2023 | arXiv |
| Generative Agents: Interactive Simulacra | Park et al. | 2023 | arXiv |
| Constitutional AI: Harmlessness from AI Feedback | Bai et al. | 2022 | arXiv |
Reading Guide: Research Frontiers
Exploration of Generative Agents, Voyager, and open research problems
Primary Paper
Generative Agents: Interactive Simulacra of Human Behavior
Park, J. S., O'Brien, J. C., et al. (2023)
arXiv arXiv
Park, J. S., O'Brien, J. C., et al. (2023)
arXiv arXiv
Secondary Papers
Exercise: Research Proposal
Develop a research proposal for agent advancement
Learning Objectives
- Create: Design novel agent approach
- Evaluate: Critically assess current limitations
- Synthesize: Propose research directions
Tasks
| Task | Points | Description |
|---|---|---|
| Literature Review | 30 | Survey current research |
| Research Proposal | 45 | Propose novel contribution |
| Feasibility Analysis | 25 | Assess implementation feasibility |
Key Concepts
Field Evolution:
- 2022: Foundation (CoT prompting, RLHF alignment)
- 2023: Emergence (ReAct, Reflexion, Generative Agents)
- 2024: Production (Claude Computer Use, GraphRAG)
- 2025+: World models, embodied agents, long-horizon planning
Open Research Problems:
- Capability Gaps: Long-horizon planning, world modeling, compositional generalization
- Safety/Alignment: Scalable oversight, goal stability, corrigibility
- Infrastructure: Evaluation benchmarks, scalable memory
Generative Agents: Simulated personas maintaining identity, memories, plans, and relationships. Enable computational social science experiments.
World Models: Internal representations of environment dynamics enabling mental simulation before acting.
Final Project Presentations
Students present their research projects on agentic AI topics.
Exercise
Write a research proposal for an open problem in agentic AI:
- Identify a meaningful, tractable problem
- Review related work and current approaches
- Propose a novel approach with clear methodology
- Design evaluation methodology and success metrics
- Assess feasibility and potential impact
Discussion Questions
- What are the most important open problems in agentic AI?
- How do we ensure agent safety as capabilities scale?
- What governance structures are needed for autonomous agents?
- Will agents with persistent world models emerge in the next 5 years?
- What role will multi-agent emergence play in future systems?
Final Project Directions
- Novel agent architecture for a specific domain
- Improved evaluation methodology for agent systems
- Safety or alignment technique for autonomous agents
- Multi-agent coordination mechanism
Additional Resources
Discussion & Questions
Join the Conversation
Have questions about this week's material? Want to discuss concepts with fellow students?