Planning where we focus at CaML requires forming views on many controversial questions. In many cases, people we've talked to have wildly different perceptions of the balance of opinions, so we thought this would be a great way for us and others to know where we're out of step on these issues. Also feel free to tell us if you think the questions are ambiguous or embed false assumptions.
The questions (voting in comments):
- Robust alignment requires alignment-relevant intervention during pretraining
- AI alignment to humans will in practice avoid moral catastrophes to animals
- AI alignment to humans will in practice avoid moral catastrophes to digital minds
- Research into digital mind suffering is sufficiently tractable to work on
- Partially aligned transformative AIs are likely to be stable under reflection
- Alignment to specific values is underrated in research relative to control
- Multipolar worlds will compete away >90% of net value that would otherwise be preserved
source & further reading
lesswrong.com — original article
Toward a Kantian refutation of Agent Foundations
Can public chat data predict real-world AI misalignments?
Rational Agentic Maximalist Philosophies