Criticism of the 80K hours AI safety argument
Points I agree with
- Timelines are shortening
- AI systems will have a large impact on many fields
- The non-power seeking risk
- AI could worsen war
- AI could be used to develop dangerous new technology (e.g. biotech)
- AI could empower totalitarian governments
- AI governance is an important neglected area
Points I find less convincing
- AI could "escape"
- AI system will probably need substantial resources to run - it seems unlikely that it could "make itself run somehwere else" or that "we couldn't turn it off"
- Real world planning systems can be guarded against far ahead of time
- Progress is fast in prompt answering DNNs but much slower in planning systems. Go, Atari games etc are special because of 100% fidelity self play.
- Now is a good time to invest substantial money/resources into technical AI safety research
- Bullet One
- Bullet Two
- Bullet Three
By Daniel Bachler