Criticism of the 80K hours AI safety argument

Points I agree with

  • Timelines are shortening
  • AI systems will have a large impact on many fields
  • The non-power seeking risk
    • AI could worsen war
    • AI could be used to develop dangerous new technology (e.g. biotech)
    • AI could empower totalitarian governments
  • AI governance is an important neglected area

Points I find less convincing

  • AI could "escape"
    • AI system will probably need substantial resources to run - it seems unlikely that it could "make itself run somehwere else" or that "we couldn't turn it off"
  • Real world planning systems can be guarded against far ahead of time
    • Progress is fast in prompt answering DNNs but much slower in planning systems. Go, Atari games etc are special because of 100% fidelity self play.
  • Now is a good time to invest substantial money/resources into technical AI safety research

Title Text

  • Bullet One
  • Bullet Two
  • Bullet Three


By Daniel Bachler