The intelligence explosion revisited

Karim Jebari and Joakim Lundborg

Outline


  •  The AI X-risk claim

  • The solution: Friendly AI

  •  Why Friendly AI is a distraction




The AI X-risk claim


  1. AI can be created
  2. Recursive improvement: AI --> ASI 
  3. ASI cannot be controlled
  4. We need Friendly AI


Recursive improvement


Human intelligence is a narrow range


Human intelligence is static

Distinction (1)


Intelligence = the ability to do stuff with your brain 


Techne = The ability to do stuff


Human techne is not static

... and it is not a narrow range



What matters is relative super intelligence


(or "super techne")

Conclusions

  • Human techne can improve very quickly
  • This makes ASI only plausible under the assumption of and intelligence explosion
  • If the human range is not narrow, then such an explosion is less likely

Distinction (2)


Agency = ability of intentional action


= behavior caused by belief/desire


unorthodox definitions


1. Belief: a representation of the world
2. Desire: a set of instructions to act

Friendly AI makes only sense if AI has wide agency


Bacteria are not "friendly"!


Minimal vs. wide agency



Minimal: "desires" are very specific 

Wide: desires are non-specific


Wide agency


Allows humans to dynamically generate instrumental goals in most contexts.


AI is not (always) stupid, its utility function is just too narrow

A wide utility function 


allows generation of new instrumental desires under a wide set of contexts

Agency is different from techne


A narrow agent cannot in its utility function contain the desire to become a wider agent, because that would require that the agent has the desire it wishes to attain.

Conclusions



Creating agent AI requires a concerted effort 

Minimal agent AI/tool AI is the greater risk


Ergo: we should focus effort at those risks



Thank you!

Karim Jebari

jebarikarim@gmail.com
politiskfilosofi.com
twitter.com/karimjebari
Made with Slides.com