From Wolfenstein to Tesla
Raúl G. Roa Gómez
Disclaimer
who's this guy?
Lead Software Architect
WE MAKE COMPUTERS DO AMAZING THINGS...
PUT STUFF WHERE THEY BELONG
make them understand
Full Stack Developer (10+ years)
C/C++, Python, Rust, MCP, MCAD and MCSD
2D/3D Game Developer (5+ years)
C/C++, UnrealScript, Lua, GLSL, HLSL, Unreal Engine, Unity
Adjunct Lecturer PUCMM (RSTA, STI) (7 years)
Software Engineering, Programming, Data Structures
DCGames (2003 ― 2005)
Game performance metrics for CS 1.6, Unreal Tournament, Quake 3 and Warcraft: Frozen Throne. Biggest video game related site in the Caribbean. Subsequently acquired by Verizon in 2005.
I've done more
what's this talk about?
intelligence
/ɪnˈtɛlɪdʒ(ə)ns/
the ability to acquire and apply knowledge and skills.
INTELLIGENCE
+
Artificial
any device that perceives its environment and takes actions that maximize its chance of success at some goal.
Russell & Norvig (2003)
Artificial Intelligence
Isn't this the same as machine learning?
not quite...
Machine learning
To be clear, machine-learning (ML) is one example of Artificial Intelligence (A.I.) It is a statistical and data-driven approach to creating A.I.
centerpiece of the 4th Industrial Revolution
all around us
- SELF DRIVING CARS
- DRONES
- VOICE/VISUAL ASSISTANTS
- CONSUMER EXPERIENCES ARE PERSONAL
- AUTOMATIC TRANSLATION, INVESTMENTS
ALL AROUND US
wait a minute...
what about games?
In video games, artificial intelligence is used to generate responsive, adaptive or intelligent behaviors primarily in non-player characters (NPCs), similar to human-like intelligence.
Wikipedia (2017)
BUT Before diving any further...
Uncanny valley
the hypothesized relationship between the degree of an object's resemblance to a human being and the emotional response to such an object
MacDorman, K. F.; Chattopadhyay, D. (2016)
The uncanny valley fuels game design
even for realistic looking games...
which makes artificial intelligence in games...
not that intelligent at all
The technical implementation is called: finite state machine.
Games tend to use algorithmic solutions to gimmick human-like behavior.
You want stuff to look somewhat realistic, but not to real, nor to hard.
A. I. in Games
Let's see a practical example
Let's take The castle of wolfenstein for a spin
the castle
bear with me...
Let's add the player...
multiple ray casts...
a SMALL little trick...
a little bit of salt....
And now, finally, the a.i.
This could be improved
BUT WHAT ABOUT THAT MACHINE LEARNING BUZZ?
Reinforcement learning
is an area of machine learning inspired by behaviorist psychology, concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward.
van Otterlo, M.; Wiering, M. (2012)
markov decision Processes
provide a mathematical framework for modeling decision making in situations where outcomes are partly random and partly under the control of a decision maker.
Bellman, R. (1957)
q-learning
Q-learning is a technique that evaluates which action to take based on an action-value function that determines the value of being in a certain state and taking a certain action at that state.
Let's be nerdy for a couple minutes
bellman's equation
UH! OH!
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
Q(1, 5)
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
Rinse and Repeat
http://mnemstudio.org/path-finding-q-learning-tutorial.htm
UPDATED Q-MATRIX
phew!
Let's see it in action!
Further reading
- Asynchronous Methods for Deep Reinforcement Learning
https://goo.gl/v6QWNY
- What’s the Difference Between Artificial Intelligence, Machine Learning, and Deep Learning?
https://goo.gl/3H6vss