Dr. Stefan Hackstein
How do Chatbots Think?
Hitler + Italy - Germany = ?
How do chatbots think?
How do chatbots think?
Same words, different meanings
"difficult"
"heavy"
modern chatbots understand meaning in context:
- heavy tasks (difficulty)
- heavy stones (weight)
How do chatbots think?
Same words, different meanings
How do chatbots think?
Same meaning, different words
I go ...
I direct my steps forward ...
indeterminate
poetic
How do chatbots think?
Meaning
How do chatbots think?
- Input is reduced to a meaning vector
Meaning
Input
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
Meaning
Input
Response
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
... response becomes part of the input ...
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
... response becomes part of the input ...
... direction adjusts to the response ...
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
... response becomes part of the input ...
... direction adjusts to the response ...
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
... response becomes part of the input ...
... direction adjusts to the response ...
... response drifts off ...
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
... response becomes part of the input ...
... direction adjusts to the response ...
... response drifts off ...
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
How do chatbots think?
- Input is reduced to a meaning vector
- Response should point in the same direction
- randomly word for word in the appropriate direction
Meaning
Input
Response
Control with explicit instruction
Chat fallen over? -> new chat
How do chatbots think?
36 + 59 = ?
How do chatbots think?
36 + 59 = ?
How do chatbots think?
36 + 59 = ?
How do chatbots think?
36 + 59 = ?
How do chatbots think?
How do chatbots think?
- Thinking = Writing
- Strength: Entire thought process transparent
- Weakness: No reflection, strong bias
- Helpful: First discussion, then response
- Harmful: First response, then discussion
- Good practice: Vary input, compare output
- think for yourself