Generative AI/Large Language Models in
Economics Teaching & Research
Andreas Park
Technology and Work in Economics
Evolution of Tech & Research in Economics
1970
1980
1990
2000
2010
2020
coding languages: C, Pascal, Cobol
symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab,
coding
+
data
Tech
Tool
Implication
faster computations
faster computations
analyze new questions
faster computations
faster and better writing & coding
AI chatbots and such
What made it to the classroom?
1970
1980
1990
2000
2010
2020
coding languages: Python & R
symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab, R
coding
+
data
Tech
Tool
good news bad news?
student lost sense of numbers
coding is too complex for many
use tools without understanding
level the playing field for writing and coding
AI chatbots and such
symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab,
Usage and Working of ChatGPT
How have I used ChatGPT?
- Gave it bullets for an exam answer and let is write a short text.
- Asked it to provide with an outline for a task.
- Asked it to write emails for me based on 2-3 content points.
- Gave it the intro to my paper and asked it to write the abstract.
- Gave it text and asked it to rewrite for clarity & style.
- Asked it to shorten text.
- Asked for short pieces of code to accomplish a task.
- Asked it to create Bibtex entires from a copy-pasted text.
How does ChatGPT work?
- (ChatGPT trained on pre-2022 data)
- not copy-paste information ⇒ unlikely plagiarism.
- extrapolates & predicts the plausible.
- gets things right & badly wrong
- future: reduced when combined with search
If the output generation is the thing that makes students learn, how can we get them to the point where they can assess an output when they can avoid producing the output?
The Problem for Instructors:
- Generative AI completes tasks for students.
- You need to understand the output to assess if it's good.
Difference Calculator to Generative AI
- calculator: deterministic
- Generative AI: probabilistic
Learnings for Homeworks
Homework Questions that need to go the Way of the Dodo
- "Find out x"
- students can just copy-paste the answer from ChatGPT
- "Synthesize the following texts"
- students can load the texts and ask for synthesis
- Explain a concept or discuss a question
- ChatGPT does this
- Case studies
- Can load the study and ask ChatGPT to answer questions.
- Soon: Math and computation questions
- ChatGPT can solve known problems and do computations
- Multiple-choice type questions
- ChatGPT can do most of them and is getting better over time
Big Lessons
- What learning occurs when and how?
- Generative AI: home-learning is harder.
- Assess factual knowledge? Do in-person!
- If in doubt, do not set a homework but switch back to in-class assessments.
Lessons for Surrogate Knowledge vs Real Understanding in Homeworks
Surrogate Knowledge = make students do one task as signal that they understood the real task
Example: in economics, solve mathy problem to signal understanding of the underlying economic mechanism
Tech can do most surrogate tasks for students
Last Words on the Future
Future of Learning, Academia, and Work
- This is totally awesome!
- makes many tasks faster and easier
- levels the playing field
- makes productive people more productive
- This makes it harder for students to learn & us to teach
- We need to put much more thought into design of courses
- NO ONE CAN TEACH AND ASSESS IN 2023 LIKE THEY DID IN 2022
Further reading:
- The AI Snake Oil blog: https://aisnakeoil.substack.com/
- https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf
I'm going to leave you with two pictures
This figure has featured in my intro class to FinTech since 2016
- 70s bank
- Mainframe computers ⇒ employment in banks ↘ dramatically?
- No! Financial sector leveraged new tech!
- Same for Generative AI.
- ⇒ We must teach students how to leverage the tech.
@financeUTM
andreas.park@rotman.utoronto.ca
slides.com/ap248
sites.google.com/site/parkandreas/
youtube.com/user/andreaspark2812/