Generative AI/Large Language Models in
Economics Teaching & Research

Andreas Park

 



 

Background

Background

  • Who am I?
    • trained in maths and economics
    • now: Professor of Finance
    • at UTM since 2003
  • Tools of trade in my field:
    • theory: formal mathematical models
    • empirics: data processing and empirical analysis, grounded in theory

Technology and Work in Economics

 Evolution of Tech & Research in Economics

1970

1980

1990

2000

2010

2020

coding languages: C, Pascal, Cobol

symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab, 

coding
+
data

Tech

Tool

Implication

faster computations

faster computations

analyze new questions

faster computations

faster and better writing & coding

AI chatbots and such

What made it to the classroom?

1970

1980

1990

2000

2010

2020

coding languages: Python & R

symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab, R

coding
+
data

Tech

Tool

good news bad news?

student lost sense of numbers

coding is too complex for many

use tools without understanding

level the playing field for writing and coding

AI chatbots and such

symbolic computation and stats software: Maple, Mathematica, SPSS, Stata, Estats, Mathlab, 

Usage and Working of ChatGPT

How have I used ChatGPT?

  • Gave it bullets for an exam answer and let is write a short text.
     
  • Asked it to provide with an outline for a task.
     
  • Asked it to write emails for me based on 2-3 content points.
     
  • Gave it the intro to my paper and asked it to write the abstract.
     
  • Gave it text and asked it to rewrite for clarity & style.
     
  • Asked it to shorten text.
     
  • Asked for short pieces of code to accomplish a task.
     
  • Asked it to create Bibtex entires from a copy-pasted text.

How does ChatGPT work?

  • (ChatGPT trained on pre-2022 data)
  • not copy-paste information \(\Rightarrow\) unlikely plagiarism.
  • extrapolates & predicts the plausible.
  • gets things right & badly wrong
  • future: reduced when combined with search

If the output generation is the thing that makes students learn, how can we get them to the point where they can assess an output when they can avoid producing the output?

The Problem for Instructors:

  1. Generative AI completes tasks for students.
  2. You need to understand the output to assess if it's good.

Difference Calculator to Generative AI

  • calculator: deterministic
  • Generative AI: probabilistic

Learnings for Homeworks

Homework Questions that need to go the Way of the Dodo

  • "Find out x"
    • students can just copy-paste the answer from ChatGPT
  • "Synthesize the following texts"
    • students can load the texts and ask for synthesis
  • Explain a concept or discuss a question
    • ChatGPT does this
  • Case studies
    • Can load the study and ask ChatGPT to answer questions.
  • Soon: Math and computation questions
    • ChatGPT can solve known problems and do computations
  • Multiple-choice type questions
    • ChatGPT can do most of them and is getting better over time

Big Lessons

  1. What learning  occurs when and how?
     
  2. Generative AI: home-learning is harder.
     
  3. Assess factual knowledge? Do in-person!
     
  4. If in doubt, do not set a homework but switch back to in-class assessments.

Lessons for Surrogate Knowledge vs Real Understanding in Homeworks

Surrogate Knowledge = make students do one task as signal that they understood the real task

Example: in economics, solve mathy problem to signal understanding of the underlying economic mechanism

Tech can do most surrogate tasks for students

Lessons about Homework and what to ask

  1. Assume students use it for everything.
  2. Do not test knowledge or understanding. 
  3. You can test knowledge by giving it a false ChatGPT output and ask them to correct it. (Downside: create "confusion" - you never know what sticks).
  4. Set homework tasks and then run in-class tests (good luck aligning with Accessibility though)
  5. On the plus: Because ChatGPT solves simple tasks you can ask students to do more complex design or conceptual tasks.
    • design and implement an algorithm
    • more ambitious papers
    • force brevity and clarity

Last Words on the Future

Future of Learning, Academia, and Work 

  1. This is totally awesome!
    • makes many tasks faster and easier
    • levels the playing field
    • makes productive people more productive
  2. This makes it harder for students to learn & us to teach
    • We need to put much more thought into design of courses
    • NO ONE CAN TEACH AND ASSESS IN 2023 LIKE THEY DID IN 2022

Further reading:

  • The AI Snake Oil blog: https://aisnakeoil.substack.com/
  • https://www.cs.princeton.edu/~arvindn/talks/MIT-STS-AI-snakeoil.pdf

I'm going to leave you with two pictures

This figure has featured in my intro class to FinTech since 2016

  • 70s bank
  • Mainframe computers \(\Rightarrow\) employment in banks  \(\searrow\) dramatically?
  • No! Financial sector leveraged new tech!
  • Same for Generative AI.
  • \(\Rightarrow\) We must teach students how to leverage the tech.

@financeUTM

andreas.park@rotman.utoronto.ca

slides.com/ap248

sites.google.com/site/parkandreas/

youtube.com/user/andreaspark2812/