Impacts of GenAI on Academic Information Discovery & Literacy

The role of librarians:

Two key parts of the work I do are

  1. Connecting patrons to resources that meet their information needs
  2. Supporting the critical appraisal of these resources

Today, I'd like to talk to you about 

  • The search & critical appraisal processes
  • How AI and GenAI have impacted these processes
  • What to consider when developing information literacy (finding & using information) in the current landscape

First: Searching

The Library Catalogue 

Databases

But also...
 

It depends on the information need!

  • What type of source?
  • How many sources?
  • How will this information be used?

Machine Learning (ML)

  • Ranking results in terms of relevancy
  • Related search suggestions

Natural Language Processing (NLP)

  • Interpreting queries
  • Understanding connections between words
  • How words relate to concepts

"Card Catalog" by yanajenn is licensed under CC BY-SA 2.0.

Why do we care?

But, bias in AI is well documented

Twitter / @Chicken3gg

And Search Engine Optimization might impede fairness

Second: Critical Appraisal

Is this resource any good?

  • Does this make sense?
  • Are there frequent errors?
  • Is this a reputable source?
  • Can I find other sources that confirm this?

Or for academic publishing:

  • Where was this published?
  • Who funded this?
  • Do I recognize any authors or their institutions?
  • Do their choice of methods/data/subjects make sense and are their conclusions logical?

While potentially unconscious, strategic searching and critical appraisal are skills you've developed.

However, this is not the case for everyone

How Students Are Actually Using Generative AI | Harvard Business Publishing Education

Without Critical Appraisal, it becomes easier for GenAI users to:

  1. Believe false information to be legitimate
  2. Unconsciously adopt biases built into training data
  3. Simplify nuanced issues
  4. Miss out on alternative voices or perspectives 

Therefore, as the methods for information retrevial changes, we need robust strategies for verifying and evaluating information

Particularly when information is synthesized for us.

Does creatine help build muscle?

  • ChatGPT (Generative AI)
  • Consensus (Search engine + Generative AI)
  • Medline (Article database)

ChatGPT

Consensus 

Ovid Medline

(creatine and ("muscle build*" or "muscle grow*")).ab,ti.

ChatGPT Consensus Medline
Natural Language
Sources
 
Transparent Results
ChatGPT Consensus Medline
Natural Language
Sources
 
Transparent
Results

increasing ease of critical appraisal 

decreasing ease of use

 Databases > Search Engines > ChatGPT

I am not saying:

I am saying:

 Databases ≠ Search Engines ≠ ChatGPT

Systematic Approach to Information Gathering

"Evidence-based Medicine (EBM)" by Tingjoh is licensed under CC BY-SA 4.0.

High value is placed on:

  • transparency
  • reproducibility 
  • accuracy
  • comprehensiveness

Therefore, article databases are utilized

Takeaways:

  • Choose the right tool
  • Using a sophisticated search tool requires an investment of time to learn
  • Sources matter; having access to full-text resources makes critical appraisal easier
  • An understanding of why results are generated can also help control bias
  • Critical appraisal is a skill that needs to be learned and practiced

Questions?

Additional Information