Managing Turbulence
How to design and continually improve our courses in order to better incorporate teaching technologies and active learning.
HANDOUTS ON BOX.IU.EDU:
https://goo.gl/xmFbH3
Erika Lee, Lecturer, MOSAIC Fellow
School of Informatics, Computing and Engineering, Indiana University
ebigalee@indiana.edu
Is this you?
"A group of students got ahold of the exam I wrote ten years ago. Guess I'll have to write another."
THEM
"This website resource didn't look like this yesterday. $%&#"
YOU
"Honey, in my field, resources from 2003 are like yesterday."
"OMG. This tool feels so 2015."
Start with backwards course design
What do I want my students to be able to think and do by the end of this course? How will my students be different by the end of the course?
Final Project
Project 1
Project 2
Project 3
Project 4
Report
Course goals
Milestone assignments based on learning outcomes. What must students be able to do and think to meet the course goals?
Lecture, Lab, Readings, Activities, Homework
WHERE IT BREAKS DOWN FOR ME
How do I figure out what teaching technologies and active learning techniques will support the concepts and skills I want my students to have?
BACKWARDS COURSE DESIGN
Decoding the Disciplines Process
-
Helps us to figure out what mental actions are involved in each of our learning outcomes
- Once we know that, we can better decide what technologies and activities will work best
Decoding the Disciplines Process
- Helps us to figure out what mental actions are involved in each of our learning outcomes
- Once we know that, we can better decide what technologies and activities will work best
PLEASE SEE HANDOUT IN OUR FALCON BOX FOLDER
Step One
Identify a bottleneck to learning
What is a bottleneck to learning in this class, a place where many students consistently fail to master crucial material?
Example bottleneck:
Teaching usability in a web design course (usability = how easy is this to use?)
Vague: Students are unable to get past their opinion in determining whether a web interface is easy to use or not.
Useful: Students (1) lack the ability to create specific tasks and scenarios to collect evidence of usability problems, (2) cannot identify the appropriate audience for a website, and (3) are unable to learn from data to strategize appropriate solutions
and future actions.
Step One
Identify a bottleneck to learning
Q: What is a bottleneck to learning in this class --
a place where many students consistently fail to master crucial material? A place where students too often get something wrong or never do quite figure it out? Describe as precisely as you can what they are getting wrong.
Take 3 minutes, describe this bottleneck as precisely as you can. Write it out on your computer, in the handout, or on a piece of paper.
Step Two
Uncover the Mental Operations that Students Must Master to Get Past the Bottleneck
Experts: Do many difficult things all at once. These mental moves are often implicit and not readily available for conscious scrutiny.
Novices: Can show us where the expert does their critical thinking -- these are our bottlenecks.
ACTION TO TAKE:
Find a trusted NOVICE to INTERVIEW YOU, the EXPERT
Step Two
Uncover the Mental Operations that Students Must Master to Get Past the Bottleneck
Results from my interview:
First, I evaluate the site for various aspects / uses -- navigation, design, content and code -- paying attention to who the intended audience is, what the site emphasizes and what I'm being directing to do.
Next, I use the problems I found to direct me in creating specific tasks and scenarios to test with users. Are the problems really problems?
Have to both analyze and evaluate
Have to connect individual findings with user tasks, and use results to make recommendations
Step Two
Uncover the Mental Operations that Students Must Master to Get Past the Bottleneck
As novices, interview each expert in turn about their bottleneck. (4 min each)
- Describe your bottleneck to the group -- what is the nature of the difficulty?
- Don't get caught up in describing specifics of how / what you teach... Explain what YOU would do as the EXPERT -- what would a specialist do -- to get past the bottleneck?
Teammates: Try to repeat the bottleneck back to the interviewee. "The learners are unable to _____"
Step Three
Modeling Mental Operations
Example model - an analogy
What I realized from the interview:
- Students are sometimes missing connections between the steps, especially when going from an individual evaluation to a study with users
- It's not about their opinion -- it’s about collecting data on users and the users’ opinions
Fitness trackers collect data by taking measurements and giving you a quantifiable picture of your health.
Like with a Fitbit, we may have an idea of how far we walk each day, but we don’t really know. We want to collect data and take measurements instead of making assumptions.
Let’s use our user tests as a sort of Fitbit.
User testing asks you to observe what people do, and measure their performance. You are the expert who will use the resulting data to make decisions about design and content.
We want to now go from a checklist of usability plusses and minuses to a set of user tasks to test and find out:
1. Does the site work as expected?
2. For aspects we thought might be issues, are they really?
Step Three
Modeling Mental Operations
One person at a time in your group --
discuss a metaphor / analogy from outside of your discipline (3 min each)
- Make sure the metaphor highlights crucial operations
- Once you have one, you'll want to test it -- does it also work for an example within your discipline?
Step Four
Creating Opportunities for Students to Practice Essential Mental Operations and Receive Feedback
How can I explicitly model these operations for students?
- What kinds of assignments, activities and tools will help my students practice these mental operations?
- In-class work? Collaborative tasks? CATs? Team-based learning? Just-in-time teaching? TopHat? Padlet? Etc....
Example from my class:
BEFORE
• Read a book
• Take quiz
• Lecture and discussion
• Perform a usability test (outside of class)
• Write a report
Assessment (medium)
Assessment (large)
AFTER
• Read book
• Padlet CAT in class
• Take quiz
• Lecture and discussion
• Perform an evaluation of a site in class (based on a rubric created in the interview process)
• Write an evaluation
• Perform a usability test (outside of class)
• Write a report
Assessment (medium)
Assessment (small)
Assessment (small)
Assessment (large)
Assessment (large)
Step Five
How can I motivate students and address the affective side of learning?
Step Six
How can I tell whether students have mastered these operations by the end of the process?
Step Seven
How can I share what I have learned with others?
Want to tame the turbulence?
Discount and Info in FALCON Box Folder
Middendorf, J., & Shopkow, L. (2017). Overcoming Student Learning Bottlenecks: Decode Your Disciplinary Critical Thinking. Stylus: Sterling, VA.
http://slides.com/ebigalee/turbulent-tech-7
Erika Lee
Lecturer, MOSAIC Fellow
School of Informatics, Computing and Engineering
Indiana University
ebigalee@indiana.edu
