AiA

Assesment in Action

Brief Chronology Talking About Student
Learning Outcomes at Swarthmore

 

Elichko, Sarah, Pamela Harris and Roberto Vargas. “Finding the Zones: Communicating Source-Based Evidence in Undergraduate Research” (poster). American Library Association Annual Conference, ACRL Assessment in Action. Orlando, FL, June 26, 2016.

Elichko, Sarah, Pamela Harris, Margaret Schaus, Peggy Seiden and Terry Snyder. “Have you thought about your thesis lately? Cognitive bottlenecks at the intersection of research & writing” (poster). Association of College and Research Libraries Annual Conference, Indianapolis, IN, April 11, 2013.

 

ALLAC ?

 

2012                         Understanding the Student Research Process in the Humanities and                                         the Social Sciences, Mellon Tri-College Faculty Forum Grant
2011                         Understanding the Student Research & Writing Process, Mellon Tri-                                         College Faculty Forum Grant
2010                          Student Scholars: Enhancing Research Knowledge in the Humanities &                                    Social Sciences, Mellon Tri-College Faculty Forum Grant

 

2007 - 2008              Self-Study: What do you expect students to know about how scholarship is constructed within your discipline? And when do you expect them to know it? 

 

 

 

 

AiA Project: Introduction

Our project aimed to identify productive zones of intervention in which librarians and faculty might focus their teaching efforts.

In partnership with the Sociology/Anthropology and Educational Studies departments, the AiA team conducted a rubric-based evaluation of senior theses and group projects.
 

We focused our analysis on the following question:
          How do students support their claims with  
          source-based evidence?

AiA: what we did

Student work:
- We read 3 final papers from EDUC 053 and selections from 10 SOAN theses
- For the SOAN theses, we read the first 3 pages of the literature review section
              - Rationale for shorter selections - Belanger et al (2015)

Rubric and evaluation:
- Used the Claremont Information Literacy in Student Work rubric
              - Selected the "Communication of Evidence" section
- Followed the norming process suggested by Oakleaf (2009) to assure inter-rater
   reliability - used an additional 3 SOAN theses

 

Analysis:
- Scored each student work using the rubric and annotated our scores
- Tried various methods to make sense of the data (see: color-coded grid)

 

AiA: results

After completion of the analysis, we ask ourselves the following question:

How can library instruction emphasize research as a process of exploration and building rather than one of accumulation?

‘Communication of evidence’ usually receives a single score on information literacy rubrics, whereas we evaluated each aspect of this concept separately. We found that individual students scored well on different combinations of skills; some students cited consistently but struggled to contextualize their sources, others vice-versa.

Our assessment highlighted some potential zones of intervention in which more students seemed to face challenges:  
              • contextualizing evidence
              • maintaining an independent voice while discussing ideas
                    from outside sources

 

MtMAPMAX  
Multi-Tier Mini-Assessment Practices Matrix

 

Tier

Practice

Reasons to do this

Potential difficulties

1st Tier

Read student work

• Opportunity to see how a range of students responds to a given assignment
• Identify common areas of difficulty
• See impacts of your instruction and advice

• Obtaining student papers
• Selecting how much to read
• Variability in the quality of student work may be due to a wide variety of factors, which you may or may not know about

2nd Tier

Use a rubric

• Can facilitate a more consistent and systematic evaluation across multiple papers
• Could call your attention to details and patterns you might otherwise miss

• Selecting, adapting or developing an appropriate rubric

• Clarifying how you’ll analyze the data that you generate

3rd Tier

Evaluate collaboratively

• By working with colleagues, you can compare your evaluations
• Gain insight from colleagues who work with other disciplines
• Opportunity to read student work from outside your disciplines

• Scheduling
• Finding time for focused reading and evaluation, as well as for discussion

4th Tier

Obtain grades

• Compare the instructor’s evaluation of student work with your own

• Grades for assignments may be more difficult to obtain

The good/valuable

  • Engage in conversation with faculty

  • Opportunity to look at student work

  • Opportunity to learn rubrics to assess information literacy  

  • Seeing assignments from student's perspective

Challenges

  • Communication with faculty!

  • Even with a narrow focus, we still generated large amounts of data  

  • Relatively little experience doing research

  • We thought rubric would analyze itself

  • Seeing assignments from student's perspectives

Using Rubrics:

Belanger, Jackie, Ning Zou, Jenny Rushing Mills, Claire Holmes, and Megan Oakleaf. “Project RAILS: Lessons Learned about Rubric Assessment of Information Literacy Skills.” Portal: Libraries and the Academy 15, no. 4 (2015): 623–644.

 

Holmes, Claire, and Megan Oakleaf. “The Official (and Unofficial) Rules for Norming Rubrics Successfully.” Journal of Academic Librarianship 39, no. 6 (2013): 599–602.

 

Oakleaf, Megan. “Using Rubrics to Assess Information Literacy: An Examination of Methodology and Interrater Reliability.” Journal of the American Society for Information Science and Technology 60, no. 5 (May 2009): 969–83. doi:10.1002/asi.21030.

BEAM & I-BEAM:

Bizup, Joseph. “BEAM: A Rhetorical Vocabulary for Teaching Research-Based Writing.” Rhetoric Review 27, no. 1 (January 4, 2008): 72–86.

Troutman, Phillip, and Mark Mullen. “I-BEAM: Instance Source Use and Research Writing Pedagogy.” Rhetoric Review 34, no. 2 (April 3, 2015): 181–99.

Assessment in Action

By Swarthmore Reference

Assessment in Action

practices for assessing student work

  • 94
Loading comments...

More from Swarthmore Reference