FYS 100

Let's make a personality test!

  • Task 1: Refine your personality test items
  • Task 2: Identify emerging clusters/dimensions
  • Task 3: Develop a scoring system
  • Task 4: Consider how you will validate our test with respect to external criteria

Plan for Today

Task 1: Refine Items

Task

Review and refine the items you developed last class. For each item, ask:

  • Is this item clearly worded?
  • Does it ask about only one thing (not double-barreled)?
  • Is it free from leading language?
  • Is it distinct from other items, or redundant?

Task 1: Refine Items

Instructions

  1. As a group, read each item aloud
  2. Discuss any problems or ambiguities
  3. Revise or eliminate problematic items
  4. Add new items for any important aspects you missed
  5. Create a final list of 20

Deliverable

Compile your refined items in a Google Document, with each item numbered and the final wording agreed upon by your group.

Task 2: Identify dimensions

Task

Go back and look at your items. Are there any clusters of items that appear to be emerging? That is, are there any dimensions that it seems multiple items measure?

Task 2: Identify Dimensions

Guiding Questions

  • What broader traits or characteristics do your items seem to be measuring?
  • Do certain items naturally group together?
  • Does your test cover the full range of personality as you understand it?
  • What important aspects of personality might your test be missing?
  • Important process question: Did you have dimensions in mind while you were creating the items?

Task 2: Identify Dimensions

Instructions

  1. Sort your items into potential clusters or dimensions
  2. Label each cluster with a descriptive name
  3. Identify any gaps in coverage
  4. Discuss how these dimensions relate to each other

Deliverable

In your Google Document, create a section titled "Dimensions" that lists each dimension you've identified, along with the items that belong to each. (Or you can just group all of the items according to the dimensions they measure.)

Task 3: Develop Scoring System

Task

Develop a comprehensive scoring system for your personality test.

Task 3: Develop Scoring System

Instructions

  1. Decide on your response format (e.g., 5-point Likert scale, True/False)
  2. Determine if any items need to be reverse-scored
  3. Establish how subscale scores will be calculated
    • Simple sum of items?
    • Average of items?
    • Weighted scoring?
  4. Define what high and low scores mean for each dimension

Task 3: Develop Scoring System

Example

 Dimension: Sociability (Items 1, 4, 7, 12)

  • Items scored on 1-5 scale (Strongly Disagree to Strongly Agree)
  • Item 7 is reverse-scored
  • Subscale score = Average of all 4 items
  • High scores (4-5) indicate high sociability; low scores (1-2) indicate preference for solitude

Task 3: Develop Scoring System

Deliverable

Add a "Scoring System" section to your Google Document that clearly explains how to calculate and interpret scores for each dimension.

Task 4: External Validation

Task

Identify 3-5 external criteria you would expect to correlate with your personality dimensions, and develop specific hypotheses.

Task 4: External Validation

Instructions

  1. Brainstorm real-world variables that might relate to your personality dimensions
    • Academic choices (major, study habits)
    • Behavioral tendencies (exercise habits, social media use)
    • Life outcomes (career interests, relationship patterns)
    • Demographic factors (age, cultural background)
  2. For each external criterion, form a specific, testable hypothesis about how scores on your test should relate to that criterion.

Task 4: External Validation

Deliverable

Add an "External Criteria and Hypotheses" section to your Google Document with your 3-5 criteria and corresponding hypotheses.

Task 4: External Validation

Example

External Criterion: Academic Major
Hypothesis: Students scoring high on our "Creativity" dimension will be more likely to major in arts than humanities, social sciences, or STEM fields.
 

External Criterion: Study Habits
Hypothesis: Students scoring high on our "Conscientiousness" dimension will report more consistent study schedules and earlier completion of assignments compared to those scoring low on this dimension.

 

Final Deliverables Checklist

  • Google Document containing:
    • Refined list of 20 numbered items
    • Identified dimensions with items grouped accordingly (doesn't have to be redundant with item list -- items can just be listed in order of dimensions)
    • Complete scoring system
    • 3-5 external criteria with specific hypotheses
  • Be prepared to share your work with the class

Common Response Formats

  • Likert scale: "Rate from 1 (Strongly Disagree) to 5 (Strongly Agree)"
  • Frequency scale: "Rarely, Sometimes, Often, Almost Always"
  • Binary choice: "True/False" or "Yes/No"
  • Semantic differential: Rating between two opposite terms (e.g., "Shy --- Outgoing")

Tips for Good Item Writing

  • Keep language simple and direct
  • Avoid double negatives
  • Ensure items are appropriate for your population
  • Balance positively and negatively worded items
  • Consider social desirability bias