Social and Political Data Science: Introduction

Methods of Data Collection and Production

Karl Ho

School of Economic, Political and Policy Sciences

University of Texas at Dallas

Qualitative Data

What is Qualitative research?

Qualitative researchers are interested in understanding the meaning people have constructed, that is, how people make sense of their world and the experiences they have in the world.

(Merriam, 2009, p. 13)

What is Qualitative research?

[Qualitative research is] research using methods such as participant observation or case studies which result in a narrative, descriptive account of a setting or practice. Sociologists using these methods typically reject positivism and adopt a form of interpretive sociology.

(Parkinson & Drislane, 2011)

Positivism: every rationally justifiable assertion can be scientifically verified or is capable of logical or mathematical proof.

- Oxford Languages

What is Qualitative research?

Qualitative research is a situated activity that locates the observer in the world. It consists of a set of interpretive, material practices that turn or convert the world into a series of representations, including field notes, interviews, conversations, photographs, recordings, and memos to the self.

(Denzin & Lincoln, 2005, p. 3)

What is Qualitative research?

[In other words], qualitative researchers study things in their natural settings, attempting to make sense of, or to interpret, phenomena in terms of
the meanings people bring to them.

(Denzin & Lincoln, 2005, p. 3)

What is Qualitative Data?

Qualitative research involves any research that uses data that do not indicate ordinal values.

(Nkwi, Nyamongo, and Ryan 2001, p. 1)

 

In short, qualitative research involves collecting and/or working with text, images, or sounds.

 

Qualitative Data Methods

  1. Participant observation

  2. In-depth interviews

  3. Focus groups

  4. Systematic elicitation

  5. Document analysis

Qualitative Research and Data

Types of Approaches

  1. Phenomenology

  2. Ethnography

  3. Inductive Thematic Analysis

  4. Grounded Theory

  5. Case Study

  6. Discourse/Conversation Analysis

  7. Narrative Analysis

  8. Mixed Methods

Type of Approach Defining Features Data Collection Implications
Phenomenology Focuses on individual experiences, beliefs, and perceptions. Questions and observations are aimed at drawing out individual experiences and perceptions.
Text used as a proxy for human experience In focus groups, group experiences and normative perceptions are typically sought out.
In-depth interviews and focus groups are ideal methods for collecting phenomenological data.
Type of Approach Defining Features Data Collection Implications
Ethnography Oriented toward studying shared meanings and practices (i.e., culture). Questions and observations are generally related to social and cultural processes and shared meanings within a given group of people.
Emphasizes the emic perspective. Traditionally, it is associated with longterm fieldwork, but some aspects are employed in applied settings.
Can have a contemporary or historical focus. Participant observation is well suited to ethnographic inquiry.
Type of Approach Defining Features Data Collection Implications
Inductive Thematic Analysis Draws on inductive analytic methods (this would be same for Grounded Theory below as well). ITA requires generation of free-flowing data.
Involves identifying and coding emergent themes within data. In-depth interviews and focus groups are the most common data collection techniques associated with ITA.
Most common analytic approach used in qualitative inquiry. Notes from participant observation activities can be analyzed using ITA, but interview/focus group data are better.
Type of Approach Defining Features Data Collection Implications
Grounded Theory Inductive data collection and analytic methods. As above, in-depth interviews and focus groups are the most common data collection techniques associated with GT.
Uses systematic and exhaustive comparison of text segments to build thematic structure and theory from a body of text. Sample sizes for grounded theory are more limited than for ITA because the analytic process is more intensive and time consuming.
Common analytic approach in qualitative studies Note: Many researchers incorrectly label all inductive thematic analyses “grounded theory,” as a default. Technically, they are not the same thing.
Type of Approach Defining Features Data Collection Implications
Case Study Analysis of one to several cases that are unique with respect to the research topic. Cases are selected based on a unique (often rarely observed) quality.
Analysis primarily focused on exploring the unique quality. Questions and observations should focus on, and delve deeply into, the unique feature of interest.
Type of Approach Defining Features Data Collection Implications
Discourse/Conversation Analysis Study of “naturally occurring” discourse These linguistically focused methods often use existing documents as data.
Can range from conversation to public events to existing documents. Conversations between individuals that spontaneously emerge within group interviews or focus groups may be studied but are not preferred.
Text and structures within discourse used as objects of analysis. Participant observation is conducive to discourse analysis if narratives from public events can be recorded.
Type of Approach Defining Features Data Collection Implications
Narrative Analysis Narratives (storytelling) used as source of data. If generating narratives (through indepth interviews), then questions/tasks need to be aimed at eliciting stories and the importance those stories, hold for participants, as well as larger cultural meaning.
Narratives from one or more sources (e.g., interviews, literature, letters, diaries).
Type of Approach Defining Features Data Collection Implications
Mixed Methods Defined as integrating quantitative and qualitative research methods in one study. Collection of qualitative data in a mixed methods study can be informed from a wide range of theoretical perspectives and analytic approaches.
Two most common designs are sequential and concurrent. Researchers must specify up front, and in detail, how, when, and why qualitative and quantitative datasets will be integrated.

Objectives of Qualitative Research

  1. Identifying and exploring

    • Building a list

  2. Describing

    • Deep data

  3. Explaining

    • Addressing whys

Qualitative Data Methods

  1. Inductive approach

  2. How and why questions

  3. Open-end exploratory questions

  4. Sequence:

    1. What do you think?
    2. Why do you think so?
    3. How do you like or dislike_____?

Qualitative Research: Human Experience

  1. Behaviors

  2. Attitudes/Opinions/Perceptions

  3. Knowledge

  4. Emotions and Values

  5. Culturally Shared Meaning

  6. Social Structures and Relationships

  7. Processes and Systems

  8. Environmental Context

Qualitative Research: Temporal dimension

  1. Single time point (cross-sectional)

  2. Longitudinal study  

  3. Panel study (cohort study) 

Qualitative Research: Level and Unit of Analysis

Open interviews

  1. Causality

  2. Face validity

  3. Reliability issue

  4. Future:

    1. AI guided internet survey
    2. Accumulated data using machine-learning

Qualitative Research and structure

Qualitative Data: Sampling

  1. What is the population?

    1. What is the representative sample?

  2. Approaches

  3. Sample size

Qualitative Data:

Sampling Strategy

  1. Estimate the size of the population of interest

  2. Control

    1. How much control you will have over your recruitment and sampling procedures
    2. How certain you are about who, what, where you need to sample for your study.

  3. Accumulative

  4. Adaptive

Future of Qualitative Research

  1. Other data than text

    1. Complex data
    2. Audio, visual and video data
    3. Machine Learning
    4. Natural Language Processing (NLP)

Mixed-methods Research

  1. Integration of qualitative and quantitative methods in a single study

  2. Strategy instead of a method

  3. Attend to structure of data

Mixed-methods Research: Sampling

  1. Temporal dimension

    1. Sequential
    2. Concurrent

  2. Relationships of Samples

    1. Identical
    2. Parallel
    3. Nested
    4. Multilevel

Mixed-methods Research

Managing Qualitative Data

Data management is “a designed structure for systematizing, categorizing, and filing materials to make them efficiently retrievable and duplicable”

- Schwandt, 1997, p. 61

Illustration: free-listing data

Free listing is a technique for gathering data about a specific domain or topic by asking people to list all the items they can think of that relate to the topic. It can be used to gather data in large group settings or in one-on-one interviews.

Illustration: free-listing data

Free listing is used by cultural anthropologist to understand particular aspects (domains) of a culture and subgroups within the culture, but it is quite useful for understanding users, tasks, terminology, and other issues important to HCI practitioners.

Illustration: free-listing data

Example: ask the audience in a customer conference  to "list all the things about "the product that frustrate you". Then plot the frequency and position of responses and use that as input to requirements and design activities.

Illustration: free-listing data

Free listing is highly cost effective. It provides a large amount of data in a few minutes and does not required trained facilitators or special materials, software, or hardware.

Illustration: free-listing data

Thompson, Eric C., and Zhang Juan. "Comparative cultural salience: measures using free-list data." Field methods 18, no. 4 (2006): 398-412.

A note on Career

  1. Readiness

    1. Web presense

    2. CV

    3. Self-evaluations

  2. Preparation

    1. Skill inventory

    2. Project showcase

  3. Character/leadership

    1. What is your mentor/supervisor looking for?

    2. Life goals