29
Matthew DeCarlo
Chapter Outline
- What are your assumptions? (16 minute read)
- Ethical and critical considerations (15 minute read)
- Social work research paradigms (25 minute read, plus 10 minute video)
- Developing your theoretical framework (15 minute read)
- Designing your project using theory and paradigm (13 minute read)
Content warning: examples in this chapter contain references to post-traumatic stress disorder and similar culture-bound syndromes related to trauma, colonization and Global North/West hegemony, racist beliefs about intelligence and racist science, sexism in medical science and STEM fields, dropping out of high school, poverty, addiction and the disease model, police violence and systemic racism, rape culture, depression, homelessness, “coming out” as a lesbian, ethnocentrism, sexual harassment, domestic violence, and oppression of TANF recipients.
7.1. What are your assumptions?
Learning Objectives
Learners will be able to…
- Ground your research project and working question in the philosophical assumptions of social science
- Define the terms ‘ontology‘ and ‘epistemology‘ and explain how they relate to quantitative and qualitative research methods
Last chapter, we reviewed the ethical commitment that social work researchers have to protect the people and communities impacted by their research. Answering the practical questions of harm, conflicts of interest, and other ethical issues will provide clear foundation of what you can and cannot do as part of your research project. In this chapter, we will transition from the real world to the conceptual world. Together, we will discover and explore the theoretical and philosophical foundations of your project. You should complete this chapter with a better sense of how theoretical and philosophical concepts help you answer your working question, and in turn, how theory and philosophy will affect the research project you design.
Embrace philosophy
The single biggest barrier to engaging with philosophy of science, at least according to some of my students, is the word philosophy. I had one student who told me that as soon as that word came up, she tuned out because she thought it was above her head. As we discussed in Chapter 1, some students already feel like research methods is too complex of a topic, and asking them to engage with philosophical concepts within research is like asking asking them to tap dance while wearing ice skates.
For those students, I would first answer that this chapter is my favorite one to write because it was the most personally impactful for me to learn during my MSW program. Finding my theoretical and philosophical home was important for me to develop as a clinician and a researcher. Following our advice in Chapter 2, you’ve hopefully chosen a topic that is important to your interests as a social work practitioner, and consider this chapter an opportunity to find your personal roots in addition to revising your working question and designing your research study.
Exploring theoretical and philosophical questions will cause your working question and research project to become clearer…eventually. Consider this chapter as something similar to getting a nice outfit for a fancy occasion. You have to try on a lot of different theories and philosophies before you find the one that fits with what you’re going for. There’s no right way to try on clothes, and there’s no one right theory or philosophy for your project. You might find a good fit with the first one you’ve tried on, or it might take a few different outfits. You have to find ideas that make sense together because they fit with how you think about your topic and how you should study it.
As you read this section, try to think about which assumptions feel right for your working question and research project. Which assumptions match what you think and believe about your topic? The goal is not to find the “right” answer, but to develop your conceptual understanding of your research topic by finding the right theoretical and philosophical fit.
Theoretical & philosophical fluency
In addition to self-discovery, theoretical and philosophical fluency is a skill that social workers must possess in order to engage in social justice work. That’s because theory and philosophy help sharpen your perceptions of the social world. Just as social workers use empirical data to support their work, they also use theoretical and philosophical foundations. More importantly, theory and philosophy help social workers build heuristics that can help identify the fundamental assumptions at the heart of social conflict and social problems. They alert you to the patterns in the underlying assumptions that different people make and how those assumptions shape their worldview, what they view as true, and what they hope to accomplish. In the next section, we will review feminist and other critical perspectives on research, and they should help inform you of how assumptions about research can reinforce existing oppression.
Understanding these deeper structures is a true gift of social work research. Because we acknowledge the usefulness and truth value of multiple philosophies and worldviews contained in this chapter, we can arrive at a deeper and more nuanced understanding of the social world. Methods can be closely associated with particular worldviews or ideologies. There are necessarily philosophical and theoretical aspects to this, and this can be intimidating at times, but it’s important to critically engage with these questions to improve the quality of research.
Building your ice float
Although it may not seem like it right now, your project will develop a from a strong connection to previous theoretical and philosophical ideas about your topic. It’s likely you already have some (perhaps unstated) philosophical or theoretical ideas that undergird your thinking on the topic. Moreover, the philosophical questions we review here should inform how you understand different theories and practice modalities in social work, as they deal with the bedrock questions about science and human knowledge.
Before we can dive into philosophy, we need to recall out conversation from Chapter 1 about objective truth and subjective truths. Let’s test your knowledge with a quick example. Is crime on the rise in the United States? A recent Five Thirty Eight article highlights the disparity between historical trends on crime that are at or near their lowest in the thirty years with broad perceptions by the public that crime is on the rise (Koerth & Thomson-DeVeaux, 2020).[1] Social workers skilled at research can marshal objective facts, much like the authors do, to demonstrate that people’s perceptions are not based on a rational interpretation of the world. Of course, that is not where our work ends. Subjective facts might seek to decenter this narrative of ever-increasing crime, deconstruct is racist and oppressive origins, or simply document how that narrative shapes how individuals and communities conceptualize their world.
Objective does not mean right, and subjective does not mean wrong. Researchers must understand what kind of truth they are searching for so they can choose a theoretical framework, methodology, and research question that matches. As we discussed in Chapter 1, researchers seeking objective truth (one of the philosophical foundations at the bottom of Figure 7.1) often employ quantitative methods (one of the methods at the top of Figure 7.1). Similarly, researchers seeking subjective truths (again, at the bottom of Figure 7.1) often employ qualitative methods (at the top of Figure 7.1). This chapter is about the connective tissue, and by the time you are done reading, you should have a first draft of a theoretical and philosophical (a.k.a. paradigmatic) framework for your study.
Ontology: Assumptions about what is real & true
In section 1.2, we reviewed the two types of truth that social work researchers seek—objective truth and subjective truths —and linked these with the methods—quantitative and qualitative—that researchers use to study the world. If those ideas aren’t fresh in your mind, you may want to navigate back to that section for an introduction.
These two types of truth rely on different assumptions about what is real in the social world—i.e., they have a different ontology. Ontology refers to the study of being (literally, it means “rational discourse about being”). In philosophy, basic questions about existence are typically posed as ontological, e.g.:
- What is there?
- What types of things are there?
- How can we describe existence?
- What kind of categories can things go into?
- Are the categories of existence hierarchical?
Objective vs. subjective ontologies
At first, it may seem silly to question whether the phenomena we encounter in the social world are real. Of course you exist, your thoughts exist, your computer exists, and your friends exist. You can see them with your eyes. This is the ontological framework of realism, which simply means that the concepts we talk about in science exist independent of observation (Burrell & Morgan, 1979).[2] Obviously, when we close our eyes, the universe does not disappear. You may be familiar with the philosophical conundrum: “If a tree falls in a forest and no one is around to hear it, does it make a sound?”
The natural sciences, like physics and biology, also generally rely on the assumption of realism. Lone trees falling make a sound. We assume that gravity and the rest of physics are there, even when no one is there to observe them. Mitochondria are easy to spot with a powerful microscope, and we can observe and theorize about their function in a cell. The gravitational force is invisible, but clearly apparent from observable facts, such as watching an apple fall from a tree. Of course, out theories about gravity have changed over the years. Improvements were made when observations could not be correctly explained using existing theories and new theories emerged that provided a better explanation of the data.
As we discussed in section 1.2, culture-bound syndromes are an excellent example of where you might come to question realism. Of course, from a Western perspective as researchers in the United States, we think that the Diagnostic and Statistical Manual (DSM) classification of mental health disorders is real and that these culture-bound syndromes are aberrations from the norm. But what about if you were a person from Korea experiencing Hwabyeong? Wouldn’t you consider the Western diagnosis of somatization disorder to be incorrect or incomplete? This conflict raises the question–do either Hwabyeong or DSM diagnoses like post-traumatic stress disorder (PTSD) really exist at all…or are they just social constructs that only exist in our minds?
If your answer is “no, they do not exist,” you are adopting the ontology of anti-realism (or relativism), or the idea that social concepts do not exist outside of human thought. Unlike the realists who seek a single, universal truth, the anti-realists perceive a sea of truths, created and shared within a social and cultural context. Unlike objective truth, which is true for all, subjective truths will vary based on who you are observing and the context in which you are observing them. The beliefs, opinions, and preferences of people are actually truths that social scientists measure and describe. Additionally, subjective truths do not exist independent of human observation because they are the product of the human mind. We negotiate what is true in the social world through language, arriving at a consensus and engaging in debate within our socio-cultural context.
These theoretical assumptions should sound familiar if you’ve studied social constructivism or symbolic interactionism in your other MSW courses, most likely in human behavior in the social environment (HBSE).[3] From an anti-realist perspective, what distinguishes the social sciences from natural sciences is human thought. When we try to conceptualize trauma from an anti-realist perspective, we must pay attention to the feelings, opinions, and stories in people’s minds. In their most radical formulations, anti-realists propose that these feelings and stories are all that truly exist.
What happens when a situation is incorrectly interpreted? Certainly, who is correct about what is a bit subjective. It depends on who you ask. Even if you can determine whether a person is actually incorrect, they think they are right. Thus, what may not be objectively true for everyone is nevertheless true to the individual interpreting the situation. Furthermore, they act on the assumption that they are right. We all do. Much of our behaviors and interactions are a manifestation of our personal subjective truth. In this sense, even incorrect interpretations are truths, even though they are true only to one person or a group of misinformed people. This leads us to question whether the social concepts we think about really exist. For researchers using subjective ontologies, they might only exist in our minds; whereas, researchers who use objective ontologies which assume these concepts exist independent of thought.
How do we resolve this dichotomy? As social workers, we know that often times what appears to be an either/or situation is actually a both/and situation. Let’s take the example of trauma. There is clearly an objective thing called trauma. We can draw out objective facts about trauma and how it interacts with other concepts in the social world such as family relationships and mental health. However, that understanding is always bound within a specific cultural and historical context. Moreover, each person’s individual experience and conceptualization of trauma is also true. Much like a client who tells you their truth through their stories and reflections, when a participant in a research study tells you what their trauma means to them, it is real even though only they experience and know it that way. By using both objective and subjective analytic lenses, we can explore different aspects of trauma—what it means to everyone, always, everywhere, and what is means to one person or group of people, in a specific place and time.
Epistemology: Assumptions about how we know things
Having discussed what is true, we can proceed to the next natural question—how can we come to know what is real and true? This is epistemology. Epistemology is derived from the Ancient Greek epistēmē which refers to systematic or reliable knowledge (as opposed to doxa, or “belief”). Basically, it means “rational discourse about knowledge,” and the focus is the study of knowledge and methods used to generate knowledge. Epistemology has a history as long as philosophy, and lies at the foundation of both scientific and philosophical knowledge.
Epistemological questions include:
- What is knowledge?
- How can we claim to know anything at all?
- What does it mean to know something?
- What makes a belief justified?
- What is the relationship between the knower and what can be known?
While these philosophical questions can seem far removed from real-world interaction, thinking about these kinds of questions in the context of research helps you target your inquiry by informing your methods and helping you revise your working question. Epistemology is closely connected to method as they are both concerned with how to create and validate knowledge. Research methods are essentially epistemologies – by following a certain process we support our claim to know about the things we have been researching. Inappropriate or poorly followed methods can undermine claims to have produced new knowledge or discovered a new truth. This can have implications for future studies that build on the data and/or conceptual framework used.
Research methods can be thought of as essentially stripped down, purpose-specific epistemologies. The knowledge claims that underlie the results of surveys, focus groups, and other common research designs ultimately rest on epistemological assumptions of their methods. Focus groups and other qualitative methods usually rely on subjective epistemological (and ontological) assumptions. Surveys and and other quantitative methods usually rely on objective epistemological assumptions. These epistemological assumptions often entail congruent subjective or objective ontological assumptions about the ultimate questions about reality.
Objective vs. subjective epistemologies
One key consideration here is the status of ‘truth’ within a particular epistemology or research method. If, for instance, some approaches emphasize subjective knowledge and deny the possibility of an objective truth, what does this mean for choosing a research method?
We began to answer this question in Chapter 1 when we described the scientific method and objective and subjective truths. Epistemological subjectivism focuses on what people think and feel about a situation, while epistemological objectivism focuses on objective facts irrelevant to our interpretation of a situation (Lin, 2015).[4]
While there are many important questions about epistemology to ask (e.g., “How can I be sure of what I know?” or “What can I not know?” see Willis, 2007[5] for more), from a pragmatic perspective most relevant epistemological question in the social sciences is whether truth is better accessed using numerical data or words and performances. Generally, scientists approaching research with an objective epistemology (and realist ontology) will use quantitative methods to arrive at scientific truth. Quantitative methods examine numerical data to precisely describe and predict elements of the social world. For example, while people can have different definitions for poverty, an objective measurement such as an annual income of “less than $25,100 for a family of four” provides a precise measurement that can be compared to incomes from all other people in any society from any time period, and refers to real quantities of money that exist in the world. Mathematical relationships are uniquely useful in that they allow comparisons across individuals as well as time and space. In this book, we will review the most common designs used in quantitative research: surveys and experiments. These types of studies usually rely on the epistemological assumption that mathematics can represent the phenomena and relationships we observe in the social world.
Although mathematical relationships are useful, they are limited in what they can tell you. While you can learn use quantitative methods to measure individuals’ experiences and thought processes, you will miss the story behind the numbers. To analyze stories scientifically, we need to examine their expression in interviews, journal entries, performances, and other cultural artifacts using qualitative methods. Because social science studies human interaction and the reality we all create and share in our heads, subjectivists focus on language and other ways we communicate our inner experience. Qualitative methods allow us to scientifically investigate language and other forms of expression—to pursue research questions that explore the words people write and speak. This is consistent with epistemological subjectivism’s focus on individual and shared experiences, interpretations, and stories.
It is important to note that qualitative methods are entirely compatible with seeking objective truth. Approaching qualitative analysis with a more objective perspective, we look simply at what was said and examine its surface-level meaning. If a person says they brought their kids to school that day, then that is what is true. A researcher seeking subjective truth may focus on how the person says the words—their tone of voice, facial expressions, metaphors, and so forth. By focusing on these things, the researcher can understand what it meant to the person to say they dropped their kids off at school. Perhaps in describing dropping their children off at school, the person thought of their parents doing the same thing. In this way, subjective truths are deeper, more personalized, and difficult to generalize.
Putting it all together
As you might guess by the structure of the next two parts of this textbook, the distinction between quantitative and qualitative is important. Because of the distinct philosophical assumptions of objectivity and subjectivity, it will inform how you define the concepts in your research question, how you measure them, and how you gather and interpret your raw data. You certainly do not need to have a final answer right now! But stop for a minute and think about which approach feels right so far. In the next section, we will consider another set of philosophical assumptions that relate to ethics and the role of research in achieving social justice.
Key Takeaways
- Philosophers of science disagree on the basic tenets of what is true and how we come to know what is true.
- Researchers searching for objective truth will likely have a different theoretical framework, research design, and methods than researchers searching for subjective truths.
- These differences are due to different assumptions about what is real and true (ontology) and how we can come to understand what is real and true (epistemology).
Exercises
Does an objective or subjective epistemological/ontological framework make the most sense for your research project?
- Are you more concerned with how people think and feel about your topic, their subjective truths—more specific to the time and place of your project?
- Or are you more concerned with objective truth, so that your results might generalize to populations beyond the ones in your study?
Using your answer to the above question, describe how either quantitative or qualitative methods make the most sense for your project.
7.2 Ethical and critical considerations
Learning Objectives
Learners will be able to…
- Apply feminist, anti-racist, and decolonization critiques of social science to your project
- Define axiology and describe the axiological assumptions of your project
So far, we have talked about knowledge as it exists in the world, but what about the process of research itself? Doesn’t the researcher bring their own biases, perspectives, and experiences to the process? The critique of science as an enterprise dominated by the perspectives of white men from North America and Europe is one that has had a profound impact on how we view knowledge. Because scientists design research studies, create measures, and interpret results, there is always the risk that a scientist’s objectivity slips and as a result, biases are expressed.
Consider this example from professional sports. The National Football League (NFL) has long downplayed the lifelong impact of concussions and traumatic brain injury. However, due to the racist science that existed when the issue was first addressed through a settlement in the 1990s, Black players were assumed to have lower cognitive function and were thus any losses in cognitive function were less significant, resulting in a lower payout or additional barriers to an eventual payout (Dale, 2021).[6] It is hard to view this “race-norming” without taking into account the impact of the Bell Curve, a racist and methodologically flawed book that purported to support white intellectual superiority (Bell, 1995).[7] According to an Associated Press report:
The NFL noted that the norms were developed in medicine “to stop bias in testing, not perpetrate it”…The binary race norms, when they are used in the testing, assumes that Black patients start with worse cognitive function than whites and other non-Blacks. That makes it harder for them to show a deficit and qualify for an award. [Two players], for instance, were denied awards but would have qualified had they been white, according to their lawsuit (Dale, 2021, para 10-13).
Part of the value in making the philosophical assumptions of your project explicit is that you can scan for sources of explicit or implicit bias you bring to the research process.
Whose truth does science establish?
Social work is concerned with the “isms” of oppression (ableism, ageism, cissexism, classism, heterosexism, racism, sexism, etc.), and so our approach to science must reconcile its history as both a tool of oppression and its exclusion of oppressed groups. Science grew out of the Enlightenment, a philosophical movement which applied reason and empirical analysis to understanding the world. While the Enlightenment brought forth tremendous achievements, the critiques of Marxian, feminist, and other critical theorists complicated the Enlightenment understanding of science. For this section, I will focus on feminist critiques of science, building upon an entry in the Stanford Encyclopedia of Philosophy (Crasnow, 2020).[8]
In its original formulation, science was an individualistic endeavor. As we learned in Chapter 1, a basic statement of the scientific method is that a researcher studies existing theories on a topic, formulates a hypothesis about what might be true, and either confirms or disconfirms their hypothesis through experiment and rigorous observation. Over time, our theories become more accurate in their predictions and more comprehensive in their conclusions. Scientists put aside their preconceptions, look at the data, and build their theories based on objective rationality.
Yet, this cannot be perfectly true. Scientists are human, after all. As a profession historically dominated by white men, scientists have dismissed women and other minorities as being psychologically unfit for the scientific profession. While attitudes have improved, science, technology, engineering, mathematics (STEM) and related fields remain dominated by white men (Grogan, 2019).[9] Biases can persist in social work theory and research when social scientists do not have similar experiences to the populations they study.
Gender bias can influence the research questions scientists choose to answer. Feminist critiques of medical science drew attention to women’s health issues, spurring research and changing standards of care. The focus on domestic violence in the empirical literature can also be seen as a result of feminist critique. Thus, critical theory helps us critique what is on the agenda for science. If science is to answer important questions, it must speak to the concerns of all people. Through the democratization in access to scientific knowledge and the means to produce it, science becomes a sister process of social development and social justice.
The goal of a diverse and participatory scientific community lies in contrast to much of what we understand to be “proper” scientific knowledge. Many of the older, classic social science theories were developed based on research which observed males or from university students in the United States or other Western nations. How these observations were made, what questions were asked, and how the data were interpreted were shaped by the same oppressive forces that existed in broader society, a process that continues into the present. In psychology, the concept of hysteria or hysterical women was believed to be caused by a wandering womb (Tasca et al., 2012).[10] Even today, there are gender biases in diagnoses of histrionic personality disorder and racial biases in psychotic disorders (Klonsky et al., 2002)[11] because the theories underlying them were created in a sexist and racist culture. In these ways, science can reinforce the truth of the white Western male perspective.
Finally, it is important to note that social science research is often conducted on populations rather than with populations. Historically, this has often meant Western men traveling to other countries and seeking to understand other cultures through a Western lens. Lacking cultural humility and failing to engage stakeholders, ethnocentric research of this sort has led to the view of non-Western cultures as inferior. Moreover, the use of these populations as research subjects rather than co-equal participants in the research process privileges the researcher’s knowledge over that from other groups or cultures. Researchers working with indigenous cultures, in particular, had a destructive habit of conducting research for a short time and then leaving, without regard for the impact their study had on the population. These critiques of Western science aim to decolonize social science and dismantle the racist ideas the oppress indigenous and non-Western peoples through research (Smith, 2013).[12]
The central concept in feminist, anti-racist, and decolonization critiques (among other critical frames) is epistemic injustice. Epistemic injustice happens when someone is treated unfairly in their capacity to know something or describe their experience of the world. As described by Fricker (2011),[13] the injustice emerges from the dismissal of knowledge from oppressed groups, discrimination against oppressed groups in scientific communities, and the resulting gap between what scientists can make sense of from their experience and the experiences of people with less power who have lived experience of the topic. We recommend this video from Edinburgh Law School which applies epistemic injustice to studying public health emergencies, disabilities, and refugee services.
Exercises
- Take a moment and reflect on how your life experiences may inform how you understand your topic. What do you already know? How might you be biased?
- Describe how previous or current studies and theories about your topic have been influenced by oppressive forces such as racism and sexism.
Self-determination and free will
When scientists observe social phenomena, they often take the perspective of determinism, meaning that what is seen is the result of processes that occurred earlier in time (i.e., cause and effect). As you will see in Chapter 9, this process is represented in the classical formulation of a research question which asks “what is the relationship between X (cause) and Y (effect)?” By framing a research question in such a way, the scientist is disregarding any reciprocal influence that Y has on X. Moreover, the scientist also excludes human agency from the equation. It is simply that a cause will necessitate an effect. For example, a researcher might find that few people living in neighborhoods with higher rates of poverty graduate from high school, and thus conclude that poverty causes adolescents to drop out of school. This conclusion, however, does not address the story behind the numbers. Each person who is counted as graduating or dropping out has a unique story of why they made the choices they did. Perhaps they had a mentor or parent that helped them succeed. Perhaps they faced the choice between employment to support family members or continuing in school.
For this reason, determinism is critiqued as reductionistic in the social sciences because people have agency over their actions. This is unlike the natural sciences like physics. While a table isn’t aware of the friction it has with the floor, parents and children are likely aware of the friction in their relationships and act based on how they interpret that conflict. The opposite of determinism is free will, that humans can choose how they act and their behavior and thoughts are not solely determined by what happened prior in a neat, cause-and-effect relationship. Researchers adopting a perspective of free will view the process of, continuing with our education example, seeking higher education as the result of a number of mutually influencing forces and the spontaneous and implicit processes of human thought. For these researchers, the picture painted by determinism is too simplistic.
A similar dichotomy can be found in the debate between individualism and holism. When you hear something like “the disease model of addiction leads to policies that pathologize and oppress people who use drugs,” the speaker is making a methodologically holistic argument. They are making a claim that abstract social forces (the disease model, policies) can cause things to change. A methodological individualist would critique this argument by saying that the disease model of addiction doesn’t actually cause anything by itself. From this perspective, it is the individuals, rather than any abstract social force, who oppress people who use drugs. The disease model itself doesn’t cause anything to change; the individuals who follow the precepts of the disease model are the agents who actually oppress people in reality. To an individualist, all social phenomena are the result of individual human action and agency. To a holist, social forces can determine outcomes for individuals without individuals playing a causal role, undercutting free will and research projects that seek to maximize human agency.
Exercises
- Which assumption, determinism or free will, makes the most sense for your project and working question?
- Is human action, or free will, central to how you understand your topic?
- Or are humans more passive and what happens to them more determined by the social forces that influence their life?
- Reflect on how your project’s assumptions may differ from your own assumptions about free will and determinism. For example, my beliefs about self-determination and free will always inform my social work practice. However, my working question and research project may rely on social theories that are deterministic and do not address human agency.
Radical change
Another assumption scientists make is around the nature of the social world. Is it an orderly place that remains relatively stable over time? Or is it a place of constant change and conflict? The view of the social world as an orderly place can help a researcher describe how things fit together to create a cohesive whole. For example, systems theory can help you understand how different systems interact with and influence one another, drawing energy from one place to another through an interconnected network with a tendency towards homeostasis. This is a more consensus-focused and status-quo-oriented perspective. Yet, this view of the social world cannot adequately explain the radical shifts and revolutions that occur. It also leaves little room for human action and free will. In this more radical space, change consists of the fundamental assumptions about how the social world works.
For example, at the time of this writing, protests are taking place across the world to remember the killing of George Floyd by Minneapolis police and other victims of police violence and systematic racism. Public support of Black Lives Matter, an anti-racist activist group that focuses on police violence and criminal justice reform, has experienced a radical shift in public support in just two weeks since the killing, equivalent to the previous 21 months of advocacy and social movement organizing (Cohn & Quealy, 2020).[14] Abolition of police and prisons, once a fringe idea, has moved into the conversation about remaking the criminal justice system from the ground-up, centering its historic and current role as an oppressive system for Black Americans. Seemingly overnight, reducing the money spent on police and giving that money to social services became a moderate political position.
A researcher centering change may choose to understand this transformation or even incorporate radical anti-racist ideas into the design and methods of their study. For an example of how to do so, see this participatory action research study working with Black and Latino youth (Bautista et al., 2013).[15] Contrastingly, a researcher centering consensus and the status quo might focus on incremental changes what people currently think about the topic. For example, see this survey of social work student attitudes on poverty and race that seeks to understand the status quo of student attitudes and suggest small changes that might change things for the better (Constance-Huggins et al., 2020).[16] To be clear, both studies contribute to racial justice. However, you can see by examining the methods section of each article how the participatory action research article addresses power and values as a core part of their research design, qualitative ethnography and deep observation over many years, in ways that privilege the voice of people with the least power. In this way, it seeks to rectify the epistemic injustice of excluding and oversimplifying Black and Latino youth. Contrast this more radical approach with the more traditional approach taken in the second article, in which they measured student attitudes using a survey developed by researchers.
Exercises
- Think about how participatory your study will be.
- Traditional studies will be less participatory. You as the researcher will determine the research question, how to measure it, data collection, etc.
- Radical studies will be more participatory. You as the researcher seek to undermine power imbalances at each stage of the research process.
- Pragmatically, more participatory studies take longer to complete and may be less suited to student projects that need to be completed in a short time frame.
Axiology: Assumptions about values
Axiology is the study of values and value judgements (literally “rational discourse about values [a xía]”). In philosophy this field is subdivided into ethics (the study of morality) and aesthetics (the study of beauty, taste and judgement). For the hard-nosed scientist, the relevance of axiology might not be obvious. After all, what difference do one’s feelings make for the data collected? Don’t we spend a long time trying to teach researchers to be objective and remove their values from the scientific method?
Like ontology and epistemology, the import of axiology is typically built into research projects and exists “below the surface”. You might not consciously engage with values in a research project, but they are still there. Similarly, you might not hear many researchers refer to their axiological commitments but they might well talk about their values and ethics, their positionality, or a commitment to social justice.
Our values focus and motivate our research. These values could include a commitment to scientific rigor, or to always act ethically as a researcher. At a more general level we might ask: What matters? Why do research at all? How does it contribute to human wellbeing? Almost all research projects are grounded in trying to answer a question that matters or has consequences. Some research projects are even explicit in their intention to improve things rather than observe them. This is most closely associated with “critical” approaches.
Critical and radical views of science focus on how to spread knowledge and information in a way that combats oppression. These questions are central for creating research projects that fight against the objective structures of oppression—like unequal pay—and their subjective counterparts in the mind—like internalized sexism. For example, a more critical research project would fight not only against statutes of limitations for sexual assault but on how women have internalized rape culture as well. Its explicit goal would be to fight oppression and to inform practice on women’s liberation. For this reason, creating change is baked into the research questions and methods used in more critical and radical research projects.
As part of studying radical change and oppression, we are likely employing a model of science that puts values front-and-center within a research project. All social work research is values-driven, as we are a values-driven profession. Historically, though, most social scientists have argued for values-free science. Scientists agree that science helps human progress, but they hold that researchers should remain as objective as possible—which means putting aside politics and personal values that might bias their results, similar to the cognitive biases we discussed in section 1.1. Over the course of last century, this perspective was challenged by scientists who approached research from an explicitly political and values-driven perspective. As we discussed earlier in this section, feminist critiques strive to understand how sexism biases research questions, samples, measures, and conclusions, while decolonization critiques try to de-center the Western perspective of science and truth.
Linking axiology, epistemology, and ontology
It is important to note that both values-central and values-neutral perspectives are useful in furthering social justice. Values-neutral science is helpful at predicting phenomena. Indeed, it matches well with objectivist ontologies and epistemologies. Let’s examine a measure of depression, the Patient Health Questionnaire (PSQ-9). The authors of this measure spent years creating a measure that accurately and reliably measures the concept of depression. This measure is assumed to measure depression in any person, and scales like this are often translated into other languages (and subsequently validated) for more widespread use . The goal is to measure depression in a valid and reliable manner. We can use this objective measure to predict relationships with other risk and protective factors, such as substance use or poverty, as well as evaluate the impact of evidence-based treatments for depression like narrative therapy.
While measures like the PSQ-9 help with prediction, they do not allow you to understand an individual person’s experience of depression. To do so, you need to listen to their stories and how they make sense of the world. The goal of understanding isn’t to predict what will happen next, but to empathically connect with the person and truly understand what’s happening from their perspective. Understanding fits best in subjectivist epistemologies and ontologies, as they allow for multiple truths (i.e. that multiple interpretations of the same situation are valid). Although all researchers addressing depression are working towards socially just ends, the values commitments researchers make as part of the research process influence them to adopt objective or subjective ontologies and epistemologies.
Exercises
What role will values play in your study?
- Are you looking to be as objective as possible, putting aside your own values?
- Or are you infusing values into each aspect of your research design?
Remember that although social work is a values-based profession, that does not mean that all social work research is values-informed. The majority of social work research is objective and tries to be value-neutral in how it approaches research.
Philosophical assumptions, as a whole
As you engage with theoretical and empirical information in social work, keep these philosophical assumptions in mind. They are useful shortcuts to understanding the deeper ideas and assumptions behind the construction of knowledge. See Table 7.1 below for a short reference list of the key assumptions we covered in sections 7.1 and 7.2. The purpose of exploring these philosophical assumptions isn’t to find out which is true and which is false. Instead, the goal is to identify the assumptions that fit with how you think about your working question and your personal worldview.
Assumptions | Central conflicts |
Ontology: assumptions about what is real | Realism vs. anti-realism (a.k.a. relativism) |
Epistemology: assumptions about how we come to know what is real | Objective truth vs. subjective truths
Math vs. language/expression Prediction vs. understanding |
Assumptions about the researcher | Researcher as unbiased vs. researcher shaped by oppression, culture, and history
Researcher as neutral force vs. researcher as oppressive force |
Assumptions about human action | Determinism vs. free will
Holism vs. individualism |
Assumptions about the social world | Orderly and consensus-focused vs. disorderly and conflict-focused |
Assumptions about the purpose of research | Study the status quo vs. create radical change
Values-neutral vs. values-informed |
Key Takeaways
- Feminist, anti-racist, and decolonization critiques of science highlight the often hidden oppressive ideas and structures in science.
- Even though social work is a values-based discipline, most social work research projects are values-neutral because those assumptions fit with the researcher’s question.
Exercises
- Using your understanding of the conflicts in Table 7.1 and explored in sections 7.1 and 7.2, critique the following (deliberately problematic) statement:
“When a scientist observes the social world, he does so objectively.”
7.3 Social work research paradigms
Learning Objectives
Learners will be able to…
- Distinguish between the three major research paradigms in social work and apply the assumptions upon which they are built to a student research project
In the previous two sections, we reviewed the three elements to the philosophical foundation of a research method: ontology, epistemology and axiology (Crotty, 1998; Guba & Lincoln, 1994; Heron & Reason, 1997).[17] In this section, you will explore how to apply these philosophical approaches to your research project. In the next section, we will do the same for theory. Keep in mind that it’s easy for us as textbook authors to lay out each step (paradigm, theory, etc.) sequentially, but in reality, research projects are not linear. Researchers rarely proceed by choosing an ontology, epistemology and axiology separately and then deciding which theory and methods to apply. As we discussed in Chapter 2 when you started conceptualizing your project, you should choose something that interests you, is feasible to conduct, and does not pose unethical risks to others. Whatever part or parts your project you have figured out right now, you’re right where you should be—in the middle of conceptualization.
How do scientific ideas change over time?
Much like your ideas develop over time as you learn more, so does the body of scientific knowledge. Kuhn’s (1962)[18] The Structure of Scientific Revolutions is one of the most influential works on the philosophy of science, and is credited with introducing the idea of competing paradigms (or “disciplinary matrices”) in research. Kuhn investigated the way that scientific practices evolve over time, arguing that we don’t have a simple progression from “less knowledge” to “more knowledge” because the way that we approach inquiry is changing over time. This can happen gradually, but the process results in moments of change where our understanding of a phenomenon changes more radically (such as in the transition from Newtonian to Einsteinian physics; or from Lamarckian to Darwinian theories of evolution). For a social work practice example, Fleuridas & Krafcik (2019)[19] trace the development of the “four forces” of psychotherapy, from psychodynamics to behaviorism to humanism as well as the competition among emerging perspectives to establish itself as the fourth force to guide psychotherapeutic practice. But how did the problems in one paradigm inspire new paradigms? Kuhn presents us with a way of understanding the history of scientific development across all topics and disciplines.
As you can see in this video from Matthew J. Brown (CC-BY), there are four stages in the cycle of science in Kuhn’s approach. Firstly, a pre-paradigmatic state where competing approaches share no consensus. Secondly, the “normal” state where there is wide acceptance of a particular set of methods and assumptions. Thirdly, a state of crisis where anomalies that cannot be solved within the existing paradigm emerge and competing theories to address them follow. Fourthly, a revolutionary phase where some new paradigmatic approach becomes dominant and supplants the old. Shnieder (2009)[20] suggests that the Kuhnian phases are characterized by different kinds of scientific activity.
Newer approaches often build upon rather than replace older ones, but they also overlap and can exist within a state of competition. Scientists working within a particular paradigm often share methods, assumptions and values. In addition to supporting specific methods, research paradigms also influence things like the ambition and nature of research, the researcher-participant relationship and how the role of the researcher is understood.
Paradigm vs. theory
The terms ‘paradigm‘ and ‘theory‘ are often used interchangeably in social science. There is not a consensus among social scientists as to whether these are identical or distinct concepts. With that said, in this text, we will make a clear distinction between the two ideas because thinking about each concept separately is more useful for our purposes.
We define paradigm a set of common philosophical (ontological, epistemological, and axiological) assumptions that inform research. The four paradigms we describe in this section refer to patterns in how groups of researchers resolve philosophical questions. Some assumptions naturally make sense together, and paradigms grow out of researchers with shared assumptions about what is important and how to study it. Paradigms are like “analytic lenses” and a provide framework on top of which we can build theoretical and empirical knowledge (Kuhn, 1962).[21] Consider this video of an interview with world-famous physicist Richard Feynman in which he explains why “when you explain a ‘why,’ you have to be in some framework that you allow something to be true. Otherwise, you are perpetually asking why.” In order to answer basic physics question like “what is happening when two magnets attract?” or a social work research question like “what is the impact of this therapeutic intervention on depression,” you must understand the assumptions you are making about social science and the social world. Paradigmatic assumptions about objective and subjective truth support methodological choices like whether to conduct interviews or send out surveys, for example.
While paradigms are broad philosophical assumptions, theory is more specific, and refers to a set of concepts and relationships scientists use to explain the social world. Theories are more concrete, while paradigms are more abstract. Look back to Figure 7.1 at the beginning of this chapter. Theory helps you identify the concepts and relationships that align with your paradigmatic understanding of the problem. Moreover, theory informs how you will measure the concepts in your research question and the design of your project.
For both theories and paradigms, Kuhn’s observation of scientific paradigms, crises, and revolutions is instructive for understanding the history of science. Researchers inherit institutions, norms, and ideas that are marked by the battlegrounds of theoretical and paradigmatic debates that stretch back hundreds of years. We have necessarily simplified this history into four paradigms: positivism, interpretivism, critical, and pragmatism. Our framework and explanation are inspired by the framework of Guba and Lincoln (1990)[22] and Burrell and Morgan (1979).[23] while also incorporating pragmatism as a way of resolving paradigmatic questions. Most of social work research and theory can be classified as belonging to one of these four paradigms, though this classification system represents only one of many useful approaches to analyzing social science research paradigms.
Building on our discussion in section 7.1 on objective vs. subjective epistemologies and ontologies, we will start with the difference between positivism and interpretivism. Afterward, we will link our discussion of axiology in section 7.2 with the critical paradigm. Finally, we will situate pragmatism as a way to resolve paradigmatic questions strategically. The difference between positivism and interpretivism is a good place to start, since the critical paradigm and pragmatism build on their philosophical insights.
It’s important to think of paradigms less as distinct categories and more as a spectrum along which projects might fall. For example, some projects may be somewhat positivist, somewhat interpretivist, and a little critical. No project fits perfectly into one paradigm. Additionally, there is no paradigm that is more correct than the other. Each paradigm uses assumptions that are logically consistent, and when combined, are a useful approach to understanding the social world using science. The purpose of this section is to acquaint you with what research projects in each paradigm look like and how they are grounded in philosophical assumptions about social science.
You should read this section to situate yourself in terms of what paradigm feels most “at home” to both you as a person and to your project. You may find, as I have, that your research projects are more conventional and less radical than what feels most like home to you, personally. In a research project, however, students should start with their working question rather than their heart. Use the paradigm that fits with your question the best, rather than which paradigm you think fits you the best.
Positivism: Researcher as “expert”
Positivism has its roots in the scientific revolution of the Enlightenment. Positivism is based on the idea that we can come to know facts about the natural world through our experiences of it. The processes that support this are the logical and analytic classification and systemization of these experiences. Through this process of empirical analysis, Positivists aim to arrive at descriptions of law-like relationships and mechanisms that govern the world we experience.
Positivists have traditionally claimed that the only authentic knowledge we have of the world is empirical and scientific. Essentially, positivism downplays any gap between our experiences of the world and the way the world really is; instead, positivism determines objective “facts” through the correct methodological combination of observation and analysis. Data collection methods typically include quantitative measurement, which is supposed to overcome the individual biases of the researcher.
Positivism aspires to high standards of validity and reliability supported by evidence, and has been applied extensively in both physical and social sciences. Its goal is familiar to all students of science: iteratively expanding the evidence base of what we know is true. We can know our observations and analysis describe real world phenomena because researchers separate themselves and objectively observe the world, placing a deep epistemological separation between “the knower” and “what is known” and reducing the possibility of bias. We can all see the logic in separating yourself as much as possible from your study so as not to bias it, even if we know we cannot do so perfectly.
However, the criticism often made of positivism with regard to human and social sciences (e.g. education, psychology, sociology) is that positivism is scientistic; which is to say that it overlooks differences between the objects in the natural world (tables, atoms, cells, etc.) and the subjects in the social work (self-aware people living in a complex socio-historical context). In pursuit of the generalizable truth of “hard” science, it fails to adequately explain the many aspects of human experience don’t conform to this way of collecting data. Furthermore, by viewing science as an idealized pursuit of pure knowledge, positivists may ignore the many ways in which power structures our access to scientific knowledge, the tools to create it, and the capital to participate in the scientific community.
Kivunja & Kuyini (2017)[24] describe the essential features of positivism as:
- A belief that theory is universal and law-like generalizations can be made across contexts
- The assumption that context is not important
- The belief that truth or knowledge is ‘out there to be discovered’ by research
- The belief that cause and effect are distinguishable and analytically separable
- The belief that results of inquiry can be quantified
- The belief that theory can be used to predict and to control outcomes
- The belief that research should follow the scientific method of investigation
- Rests on formulation and testing of hypotheses
- Employs empirical or analytical approaches
- Pursues an objective search for facts
- Believes in ability to observe knowledge
- The researcher’s ultimate aim is to establish a comprehensive universal theory, to account for human and social behavior
- Application of the scientific method
Many quantitative researchers now identify as postpositivist. Postpositivism retains the idea that truth should be considered objective, but asserts that our experiences of such truths are necessarily imperfect because they are ameliorated by our values and experiences. Understanding how postpositivism has updated itself in light of the developments in other research paradigms is instructive for developing your own paradigmatic framework. Epistemologically, postpositivists operate on the assumption that human knowledge is based not on the assessments from an objective individual, but rather upon human conjectures. As human knowledge is thus unavoidably conjectural and uncertain, though assertions about what is true and why it is true can be modified or withdrawn in the light of further investigation. However, postpositivism is not a form of relativism, and generally retains the idea of objective truth.
These epistemological assumptions are based on ontological assumptions that an objective reality exists, but contra positivists, they believe reality can be known only imperfectly and probabilistically. While positivists believe that research is or can be value-free or value-neutral, postpositivists take the position that bias is undesired but inevitable, and therefore the investigator must work to detect and try to correct it. Postpositivists work to understand how their axiology (i.e., values and beliefs) may have influenced their research, including through their choice of measures, populations, questions, and definitions, as well as through their interpretation and analysis of their work. Methodologically, they use mixed methods and both quantitative and qualitative methods, accepting the problematic nature of “objective” truths and seeking to find ways to come to a better, yet ultimately imperfect understanding of what is true. A popular form of postpositivism is critical realism, which lies between positivism and interpretivism.
Is positivism right for your project?
Positivism is concerned with understanding what is true for everybody. Social workers whose working question fits best with the positivist paradigm will want to produce data that are generalizable and can speak to larger populations. For this reason, positivistic researchers favor quantitative methods—probability sampling, experimental or survey design, and multiple, and well-established instruments to measure key concepts.
A positivist orientation to research is appropriate when your research question asks for generalizable truths. For example, your working question may look something like: does my agency’s housing intervention lead to fewer periods of homelessness for our clients? It is necessary to study such a relationship quantitatively and objectively. When social workers speak about social problems impacting societies and individuals, they reference positivist research, including experiments and surveys of the general populations. Positivist research is exceptionally good at producing cause-and-effect explanations that apply across many different situations and groups of people. There are many good reasons why positivism is the dominant research paradigm in the social sciences.
Critiques of positivism stem from two major issues. First and foremost, positivism may not fit the messy, contradictory, and circular world of human relationships. A positivistic approach does not allow the researcher to understand another person’s subjective mental state in detail. This is because the positivist orientation focuses on quantifiable, generalizable data—and therefore encompasses only a small fraction of what may be true in any given situation. This critique is emblematic of the interpretivist paradigm, which we will describe next.
In the section after that, we will describe the critical paradigm, which critiques the positivist paradigm (and the interpretivist paradigm) for focusing too little on social change, values, and oppression. Positivists assume they know what is true, but they often do not incorporate the knowledge and experiences of oppressed people, even when those community members are directly impacted by the research. Positivism has been critiqued as ethnocentrist, patriarchal, and classist (Kincheloe & Tobin, 2009).[25] This leads them to do research on, rather than with populations by excluding them from the conceptualization, design, and impact of a project, a topic we discussed in section 2.4. It also leads them to ignore the historical and cultural context that is important to understanding the social world. The result can be a one-dimensional and reductionist view of reality.
Exercises
- From your literature search, identify an empirical article that uses quantitative methods to answer a research question similar to your working question or about your research topic.
- Review the assumptions of the positivist research paradigm.
- Discuss in a few sentences how the author’s conclusions are based on some of these paradigmatic assumptions. How might a researcher operating from a different paradigm (e.g., interpretivism, critical) critique these assumptions as well as the conclusions of this study?
Interpretivism: Researcher as “empathizer”
Positivism is focused on generalizable truth. Interpretivism, by contrast, develops from the idea that we want to understand the truths of individuals, how they interpret and experience the world, their thought processes, and the social structures that emerge from sharing those interpretations through language and behavior. The process of interpretation (or social construction) is guided by the empathy of the researcher to understand the meaning behind what other people say.
Historically, interpretivism grew out of a specific critique of positivism: that knowledge in the human and social sciences cannot conform to the model of natural science because there are features of human experience that cannot objectively be “known”. The tools we use to understand objects that have no self-awareness may not be well-attuned to subjective experiences like emotions, understandings, values, feelings, socio-cultural factors, historical influences, and other meaningful aspects of social life. Instead of finding a single generalizable “truth,” the interpretivist researcher aims to generate understanding and often adopts a relativist position.
While positivists seek “the truth,” the social constructionist framework argues that “truth” varies. Truth differs based on who you ask, and people change what they believe is true based on social interactions. These subjective truths also exist within social and historical contexts, and our understanding of truth varies across communities and time periods. This is because we, according to this paradigm, create reality ourselves through our social interactions and our interpretations of those interactions. Key to the interpretivist perspective is the idea that social context and interaction frame our realities.
Researchers operating within this framework take keen interest in how people come to socially agree, or disagree, about what is real and true. Consider how people, depending on their social and geographical context, ascribe different meanings to certain hand gestures. When a person raises their middle finger, those of us in Western cultures will probably think that this person isn’t very happy (not to mention the person at whom the middle finger is being directed!). In other societies around the world, a thumbs-up gesture, rather than a middle finger, signifies discontent (Wong, 2007).[26] The fact that these hand gestures have different meanings across cultures aptly demonstrates that those meanings are socially and collectively constructed. What, then, is the “truth” of the middle finger or thumbs up? As we’ve seen in this section, the truth depends on the intention of the person making the gesture, the interpretation of the person receiving it, and the social context in which the action occurred.
Qualitative methods are preferred as ways to investigate these phenomena. Data collected might be unstructured (or “messy”) and correspondingly a range of techniques for approaching data collection have been developed. Interpretivism acknowledges that it is impossible to remove cultural and individual influence from research, often instead making a virtue of the positionality of the researcher and the socio-cultural context of a study.
One common objection positivists levy against interpretivists is that interpretivism tends to emphasize the subjective over the objective. If the starting point for an investigation is that we can’t fully and objectively know the world, how can we do research into this without everything being a matter of opinion? For the positivist, this risk for confirmation bias as well as invalid and unreliable measures makes interpretivist research unscientific. Clearly, we disagree with this assessment, and you should, too. Positivism and interpretivism have different ontologies and epistemologies with contrasting notions of rigor and validity (for more information on assumptions about measurement, see Chapter 11 for quantitative validity and reliability and Chapter 20 for qualitative rigor). Nevertheless, both paradigms apply the values and concepts of the scientific method through systematic investigation of the social world, even if their assumptions lead them to do so in different ways. Interpretivist research often embraces a relativist epistemology, bringing together different perspectives in search of a trustworthy and authentic understanding or narrative.
Kivunja & Kuyini (2017)[27] describe the essential features of interpretivism as:
- The belief that truths are multiple and socially constructed
- The acceptance that there is inevitable interaction between the researcher and his or her research participants
- The acceptance that context is vital for knowledge and knowing
- The belief that knowledge can be value laden and the researcher’s values need to be made explicit
- The need to understand specific cases and contexts rather deriving universal laws that apply to everyone, everywhere.
- The belief that causes and effects are mutually interdependent, and that causality may be circular or contradictory
- The belief that contextual factors need to be taken into consideration in any systematic pursuit of understanding
One important clarification: it’s important to think of the interpretivist perspective as not just about individual interpretations but the social life of interpretations. While individuals may construct their own realities, groups—from a small one such as a married couple to large ones such as nations—often agree on notions of what is true and what “is” and what “is not.” In other words, the meanings that we construct have power beyond the individuals who create them. Therefore, the ways that people and communities act based on such meanings is of as much interest to interpretivists as how they were created in the first place. Theories like social constructionism, phenomenology, and symbolic interactionism are often used in concert with interpretivism.
Is interpretivism right for your project?
An interpretivist orientation to research is appropriate when your working question asks about subjective truths. The cause-and-effect relationships that interpretivist studies produce are specific to the time and place in which the study happened, rather than a generalizable objective truth. More pragmatically, if you picture yourself having a conversation with participants like an interview or focus group, then interpretivism is likely going to be a major influence for your study.
Positivists critique the interpretivist paradigm as non-scientific. They view the interpretivist focus on subjectivity and values as sources of bias. Positivists and interpretivists differ on the degree to which social phenomena are like natural phenomena. Positivists believe that the assumptions of the social sciences and natural sciences are the same, while interpretivists strongly believe that social sciences differ from the natural sciences because their subjects are social creatures.
Similarly, the critical paradigm finds fault with the interpretivist focus on the status quo rather than social change. Although interpretivists often proceed from a feminist or other standpoint theory, the focus is less on liberation than on understanding the present from multiple perspectives. Other critical theorists may object to the consensus orientation of interpretivist research. By searching for commonalities between people’s stories, they may erase the uniqueness of each individual’s story. For example, while interpretivists may arrive at a consensus definition of what the experience of “coming out” is like for people who identify as lesbian, gay, bisexual, transgender, or queer, it cannot represent the diversity of each person’s unique “coming out” experience and what it meant to them. For example, see Rosario and colleagues’ (2009)[28] critique the literature on lesbians “coming out” because previous studies did not addressing how appearing, behaving, or identifying as a butch or femme impacted the experience of “coming out” for lesbians.
Exercises
- From your literature search, identify an empirical article that uses qualitative methods to answer a research question similar to your working question or about your research topic.
- Review the assumptions of the interpretivist research paradigm.
- Discuss in a few sentences how the author’s conclusions are based on some of these paradigmatic assumptions. How might a researcher operating from a different paradigm (like positivism or the critical paradigm) critique the conclusions of this study?
Critical paradigm: Researcher as “activist”
As we’ve discussed a bit in the preceding sections, the critical paradigm focuses on power, inequality, and social change. Although some rather diverse perspectives are included here, the critical paradigm, in general, includes ideas developed by early social theorists, such as Max Horkheimer (Calhoun et al., 2007),[29] and later works developed by feminist scholars, such as Nancy Fraser (1989).[30] Unlike the positivist paradigm, the critical paradigm assumes that social science can never be truly objective or value-free. Furthermore, this paradigm operates from the perspective that scientific investigation should be conducted with the express goal of social change. Researchers in the critical paradigm foreground axiology, positionality and values . In contrast with the detached, “objective” observations associated with the positivist researcher, critical approaches make explicit the intention for research to act as a transformative or emancipatory force within and beyond the study.
Researchers in the critical paradigm might start with the knowledge that systems are biased against certain groups, such as women or ethnic minorities, building upon previous theory and empirical data. Moreover, their research projects are designed not only to collect data, but to impact the participants as well as the systems being studied. The critical paradigm applies its study of power and inequality to change those power imbalances as part of the research process itself. If this sounds familiar to you, you may remember hearing similar ideas when discussing social conflict theory in your human behavior in the social environment (HBSE) class.[31] Because of this focus on social change, the critical paradigm is a natural home for social work research. However, we fall far short of adopting this approach widely in our profession’s research efforts.
Is the critical paradigm right for your project?
Every social work research project impacts social justice in some way. What distinguishes critical research is how it integrates an analysis of power into the research process itself. Critical research is appropriate for projects that are activist in orientation. For example, critical research projects should have working questions that explicitly seek to raise the consciousness of an oppressed group or collaborate equitably with community members and clients to addresses issues of concern. Because of their transformative potential, critical research projects can be incredibly rewarding to complete. However, partnerships take a long time to develop and social change can evolve slowly on an issue, making critical research projects a more challenging fit for student research projects which must be completed under a tight deadline with few resources.
Positivists critique the critical paradigm on multiple fronts. First and foremost, the focus on oppression and values as part of the research process is seen as likely to bias the research process, most problematically, towards confirmation bias. If you start out with the assumption that oppression exists and must be dealt with, then you are likely to find that regardless of whether it is truly there or not. Similarly, positivists may fault critical researchers for focusing on how the world should be, rather than how it truly is. In this, they may focus too much on theoretical and abstract inquiry and less on traditional experimentation and empirical inquiry. Finally, the goal of social transformation is seen as inherently unscientific, as science is not a political practice.
Interpretivists often find common cause with critical researchers. Feminist studies, for example, may explore the perspectives of women while centering gender-based oppression as part of the research process. In interpretivist research, the focus is less on radical change as part of the research process and more on small, incremental changes based on the results and conclusions drawn from the research project. Additionally, some critical researchers’ focus on individuality of experience is in stark contrast to the consensus-orientation of interpretivists. Interpretivists seek to understand people’s true selves. Some critical theorists argue that people have multiple selves or no self at all.
Exercises
- From your literature search, identify an article relevant to your working question or broad research topic that uses a critical perspective. You should look for articles where the authors are clear that they are applying a critical approach to research like feminism, anti-racism, Marxism and critical theory, decolonization, anti-oppressive practice, or other social justice-focused theoretical perspectives. To target your search further, include keywords in your queries to research methods commonly used in the critical paradigm like participatory action research and community-based participatory research. If you have trouble identifying an article for this exercise, consult your professor for some help. These articles may be more challenging to find, but reviewing one is necessary to get a feel for what research in this paradigm is like.
- Review the assumptions of the critical research paradigm.
- Discuss in a few sentences how the author’s conclusions are based on some of these paradigmatic assumptions. How might a researcher operating from different assumptions (like values-neutrality or researcher as neutral and unbiased) critique the conclusions of this study?
Pragmatism: Researcher as “strategist”
“Essentially, all models are wrong but some are useful.” (Box, 1976)[32]
Pragmatism is a research paradigm that suspends questions of philosophical ‘truth’ and focuses more on how different philosophies, theories, and methods can be used strategically to provide a multidimensional view of a topic. Researchers employing pragmatism will mix elements of positivist, interpretivist, and critical research depending on the purpose of a particular project and the practical constraints faced by the researcher and their research context. We favor this approach for student projects because it avoids getting bogged down in choosing the “right” paradigm and instead focuses on the assumptions that help you answer your question, given the limitations of your research context. Student research projects are completed quickly and moving in the direction of pragmatism can be a route to successfully completing a project. Your project is a representation of what you think is feasible, ethical, and important enough for you to study.
The crucial consideration for the pragmatist is whether the outcomes of research have any real-world application, rather than whether they are “true.” The methods, theories, and philosophies chosen by pragmatic researchers are guided by their working question. There are no distinctively pragmatic research methods since this approach is about making judicious use whichever methods fit best with the problem under investigation. Pragmatic approaches may be less likely to prioritize ontological, epistemological or axiological consistency when combining different research methods. Instead, the emphasis is on solving a pressing problem and adapting to the limitations and opportunities in the researchers’ context.
Adopt a multi-paradigmatic perspective
Believe it or not, there is a long literature of acrimonious conflict between scientists from positivist, interpretivist, and critical camps (see Heineman-Pieper et al., 2002[33] for a longer discussion). Pragmatism is an old idea, but it is appealing precisely because it attempts to resolve the problem of multiple incompatible philosophical assumptions in social science. To a pragmatist, there is no “correct” paradigm. All paradigms rely on assumptions about the social world that are the subject of philosophical debate. Each paradigm is an incomplete understanding of the world, and it requires a scientific community using all of them to gain a comprehensive view of the social world. This multi-paradigmatic perspective is a unique gift of social work research, as our emphasis on empathy and social change makes us more critical of positivism, the dominant paradigm in social science.
We offered the metaphors of expert, empathizer, activist, and strategist for each paradigm. It’s important not to take these labels too seriously. For example, some may view that scientists should be experts or that activists are biased and unscientific. Nevertheless, we hope that these metaphors give you a sense of what it feels like to conduct research within each paradigm.
One of the unique aspects of paradigmatic thinking is that often where you think you are most at home may actually be the opposite of where your research project is. For example, in my graduate and doctoral education, I thought I was a critical researcher. In fact, I thought I was a radical researcher focused on social change and transformation. Yet, often times when I sit down to conceptualize and start a research project, I find myself squarely in the positivist paradigm, thinking through neat cause-and-effect relationships that can be mathematically measured. There is nothing wrong with that! Your task for your research project is to find the paradigm that best matches your research question. Think through what you really want to study and how you think about the topic, then use assumptions of that paradigm to guide your inquiry.
Another important lesson is that no research project fits perfectly in one paradigm or another. Instead, there is a spectrum along which studies are, to varying degrees, interpretivist, positivist, and critical. For example, all social work research is a bit activist in that our research projects are designed to inform action for change on behalf of clients and systems. However, some projects will focus on the conclusions and implications of projects informing social change (i.e., positivist and interpretivist projects) while others will partner with community members and design research projects collaboratively in a way that leads to social change (i.e. critical projects). In section 7.5, we will describe a pragmatic approach to research design guided by your paradigmatic and theoretical framework.
Key Takeaways
- Social work research falls, to some degree, in each of the four paradigms: positivism, interpretivism, critical, and pragmatist.
- Adopting a pragmatic, multi-paradigmatic approach to research makes sense for student researchers, as it directs students to use the philosophical assumptions and methodological approaches that best match their research question and research context.
- Research in all paradigms is necessary to come to a comprehensive understanding of a topic, and social workers must be able to understand and apply knowledge from each research paradigm.
Exercises
- Describe which paradigm best fits your perspective on the world and which best fits with your project.
- Identify any similarities and differences in your personal assumptions and the assumption your research project relies upon. For example, are you a more critical and radical thinker but have chosen a more “expert” role for yourself in your research project?
7.4 Developing your theoretical framework
Learning Objectives
Learners will be able to…
- Differentiate between theories that explain specific parts of the social world versus those that are more broad and sweeping in their conclusions
- Identify the theoretical perspectives that are relevant to your project and inform your thinking about it
- Define key concepts in your working question and develop a theoretical framework for how you understand your topic.
Much like paradigms, theories provide a way of looking at the world and of understanding human interaction. Paradigms are grounded in big assumptions about the world—what is real, how do we create knowledge—whereas theories describe more specific phenomena. Well, we are still oversimplifying a bit. Some theories try to explain the whole world, while others only try to explain a small part. Some theories can be grouped together based on common ideas but retain their own individual and unique features. Our goal is to help you find a theoretical framework that helps you understand your topic more deeply and answer your working question.
Theories: Big and small
In your human behavior and the social environment (HBSE) class, you were introduced to the major theoretical perspectives that are commonly used in social work. These are what we like to call big-T ‘T’heories. When you read about systems theory, you are actually reading a synthesis of decades of distinct, overlapping, and conflicting theories that can be broadly classified within systems theory. For example, within systems theory, some approaches focus more on family systems while others focus on environmental systems, though the core concepts remain similar.
Different theorists define concepts in their own way, and as a result, their theories may explore different relationships with those concepts. For example, Deci and Ryan’s (1985)[34] self-determination theory discusses motivation and establishes that it is contingent on meeting one’s needs for autonomy, competency, and relatedness. By contrast, ecological self-determination theory, as written by Abery & Stancliffe (1996),[35] argues that self-determination is the amount of control exercised by an individual over aspects of their lives they deem important across the micro, meso, and macro levels. If self-determination were an important concept in your study, you would need to figure out which of the many theories related to self-determination helps you address your working question.
Theories can provide a broad perspective on the key concepts and relationships in the world or more specific and applied concepts and perspectives. Table 7.2 summarizes two commonly used lists of big-T Theoretical perspectives in social work. See if you can locate some of the theories that might inform your project.
Payne’s (2014)[36] practice theories | Hutchison’s (2014)[37] theoretical perspectives |
Psychodynamic | Systems |
Crisis and task-centered | Conflict |
Cognitive-behavioral | Exchange and choice |
Systems/ecological | Social constructionist |
Macro practice/social development/social pedagogy | Psychodynamic |
Strengths/solution/narrative | Developmental |
Humanistic/existential/spiritual | Social behavioral |
Critical | Humanistic |
Feminist | |
Anti-discriminatory/multi-cultural sensitivity |
Competing theoretical explanations
Within each area of specialization in social work, there are many other theories that aim to explain more specific types of interactions. For example, within the study of sexual harassment, different theories posit different explanations for why harassment occurs.
One theory, first developed by criminologists, is called routine activities theory. It posits that sexual harassment is most likely to occur when a workplace lacks unified groups and when potentially vulnerable targets and motivated offenders are both present (DeCoster, Estes, & Mueller, 1999).[38]
Other theories of sexual harassment, called relational theories, suggest that one’s existing relationships are the key to understanding why and how workplace sexual harassment occurs and how people will respond when it does occur (Morgan, 1999).[39] Relational theories focus on the power that different social relationships provide (e.g., married people who have supportive partners at home might be more likely than those who lack support at home to report sexual harassment when it occurs).
Finally, feminist theories of sexual harassment take a different stance. These theories posit that the organization of our current gender system, wherein those who are the most masculine have the most power, best explains the occurrence of workplace sexual harassment (MacKinnon, 1979).[40] As you might imagine, which theory a researcher uses to examine the topic of sexual harassment will shape the questions asked about harassment. It will also shape the explanations the researcher provides for why harassment occurs.
For a graduate student beginning their study of a new topic, it may be intimidating to learn that there are so many theories beyond what you’ve learned in your theory classes. What’s worse is that there is no central database of theories on your topic. However, as you review the literature in your area, you will learn more about the theories scientists have created to explain how your topic works in the real world. There are other good sources for theories, in addition to journal articles. Books often contain works of theoretical and philosophical importance that are beyond the scope of an academic journal. Do a search in your university library for books on your topic, and you are likely to find theorists talking about how to make sense of your topic. You don’t necessarily have to agree with the prevailing theories about your topic, but you do need to be aware of them so you can apply theoretical ideas to your project.
Applying big-T theories to your topic
The key to applying theories to your topic is learning the key concepts associated with that theory and the relationships between those concepts, or propositions. Again, your HBSE class should have prepared you with some of the most important concepts from the theoretical perspectives listed in Table 7.2. For example, the conflict perspective sees the world as divided into dominant and oppressed groups who engage in conflict over resources. If you were applying these theoretical ideas to your project, you would need to identify which groups in your project are considered dominant or oppressed groups, and which resources they were struggling over. This is a very general example. Challenge yourself to find small-t theories about your topic that will help you understand it in much greater detail and specificity. If you have chosen a topic that is relevant to your life and future practice, you will be doing valuable work shaping your ideas towards social work practice.
Integrating theory into your project can be easy, or it can take a bit more effort. Some people have a strong and explicit theoretical perspective that they carry with them at all times. For me, you’ll probably see my work drawing from exchange and choice, social constructionist, and critical theory. Maybe you have theoretical perspectives you naturally employ, like Afrocentric theory or person-centered practice. If so, that’s a great place to start since you might already be using that theory (even subconsciously) to inform your understanding of your topic. But if you aren’t aware of whether you are using a theoretical perspective when you think about your topic, try writing a paragraph off the top of your head or talking with a friend explaining what you think about that topic. Try matching it with some of the ideas from the broad theoretical perspectives from Table 7.2. This can ground you as you search for more specific theories. Some studies are designed to test whether theories apply the real world while others are designed to create new theories or variations on existing theories. Consider which feels more appropriate for your project and what you want to know.
Another way to easily identify the theories associated with your topic is to look at the concepts in your working question. Are these concepts commonly found in any of the theoretical perspectives in Table 7.2? Take a look at the Payne and Hutchison texts and see if any of those look like the concepts and relationships in your working question or if any of them match with how you think about your topic. Even if they don’t possess the exact same wording, similar theories can help serve as a starting point to finding other theories that can inform your project. Remember, HBSE textbooks will give you not only the broad statements of theories but also sources from specific theorists and sub-theories that might be more applicable to your topic. Skim the references and suggestions for further reading once you find something that applies well.
Exercises
Choose a theoretical perspective from Hutchison, Payne, or another theory textbook that is relevant to your project. Using their textbooks or other reputable sources, identify :
- At least five important concepts from the theory
- What relationships the theory establishes between these important concepts (e.g., as x increases, the y decreases)
- How you can use this theory to better understand the concepts and variables in your project?
Developing your own theoretical framework
Hutchison’s and Payne’s frameworks are helpful for surveying the whole body of literature relevant to social work, which is why they are so widely used. They are one framework, or way of thinking, about all of the theories social workers will encounter that are relevant to practice. Social work researchers should delve further and develop a theoretical or conceptual framework of their own based on their reading of the literature. In Chapter 8, we will develop your theoretical framework further, identifying the cause-and-effect relationships that answer your working question. Developing a theoretical framework is also instructive for revising and clarifying your working question and identifying concepts that serve as keywords for additional literature searching. The greater clarity you have with your theoretical perspective, the easier each subsequent step in the research process will be.
Getting acquainted with the important theoretical concepts in a new area can be challenging. While social work education provides a broad overview of social theory, you will find much greater fulfillment out of reading about the theories related to your topic area. We discussed some strategies for finding theoretical information in Chapter 3 as part of literature searching. To extend that conversation a bit, some strategies for searching for theories in the literature include:
- Using keywords like “theory,” “conceptual,” or “framework” in queries to better target the search at sources that talk about theory.
- Consider searching for these keywords in the title or abstract, specifically
- Looking at the references and cited by links within theoretical articles and textbooks
- Looking at books, edited volumes, and textbooks that discuss theory
- Talking with a scholar on your topic, or asking a professor if they can help connect you to someone
- Looking at how researchers use theory in their research projects
- Nice authors are clear about how they use theory to inform their research project, usually in the introduction and discussion section.
- Starting with a Big-T Theory and looking for sub-theories or specific theorists that directly address your topic area
- For example, from the broad umbrella of systems theory, you might pick out family systems theory if you want to understand the effectiveness of a family counseling program.
It’s important to remember that knowledge arises within disciplines, and that disciplines have different theoretical frameworks for explaining the same topic. While it is certainly important for the social work perspective to be a part of your analysis, social workers benefit from searching across disciplines to come to a more comprehensive understanding of the topic. Reaching across disciplines can provide uncommon insights during conceptualization, and once the study is completed, a multidisciplinary researcher will be able to share results in a way that speaks to a variety of audiences. A study by An and colleagues (2015)[41] uses game theory from the discipline of economics to understand problems in the Temporary Assistance for Needy Families (TANF) program. In order to receive TANF benefits, mothers must cooperate with paternity and child support requirements unless they have “good cause,” as in cases of domestic violence, in which providing that information would put the mother at greater risk of violence. Game theory can help us understand how TANF recipients and caseworkers respond to the incentives in their environment, and highlight why the design of the “good cause” waiver program may not achieve its intended outcome of increasing access to benefits for survivors of family abuse.
Of course, there are natural limits on the depth with which student researchers can and should engage in a search for theory about their topic. At minimum, you should be able to draw connections across studies and be able to assess the relative importance of each theory within the literature. Just because you found one article applying your theory (like game theory, in our example above) does not mean it is important or often used in the domestic violence literature. Indeed, it would be much more common in the family violence literature to find psychological theories of trauma, feminist theories of power and control, and similar theoretical perspectives used to inform research projects rather than game theory, which is equally applicable to survivors of family violence as workers and bosses at a corporation. Consider using the Cited By feature to identify articles, books, and other sources of theoretical information that are seminal or well-cited in the literature. Similarly, by using the name of a theory in the keywords of a search query (along with keywords related to your topic), you can get a sense of how often the theory is used in your topic area. You should have a sense of what theories are commonly used to analyze your topic, even if you end up choosing a different one to inform your project.
Theories that are not cited or used as often are still immensely valuable. As we saw before with TANF and “good cause” waivers, using theories from other disciplines can produce uncommon insights and help you make a new contribution to the social work literature. Given the privileged position that the social work curriculum places on theories developed by white men, students may want to explore Afrocentricity as a social work practice theory (Pellebon, 2007)[42] or abolitionist social work (Jacobs et al., 2021)[43] when deciding on a theoretical framework for their research project that addresses concepts of racial justice. Start with your working question, and explain how each theory helps you answer your question. Some explanations are going to feel right, and some concepts will feel more salient to you than others. Keep in mind that this is an iterative process. Your theoretical framework will likely change as you continue to conceptualize your research project, revise your research question, and design your study.
By trying on many different theoretical explanations for your topic area, you can better clarify your own theoretical framework. Some of you may be fortunate enough to find theories that match perfectly with how you think about your topic, are used often in the literature, and are therefore relatively straightforward to apply. However, many of you may find that a combination of theoretical perspectives is most helpful for you to investigate your project. For example, maybe the group counseling program for which you are evaluating client outcomes draws from both motivational interviewing and cognitive behavioral therapy. In order to understand the change happening in the client population, you would need to know each theory separately as well as how they work in tandem with one another. Because theoretical explanations and even the definitions of concepts are debated by scientists, it may be helpful to find a specific social scientist or group of scientists whose perspective on the topic you find matches with your understanding of the topic. Of course, it is also perfectly acceptable to develop your own theoretical framework, though you should be able to articulate how your framework fills a gap within the literature.
If you are adapting theoretical perspectives in your study, it is important to clarify the original authors’ definitions of each concept. Jabareen (2009)[44] offers that conceptual frameworks are not merely collections of concepts but, rather, constructs in which each concept plays an integral role.[45] A conceptual framework is a network of linked concepts that together provide a comprehensive understanding of a phenomenon. Each concept in a conceptual framework plays an ontological or epistemological role in the framework, and it is important to assess whether the concepts and relationships in your framework make sense together. As your framework takes shape, you will find yourself integrating and grouping together concepts, thinking about the most important or least important concepts, and how each concept is causally related to others.
Much like paradigm, theory plays a supporting role for the conceptualization of your research project. Recall the ice float from Figure 7.1. Theoretical explanations support the design and methods you use to answer your research question. In student projects that lack a theoretical framework, I often see the biases and errors in reasoning that we discussed in Chapter 1 that get in the way of good social science. That’s because theories mark which concepts are important, provide a framework for understanding them, and measure their interrelationships. If you are missing this foundation, you will operate on informal observation, messages from authority, and other forms of unsystematic and unscientific thinking we reviewed in Chapter 1.
Theory-informed inquiry is incredibly helpful for identifying key concepts and how to measure them in your research project, but there is a risk in aligning research too closely with theory. The theory-ladenness of facts and observations produced by social science research means that we may be making our ideas real through research. This is a potential source of confirmation bias in social science. Moreover, as Tan (2016)[46] demonstrates, social science often proceeds by adopting as true the perspective of Western and Global North countries, and cross-cultural research is often when ethnocentric and biased ideas are most visible. In her example, a researcher from the West studying teacher-centric classrooms in China that rely partially on rote memorization may view them as less advanced than student-centered classrooms developed in a Western country simply because of Western philosophical assumptions about the importance of individualism and self-determination. Developing a clear theoretical framework is a way to guard against biased research, and it will establish a firm foundation on which you will develop the design and methods for your study.
Key Takeaways
- Just as empirical evidence is important for conceptualizing a research project, so too are the key concepts and relationships identified by social work theory.
- Using theory your theory textbook will provide you with a sense of the broad theoretical perspectives in social work that might be relevant to your project.
- Try to find small-t theories that are more specific to your topic area and relevant to your working question.
Exercises
In Chapter 2, you developed a concept map for your proposal. Take a moment to revisit your concept map now as your theoretical framework is taking shape. Make any updates to the key concepts and relationships in your concept map.
. If you need a refresher, we have embedded a short how-to video from the University of Guelph Library (CC-BY-NC-SA 4.0) that we also used in Chapter 2.
7.5 Designing your project using theory and paradigm
Learning Objectives
Learners will be able to…
- Apply the assumptions of each paradigm to your project
- Summarize what aspects of your project stem from positivist, interpretivist, or critical assumptions
In the previous sections, we reviewed the major paradigms and theories in social work research. In this section, we will provide an example of how to apply theory and paradigm in research. This process is depicted in Figure 7.2 below with some quick summary questions for each stage. Some questions in the figure below have example answers like designs (i.e., experimental, survey) and data analysis approaches (i.e., discourse analysis). These examples are arbitrary. There are a lot of options that are not listed. So, don’t feel like you have to memorize them or use them in your study.
This diagram (taken from an archived Open University (UK) course entitled E89-Educational Inquiry) shows one way to visualize the research design process. While research is far from linear, in general, this is how research projects progress sequentially. Researchers begin with a working question, and through engaging with the literature, develop and refine those questions into research questions (a process we will finalize in Chapter 9). But in order to get to the part where you gather your sample, measure your participants, and analyze your data, you need to start with paradigm. Based on your work in section 7.3, you should have a sense of which paradigm or paradigms are best suited to answering your question. The approach taken will often reflect the nature of the research question; the kind of data it is possible to collect; and work previously done in the area under consideration. When evaluating paradigm and theory, it is important to look at what other authors have done previously and the framework used by studies that are similar to the one you are thinking of conducting.
Once you situate your project in a research paradigm, it becomes possible to start making concrete choices about methods. Depending on the project, this will involve choices about things like:
- What is my final research question?
- What are the key variables and concepts under investigation, and how will I measure them?
- How do I find a representative sample of people who experience the topic I’m studying?
- What design is most appropriate for my research question?
- How will I collect and analyze data?
- How do I determine whether my results describe real patterns in the world or are the result of bias or error?
The data collection phase can begin once these decisions are made. It can be very tempting to start collecting data as soon as possible in the research process as this gives a sense of progress. However, it is usually worth getting things exactly right before collecting data as an error found in your approach further down the line can be harder to correct or recalibrate around.
Designing a study using paradigm and theory: An example
Paradigm and theory have the potential to turn some people off since there is a lot of abstract terminology and thinking about real-world social work practice contexts. In this section, I’ll use an example from my own research, and I hope it will illustrate a few things. First, it will show that paradigms are really just philosophical statements about things you already understand and think about normally. It will also show that no project neatly sits in one paradigm and that a social work researcher should use whichever paradigm or combination of paradigms suit their question the best. Finally, I hope it is one example of how to be a pragmatist and strategically use the strengths of different theories and paradigms to answering a research question. We will pick up the discussion of mixed methods in the next chapter.
Thinking as an expert: Positivism
In my undergraduate research methods class, I used an open textbook much like this one and wanted to study whether it improved student learning. You can read a copy of the article we wrote on based on our study. We’ll learn more about the specifics of experiments and evaluation research in Chapter 13, but you know enough to understand what evaluating an intervention might look like. My first thought was to conduct an experiment, which placed me firmly within the positivist or “expert” paradigm.
Experiments focus on isolating the relationship between cause and effect. For my study, this meant studying an open textbook (the cause, or intervention) and final grades (the effect, or outcome). Notice that my position as “expert” lets me assume many things in this process. First, it assumes that I can distill the many dimensions of student learning into one number—the final grade. Second, as the “expert,” I’ve determined what the intervention is: indeed, I created the book I was studying, and applied a theory from experts in the field that explains how and why it should impact student learning.
Theory is part of applying all paradigms, but I’ll discuss its impact within positivism first. Theories grounded in positivism help explain why one thing causes another. More specifically, these theories isolate a causal relationship between two (or more) concepts while holding constant the effects of other variables that might confound the relationship between the key variables. That is why experimental design is so common in positivist research. The researcher isolates the environment from anything that might impact or bias the cause and effect relationship they want to investigate.
But in order for one thing to lead to change in something else, there must be some logical, rational reason why it would do so. In open education, there are a few hypotheses (though no full-fledged theories) on why students might perform better using open textbooks. The most common is the access hypothesis, which states that students who cannot afford expensive textbooks or wouldn’t buy them anyway can access open textbooks because they are free, which will improve their grades. It’s important to note that I held this theory prior to starting the experiment, as in positivist research you spell out your hypotheses in advance and design an experiment to support or refute that hypothesis.
Notice that the hypothesis here applies not only to the people in my experiment, but to any student in higher education. Positivism seeks generalizable truth, or what is true for everyone. The results of my study should provide evidence that anyone who uses an open textbook would achieve similar outcomes. Of course, there were a number of limitations as it was difficult to tightly control the study. I could not randomly assign students or prevent them from sharing resources with one another, for example. So, while this study had many positivist elements, it was far from a perfect positivist study because I was forced to adapt to the pragmatic limitations of my research context (e.g., I cannot randomly assign students to classes) that made it difficult to establish an objective, generalizable truth.
Thinking like an empathizer: Interpretivism
One of the things that did not sit right with me about the study was the reliance on final grades to signify everything that was going on with students. I added another quantitative measure that measured research knowledge, but this was still too simplistic. I wanted to understand how students used the book and what they thought about it. I could create survey questions that ask about these things, but to get at the subjective truths here, I thought it best to use focus groups in which students would talk to one another with a researcher moderating the discussion and guiding it using predetermined questions. You will learn more about focus groups in Chapter 18.
Researchers spoke with small groups of students during the last class of the semester. They prompted people to talk about aspects of the textbook they liked and didn’t like, compare it to textbooks from other classes, describe how they used it, and so forth. It was this focus on understanding and subjective experience that brought us into the interpretivist paradigm. Alongside other researchers, I created the focus group questions but encouraged researchers who moderated the focus groups to allow the conversation to flow organically.
We originally started out with the assumption, for which there is support in the literature, that students would be angry with the high-cost textbook that we used prior to the free one, and this cost shock might play a role in students’ negative attitudes about research. But unlike the hypotheses in positivism, these are merely a place to start and are open to revision throughout the research process. This is because the researchers are not the experts, the participants are! Just like your clients are the experts on their lives, so were the students in my study. Our job as researchers was to create a group in which they would reveal their informed thoughts about the issue, coming to consensus around a few key themes.
When we initially analyzed the focus groups, we uncovered themes that seemed to fit the data. But the overall picture was murky. How were themes related to each other? And how could we distill these themes and relationships into something meaningful? We went back to the data again. We could do this because there isn’t one truth, as in positivism, but multiple truths and multiple ways of interpreting the data. When we looked again, we focused on some of the effects of having a textbook customized to the course. It was that customization process that helped make the language more approachable, engaging, and relevant to social work practice.
Ultimately, our data revealed differences in how students perceived a free textbook versus a free textbook that is customized to the class. When we went to interpret this finding, the remix hypothesis of open textbook was helpful in understanding that relationship. It states that the more faculty incorporate editing and creating into the course, the better student learning will be. Our study helped flesh out that theory by discussing the customization process and how students made sense of a customized resource.
In this way, theoretical analysis operates differently in interpretivist research. While positivist research tests existing theories, interpretivist research creates theories based on the stories of research participants. However, it is difficult to say if this theory was totally emergent in the dataset or if my prior knowledge of the remix hypothesis influenced my thinking about the data. Interpretivist researchers are encouraged to put a box around their prior experiences and beliefs, acknowledging them, but trying to approach the data with fresh eyes. Interpretivists know that this is never perfectly possible, though, as we are always influenced by our previous experiences when interpreting data and conducting scientific research projects.
Thinking like an activist: Critical
Although adding focus groups helped ease my concern about reducing student learning down to just final grades by providing a more rich set of conversations to analyze. However, my role as researcher and “expert” was still an important part of the analysis. As someone who has been out of school for a while, and indeed has taught this course for years, I have lost touch with what it is like to be a student taking research methods for the first time. How could I accurately interpret or understand what students were saying? Perhaps I would overlook things that reflected poorly on my teaching or my book. I brought other faculty researchers on board to help me analyze the data, but this still didn’t feel like enough.
By luck, an undergraduate student approached me about wanting to work together on a research project. I asked her if she would like to collaborate on evaluating the textbook with me. Over the next year, she assisted me with conceptualizing the project, creating research questions, as well as conducting and analyzing the focus groups. Not only would she provide an “insider” perspective on coding the data, steeped in her lived experience as a student, but she would serve as a check on my power through the process.
Including people from the group you are measuring as part of your research team is a common component of critical research. Ultimately, critical theorists would find my study to be inadequate in many ways. I still developed the research question, created the intervention, and wrote up the results for publication, which privileges my voice and role as “expert.” Instead, critical theorists would emphasize the role of students (community members) in identifying research questions, choosing the best intervention to used, and so forth. But collaborating with students as part of a research team did address some of the power imbalances in the research process.
Critical research projects also aim to have an impact on the people and systems involved in research. No students or researchers had profound personal realizations as a result of my study, nor did it lessen the impact of oppressive structures in society. I can claim some small victory that my department switched to using my textbook after the study was complete (changing a system), though this was likely the result of factors other than the study (my advocacy for open textbooks).
Social work research is almost always designed to create change for people or systems. To that end, every social work project is at least somewhat critical. However, the additional steps of conducting research with people rather than on people reveal a depth to the critical paradigm. By bringing students on board the research team, study had student perspectives represented in conceptualization, data collection, and analysis. That said, there was much to critique about this study from a critical perspective. I retained a lot of the power in the research process, and students did not have the ability to determine the research question or purpose of the project. For example, students might likely have said that textbook costs and the quality of their research methods textbook were less important than student debt, racism, or other potential issues experienced by students in my class. Instead of a ground-up research process based in community engagement, my research included some important participation by students on project created and led by faculty.
Conceptualization is an iterative process
I hope this conversation was useful in applying paradigms to a research project. While my example discusses education research, the same would apply for social work research about social welfare programs, clinical interventions, or other topics. Paradigm and theory are covered at the beginning of the conceptualization of your project because these assumptions will structure the rest of your project. Each of the research steps that occur after this chapter (e.g., forming a question, choosing a design) rely upon philosophical and theoretical assumptions. As you continue conceptualizing your project over the next few weeks, you may find yourself shifting between paradigms. That is normal, as conceptualization is not a linear process. As you move through the next steps of conceptualizing and designing a project, you’ll find philosophies and theories that best match how you want to study your topic.
Viewing theoretical and empirical arguments through this lens is one of the true gifts of the social work approach to research. The multi-paradigmatic perspective is a hallmark of social work research and one that helps us contribute something unique on research teams and in practice.
Key Takeaways
- Multi-paradigmatic research is a distinguishing hallmark of social work research. Understanding the limitations and strengths of each paradigm will help you justify your research approach and strategically choose elements from one or more paradigms to answer your question.
- Paradigmatic assumptions help you understand the “blind spots” in your research project and how to adjust and address these areas. Keep in mind, it is not necessary to address all of your blind spots, as all projects have limitations.
Exercises
- Sketch out which paradigm applies best to your project. Second, building on your answer to the exercise in section 7.3, identify how the theory you chose and the paradigm in which you find yourself are consistent or are in conflict with one another. For example, if you are using systems theory in a positivist framework, you might talk about how they both rely on a deterministic approach to human behavior with a focus on the status-quo and social order.
- Koerth, M. & Thomson-DeVeaux, A. (2020, August 3). Many Americans are convinced crime is rising in the U.S. They're wrong. FiveThirtyEight. Retrieved from: https://fivethirtyeight.com/features/many-americans-are-convinced-crime-is-rising-in-the-u-s-theyre-wrong ↵
- Burrell, G. & Morgan, G. (1979). Sociological paradigms and organizational analysis. Routledge. ↵
- Here are links to two HBSE open textbooks, if you are unfamiliar with social work theories. https://uark.pressbooks.pub/hbse1/ and https://uark.pressbooks.pub/humanbehaviorandthesocialenvironment2/ ↵
- Lin, C. T. (2016). A critique of epistemic subjectivity. Philosophia, 44(3), 915-920. ↵
- Wills, J. W. (2007). World views, paradigms and the practice of social science research. Thousand Oaks, CA: Sage. ↵
- Dale, M. (2021, June 2). NFL pledges to half 'race-norming,' review Black claims. Associated Press. Retrieved from: https://apnews.com/article/pa-state-wire-race-and-ethnicity-health-nfl-sports-205b304c0c3724532d74fc54e58b4d1d ↵
- Bell, D. A. (1995). Who's afraid of critical race theory. University of Illinois Law Review, 1995(4), 893-910. ↵
- Crasnow, S. (2020). Feminist perspectives on science. In E. N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2020 Edition). Retrieved from: https://plato.stanford.edu/entries/feminist-science/ ↵
- Grogan, K.E. (2019) How the entire scientific community can confront gender bias in the workplace. Nature Ecology & Evolution, 3, 3–6. doi:10.1038/s41559-018-0747-4 ↵
- Tasca, C., Rapetti, M., Carta, M. G., & Fadda, B. (2012). Women and hysteria in the history of mental health. Clinical practice and epidemiology in mental health: Clinical practice & epidemiology in mental health, 8, 110-119. ↵
- Klonsky, E. D., Jane, J. S., Turkheimer, E., & Oltmanns, T. F. (2002). Gender role and personality disorders. Journal of personality disorders, 16(5), 464-476. ↵
- Smith, L. T. (2013). Decolonizing methodologies: Research and indigenous peoples. Zed Books Ltd. ↵
- Fricker, M. (2011). Epistemic injustice: Power and the ethics of knowing. Oxford University Press. ↵
- Cohn, N. & Quealy, K. (2020, June 10). How public opinion has moved on Black Lives Matter. The New York Times. Retrieved from: https://www.nytimes.com/interactive/2020/06/10/upshot/black-lives-matter-attitudes.html ↵
- Bautista, M., Bertrand, M., Morrell, E., Scorza, D. A., & Matthews, C. (2013). Participatory action research and city youth: Methodological insights from the Council of Youth Research. Teachers College Record, 115(10), 1-23. ↵
- Constance-Huggins, M., Davis, A., & Yang, J. (2020). Race Still Matters: The Relationship Between Racial and Poverty Attitudes Among Social Work Students. Advances in Social Work, 20(1), 132-151. ↵
- Crotty, M. (1998). The foundations of social research: Meaning and perspective in the research process. London: SAGE.; Guba E., & Lincoln, Y., (1994). Competing paradigms in qualitative research (pp. 105-118). In Denzin, N. & Lincoln, Y (eds.) Handbook on qualitative research. Thousand Oaks, Ca: Sage.; Heron, J. & Reason, P. (1997). A participatory inquiry paradigm. Qualitative Inquiry. 3(3), 274-294. ↵
- Kuhn, T. (1962). The structure of scientific revolutions. Chicago: University of Chicago Press. ↵
- Fleuridas, C., & Krafcik, D. (2019). Beyond four forces: The evolution of psychotherapy. Sage Open, 9(1), 2158244018824492. ↵
- Shneider, A. M. (2009). Four stages of a scientific discipline; four types of scientist. Trends in Biochemical Sciences 34 (5), 217-233. https://doi.org/10.1016/j.tibs.2009.02.00 ↵
- Burrell, G. & Morgan, G. (1979). Sociological paradigms and organizational analysis. Routledge. Guba, E. (ed.) (1990). The paradigm dialog. SAGE. ↵
- Routledge. Guba, E. (ed.) (1990). The paradigm dialog. SAGE. ↵
- Burrell, G. & Morgan, G. (1979). Sociological paradigms and organizational analysis. Here is a summary of Burrell & Morgan from Babson College, and our classification collapses radical humanism and radical structuralism into the critical paradigm, following Guba and Lincoln's three-paradigm framework. We feel this approach is more parsimonious and easier for students to understand on an introductory level. ↵
- Kivuna, C. & Kuyini, A. B. (2017). Understanding and applying research paradigms in educational contexts. International Journal of Higher Education, 6(5), 26-41. https://eric.ed.gov/?id=EJ1154775 ↵
- Kincheloe, J. L. & Tobin, K. (2009). The much exaggerated death of positivism. Cultural studies of science education, 4, 513-528. ↵
- For more about how the meanings of hand gestures vary by region, you might read the following blog entry: Wong, W. (2007). The top 10 hand gestures you’d better get right. Retrieved from: http://www.languagetrainers.co.uk/blog/2007/09/24/top-10-hand-gestures ↵
- Kivuna, C. & Kuyini, A. B. (2017). Understanding and applying research paradigms in educational contexts. International Journal of Higher Education, 6(5), 26-41. https://eric.ed.gov/?id=EJ1154775 ↵
- Rosario, M., Schrimshaw, E. W., Hunter, J., & Levy-Warren, A. (2009). The coming-out process of young lesbian and bisexual women: Are there butch/femme differences in sexual identity development?. Archives of sexual behavior, 38(1), 34-49. ↵
- Calhoun, C., Gerteis, J., Moody, J., Pfaff, S., & Virk, I. (Eds.). (2007). Classical sociological theory (2nd ed.). Malden, MA: Blackwell. ↵
- Fraser, N. (1989). Unruly practices: Power, discourse, and gender in contemporary social theory. Minneapolis, MN: University of Minnesota Press. ↵
- Here are links to two HBSE open textbooks, if you are unfamiliar with social work theories and would like more background. https://uark.pressbooks.pub/hbse1/ and https://uark.pressbooks.pub/humanbehaviorandthesocialenvironment2/ ↵
- Box, G. E. P.. (1976). Science and statistics. Journal of the American Statistical Association, 71(356), 791. ↵
- Heineman-Pieper, J., Tyson, K., & Pieper, M. H. (2002). Doing good science without sacrificing good values: Why the heuristic paradigm is the best choice for social work. Families in Society, 83(1), 15-28. ↵
- Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Self-determination in personality. Journal of research in personality, 19(2), 109-134. ↵
- Abery, B., & Stancliffe, R. (1996). The ecology of self-determination. in Self-determination across the life span: Independence and choice for people with disabilities (pp. 111-145.) Baltimore, MD: Paul H. Brookes Publishing Company ↵
- Payne, M. (2014). Modern social work theory. Oxford University Press. ↵
- Hutchison, E. D. (2014). Dimensions of human behavior: Person and environment. Sage Publications. ↵
- DeCoster, S., Estes, S. B., & Mueller, C. W. (1999). Routine activities and sexual harassment in the workplace. Work and Occupations, 26, 21–49. ↵
- Morgan, P. A. (1999). Risking relationships: Understanding the litigation choices of sexually harassed women. The Law and Society Review, 33, 201–226. ↵
- MacKinnon, C. (1979). Sexual harassment of working women: A case of sex discrimination. New Haven, CT: Yale University Press. ↵
- An, S., Yoo, J., & Nackerud, L. G. (2015). Using game theory to understand screening for domestic violence under the TANF family violence option. Advances in Social Work, 16(2), 338-357. ↵
- Pellebon, D. A. (2007). An analysis of Afrocentricity as theory for social work practice. Advances in Social Work, 8(1), 169-183. ↵
- Jacobs, L. A., Kim, M. E., Whitfield, D. L., Gartner, R. E., Panichelli, M., Kattari, S. K., ... & Mountz, S. E. (2021). Defund the police: Moving towards an anti-carceral social work. Journal of Progressive Human Services, 32(1), 37-62. ↵
- Jabareen, Y. (2009). Building a conceptual framework: philosophy, definitions, and procedure. International journal of qualitative methods, 8(4), 49-62. ↵
- Jabareen distinguishes between theoretical and conceptual frameworks. We agree with this distinction, but feel that this additional detail is not needed here. ↵
- Tan, C. (2016). Investigator bias and theory-ladenness in cross-cultural research: Insights from Wittgenstein. Current Issues in Comparative Education, 18(1), 83-95. ↵
A composite scale using a series of items arranged in increasing order of intensity of the construct of interest, from least intense to most intense.
(also known as bias) refers to when a measure consistently outputs incorrect data, usually in one direction and due to an identifiable process
This is the document where you list your variable names, what the variables actually measure or represent, what each of the values of the variable mean if the meaning isn't obvious.
Data someone else has collected that you have permission to use in your research.
The rows in your data set. In social work, these are often your study participants (people), but can be anything from census tracts to black bears to trains.
The name of your variable.
Chapter Outline
- Where do I start with quantitative data analysis? (12 minute read)
- Measures of central tendency (17 minute read, including 5-minute video)
- Frequencies and variability (13 minute read)
Content warning: examples in this chapter contain references to depression and self-esteem.
People often dread quantitative data analysis because—oh no—it's math. And true, you're going to have to work with numbers. For years, I thought I was terrible at math, and then I started working with data and statistics, and it turned out I had a real knack for it. (I have a statistician friend who claims statistics is not math, which is a math joke that's way over my head, but there you go.) This chapter, and the subsequent quantitative analysis chapters, are going to focus on helping you understand descriptive statistics and a few statistical tests, NOT calculate them (with a couple of exceptions). Future research classes will focus on teaching you to calculate these tests for yourself. So take a deep breath and clear your mind of any doubts about your ability to understand and work with numerical data.
In this chapter, we're going to discuss the first step in analyzing your quantitative data: univariate data analysis. Univariate data analysis is a quantitative method in which a variable is examined individually to determine its distribution, or the way the scores are distributed across the levels of that variable. When we talk about levels, what we are talking about are the possible values of the variable—like a participant's age, income or gender. (Note that this is different than our earlier discussion in Chaper 10 of levels of measurement, but the level of measurement of your variables absolutely affects what kinds of analyses you can do with it.) Univariate analysis is non-relational, which just means that we're not looking into how our variables relate to each other. Instead, we're looking at variables in isolation to try to understand them better. For this reason, univariate analysis is best for descriptive research questions.
So when do you use univariate data analysis? Always! It should be the first thing you do with your quantitative data, whether you are planning to move on to more sophisticated statistical analyses or are conducting a study to describe a new phenomenon. You need to understand what the values of each variable look like—what if one of your variables has a lot of missing data because participants didn't answer that question on your survey? What if there isn't much variation in the gender of your sample? These are things you'll learn through univariate analysis.
14.1 Where do I start with quantitative data analysis?
Learning Objectives
Learners will be able to...
- Define and construct a data analysis plan
- Define key data management terms—variable name, data dictionary, primary and secondary data, observations/cases
No matter how large or small your data set is, quantitative data can be intimidating. There are a few ways to make things manageable for yourself, including creating a data analysis plan and organizing your data in a useful way. We'll discuss some of the keys to these tactics below.
The data analysis plan
As part of planning for your research, and to help keep you on track and make things more manageable, you should come up with a data analysis plan. You've basically been working on doing this in writing your research proposal so far. A data analysis plan is an ordered outline that includes your research question, a description of the data you are going to use to answer it, and the exact step-by-step analyses, that you plan to run to answer your research question. This last part—which includes choosing your quantitative analyses—is the focus of this and the next two chapters of this book.
A basic data analysis plan might look something like what you see in Table 14.1. Don't panic if you don't yet understand some of the statistical terms in the plan; we're going to delve into them throughout the next few chapters. Note here also that this is what operationalizing your variables and moving through your research with them looks like on a basic level.
Research question: What is the relationship between a person's race and their likelihood to graduate from high school? |
Data: Individual-level U.S. American Community Survey data for 2017 from IPUMS, which includes race/ethnicity and other demographic data (i.e., educational attainment, family income, employment status, citizenship, presence of both parents, etc.). Only including individuals for which race and educational attainment data is available. |
Steps in Data Analysis Plan
|
An important point to remember is that you should never get stuck on using a particular statistical method because you or one of your co-researchers thinks it's cool or it's the hot thing in your field right now. You should certainly go into your data analysis plan with ideas, but in the end, you need to let your research question and the actual content of your data guide what statistical tests you use. Be prepared to be flexible if your plan doesn't pan out because the data is behaving in unexpected ways.
Managing your data
Whether you've collected your own data or are using someone else's data, you need to make sure it is well-organized in a database in a way that's actually usable. "Database" can be kind of a scary word, but really, I just mean an Excel spreadsheet or a data file in whatever program you're using to analyze your data (like SPSS, SAS, or r). (I would avoid Excel if you've got a very large data set—one with millions of records or hundreds of variables—because it gets very slow and can only handle a certain number of cases and variables, depending on your version. But if your data set is smaller and you plan to keep your analyses simple, you can definitely get away with Excel.) Your database or data set should be organized with variables as your columns and observations/cases as your rows. For example, let's say we did a survey on ice cream preferences and collected the following information in Table 14.2:
Name | Age | Gender | Hometown | Fav_Ice_Cream |
Tom | 54 | 0 | 1 | Rocky Road |
Jorge | 18 | 2 | 0 | French Vanilla |
Melissa | 22 | 1 | 0 | Espresso |
Amy | 27 | 1 | 0 | Black Cherry |
There are a few key data management terms to understand:
- Variable name: Just what it sounds like—the name of your variable. Make sure this is something useful, short and, if you're using something other than Excel, all one word. Most statistical programs will automatically rename variables for you if they aren't one word, but the names are usually a little ridiculous and long.
- Observations/cases: The rows in your data set. In social work, these are often your study participants (people), but can be anything from census tracts to black bears to trains. When we talk about sample size, we're talking about the number of observations/cases. In our mini data set, each person is an observation/case.
- Primary data: Data you have collected yourself.
- Secondary data: Data someone else has collected that you have permission to use in your research. For example, for my student research project in my MSW program, I used data from a local probation program to determine if a shoplifting prevention group was reducing the rate at which people were re-offending. I had data on who participated in the program and then received their criminal history six months after the end of their probation period. This was secondary data I used to determine whether the shoplifting prevention group had any effect on an individual's likelihood of re-offending.
- Data dictionary (sometimes called a code book): This is the document where you list your variable names, what the variables actually measure or represent, what each of the values of the variable mean if the meaning isn't obvious (i.e., if there are numbers assigned to gender), the level of measurement and anything special to know about the variables (for instance, the source if you mashed two data sets together). If you're using secondary data, the data dictionary should be available to you.
When considering what data you might want to collect as part of your project, there are two important considerations that can create dilemmas for researchers. You might only get one chance to interact with your participants, so you must think comprehensively in your planning phase about what information you need and collect as much relevant data as possible. At the same time, though, especially when collecting sensitive information, you need to consider how onerous the data collection is for participants and whether you really need them to share that information. Just because something is interesting to us doesn't mean it's related enough to our research question to chase it down. Work with your research team and/or faculty early in your project to talk through these issues before you get to this point. And if you're using secondary data, make sure you have access to all the information you need in that data before you use it.
Let's take that mini data set we've got up above and I'll show you what your data dictionary might look like in Table 14.3.
Variable name | Description | Values/Levels | Level of measurement | Notes |
Name | Participant's first name | n/a | n/a | First names only. If names appear more than once, a random number has been attached to the end of the name to distinguish. |
Age | Participant's age at time of survey | n/a | Interval/Ratio | Self-reported |
Gender | Participant's self-identified gender | 0=cisgender female1=cisgender male2=non-binary3=transgender female4=transgender male5=another gender | Nominal | Self-reported |
Hometown | Participant's hometown—this town or another town | 0=This town
1=Another town |
Nominal | Self-reported |
Fav_Ice_Cream | Participant's favorite ice cream | n/a | n/a | Self-reported |
Key Takeaways
- Getting organized at the beginning of your project with a data analysis plan will help keep you on track. Data analysis plans should include your research question, a description of your data, and a step-by-step outline of what you're going to do with it.
- Be flexible with your data analysis plan—sometimes data surprises us and we have to adjust the statistical tests we are using.
- Always make a data dictionary or, if using secondary data, get a copy of the data dictionary so you (or someone else) can understand the basics of your data.
Exercises
- Make a data analysis plan for your project. Remember this should include your research question, a description of the data you will use, and a step-by-step outline of what you're going to do with your data once you have it, including statistical tests (non-relational and relational) that you plan to use. You can do this exercise whether you're using quantitative or qualitative data! The same principles apply.
- Make a data dictionary for the data you are proposing to collect as part of your study. You can use the example above as a template.
14.2 Measures of central tendency
Learning Objectives
Learners will be able to...
- Explain measures of central tendency—mean, median and mode—and when to use them to describe your data
- Explain the importance of examining the range of your data
- Apply the appropriate measure of central tendency to a research problem or question
A measure of central tendency is one number that can give you an idea about the distribution of your data. The video below gives a more detailed introduction to central tendency. Then we'll talk more specifically about our three measures of central tendency—mean, median and mode.
One quick note: the narrator in the video mentions skewness and kurtosis. Basically, these refer to a particular shape for a distribution when you graph it out. That gets into some more advanced multivariate analysis that we aren't tackling in this book, so just file them away for a more advanced class, if you ever take on additional statistics coursework.
There are three key measures of central tendency, which we'll go into now.
Mean
The mean, also called the average, is calculated by adding all your cases and dividing the sum by the number of cases. You've undoubtedly calculated a mean at some point in your life. The mean is the most widely used measure of central tendency because it's easy to understand and calculate. It can only be used with interval/ratio variables, like age, test scores or years of post-high school education. (If you think about it, using it with a nominal or ordinal variable doesn't make much sense—why do we care about the average of our numerical values we assigned to certain races?)
The biggest drawback of using the mean is that it's extremely sensitive to outliers, or extreme values in your data. And the smaller your data set is, the more sensitive your mean is to these outliers. One thing to remember about outliers—they are not inherently bad, and can sometimes contain really important information. Don't automatically discard them because they skew your data.
Let's take a minute to talk about how to locate outliers in your data. If your data set is very small, you can just take a look at it and see outliers. But in general, you're probably going to be working with data sets that have at least a couple dozen cases, which makes just looking at your values to find outliers difficult. The best way to quickly look for outliers is probably to make a scatter plot with excel or whatever database management program you're using.
Let's take a very small data set as an example. Oh hey, we had one before! I've re-created it in Table 14.5. We're going to add some more cases to it so it's a little easier to illustrate what we're doing.
Name | Age | Gender | Hometown | Fav_Ice_Cream |
Tom | 54 | 0 | 1 | Rocky Road |
Jorge | 18 | 2 | 0 | French Vanilla |
Melissa | 22 | 1 | 0 | Espresso |
Amy | 27 | 1 | 0 | Black Cherry |
Akiko | 28 | 3 | 0 | Chocolate |
Michael | 32 | 0 | 1 | Pistachio |
Jess | 29 | 1 | 0 | Chocolate |
Subasri | 34 | 1 | 0 | Vanilla Bean |
Brian | 21 | 0 | 1 | Moose Tracks |
Crystal | 18 | 1 | 0 | Strawberry |
Let's say we're interested in knowing more about the distribution of participant age. Let's see a scatterplot of age (Figure 14.1). On our y-axis (the vertical one) is the value of age, and on our x-axis (the horizontal one) is the frequency of each age, or the number of times it appears in our data set.
Do you see any outliers in the scatter plot? There is one participant who is significantly older than the rest at age 54. Let's think about what happens when we calculate our mean with and without that outlier. Complete the two exercises below by using the ages listed in our mini-data set in this section.
Next, let's try it without the outlier.
With our outlier, the average age of our participants is 28, and without it, the average age is 25. That might not seem enormous, but it illustrates the effects of outliers on the mean.
Just because Tom is an outlier at age 54 doesn't mean you should exclude him. The most important thing about outliers is to think critically about them and how they could affect your analysis. Finding outliers should prompt a couple of questions. First, could the data have been entered incorrectly? Is Tom actually 24, and someone just hit the "5" instead of the "2" on the number pad? What might be special about Tom that he ended up in our group, given how that he is different? Are there other relevant ways in which Tom differs from our group (is he an outlier in other ways)? Does it really matter than Tom is much older than our other participants? If we don't think age is a relevant factor in ice cream preferences, then it probably doesn't. If we do, then we probably should have made an effort to get a wider range of ages in our participants.
Median
The median (also called the 50th percentile) is the middle value when all our values are placed in numerical order. If you have five values and you put them in numerical order, the third value will be the median. When you have an even number of values, you'll have to take the average of the middle two values to get the median. So, if you have 6 values, the average of values 3 and 4 will be the median. Keep in mind that for large data sets, you're going to want to use either Excel or a statistical program to calculate the median—otherwise, it's nearly impossible logistically.
Like the mean, you can only calculate the median with interval/ratio variables, like age, test scores or years of post-high school education. The median is also a lot less sensitive to outliers than the mean. While it can be more time intensive to calculate, the median is preferable in most cases to the mean for this reason. It gives us a more accurate picture of where the middle of our distribution sits in most cases. In my work as a policy analyst and researcher, I rarely, if ever, use the mean as a measure of central tendency. Its main value for me is to compare it to the median for statistical purposes. So get used to the median, unless you're specifically asked for the mean. (When we talk about t-tests in the next chapter, we'll talk about when the mean can be useful.)
Let's go back to our little data set and calculate the median age of our participants (Table 14.6).
Name | Age | Gender | Hometown | Fav_Ice_Cream |
Tom | 54 | 0 | 1 | Rocky Road |
Jorge | 18 | 2 | 0 | French Vanilla |
Melissa | 22 | 1 | 0 | Espresso |
Amy | 27 | 1 | 0 | Black Cherry |
Akiko | 28 | 3 | 0 | Chocolate |
Michael | 32 | 0 | 1 | Pistachio |
Jess | 29 | 1 | 0 | Chocolate |
Subasri | 34 | 1 | 0 | Vanilla Bean |
Brian | 21 | 0 | 1 | Moose Tracks |
Crystal | 18 | 1 | 0 | Strawberry |
Remember, to calculate the median, you put all the values in numerical order and take the number in the middle. When there's an even number of values, take the average of the two middle values.
What happens if we remove Tom, the outlier?
With Tom in our group, the median age is 27.5, and without him, it's 27. You can see that the median was far less sensitive to him being included in our data than the mean was.
Mode
The mode of a variable is the most commonly occurring value. While you can calculate the mode for interval/ratio variables, it's mostly useful when examining and describing nominal or ordinal variables. Think of it this way—do we really care that there are two people with an income of $38,000 per year, or do we care that these people fall into a certain category related to that value, like above or below the federal poverty level?
Let's go back to our ice cream survey (Table 14.7).
Name | Age | Gender | Hometown | Fav_Ice_Cream |
Tom | 54 | 0 | 1 | Rocky Road |
Jorge | 18 | 2 | 0 | French Vanilla |
Melissa | 22 | 1 | 0 | Espresso |
Amy | 27 | 1 | 0 | Black Cherry |
Akiko | 28 | 3 | 0 | Chocolate |
Michael | 32 | 0 | 1 | Pistachio |
Jess | 29 | 1 | 0 | Chocolate |
Subasri | 34 | 1 | 0 | Vanilla Bean |
Brian | 21 | 0 | 1 | Moose Tracks |
Crystal | 18 | 1 | 0 | Strawberry |
We can use the mode for a few different variables here: gender, hometown and fav_ice_cream. The cool thing about the mode is that you can use it for numeric/quantitative and text/quantitative variables.
So let's find some modes. For hometown—or whether the participant's hometown is the one in which the survey was administered or not—the mode is 0, or "no" because that's the most common answer. For gender, the mode is 0, or "female." And for fav_ice_cream, the mode is Chocolate, although there's a lot of variation there. Sometimes, you may have more than one mode, which is still useful information.
One final thing I want to note about these three measures of central tendency: if you're using something like a ranking question or a Likert scale, depending on what you're measuring, you might use a mean or median, even though these look like they will only spit out ordinal variables. For example, say you're a car designer and want to understand what people are looking for in new cars. You conduct a survey asking participants to rank the characteristics of a new car in order of importance (an ordinal question). The most commonly occurring answer—the mode—really tells you the information you need to design a car that people will want to buy. On the flip side, if you have a scale of 1 through 5 measuring a person's satisfaction with their most recent oil change, you may want to know the mean score because it will tell you, relative to most or least satisfied, where most people fall in your survey. To know what's most helpful, think critically about the question you want to answer and about what the actual values of your variable can tell you.
Key Takeaways
- The mean is the average value for a variable, calculated by adding all values and dividing the total by the number of cases. While the mean contains useful information about a variable's distribution, it's also susceptible to outliers, especially with small data sets.
- In general, the mean is most useful with interval/ratio variables.
- The median, or 50th percentile, is the exact middle of our distribution when the values of our variable are placed in numerical order. The median is usually a more accurate measurement of the middle of our distribution because outliers have a much smaller effect on it.
- In general, the median is only useful with interval/ratio variables.
- The mode is the most commonly occurring value of our variable. In general, it is only useful with nominal or ordinal variables.
Exercises
- Say you want to know the income of the typical participant in your study. Which measure of central tendency would you use? Why?
- Find an interval/ratio variable and calculate the mean and median. Make a scatter plot and look for outliers.
- Find a nominal variable and calculate the mode.
14.3 Frequencies and variability
Learning Objectives
Learners will be able to...
- Define descriptive statistics and understand when to use these methods.
- Produce and describe visualizations to report quantitative data.
Descriptive statistics refer to a set of techniques for summarizing and displaying data. We've already been through the measures of central tendency, (which are considered descriptive statistics) which got their own chapter because they're such a big topic. Now, we're going to talk about other descriptive statistics and ways to visually represent data.
Frequency tables
One way to display the distribution of a variable is in a frequency table. Table 14.2, for example, is a frequency table showing a hypothetical distribution of scores on the Rosenberg Self-Esteem Scale for a sample of 40 college students. The first column lists the values of the variable—the possible scores on the Rosenberg scale—and the second column lists the frequency of each score. This table shows that there were three students who had self-esteem scores of 24, five who had self-esteem scores of 23, and so on. From a frequency table like this, one can quickly see several important aspects of a distribution, including the range of scores (from 15 to 24), the most and least common scores (22 and 17, respectively), and any extreme scores that stand out from the rest.
Self-esteem score (out of 30) | Frequency |
24 | 3 |
23 | 5 |
22 | 10 |
21 | 8 |
20 | 5 |
19 | 3 |
18 | 3 |
17 | 0 |
16 | 2 |
15 | 1 |
There are a few other points worth noting about frequency tables. First, the levels listed in the first column usually go from the highest at the top to the lowest at the bottom, and they usually do not extend beyond the highest and lowest scores in the data. For example, although scores on the Rosenberg scale can vary from a high of 30 to a low of 0, Table 14.8 only includes levels from 24 to 15 because that range includes all the scores in this particular data set. Second, when there are many different scores across a wide range of values, it is often better to create a grouped frequency table, in which the first column lists ranges of values and the second column lists the frequency of scores in each range. Table 14.9, for example, is a grouped frequency table showing a hypothetical distribution of simple reaction times for a sample of 20 participants. In a grouped frequency table, the ranges must all be of equal width, and there are usually between five and 15 of them. Finally, frequency tables can also be used for nominal or ordinal variables, in which case the levels are category labels. The order of the category labels is somewhat arbitrary, but they are often listed from the most frequent at the top to the least frequent at the bottom.
Reaction time (ms) | Frequency |
241–260 | 1 |
221–240 | 2 |
201–220 | 2 |
181–200 | 9 |
161–180 | 4 |
141–160 | 2 |
Histograms
A histogram is a graphical display of a distribution. It presents the same information as a frequency table but in a way that is grasped more quickly and easily. The histogram in Figure 14.2 presents the distribution of self-esteem scores in Table 14.8. The x-axis (the horizontal one) of the histogram represents the variable and the y-axis (the vertical one) represents frequency. Above each level of the variable on the x-axis is a vertical bar that represents the number of individuals with that score. When the variable is quantitative, as it is in this example, there is usually no gap between the bars. When the variable is nominal or ordinal, however, there is usually a small gap between them. (The gap at 17 in this histogram reflects the fact that there were no scores of 17 in this data set.)
Distribution shapes
When the distribution of a quantitative variable is displayed in a histogram, it has a shape. The shape of the distribution of self-esteem scores in Figure 14.2 is typical. There is a peak somewhere near the middle of the distribution and “tails” that taper in either direction from the peak. The distribution of Figure 14.2 is unimodal, meaning it has one distinct peak, but distributions can also be bimodal, as in Figure 14.3, meaning they have two distinct peaks. Figure 14.3, for example, shows a hypothetical bimodal distribution of scores on the Beck Depression Inventory. I know we talked about the mode mostly for nominal or ordinal variables, but you can actually use histograms to look at the distribution of interval/ratio variables, too, and still have a unimodal or bimodal distribution even if you aren't calculating a mode. Distributions can also have more than two distinct peaks, but these are relatively rare in social work research.
Another characteristic of the shape of a distribution is whether it is symmetrical or skewed. The distribution in the center of Figure 14.4 is symmetrical. Its left and right halves are mirror images of each other. The distribution on the left is negatively skewed, with its peak shifted toward the upper end of its range and a relatively long negative tail. The distribution on the right is positively skewed, with its peak toward the lower end of its range and a relatively long positive tail.
Chapter Outline
- Ethical responsibility and cultural respectfulness (7 minute read)
- Critical considerations (8 minute read)
- Find the right qualitative data to answer my research question (17 minute read)
- How to gather a qualitative sample (21 minute read)
- What should my sample look like? (9 minute read)
Content warning: examples in this chapter contain references to substance use, ageism, injustices against the Black community in research (e.g. Henrietta Lacks and Tuskegee Syphillis Study), children and their educational experiences, mental health, research bias, job loss and business closure, mobility limitations, politics, media portrayals of LatinX families, labor protests, neighborhood crime, Batten Disease (childhood disorder), transgender youth, cancer, child welfare including kinship care and foster care, Planned Parenthood, trauma and resilience, sexual health behaviors.
Now let's change things up! In the previous chapters, we were exploring steps to create and carry out a quantitative research study. Quantitative studies are great when we want to summarize data and examine or test relationships between ideas using numbers and the power of statistics. However, qualitative research offers us a different and equally important tool. Sometimes the aim of research is to explore meaning and experience. If these are the goals of our research proposal, we are going to turn to qualitative research. Qualitative research relies on the power of human expression through words, pictures, movies, performance and other artifacts that represent these things. All of these tell stories about the human experience and we want to learn from them and have them be represented in our research. Generally speaking, qualitative research is about the gathering up of these stories, breaking them into pieces so we can examine the ideas that make them up, and putting them back together in a way that allows us to tell a common or shared story that responds to our research question. Back in Chapter 7 we talked about different paradigms.
Before plunging further into our exploration of qualitative research, I would like to suggest that we begin by thinking about some ethical, cultural and empowerment-related considerations as you plan your proposal. This is by no means a comprehensive discussion of these topics as they relate to qualitative research, but my intention is to have you think about a few issues that are relevant at each step of the qualitative process. I will begin each of our qualitative chapters with some discussion about these topics as they relate to each of these steps in the research process. These sections are specially situated at the beginning of the chapters so that you can consider how these principles apply throughout the proceeding discussion. At the end of this chapter there will be an opportunity to reflect on these areas as they apply specifically to your proposal. Now, we have already discussed research ethics back in Chapter 8. However, as qualitative researchers we have some unique ethical commitments to participants and to the communities that they represent. Our work as qualitative researchers often requires us to represent the experiences of others, which also means that we need to be especially attentive to how culture is reflected in our research. Cultural respectfulness suggests that we approach our work and our participants with a sense of humility. This means that we maintain an open mind, a desire to learn about the cultural context of participants' lives, and that we preserve the integrity of this context as we share our findings.
17.1 Ethical responsibility and cultural respectfulness
Learning Objectives
Learners will be able to...
- Explain how our ethical responsibilities as researchers translate into decisions regarding qualitative sampling
- Summarize how aspects of culture and identity may influence recruitment for qualitative studies
Representation
Representation reflects two important aspects of our work as qualitative researchers, who is present and how are they presented. First, we need to consider who we are including or excluding in or sample. Recruitment and sampling is especially tied to our ethical mandate as researchers to uphold the principle of justice under the Belmont Report[2] (see Chapter 6 for additional information). Within this context we need to:
- Assure there is fair distribution of risks and benefits related to our research
- Be conscientious in our recruitment efforts to support equitable representation
- Ensure that special protections to vulnerable groups involved in research activities are in place
As you plan your qualitative research study, make sure to consider who is invited and able to participate and who is not. These choices have important implications for your findings and how well your results reflect the population you are seeking to represent. There may be explicit exclusions that don't allow certain people to participate, but there may also be unintended reasons people are excluded (e.g. transportation, language barriers, access to technology, lack of time).
The second part of representation has to do with how we disseminate our findings and how this reflects on the population we are studying. We will speak further about this aspect of representation in Chapter 21, which is specific to qualitative research dissemination. For now, it is enough to know that we need to be thoughtful about who we attempt to recruit and how effectively our resultant sample reflects our population.
Being mindful of history
As you plan for the recruitment of your sample, be mindful of the history of how this group (and/or the individuals you may be interacting with) has been treated – not just by the research community, but by others in positions of power. As researchers, we usually represent an outside influence and the people we are seeking to recruit may have significant reservations about trusting us and being willing to participate in our study (often grounded in good historical reasons—see Chapter 6 for additional information). Because of this, be very intentional in your efforts to be transparent about the purpose of your research and what it involves, why it is important to you, as well as how it can impact the community. Also, in helping to address this history, we need to make concerted efforts to get to know the communities that we research with well, including what is important to them.
Stories as sacred: How are we requesting them?
Finally, it is worth pointing out that as qualitative researchers, we have an extra layer of ethical and cultural responsibility. While quantitative research deals with numbers, as qualitative researchers, we are generally asking people to share their stories. Stories are intimate, full of depth and meaning, and can reveal tremendous amounts about who we are and what makes us tick. Because of this, we need to take special care to treat these stories as sacred. I will come back to this point in subsequent chapters, but as we go about asking for people to share their stories, we need to do so humbly.
Key Takeaways
- As researchers, we need to consider how our participant communities have been treated historically, how we are representing them in the present through our research, and the implications this representation could have (intended and unintended) for their lives. We need to treat research participants and their stories with respect and humility.
- When conducting qualitative research, we are asking people to share their stories with us. These "data" are personal, intimate, and often reflect the very essence of who our participants are. As researchers, we need to treat research participants and their stories with respect and humility.
17.2 Critical considerations
Learning Objectives
Learners will be able to...
- Assess dynamics of power in sampling design and recruitment for individual participants and participant communities
- Create opportunities for empowerment through early choice points in key research design elements
Power
Related to the previous discussion regarding being mindful of history, we also need to consider the current dynamics of power between researcher and potential participant. While we may not always recognize or feel like we are in a position of power, as researchers we hold specialized knowledge, a particular skill set, and what we do can with the data we collect can have important implications and consequences for individuals, groups, and communities. All of these contribute to the formation of a role ascribed with power. It is important for us to consider how this power is perceived and whenever possible, how we can build opportunities for empowerment that can be built into our research design. Examples of some strategies include:
- Recruiting and meeting in spaces that are culturally acceptable
- Finding ways to build participant choice into the research process
- Working with a community advisory group during the research process (explained further in the example box below)
- Designing informative and educational materials that help to thoroughly explain the research process in meaningful ways
- Regularly checking with participants for understanding
- Asking participants what they would like to get out of their participation and what it has been like to participate in our research
- Determining if there are ways that we can contribute back to communities beyond our research (developing an ongoing commitment to collaboration and reciprocity)
While it may be beyond the scope of a student research project to address all of these considerations, I do think it is important that we start thinking about these more in our research practices. As social work researchers, we should be modeling empowerment practices in the field of social science research, but we often fail to meet this standard.
Example. A community advisory group can be a tremendous asset throughout our research process, but especially in early stages of planning, including recruitment. I was fortunate enough to have a community advisory group for one of the projects I worked on. They were incredibly helpful as I considered different perspectives I needed to include in my study, helping me to think through a respectful way to approach recruitment, and how we might make the research arrangement a bit more reciprocal so community members might benefit as well.
Intersectional identity
As qualitative researchers, we are often not looking to prove a hypothesis or uncover facts. Instead, we are generally seeking to expand our understanding of the breadth and depth of human experience. Breadth is reflected as we seek to uncover variation across participants and depth captures variation or detail within each participants' story. Both are important for generating the fullest picture possible for our findings. For example, we might be interested in learning about people's experience living in an assisted living facility by interviewing residents. We would want to capture a range of different residents' experiences (breadth) and for each resident, we would seek as much detail as possible (depth). Do note, sometimes our research may only involve one person, such as in a case study. However, in these instances we are usually trying to understand many aspects or dimensions of that single case.
To capture this breadth and depth we need to remember that people are made of multiple stories formed by intersectional identities. This means that our participants never just represent one homogeneous social group. We need to consider the various aspects of our population that will help to give the most complete representation in our sample as we go about recruitment.
Exercises
Identify a population you are interested in studying. This might be a population you are working with at your field placement (either directly or indirectly), a group you are especially interested in learning more about, or a community you want to serve in the future. As you formulate your question, you may draw your sample directly from clients that are being served, others in their support network, service providers that are providing services, or other stakeholders that might be invested in the well-being of this group or community. Below, list out two populations you are interested in studying and then for each one, think about two groups connected with this population that you might focus your study on.
Population | Group |
1. | 1a. |
1b. | |
2. | 2a. |
2b. |
Next, think about what would kind of information might help you understand this group better. If you had the chance to sit down and talk with them, what kinds of things would you want to ask? What kinds of things would help you understand their perspective or their worldview more clearly? What kinds of things do we need to learn from them and their experiences that could help us to be better social workers? For each of the groups you identified above, write out something you would like to learn from their experience.
Population | Group | What I would like to learn from their experience |
1 | 1a. | 1a. |
1b. | 1b. | |
2. | 2a. | 2a. |
2b. | 2b. |
Finally, consider how this group might perceive a request to participate. For the populations and the groups that you have identified, think about the following questions:
- How have these groups been represented in the news?
- How have these groups been represented in popular culture and popular media?
- What historical or contemporary factors might influence these group members' opinions of research and researchers?
- In what ways have these groups been oppressed and how might research or academic institutions have contributed to this oppression?
Our impact on the qualitative process
It is important for qualitative research to thoughtfully plan for and attempt to capture our own impact on the research process. This influence that we can have on the research process represents what is known as researcher bias. This requires that we consider how we, as human beings, influence the research we conduct. This starts at the very beginning of the research process, including how we go about sampling. Our choices throughout the research process are driven by our unique values, experiences, and existing knowledge of how the world works. To help capture this contribution, qualitative researchers may plan to use tools like a reflexive journal, which is a research journal that helps the researcher to reflect on and consider their thoughts and reactions to the research process and how these may influence or shape a study (there will be more about this tool in Chapter 20 when we discuss the concept of rigor). While this tool is not specific to the sampling process, the next few chapters will suggest reflexive journal questions to help you think through how it might be used as you develop a qualitative proposal.
Example. To help demonstrate the potential for researcher bias, consider a number of students that I work with who are placed in school systems for their field experience and choose to focus their research proposal in this area. Some are interested in understanding why parents or guardians aren't more involved in their children's educational experience. While this might be an interesting topic, I would encourage students to consider what kind of biases they might have around this issue.
- What expectations do they have about parenting?
- What values do they attach to education and how it should be supported in the home?
- How has their own upbringing shaped their expectations?
- What do they know about the families that the school district serves and how did they come by this information?
- How are these families' life experiences different from their own?
The answers to these questions may unconsciously shape the early design of the study, including the research question they ask and the sources of data they seek out. For instance, their study may only focus on the behaviors and the inclinations of the families, but do little to investigate the role that the school plays in engagement and other structural barriers that might exist (e.g. language, stigma, accessibility, child-care, financial constraints, etc.).
Key Takeaways
- As researchers, we wield (sometimes subtle) power and we need to be conscientious of how we use and distribute this power.
- Qualitative study findings represent complex human experiences. As good as we may be, we are only going to capture a relatively small window into these experiences (and need to be mindful of this when discussing our findings).
Exercises
In the early stages of your research process, it is a good idea to start your reflexive journal. Starting a reflexive journal is as easy as opening up a new word document, titling it and chronologically dating your entries. If you are more tactile-oriented, you can also keep your reflexive journal in paper bound journal.
To prompt your initial entry, put your thoughts down in response to the following questions:
- What led you to be interested in this topic?
- What experience(s) do you have in this area?
- What knowledge do you have about this issue and how did you come by this knowledge?
- In what ways might you be biased about this topic?
Don't answer this last question too hastily! Our initial reaction is often—"Biased!?! Me—I don't have a biased bone in my body! I have an open-mind about everything, toward everyone!" After all, much of our social work training directs us towards acceptance and working to understand the perspectives of others. However, WE ALL HAVE BIASES. These are conscious or subconscious preferences that lead us to favor some things over others. These preferences influence the choices we make throughout the research process. The reflexive journal helps us to reflect on these preferences, where they might stem from, and how they might be influencing our research process. For instance, I conduct research in the area of mental health. Before I became a researcher, I was a mental health clinician, and my years as a mental health practitioner created biases for me that influence my approach to research. For instance, I may be biased in perceiving mental health services as being well-intentioned and helpful. However, participants may well have very different perceptions based on their experiences or beliefs (or those of their loved ones).
17.3 Finding the right qualitative data to answer my research question
Learning Objectives
Learners will be able to...
- Compare different types of qualitative data
- Begin to formulate decisions as they build their qualitative research proposal, specially in regards to selecting types of data that can effectively answer their research question
Sampling starts with deciding on the type of data you will be using. Qualitative research may use data from a variety of sources. Sources of qualitative data may come from interviews or focus groups, observations, a review of written documents, administrative data, or other forms of media, and performances. While some qualitative studies rely solely on one source of data, others incorporate a variety.
You should now be well acquainted with the term triangulation. When thinking about triangulation in qualitative research, we are often referring to our use of multiple sources of data among those listed above to help strengthen the confidence we have in our findings. Drawing on a journalism metaphor, this allows us to "fact check" our data to help ensure that we are getting the story correct. This can mean that we use one type of data (like interviews), but we intentionally plan to get a diverse range of perspectives from people we know will see things differently. In this case we are using triangulation of perspectives. In addition, we may also you a variety of different types of data, like including interviews, data from case records, and staff meeting minutes all as data sources in the same study. This reflects triangulation through types of data.
As a student conducting research, you may not always have access to vulnerable groups or communities in need, or it may be unreasonable for you to collect data from them directly due to time, resource, or knowledge constraints. Because of this, as you are reviewing the sections below, think about accessible alternative sources of data that will still allow you to answer your research question practically, and I will provide some examples along the way to get you started. In the above example, local media coverage might be a means of obtaining data that does not involve vulnerable directly collecting data from potentially vulnerable participants.
Verbal data
Perhaps the bread and butter of the qualitative researcher, we often rely on what people tell us as a primary source of information for qualitative studies in the form of verbal data. The researcher who schedules interviews with recipients of public assistance to capture their experience after legislation drastically changes requirements for benefits relies on the communication between the researcher and the impacted recipients of public assistance. Focus groups are another frequently used method of gathering verbal data. Focus groups bring together a group of participants to discuss their unique perspectives and explore commonalities on a given topic. One such example is a researcher who brings together a group of child welfare workers who have been in the field for one to two years to ask them questions regarding their preparation, experiences, and perceptions regarding their work.
A benefit of utilizing verbal data is that it offers an opportunity for researchers to hear directly from participants about their experiences, opinions, or understanding of a given topic. Of course, this requires that participants be willing to share this information with a researcher and that the information shared is genuine. If groups of participants are unwilling to participate in sharing verbal data or if participants share information that somehow misrepresents their feelings (perhaps because they feel intimidated by the research process), then our qualitative sample can become biased and lead to inaccurate or partially accurate findings.
As noted above, participant willingness and honesty can present challenges for qualitative researchers. You may face similar challenges as a student gathering verbal data directly from participants who have been personally affected by your research topic. Because of this, you might want to gather verbal data from other sources. Many of the students I work with are placed in schools. It is not feasible for them to interview the youth they work with directly, so frequently they will interview other professionals in the school, such as teachers, counselors, administration, and other staff. You might also consider interviewing other social work students about their perceptions or experiences working with a particular group.
Again, because it may be problematic or unrealistic for you to obtain verbal data directly from vulnerable groups as a student researcher, you might consider gathering verbal data from the following sources:
- Interviews and focus groups with providers, social work students, faculty, the general public, administrators, local politicians, advocacy groups
- Public blogs of people invested in your topic
- Publicly available transcripts from interviews with experts in the area or people reporting experiences in popular media
Make sure to consult with your professor to ensure that what you are planning will be realistic for the purposes of your study.
Observational data
As researchers, we sometimes rely on our own powers of observation to gather data on a particular topic. We may observe a person’s behavior, an interaction, setting, context, and maybe even our own reactions to what we are observing (i.e. what we are thinking or feeling). When observational data is used for quantitative purposes, it involves a count, such as how many times a certain behavior occurs for a child in a classroom. However, when observational data is used for qualitative purposes, it involves the researcher providing a detailed description. For instance, a qualitative researcher may conduct observations of how mothers and children interact in child and adolescent cancer units, and take notes about where exchanges take place, topics of conversation, nonverbal information, and data about the setting itself – what the unit looks like, how it is arranged, the lighting, photos on the wall, etc.
Observational data can provide important contextual information that may not be captured when we rely solely on verbal data. However, using this form of data requires us, as researchers, to correctly observe and interpret what is going on. As we don’t have direct access to what participants may be thinking or feeling to aid us (which can lead us to misinterpret or create a biased representation of what we are observing), our take on this situation may vary drastically from that of another person observing the same thing. For instance, if we observe two people talking and one begins crying, how do we know if these are tears of joy or sorrow? When you observe someone being abrupt in a conversation, I might interpret that as the person being rude while you might perceive that the person is distracted or preoccupied with something. The point is, we can't know for sure. Perhaps one of the most challenging aspects of gathering observational data is collecting neutral, objective observations, that are not laden with our subjective value judgments placed on them. Students often find this out in class during one of our activities. For this activity, they have to go out to public space and write down observations about what they observe. When they bring them back to class and we start discussing them together, we quickly realize how often we make (unfounded) judgments. Frequent examples from our class include determining the race/ethnicity of people they observe or the relationships between people, without any confirmational knowledge. Additionally, they often describe scenarios with adverbs and adjectives that often reflect judgments and values they are placing on their data. I'm not sharing this to call them out, in fact, they do a great job with the assignment. I just want to demonstrate that as human beings, we are often less objective than we think we are! These are great examples of research bias.
Again, gaining access to observational spaces, especially private ones, might be a challenge for you as a student. As such, you might consider if observing public spaces might be an option. If you do opt for this, make sure you are not violating anyone's right to privacy. For instance, gathering information in a narcotics anonymous meeting or a religious celebration might be perceived as offensive, invasive or in direct opposition to values (like anonymity) of participants. When making observations in public spaces be careful not to gather any information that might identify specific individuals or organizations. Also, it is important to consider the influence your presence may have on a community, particularly if your observation makes you stand out among those typically present in that setting. Always consider the needs of the individual and the communities in formulating a plan for observing public behavior. Public spaces might include commercial spaces or events open to the public as well as municipal parks. Below we will have an expanded discussion about different varieties of non-probability sampling strategies that apply to qualitative research. Recruiting in public spaces like these may work for strategies such as convenience sampling or quota sampling, but would not be a good choice for snowball sampling or purposive sampling.
As with the cautionary note for student researchers under verbal data, you may experience restricted access to spaces in which you are able to gather observational data. However, if you do determine that observational data might be a good fit for your student proposal, you might consider the following spaces:
- Shopping malls
- Public parks or beaches
- Public meetings or rallies
- Public transportation
Artifacts (documents & other media)
Existing artifacts can also be very useful for the qualitative researcher. Examples include newspapers, blogs, websites, podcasts, television shows, movies, pictures, video recordings, artwork, and live performances. While many of these sources may provide indirect information on a topic, this information can still be quite valuable in capturing the sentiment of popular culture and thereby help researchers enhance their understanding of (dominant) societal values and opinions. Conversely, researchers can intentionally choose to seek out divergent, unique or controversial perspectives by searching for artifacts that tend to take up positions that differ from the mainstream, such as independent publications and (electronic) newsletters. While we will explore this further below, it is important to understand that data and research, in all its forms, is political. Among many other purposes, it is used to create, critique, and change policy; to engage in activism; to support and refute how organizations function; and to sway public opinion.
When utilizing documents and other media as artifacts, researchers may choose to use an entire source (such as a book or movie), or they may use a segment or portion of that artifact (such as the front-page stories from newspapers, or specific scenes in a television series). Your choice of which artifacts you choose to include will be driven by your question, and remember, you want your sample of artifacts to reflect the diversity of perspectives that may exist in the population you are interested in. For instance, perhaps I am interested in studying how various forms of media portray substance use treatment. I might intentionally include a range of liberal to conservative views that are portrayed across a number of media sources.
As qualitative researchers using artifacts, we often need to do some digging to understand the context of said artifact. We do this because data is almost always affiliated or aligned with some position (again, data is political). To help us consider this, it may be helpful to reflect on the following questions:
- Who owns the artifact or where is it housed
- What values does the owner (organization or person) hold
- How might the position or identity of the owner influence what information is shared or how it is portrayed
- What is the purpose of the artifact
- Who is the audience for which the artifact is intended
Answers to questions such as these can help us to better understand and give meaning to the content of the artifacts. Content is the substance of the artifact (e.g. the words, picture, scene). While context is the circumstances surrounding content. Both work together to help provide meaning, and further understanding of what can be derived from an artifact. As an example to illustrate this point, let's say that you are including meeting minutes from an organizing network as a source of data for your study. The narrative description in these minutes will certainly be important, however, they may not tell the whole story. For instance, you might not know from the text that the organization has recently voted in a new president and this has created significant division within the network. Knowing this information might help you to interpret the agenda and the discussion contained in the minutes very differently.
As student researchers, using documents and other artifacts may be a particularly appealing source of data for your study. This is because this data already exists (you aren't creating new data) and depending on what you select, it might be relatively easy to access. Examples of utilizing existing artifacts might include studying the cultural context of movie portrayals of Latinx families or analyzing publicly available town hall meeting minutes to explore expressions of social capital. Below is a list of sources of data from documents or other media sources to consider for your student proposal:
- Movies or TV shows
- Music or music videos
- Public blogs
- Policies or other organizational documents
- Meeting minutes
- Comments in online forums
- Books, newspapers, magazines, or other print/virtual text-based materials
- Recruitment, training, or educational materials
- Musical or artistic expressions
Photovoice
Finally, Photovoice is a technique that merges pictures with narrative (word or voice) data that helps interpret the meaning or significance of the visual. Photovoice is often used for qualitative work that is conducted as part of Community Based Participatory Research (CBPR), wherein community members act as both participants and as co-researchers. These community members are provided with a means of capturing images that reflect their understanding of some topic, prompt or question, and then they are asked to provide a narrative description or interpretation to help give meaning to the image(s). Both the visual and narrative information are used as qualitative data to include in the study. Dissemination of Photovoice projects often involve a public display of the works, such as through a demonstration or art exhibition to raise awareness or to produce some specific change that is desired by participants. Because this form of study is often intentionally persuasive in nature, we need to recognize that this form of data will be inherently subjective. As a student, it may be particularly challenging to implement a Photovoice project, especially due to its time-intensive nature, as well as the additional commitments of needing to engage, train, and collaborate with community partners.
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Verbal |
|
Strengths
Challenges
|
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Observational | Observations in:
|
Strengths
Challenges
|
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Documents & Other Media |
|
Strengths
Challenges
|
How many kinds of data?
You will need to consider whether you will rely on one kind of data or multiple. While many qualitative studies solely use one type of data, such as interviews or focus groups, others may use multiple sources. The decision to use multiple sources is often made to help strengthen the confidence we have in our findings or to help us to produce a richer, more detailed description of our results. For instance, if we are conducting a case study of what the family experience is for a child with a very rare disorder like Batten Disease, we may use multiple sources of data. These can include observing family and community interactions, conducting interviews with family members and others connected to the family (such as service providers,) and examining journal entries families were asked to keep over the course of the study. By collecting data from a variety of sources such as this, we can more broadly represent a range of perspectives when answering our research question, which will hopefully provide a more holistic picture of the family experience. However, if we are trying to examine the decision-making processes of adult protective workers, it may make the most sense to rely on just one type of data, such as interviews with adult protective workers.
Key Takeaways
- There are numerous types of qualitative data (verbal, observational, artifacts) that we may be able to access when planning a qualitative study. As we plan, we need to consider the strengths and challenges that each possess and how well each type might answer our research question.
- The use of multiple types of qualitative data does add complexity to a study, but this complication may well be worth it to help us explore multiple dimensions of our topic and thereby enrich our findings.
Exercises
Reflexive Journal Entry Prompt
For your next entry, consider responding to the following:
- What types of data appeal to you?
- Why do you think you are drawn to them?
- How well does this type of data "fit" as a means of answering your question? Why?
17.4 How to gather a qualitative sample
Learning Objectives
Learners will be able to...
- Compare and contrast various non-probability sampling approaches
- Select a sampling strategy that ideologically fits the research question and is practical/actionable
Before we launch into how to plan our sample, I'm going to take a brief moment to remind us of the philosophical basis surrounding the purpose of qualitative research—not to punish you, but because it has important implications for sampling.
Nomothetic vs. idiographic
As a quick reminder, as we discussed in Chapter 8 idiographic research aims to develop a rich or deep understanding of the individual or the few. The focus is on capturing the uniqueness of a smaller sample in a comprehensive manner. For example, an idiographic study might be a good approach for a case study examining the experiences of a transgender youth and her family living in a rural Midwestern state. Data for this idiographic study would be collected from a range of sources, including interviews with family members, observations of family interactions at home and in the community, a focus group with the youth and her friend group, another focus group with the mother and her social network, etc. The aim would be to gain a very holistic picture of this family's experiences.
On the other hand, nomothetic research is invested in trying to uncover what is ‘true’ for many. It seeks to develop a general understanding of a very specific relationship between variables. The aim is to produce generalizable findings, or findings that apply to a large group of people. This is done by gathering a large sample and looking at a limited or restricted number of aspects. A nomothetic study might involve a national survey of heath care providers in which thousands of providers are surveyed regarding their current knowledge and competence in treating transgender individuals. It would gather data from a very large number of people, and attempt to highlight some general findings across this population on a very focused topic.
Idiographic and nomothetic research represent two different research categories existing at opposite extremes on a continuum. Qualitative research generally exists on the idiographic end of this continuum. We are most often seeking to obtain a rich, deep, detailed understanding from a relatively small group of people.
Non-probability sampling
Non-probability sampling refers to sampling techniques for which a person’s (or event’s) likelihood of being selected for membership in the sample is unknown. Because we don’t know the likelihood of selection, we don’t know whether a sample represents a larger population or not. But that’s okay, because representing the population is not the goal of nonprobability samples. That said, the fact that nonprobability samples do not represent a larger population does not mean that they are drawn arbitrarily or without any specific purpose in mind. We typically use nonprobability samples in research projects that are qualitative in nature. We will examine several types of nonprobability samples. These include purposive samples, snowball samples, quota samples, and convenience samples.
Convenience or availability
Convenience sampling, also known as availability sampling, is a nonprobability sampling strategy that is employed by both qualitative and quantitative researchers. To draw a convenience sample, we would simply collect data from those people or other relevant elements to which we have the most convenient access. While convenience samples offer one major benefit—convenience—we should be cautious about generalizing from research that relies on convenience samples because we have no confidence that the sample is representative of a broader population. If you are a social work student who needs to conduct a research project at your field placement setting and you decide to conduct a focus group with the staff at your agency, you are using a convenience sampling approach – you are recruiting participants that are easily accessible to you. In addition, if you elect to analyze existing data that your social work program has collected as part of their graduation exit surveys, you are using data that you readily have access to for your project; again, you have a convenience sample. The vast majority of students I work with on their proposal design rely on convenience data due to time constraints and limited resources.
Purposive
To draw a purposive sample, we begin with specific perspectives or purposive criteria in mind that we want to examine. We would then seek out research participants who cover that full range of perspectives. For example, if you are studying mental health supports on your campus, you may want to be sure to include not only students, but mental health practitioners and student affairs administrators as well. You might also select students who currently use mental health supports, those who dropped out of supports, and those who are waiting to receive supports. The "purposive" part of purposive sampling comes from selecting specific participants on purpose because you already know they have certain characteristics—being an administrator, dropping out of mental health supports, for example—that you need in your sample.
Note that these differ from inclusion criteria, which are more general requirements a person must possess to be a part of your sample; to be a potential participant that may or may not be sampled. For example, one of the inclusion criteria for a study of your campus’ mental health supports might be that participants had to have visited the mental health center in the past year. That differs from purposive sampling. In purposive sampling, you know characteristics of individuals and recruit them because of those characteristics. For example, I might recruit Jane because she stopped seeking supports this month, because she has worked at the center for many years, and so forth.
Also, it’s important to recognize that purposive sampling requires you to have prior information about your participants before recruiting them because you need to know their perspectives or experiences before you know whether you want them in your sample. This is a common mistake that many students make. What I often hear is, “I’m using purposive sampling because I’m recruiting people from the health center,” or something like that. That’s not purposive sampling. In most instances they really mean they are going to use convenience sampling-taking whoever they can recruit that fit the inclusion criteria (i.e. have attended the mental health center). Purposive sampling is recruiting specific people because of the various characteristics and perspectives they bring to your sample. Imagine we were creating a focus group. A purposive sample might gather clinicians, patients, administrators, staff, and former patients together so they can talk as a group. Purposive sampling would seek out people that have each of those attributes.
If you are considering using a purposive sampling approach for your research proposal, you will need to determine what your purposive criteria involves. There are a range of different purposive strategies that might be employed, including: maximum variation, typical case, extreme case, or political case, and you want to be thoughtful in thinking about which one(s) you select and why.
Purposive Strategy | Description | Student Example |
Maximum Variation | Case(s) selected to represent a range of very different perspectives on a topic | You interview student leaders from the schools of social work, business, the arts, math & science, education, history & anthropology and health studies to ensure that you have the perspective of a variety of disciplines |
Typical Case | Case(s) selected to reflect a commonly held perspective. | You interview a child welfare worker specifically because many of their characteristics fit the state statistical profile for providers in that service area. |
Extreme Case | Case(s) selected to represent extreme or underrepresented perspectives. | You examine websites devoted to rare cancer survivor support. |
Political Case | Case(s) selected to represent a contemporary politicized issue | You analyze media interviews with Planned Parenthood providers, employees, and clients from 2010 to present. |
Expert Case | Case(s) selected based on specialized content knowledge or expertise | You are interested in studying resilience in trauma providers, so you research and reach out to a handful of authorities in this area. |
Theory-Based Case | Case(s) selected based on their representation of a specific theoretical orientation or some aspect of a given theory | You are interested in studying how training methods vary by practitioner according to their theoretical orientation. You specifically reach out to a clinician who identifies as a Cognitive Behavioral clinician, one who identifies as Bowenian, and one who identifies as Structural Family. |
Critical Case | Case(s) selected based on the likelihood that the case will yield the desired information | You examine a public gaming network forum on social media to see how participants offer support to one another. |
It can be a bit tricky determining how to approach or formulate your purposive cases. Below are a couple additional resources to explore this strategy further.
For more information on purposive sampling consult this webpage from Laerd Statistics on purposive sampling and this webpage from the University of Connecticut on education research.
Snowball
When using snowball sampling, we might know one or two people we’d like to include in our study but then we have to rely on those initial participants to help identify additional participants. Thus, our sample builds and grows as the study continues, much as a snowball builds and becomes larger as it rolls through the snow. Snowball sampling is an especially useful strategy when you wish to study a stigmatized group or behavior. These groups may have limited visibility and accessibility for a variety of reasons, including safety.
Malebranche and colleagues (2010)[6] were interested in studying sexual health behaviors of Black, bisexual men. Anticipating that this may be a challenging group to recruit, they utilized a snowball sampling approach. They recruited initial contacts through activities such as advertising on websites and distributing fliers strategically (e.g. barbershops, nightclubs). These initial recruits were compensated with $50 and received study information sheets and five contact cards to distribute to people in their social network that fit the study criteria. Eventually the research team was able to recruit a sample of 38 men who fit the study criteria.
Snowball sampling may present some ethical quandaries for us. Since we are essentially relying on others to help advertise for us, we are giving up some of our control over the process of recruitment. We may be worried about coercion, or having people put undue pressure to have others' they know participate in your study. To help mitigate this, we would want to make sure that any participant we recruit understands that participation is completely voluntary and if they tell others about the studies, they should also make them aware that it is voluntary, too. In addition to coercion, we also want to make sure that people's privacy is not violated when we take this approach. For this reason, it is good practice when using a snowball approach to provide people with our contact information as the researchers and ask that they get in touch with us, rather than the other way around. This may also help to protect again potential feelings of exploitation or feeling taken advantage of. Because we often turn to snowball sampling when our population is difficult to reach or engage, we need to be especially sensitive to why this is. It is often because they have been exploited in the past and participating in research may feel like an extension of this. To address this, we need to have a very clear and transparent informed consent process and to also think about how we can use or research to benefit the people we work in the most meaningful and tangible ways.
Quota
Quota sampling is another nonprobability sampling strategy. This type of sampling is actually employed by both qualitative and quantitative researchers, but because it is a nonprobability method, we’ll discuss it in this section. When conducting quota sampling, we identify categories that are important to our study and for which there is likely to be some variation. Subgroups are created based on each category and the researcher decides how many people (or whatever element happens to be the focus of the research) to include from each subgroup and collects data from that number for each subgroup. To demonstrate, perhaps we are interested in studying support needs for children in the foster care system. We decide that we want to examine equal numbers (seven each) of children placed in a kinship placement, a non-kinship foster placement, group home, and residential placements. We expect that the experiences and needs across these settings may differ significantly, so we want to have good representation of each one, thus setting a quota of seven for each type of placement.
Sampling Type | Brief Description |
Convenience/ Availability | You gather data from whatever cases/people/documents happen to be convenient |
Purposive | You seek out elements that meet specific criteria, representing different perspectives |
Snowball | You rely on participant referrals to recruit new participants |
Quota | You select a designated number of cases from specified subgroups |
As you continue to plan for your proposal, below you will find some of the strengths and challenges presented by each of these types of sampling.
Sampling Type | Strengths | Challenges |
Convenience/ Availability | Allows us to draw sample from participants who are most readily available/accessible | Sample may be biased and may represent limited or skewed diversity in characteristics of participants |
Purposive | Ensures that specific expertise, positions, or experiences are represented in sample participants | It may be challenging to define purposive criteria or to locate cases that represent these criteria; restricts our potential sampling pool |
Snowball | Accesses participant social network and community knowledge
Can be helpful in gaining access to difficult to reach populations |
May be hard to locate initial small group of participants, concerns over privacy—people might not want to share contacts, process may be slow or drawn-out |
Quota | Helps to ensure specific characteristics are represented and defines quantity of each | Can be challenging to fill quotas, especially for subgroups that might be more difficult to locate or reluctant to participate |
Wait a minute, we need a plan!
Both qualitative and quantitative research should be planful and systematic. We've actually covered a lot of ground already and before we get any further, we need to start thinking about what the plan for your qualitative research proposal will look like. This means that as you develop your research proposal, you need to consider what you will be doing each step of the way: how you will find data, how you will capture it, how you will organize it, and how you will store it. If you have multiple types of data, you need to have a plan in place for each type. The plan that you develop is your data collection protocol. If you have a team of researchers (or are part of a research team), the data collection protocol is an important communication tool, making sure that everyone is clear what is going on as the research proceeds. This plan is important to help keep you and others involved in your research consistent and accountable. Throughout this chapter and the next (Chapter 18—qualitative data gathering) we will walk through points you will want to include in your data collection protocol. While I've spent a fair amount of time talking about the importance of having a plan here, qualitative design often does embrace some degree of flexibility. This flexibility is related to the concept of emergent design that we find in qualitative studies. Emergent design is the idea that some decision in our design will be dynamic and fluid as our understanding of the research question evolves. The more we learn about the topic, the more we want to understand it thoroughly.
Exercises
A research protocol is a document that not only defines your research project and its aims, but also comprehensively plans how you will carry it out. If this sounds like the function of a research proposal, you are right, they are similar. What differentiates a protocol from a proposal is the level of detail. A proposal is more conceptual; a protocol is more practical (right down to the dollars and cents!). A protocol offers explicit instructions for you and your research team, any funders that may be involved in your research, and any oversight bodies that might be responsible for overseeing your study. Not every study requires a research protocol, but what I'm suggesting here is that you consider constructing at least a limited one to help though the decisions you will need to make to construct your qualitative study.
Al-Jundi and Sakka (2016)[7] provide the following elements for a research protocol:
- What is the question? (Hypothesis) What is to be investigated?
- Why is the study important (Significance)
- Where and when will it take place?
- What is the methodology? (Procedures and methods to be used).
- How are you going to implement it? (Research design)
- What is the proposed time table and budget?
- What are the resources required (technical, scientific, and financial)?
While your research proposal in its entirety will focus on many of these areas, our attention for developing your qualitative research protocol will hone in on the two highlighted above. As we go through these next couple chapters, there will be a number of exercises that walk you though decision points that will form your qualitative research protocol.
To begin developing your qualitative research protocol:
- Select the question you have decided is the best to frame your research proposal.
- Write a brief paragraph about the aim of your study, ending it with the research question you have selected.
Here are a few additional resources on developing a research protocol:
Cameli et al., (2018) How to write a research protocol: Tips and tricks.
Ohio State University, Institutional Review Board (n.d.). Research protocol.
World Health Organization (n.d.). Recommended format for a research protocol.
Exercises
Decision Point: What types of data will you be using?
- After having considered the different types of data that have been reviewed here, what type(s) are you planning on using?
- Why is this a good choice, given your research question?
- Are you thinking about using more than one type?
- If so, provide support for this decision.
Exercises
Decision Point: Which non-probability sampling strategy will you employ?
- Thinking about the four non-probability sampling strategies we reviewed, which one makes the most sense for your proposal?
- Why is this is a good fit?
- What challenges or limitations does this present for your study?
- What steps might your take to address these challenges?
Recruiting strategies
Much like quantitative research, recruitment for qualitative studies can take many different approaches. When considering how to draw your qualitative sample, it may be helpful to first consider which of these three general strategies will best fit your research question and general study design: public, targeted, or membership-based. While all will lead to a sample, the process for getting you there will look very different, depending on the strategy you select.
Public
Taking a public approach to recruitment offers you access to the broadest swath of potential participants. With this approach, you are taking advantage of public spaces in an attempt to gain the attention of the general population of people that frequent that space so that they can learn about your study. These spaces can be in-person (e.g. libraries, coffee shops, grocery stores, health care settings, parks) or virtual (e.g. open chat forums, e-bulletin boards, news feeds). Furthermore, a public approach can be static (such as hanging a flier), or dynamic (such as talking to people and directly making requests to participate). While a public approach may offer broad coverage in that it attempts to appeal to an array of people, it may be perceived as impersonal or easily able to be overlooked, due to the potential presence of other announcements that may be featured in public spaces. Public recruitment is most likely to be associated with convenience or quota sampling and is unlikely to be used with purposive or snowball sampling, where we would need some advance knowledge of people and the characteristics they possess.
Targeted
As an alternative, you may elect to take a targeted approach to recruitment. By targeting a select group, you are restricting your sampling frame to those individuals or groups who are potentially most well-suited to answer your research question. Additionally, you may be targeting specific people to help craft a diverse sample, particularly with respect to personal characteristics and/or opinions.
You can target your recruitment through the use of different strategies. First, you might consider the use of knowledgeable and well-connected community members. These are people who may possess a good amount of social capital in their community, which can aid in recruitment efforts. If you are considering the use of community members in this role, make sure to be thoughtful in your approach, as you are essentially asking them to share some of their social capital with you. This means learning about the community or group, approaching community members with a sense of humility, and making sure to demonstrate transparency and authenticity in your interactions. These community members may also be champions for the topic you are researching. A champion is someone who helps to draw the interest of a particular group of people. The champion often comes from within the group itself. As an example, let's say you're interested in studying the experiences of family members who have a loved one struggling with substance use. To aid in your recruitment for this study, you enlist the help of a local person who does a lot of work with Al-Anon, an organization facilitating mutual support groups for individuals and families affected by alcoholism.
A targeted approach can certainly help ensure that we are talking to people who are knowledgeable about the topic we are interested in, however, we still need to be aware of the potential for bias. If we target our recruitment based on connection to a particular person, event, or passion for the topic, these folks may share information that they think is viewed as favorable or that disproportionately reflects a particular perspective. This phenomenon is due to the fact that we often spend time with people who are like-minded or share many of our views. A targeted approach may be helpful for any type of non-probability sampling, but can be especially useful for purposive, quota, or snowball sampling, where we are trying to access people or groups of people with specific characteristics or expertise.
Membership-based
Finally, you might consider a membership-based approach. This approach is really a form of targeted recruitment, but may benefit from some individual attention. When using a membership-based approach, your sampling frame is the membership list of a particular organization or group. As you might have guessed, this organization or group must be well-suited for helping to answer your research question. You will need permission to access membership, and the identity of the person authorized to grant permission will depend on the organizational structure. When contacting members regarding recruitment, you may consider using directories, newsletters, listservs or membership meetings. When utilizing a membership-based approach, we often know that members possess specific inclusion criteria we need, however, because they are all associated with that particular group or organization, they may be homogenous or like-minded in other ways. This may limit the diversity in our sample and is something to be mindful of when interpreting our findings. Membership-based recruiting can be helpful when we have a membership group that fulfills our inclusion criteria. For instance, if you want to conduct research with social workers, you might attempt to recruit through the NASW membership distribution list (but this access will come with stipulations and a price tag). Membership-based recruitment may be helpful for any non-probability sampling approach, given that the membership criteria and study inclusion criteria are a close fit. Table 17.5 offers some additional considerations for each of these strategies with examples to help demonstrate sources that might correspond with them.
Recruitment Strategy | Strengths/Challenges | Example of Source for Recruitment |
Public | Strengths: Easier to gain access; Exposure to large numbers of people
Challenges: Can be impersonal, Difficult to cultivate interest |
Advertising in public events & spaces
Accessing materials in local libraries or museums Finding public web-based resources and sources of data (websites, blogs, open forums)
|
Targeted | Strengths: Prior knowledge of potential audience, More focused use of resources
Challenges: May be hard to locate/access target group(s), Groups may be suspicious of/or resistant to being targeted |
Working with advocacy group for issue you are studying to aid recruitment
Contacting local expert (historian) to help you locate relevant documents Advertising in places that your population may frequent
|
Membership-Based | Strengths: Shared interest (through common membership), Potentially existing infrastructure for outreach
Challenges: Organization may be highly sensitive to protecting members, Members may be similar in perspectives and limit diversity of ideas |
Membership newsletters
Listserv or Facebook groups Advertising at membership meetings or events |
Key Takeaways
- Qualitative research predominately relies on non-probability sampling techniques. There are a number of these techniques to choose from (convenience/availability, purposive, snowball, quota), each with advantages and limitations to consider. As we consider these, we need to reflect on both our research question and the resources we have available to us in developing a sampling strategy.
- As we consider where and how we will recruit our sample, there are a range of general approaches, including public, targeted, and membership-based.
Exercises
Decision Point: How will you recruit or gain access to your sample?
- Now you've decided your data type and sampling strategy...
- If you are recruiting people, how will you identify them? If necessary (and it often is), how will gain permission to do this?
- If you are using documents or other artifacts for your study, how will you gain access to these? If necessary (and it often is), how will gain permission to do this?
17.5 What should my sample look like?
Learning Objectives
Learners will be able to...
- Explain key factors that influence the makeup of a qualitative sample
- Develop and critique a sampling strategy to support their qualitative proposal
Once you have started your recruitment, you also need to know when to stop. Knowing when to stop recruiting for a qualitative research study generally involves a dynamic and reflective process. This means that you will actively be involved in a process of recruiting, collecting data, beginning to review your preliminary data, and conducting more recruitment to gather more data. You will continue this process until you have gathered enough data and included sufficient perspectives to answer your research question in rich and meaningful way.
The sample size of qualitative studies can vary significantly. For instance, case studies may involve only one participant or event, while some studies may involve hundreds of interviews or even thousands of documents. Generally speaking, when compared to quantitative research, qualitative studies have a considerably smaller sample. Your decision regarding sample size should be guided by a few considerations, described below.
Amount of data
When gathering quantitative data, the amount of data we are gathering is often specified at the start (e.g. a fixed number of questions on a survey or a set number of indicators on a tracking form). However, when gathering qualitative data, we are often asking people to expand on and explore their thoughts and reactions to certain things. This can produce A LOT of data. If you have ever had to transcribe an interview (type out the conversation while listening to an audio recorded interview), you quickly learn that a 15-minute discussion turns into many pages of dialogue. As such, each interview or focus group you conduct represents multi-page transcripts, all of which becomes your data. If you are conducting interviews or focus groups, you will know you have collected enough data from each interaction when you have covered all your questions and allowed the participant(s) to share any and all ideas they have related to the topic. If you are using observational data, you need to spend sufficient time making observations and capturing data to offer a genuine and holistic representation of the thing you are observing (at least to the best of your ability). When using documents and other sources of media, again, you want to ensure that diverse perspectives are represented through your artifact choices so that your data reflects a well-rounded representation of the issue you are studying. For any of these data sources, this involves a judgment call on the researcher's part. Your judgment should be informed by what you have read in the existing literature and consultation with your professor.
As part of your analysis, you will likely eventually break these larger hunks of data apart into words or small phrases, giving you potentially thousands of pieces of data. If you are relying on documents or other artifacts, the amount of data contained in each of these pieces is determined in advance, as they already exist. However, you will need to determine how many to include. With interviews, focus groups, or other forms of data generation (e.g. taking pictures for a photovoice project), we don’t necessarily know how much data will be generated with each encounter, as it will depend on the questions that are asked, the information that is shared, and how well we capture it.
Type of study
A variety of types of qualitative studies will be discussed in greater detail in Chapter 22. While you don't necessarily need to have an extensive understanding of them all at this point in time, it is important that you understand which of the different design types are best for answering certain research questions. For instance, if our question involves understanding some type of experience, that is often best answered by a phenomenological design. Or, if we want to better understand some process, a grounded theory study may be best suited. While there are no hard and fast rules regarding qualitative sample size, each of these different types of designs has different guidelines for what is considered an acceptable or reasonable number to include in your sample. So drawing on the previous examples, your grounded theory study might include 45 participants because you need more people to gain a clearer picture of each step of the process, while your phenomenological study includes 20 because that provides a good representation of the experience you are interested in. Both would be reasonable targets based on the respective study design type. So as you consider your research question and which specific type of qualitative design this leads you to, you will need to do some investigation to see what size samples are recommended for that particular type of qualitative design.
Diversity of perspectives
As you consider your research question, you also may want to think about the potential variation in how your study population might view this topic. If you are conducting a case study of one person, this obviously isn’t a concern, but if you are interested in exploring a range of experiences, you want to plan to intentionally recruit so this level of diversity is reflected in your sample. The level of variation you seek will have direct implications for how big your sample might be. In the example provided above in the section on quota sampling, we wanted to ensure we had equal representation across a host of placement dispositions for children in foster care. This helped us define our target sample size: (4) settings a quota of (7) participants from each type of setting = a target sample size of (28).
In Chapter 18, we will be talking about different approaches to data gathering, which may help to dictate the range of perspectives you want to represent. For instance, if you conduct a focus group, you want all of your participants to have some experience with the thing that you are studying, but you hope that their perspectives differ from one another. Furthermore, you may want to avoid groups of participants who know each other well in the same focus group (if possible), as this may lead to groupthink or level of familiarity that doesn't really encourage differences being expressed. Ideally, we want to encourage a discussion where a variety of ideas are shared, offering a more complete understanding of how the topic is experienced. This is true in all forms of qualitative data, in that your findings are likely to be more well-rounded and offer a broader understanding fo the issue if you recruit a sample with diverse perspectives.
Saturation
Finally, the concept of saturation has important implications for both qualitative sample size and data analysis. To understand the idea of saturation, it is first important to understand that unlike most quantitative research, with qualitative research we often at least begin the process of data analysis while we are still actively collecting data. This is called an iterative approach to data analysis. So, if you are a qualitative researcher conducting interviews, you may be aiming to complete 30 interviews. After you have completed your first five interviews, you may begin reviewing and coding (a term that refers to labeling the different ideas found in your transcripts) these interviews while you are still conducting more interviews. You go on to review each new interview that you conduct and code it for the ideas that are reflected there. Eventually, you will reach a point where conducting more interviews isn’t producing any new ideas, and this is the point of saturation. Reaching saturation is an indication that we can stop data collection. This may come before or after you hit 30, but as you can see, it is driven by the presence of new ideas or concepts in your interviews, not a specific number.
This chapter represents our transition in the text to a focus on qualitative methods in research. Throughout this chapter we have explored a number of topics including various types of qualitative data, approaches to qualitative sampling, and some considerations for recruitment and sample composition. It bears repeating that your plan for sampling should be driven by a number of things: your research question, what is feasible for you, especially as a student researcher, best practices in qualitative research. Finally, in subsequent chapters, we will continue the discussion about reflexivity as it relates to the qualitative research process that we began here.
Key Takeaways
- The composition of our qualitative sample comes with some important decisions to consider, including how large should our sample be and what level and type of diversity it should reflect. These decisions are guided by the purposes or aims of our study, as well as access to resources and our population.
- The concept of saturation is important for qualitative research. It helps us to determine when we have sufficiently collected a range of perspectives on the topic we are studying.
Exercises
Decision Point(s): What should your sample look like (sample composition)?
- How will you determine you have gathered enough data?
- Will you start in advance with a set a number of data sources (people or artifacts)?
- If so, how many?
- How was this number determined?
- OR will you use the concept of saturation to determine when to stop?
- Will you start in advance with a set a number of data sources (people or artifacts)?
- How diverse should your sample be and in what ways?
- What supports your decision in regards to the previous question?
Exercises
This isn't so much a decision point, but a chance for you to reflect on the choices you've made thus far in your protocol with regards to your: (1) ethical responsibility, (2) commitment to cultural humility, and (3) respect for empowerment of individuals and groups as a social work researcher. Think about each of the decisions you've made thus far and work across this grid to identify any important considerations that you need to take into account.
Decision Point | Ethical Responsibility | Cultural Humility | Empowerment |
Research Question | |||
Type of Data | |||
Sampling Approach | |||
Recruitment/ Access | |||
Sample Composition |
Exercises
Reflexive Journal Entry Prompt
You have been prompted to make a number of choices regarding how you will proceed with gathering your qualitative sample. Based on what you have learned and what you are planning, respond to the following questions below.
- What are the strengths of your sampling plan in respect to being able to answer your qualitative research question?
- How feasible is it for you, as a student researcher, to be able to carry out your sampling plan?
- What reservations or questions do you still need to have answered to adequately plan for your sample?
- What excites you about your proposal thus far?
- What worries you about your proposal thus far?
Univariate data analysis is a quantitative method in which a variable is examined individually to determine its distribution.
A convenience sample is formed by collecting data from those people or other relevant elements to which we have the most convenient access. Essentially, we take who we can get.
A quota sample involves the researcher identifying a subgroups within a population that they want to make sure to include in their sample, and then identifies a quota or target number to recruit that represent each of these subgroups.
Chapter Outline
- Ethical responsibility and cultural respectfulness (7 minute read)
- Critical considerations (8 minute read)
- Find the right qualitative data to answer my research question (17 minute read)
- How to gather a qualitative sample (21 minute read)
- What should my sample look like? (9 minute read)
Content warning: examples in this chapter contain references to substance use, ageism, injustices against the Black community in research (e.g. Henrietta Lacks and Tuskegee Syphillis Study), children and their educational experiences, mental health, research bias, job loss and business closure, mobility limitations, politics, media portrayals of LatinX families, labor protests, neighborhood crime, Batten Disease (childhood disorder), transgender youth, cancer, child welfare including kinship care and foster care, Planned Parenthood, trauma and resilience, sexual health behaviors.
Now let's change things up! In the previous chapters, we were exploring steps to create and carry out a quantitative research study. Quantitative studies are great when we want to summarize data and examine or test relationships between ideas using numbers and the power of statistics. However, qualitative research offers us a different and equally important tool. Sometimes the aim of research is to explore meaning and experience. If these are the goals of our research proposal, we are going to turn to qualitative research. Qualitative research relies on the power of human expression through words, pictures, movies, performance and other artifacts that represent these things. All of these tell stories about the human experience and we want to learn from them and have them be represented in our research. Generally speaking, qualitative research is about the gathering up of these stories, breaking them into pieces so we can examine the ideas that make them up, and putting them back together in a way that allows us to tell a common or shared story that responds to our research question. Back in Chapter 7 we talked about different paradigms.
Before plunging further into our exploration of qualitative research, I would like to suggest that we begin by thinking about some ethical, cultural and empowerment-related considerations as you plan your proposal. This is by no means a comprehensive discussion of these topics as they relate to qualitative research, but my intention is to have you think about a few issues that are relevant at each step of the qualitative process. I will begin each of our qualitative chapters with some discussion about these topics as they relate to each of these steps in the research process. These sections are specially situated at the beginning of the chapters so that you can consider how these principles apply throughout the proceeding discussion. At the end of this chapter there will be an opportunity to reflect on these areas as they apply specifically to your proposal. Now, we have already discussed research ethics back in Chapter 8. However, as qualitative researchers we have some unique ethical commitments to participants and to the communities that they represent. Our work as qualitative researchers often requires us to represent the experiences of others, which also means that we need to be especially attentive to how culture is reflected in our research. Cultural respectfulness suggests that we approach our work and our participants with a sense of humility. This means that we maintain an open mind, a desire to learn about the cultural context of participants' lives, and that we preserve the integrity of this context as we share our findings.
17.1 Ethical responsibility and cultural respectfulness
Learning Objectives
Learners will be able to...
- Explain how our ethical responsibilities as researchers translate into decisions regarding qualitative sampling
- Summarize how aspects of culture and identity may influence recruitment for qualitative studies
Representation
Representation reflects two important aspects of our work as qualitative researchers, who is present and how are they presented. First, we need to consider who we are including or excluding in or sample. Recruitment and sampling is especially tied to our ethical mandate as researchers to uphold the principle of justice under the Belmont Report[8] (see Chapter 6 for additional information). Within this context we need to:
- Assure there is fair distribution of risks and benefits related to our research
- Be conscientious in our recruitment efforts to support equitable representation
- Ensure that special protections to vulnerable groups involved in research activities are in place
As you plan your qualitative research study, make sure to consider who is invited and able to participate and who is not. These choices have important implications for your findings and how well your results reflect the population you are seeking to represent. There may be explicit exclusions that don't allow certain people to participate, but there may also be unintended reasons people are excluded (e.g. transportation, language barriers, access to technology, lack of time).
The second part of representation has to do with how we disseminate our findings and how this reflects on the population we are studying. We will speak further about this aspect of representation in Chapter 21, which is specific to qualitative research dissemination. For now, it is enough to know that we need to be thoughtful about who we attempt to recruit and how effectively our resultant sample reflects our population.
Being mindful of history
As you plan for the recruitment of your sample, be mindful of the history of how this group (and/or the individuals you may be interacting with) has been treated – not just by the research community, but by others in positions of power. As researchers, we usually represent an outside influence and the people we are seeking to recruit may have significant reservations about trusting us and being willing to participate in our study (often grounded in good historical reasons—see Chapter 6 for additional information). Because of this, be very intentional in your efforts to be transparent about the purpose of your research and what it involves, why it is important to you, as well as how it can impact the community. Also, in helping to address this history, we need to make concerted efforts to get to know the communities that we research with well, including what is important to them.
Stories as sacred: How are we requesting them?
Finally, it is worth pointing out that as qualitative researchers, we have an extra layer of ethical and cultural responsibility. While quantitative research deals with numbers, as qualitative researchers, we are generally asking people to share their stories. Stories are intimate, full of depth and meaning, and can reveal tremendous amounts about who we are and what makes us tick. Because of this, we need to take special care to treat these stories as sacred. I will come back to this point in subsequent chapters, but as we go about asking for people to share their stories, we need to do so humbly.
Key Takeaways
- As researchers, we need to consider how our participant communities have been treated historically, how we are representing them in the present through our research, and the implications this representation could have (intended and unintended) for their lives. We need to treat research participants and their stories with respect and humility.
- When conducting qualitative research, we are asking people to share their stories with us. These "data" are personal, intimate, and often reflect the very essence of who our participants are. As researchers, we need to treat research participants and their stories with respect and humility.
17.2 Critical considerations
Learning Objectives
Learners will be able to...
- Assess dynamics of power in sampling design and recruitment for individual participants and participant communities
- Create opportunities for empowerment through early choice points in key research design elements
Power
Related to the previous discussion regarding being mindful of history, we also need to consider the current dynamics of power between researcher and potential participant. While we may not always recognize or feel like we are in a position of power, as researchers we hold specialized knowledge, a particular skill set, and what we do can with the data we collect can have important implications and consequences for individuals, groups, and communities. All of these contribute to the formation of a role ascribed with power. It is important for us to consider how this power is perceived and whenever possible, how we can build opportunities for empowerment that can be built into our research design. Examples of some strategies include:
- Recruiting and meeting in spaces that are culturally acceptable
- Finding ways to build participant choice into the research process
- Working with a community advisory group during the research process (explained further in the example box below)
- Designing informative and educational materials that help to thoroughly explain the research process in meaningful ways
- Regularly checking with participants for understanding
- Asking participants what they would like to get out of their participation and what it has been like to participate in our research
- Determining if there are ways that we can contribute back to communities beyond our research (developing an ongoing commitment to collaboration and reciprocity)
While it may be beyond the scope of a student research project to address all of these considerations, I do think it is important that we start thinking about these more in our research practices. As social work researchers, we should be modeling empowerment practices in the field of social science research, but we often fail to meet this standard.
Example. A community advisory group can be a tremendous asset throughout our research process, but especially in early stages of planning, including recruitment. I was fortunate enough to have a community advisory group for one of the projects I worked on. They were incredibly helpful as I considered different perspectives I needed to include in my study, helping me to think through a respectful way to approach recruitment, and how we might make the research arrangement a bit more reciprocal so community members might benefit as well.
Intersectional identity
As qualitative researchers, we are often not looking to prove a hypothesis or uncover facts. Instead, we are generally seeking to expand our understanding of the breadth and depth of human experience. Breadth is reflected as we seek to uncover variation across participants and depth captures variation or detail within each participants' story. Both are important for generating the fullest picture possible for our findings. For example, we might be interested in learning about people's experience living in an assisted living facility by interviewing residents. We would want to capture a range of different residents' experiences (breadth) and for each resident, we would seek as much detail as possible (depth). Do note, sometimes our research may only involve one person, such as in a case study. However, in these instances we are usually trying to understand many aspects or dimensions of that single case.
To capture this breadth and depth we need to remember that people are made of multiple stories formed by intersectional identities. This means that our participants never just represent one homogeneous social group. We need to consider the various aspects of our population that will help to give the most complete representation in our sample as we go about recruitment.
Exercises
Identify a population you are interested in studying. This might be a population you are working with at your field placement (either directly or indirectly), a group you are especially interested in learning more about, or a community you want to serve in the future. As you formulate your question, you may draw your sample directly from clients that are being served, others in their support network, service providers that are providing services, or other stakeholders that might be invested in the well-being of this group or community. Below, list out two populations you are interested in studying and then for each one, think about two groups connected with this population that you might focus your study on.
Population | Group |
1. | 1a. |
1b. | |
2. | 2a. |
2b. |
Next, think about what would kind of information might help you understand this group better. If you had the chance to sit down and talk with them, what kinds of things would you want to ask? What kinds of things would help you understand their perspective or their worldview more clearly? What kinds of things do we need to learn from them and their experiences that could help us to be better social workers? For each of the groups you identified above, write out something you would like to learn from their experience.
Population | Group | What I would like to learn from their experience |
1 | 1a. | 1a. |
1b. | 1b. | |
2. | 2a. | 2a. |
2b. | 2b. |
Finally, consider how this group might perceive a request to participate. For the populations and the groups that you have identified, think about the following questions:
- How have these groups been represented in the news?
- How have these groups been represented in popular culture and popular media?
- What historical or contemporary factors might influence these group members' opinions of research and researchers?
- In what ways have these groups been oppressed and how might research or academic institutions have contributed to this oppression?
Our impact on the qualitative process
It is important for qualitative research to thoughtfully plan for and attempt to capture our own impact on the research process. This influence that we can have on the research process represents what is known as researcher bias. This requires that we consider how we, as human beings, influence the research we conduct. This starts at the very beginning of the research process, including how we go about sampling. Our choices throughout the research process are driven by our unique values, experiences, and existing knowledge of how the world works. To help capture this contribution, qualitative researchers may plan to use tools like a reflexive journal, which is a research journal that helps the researcher to reflect on and consider their thoughts and reactions to the research process and how these may influence or shape a study (there will be more about this tool in Chapter 20 when we discuss the concept of rigor). While this tool is not specific to the sampling process, the next few chapters will suggest reflexive journal questions to help you think through how it might be used as you develop a qualitative proposal.
Example. To help demonstrate the potential for researcher bias, consider a number of students that I work with who are placed in school systems for their field experience and choose to focus their research proposal in this area. Some are interested in understanding why parents or guardians aren't more involved in their children's educational experience. While this might be an interesting topic, I would encourage students to consider what kind of biases they might have around this issue.
- What expectations do they have about parenting?
- What values do they attach to education and how it should be supported in the home?
- How has their own upbringing shaped their expectations?
- What do they know about the families that the school district serves and how did they come by this information?
- How are these families' life experiences different from their own?
The answers to these questions may unconsciously shape the early design of the study, including the research question they ask and the sources of data they seek out. For instance, their study may only focus on the behaviors and the inclinations of the families, but do little to investigate the role that the school plays in engagement and other structural barriers that might exist (e.g. language, stigma, accessibility, child-care, financial constraints, etc.).
Key Takeaways
- As researchers, we wield (sometimes subtle) power and we need to be conscientious of how we use and distribute this power.
- Qualitative study findings represent complex human experiences. As good as we may be, we are only going to capture a relatively small window into these experiences (and need to be mindful of this when discussing our findings).
Exercises
In the early stages of your research process, it is a good idea to start your reflexive journal. Starting a reflexive journal is as easy as opening up a new word document, titling it and chronologically dating your entries. If you are more tactile-oriented, you can also keep your reflexive journal in paper bound journal.
To prompt your initial entry, put your thoughts down in response to the following questions:
- What led you to be interested in this topic?
- What experience(s) do you have in this area?
- What knowledge do you have about this issue and how did you come by this knowledge?
- In what ways might you be biased about this topic?
Don't answer this last question too hastily! Our initial reaction is often—"Biased!?! Me—I don't have a biased bone in my body! I have an open-mind about everything, toward everyone!" After all, much of our social work training directs us towards acceptance and working to understand the perspectives of others. However, WE ALL HAVE BIASES. These are conscious or subconscious preferences that lead us to favor some things over others. These preferences influence the choices we make throughout the research process. The reflexive journal helps us to reflect on these preferences, where they might stem from, and how they might be influencing our research process. For instance, I conduct research in the area of mental health. Before I became a researcher, I was a mental health clinician, and my years as a mental health practitioner created biases for me that influence my approach to research. For instance, I may be biased in perceiving mental health services as being well-intentioned and helpful. However, participants may well have very different perceptions based on their experiences or beliefs (or those of their loved ones).
17.3 Finding the right qualitative data to answer my research question
Learning Objectives
Learners will be able to...
- Compare different types of qualitative data
- Begin to formulate decisions as they build their qualitative research proposal, specially in regards to selecting types of data that can effectively answer their research question
Sampling starts with deciding on the type of data you will be using. Qualitative research may use data from a variety of sources. Sources of qualitative data may come from interviews or focus groups, observations, a review of written documents, administrative data, or other forms of media, and performances. While some qualitative studies rely solely on one source of data, others incorporate a variety.
You should now be well acquainted with the term triangulation. When thinking about triangulation in qualitative research, we are often referring to our use of multiple sources of data among those listed above to help strengthen the confidence we have in our findings. Drawing on a journalism metaphor, this allows us to "fact check" our data to help ensure that we are getting the story correct. This can mean that we use one type of data (like interviews), but we intentionally plan to get a diverse range of perspectives from people we know will see things differently. In this case we are using triangulation of perspectives. In addition, we may also you a variety of different types of data, like including interviews, data from case records, and staff meeting minutes all as data sources in the same study. This reflects triangulation through types of data.
As a student conducting research, you may not always have access to vulnerable groups or communities in need, or it may be unreasonable for you to collect data from them directly due to time, resource, or knowledge constraints. Because of this, as you are reviewing the sections below, think about accessible alternative sources of data that will still allow you to answer your research question practically, and I will provide some examples along the way to get you started. In the above example, local media coverage might be a means of obtaining data that does not involve vulnerable directly collecting data from potentially vulnerable participants.
Verbal data
Perhaps the bread and butter of the qualitative researcher, we often rely on what people tell us as a primary source of information for qualitative studies in the form of verbal data. The researcher who schedules interviews with recipients of public assistance to capture their experience after legislation drastically changes requirements for benefits relies on the communication between the researcher and the impacted recipients of public assistance. Focus groups are another frequently used method of gathering verbal data. Focus groups bring together a group of participants to discuss their unique perspectives and explore commonalities on a given topic. One such example is a researcher who brings together a group of child welfare workers who have been in the field for one to two years to ask them questions regarding their preparation, experiences, and perceptions regarding their work.
A benefit of utilizing verbal data is that it offers an opportunity for researchers to hear directly from participants about their experiences, opinions, or understanding of a given topic. Of course, this requires that participants be willing to share this information with a researcher and that the information shared is genuine. If groups of participants are unwilling to participate in sharing verbal data or if participants share information that somehow misrepresents their feelings (perhaps because they feel intimidated by the research process), then our qualitative sample can become biased and lead to inaccurate or partially accurate findings.
As noted above, participant willingness and honesty can present challenges for qualitative researchers. You may face similar challenges as a student gathering verbal data directly from participants who have been personally affected by your research topic. Because of this, you might want to gather verbal data from other sources. Many of the students I work with are placed in schools. It is not feasible for them to interview the youth they work with directly, so frequently they will interview other professionals in the school, such as teachers, counselors, administration, and other staff. You might also consider interviewing other social work students about their perceptions or experiences working with a particular group.
Again, because it may be problematic or unrealistic for you to obtain verbal data directly from vulnerable groups as a student researcher, you might consider gathering verbal data from the following sources:
- Interviews and focus groups with providers, social work students, faculty, the general public, administrators, local politicians, advocacy groups
- Public blogs of people invested in your topic
- Publicly available transcripts from interviews with experts in the area or people reporting experiences in popular media
Make sure to consult with your professor to ensure that what you are planning will be realistic for the purposes of your study.
Observational data
As researchers, we sometimes rely on our own powers of observation to gather data on a particular topic. We may observe a person’s behavior, an interaction, setting, context, and maybe even our own reactions to what we are observing (i.e. what we are thinking or feeling). When observational data is used for quantitative purposes, it involves a count, such as how many times a certain behavior occurs for a child in a classroom. However, when observational data is used for qualitative purposes, it involves the researcher providing a detailed description. For instance, a qualitative researcher may conduct observations of how mothers and children interact in child and adolescent cancer units, and take notes about where exchanges take place, topics of conversation, nonverbal information, and data about the setting itself – what the unit looks like, how it is arranged, the lighting, photos on the wall, etc.
Observational data can provide important contextual information that may not be captured when we rely solely on verbal data. However, using this form of data requires us, as researchers, to correctly observe and interpret what is going on. As we don’t have direct access to what participants may be thinking or feeling to aid us (which can lead us to misinterpret or create a biased representation of what we are observing), our take on this situation may vary drastically from that of another person observing the same thing. For instance, if we observe two people talking and one begins crying, how do we know if these are tears of joy or sorrow? When you observe someone being abrupt in a conversation, I might interpret that as the person being rude while you might perceive that the person is distracted or preoccupied with something. The point is, we can't know for sure. Perhaps one of the most challenging aspects of gathering observational data is collecting neutral, objective observations, that are not laden with our subjective value judgments placed on them. Students often find this out in class during one of our activities. For this activity, they have to go out to public space and write down observations about what they observe. When they bring them back to class and we start discussing them together, we quickly realize how often we make (unfounded) judgments. Frequent examples from our class include determining the race/ethnicity of people they observe or the relationships between people, without any confirmational knowledge. Additionally, they often describe scenarios with adverbs and adjectives that often reflect judgments and values they are placing on their data. I'm not sharing this to call them out, in fact, they do a great job with the assignment. I just want to demonstrate that as human beings, we are often less objective than we think we are! These are great examples of research bias.
Again, gaining access to observational spaces, especially private ones, might be a challenge for you as a student. As such, you might consider if observing public spaces might be an option. If you do opt for this, make sure you are not violating anyone's right to privacy. For instance, gathering information in a narcotics anonymous meeting or a religious celebration might be perceived as offensive, invasive or in direct opposition to values (like anonymity) of participants. When making observations in public spaces be careful not to gather any information that might identify specific individuals or organizations. Also, it is important to consider the influence your presence may have on a community, particularly if your observation makes you stand out among those typically present in that setting. Always consider the needs of the individual and the communities in formulating a plan for observing public behavior. Public spaces might include commercial spaces or events open to the public as well as municipal parks. Below we will have an expanded discussion about different varieties of non-probability sampling strategies that apply to qualitative research. Recruiting in public spaces like these may work for strategies such as convenience sampling or quota sampling, but would not be a good choice for snowball sampling or purposive sampling.
As with the cautionary note for student researchers under verbal data, you may experience restricted access to spaces in which you are able to gather observational data. However, if you do determine that observational data might be a good fit for your student proposal, you might consider the following spaces:
- Shopping malls
- Public parks or beaches
- Public meetings or rallies
- Public transportation
Artifacts (documents & other media)
Existing artifacts can also be very useful for the qualitative researcher. Examples include newspapers, blogs, websites, podcasts, television shows, movies, pictures, video recordings, artwork, and live performances. While many of these sources may provide indirect information on a topic, this information can still be quite valuable in capturing the sentiment of popular culture and thereby help researchers enhance their understanding of (dominant) societal values and opinions. Conversely, researchers can intentionally choose to seek out divergent, unique or controversial perspectives by searching for artifacts that tend to take up positions that differ from the mainstream, such as independent publications and (electronic) newsletters. While we will explore this further below, it is important to understand that data and research, in all its forms, is political. Among many other purposes, it is used to create, critique, and change policy; to engage in activism; to support and refute how organizations function; and to sway public opinion.
When utilizing documents and other media as artifacts, researchers may choose to use an entire source (such as a book or movie), or they may use a segment or portion of that artifact (such as the front-page stories from newspapers, or specific scenes in a television series). Your choice of which artifacts you choose to include will be driven by your question, and remember, you want your sample of artifacts to reflect the diversity of perspectives that may exist in the population you are interested in. For instance, perhaps I am interested in studying how various forms of media portray substance use treatment. I might intentionally include a range of liberal to conservative views that are portrayed across a number of media sources.
As qualitative researchers using artifacts, we often need to do some digging to understand the context of said artifact. We do this because data is almost always affiliated or aligned with some position (again, data is political). To help us consider this, it may be helpful to reflect on the following questions:
- Who owns the artifact or where is it housed
- What values does the owner (organization or person) hold
- How might the position or identity of the owner influence what information is shared or how it is portrayed
- What is the purpose of the artifact
- Who is the audience for which the artifact is intended
Answers to questions such as these can help us to better understand and give meaning to the content of the artifacts. Content is the substance of the artifact (e.g. the words, picture, scene). While context is the circumstances surrounding content. Both work together to help provide meaning, and further understanding of what can be derived from an artifact. As an example to illustrate this point, let's say that you are including meeting minutes from an organizing network as a source of data for your study. The narrative description in these minutes will certainly be important, however, they may not tell the whole story. For instance, you might not know from the text that the organization has recently voted in a new president and this has created significant division within the network. Knowing this information might help you to interpret the agenda and the discussion contained in the minutes very differently.
As student researchers, using documents and other artifacts may be a particularly appealing source of data for your study. This is because this data already exists (you aren't creating new data) and depending on what you select, it might be relatively easy to access. Examples of utilizing existing artifacts might include studying the cultural context of movie portrayals of Latinx families or analyzing publicly available town hall meeting minutes to explore expressions of social capital. Below is a list of sources of data from documents or other media sources to consider for your student proposal:
- Movies or TV shows
- Music or music videos
- Public blogs
- Policies or other organizational documents
- Meeting minutes
- Comments in online forums
- Books, newspapers, magazines, or other print/virtual text-based materials
- Recruitment, training, or educational materials
- Musical or artistic expressions
Photovoice
Finally, Photovoice is a technique that merges pictures with narrative (word or voice) data that helps interpret the meaning or significance of the visual. Photovoice is often used for qualitative work that is conducted as part of Community Based Participatory Research (CBPR), wherein community members act as both participants and as co-researchers. These community members are provided with a means of capturing images that reflect their understanding of some topic, prompt or question, and then they are asked to provide a narrative description or interpretation to help give meaning to the image(s). Both the visual and narrative information are used as qualitative data to include in the study. Dissemination of Photovoice projects often involve a public display of the works, such as through a demonstration or art exhibition to raise awareness or to produce some specific change that is desired by participants. Because this form of study is often intentionally persuasive in nature, we need to recognize that this form of data will be inherently subjective. As a student, it may be particularly challenging to implement a Photovoice project, especially due to its time-intensive nature, as well as the additional commitments of needing to engage, train, and collaborate with community partners.
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Verbal |
|
Strengths
Challenges
|
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Observational | Observations in:
|
Strengths
Challenges
|
Type of Data | Potential Sources of Student Data | Some Potential Strengths and Challenges with this Type of Data |
Documents & Other Media |
|
Strengths
Challenges
|
How many kinds of data?
You will need to consider whether you will rely on one kind of data or multiple. While many qualitative studies solely use one type of data, such as interviews or focus groups, others may use multiple sources. The decision to use multiple sources is often made to help strengthen the confidence we have in our findings or to help us to produce a richer, more detailed description of our results. For instance, if we are conducting a case study of what the family experience is for a child with a very rare disorder like Batten Disease, we may use multiple sources of data. These can include observing family and community interactions, conducting interviews with family members and others connected to the family (such as service providers,) and examining journal entries families were asked to keep over the course of the study. By collecting data from a variety of sources such as this, we can more broadly represent a range of perspectives when answering our research question, which will hopefully provide a more holistic picture of the family experience. However, if we are trying to examine the decision-making processes of adult protective workers, it may make the most sense to rely on just one type of data, such as interviews with adult protective workers.
Key Takeaways
- There are numerous types of qualitative data (verbal, observational, artifacts) that we may be able to access when planning a qualitative study. As we plan, we need to consider the strengths and challenges that each possess and how well each type might answer our research question.
- The use of multiple types of qualitative data does add complexity to a study, but this complication may well be worth it to help us explore multiple dimensions of our topic and thereby enrich our findings.
Exercises
Reflexive Journal Entry Prompt
For your next entry, consider responding to the following:
- What types of data appeal to you?
- Why do you think you are drawn to them?
- How well does this type of data "fit" as a means of answering your question? Why?
17.4 How to gather a qualitative sample
Learning Objectives
Learners will be able to...
- Compare and contrast various non-probability sampling approaches
- Select a sampling strategy that ideologically fits the research question and is practical/actionable
Before we launch into how to plan our sample, I'm going to take a brief moment to remind us of the philosophical basis surrounding the purpose of qualitative research—not to punish you, but because it has important implications for sampling.
Nomothetic vs. idiographic
As a quick reminder, as we discussed in Chapter 8 idiographic research aims to develop a rich or deep understanding of the individual or the few. The focus is on capturing the uniqueness of a smaller sample in a comprehensive manner. For example, an idiographic study might be a good approach for a case study examining the experiences of a transgender youth and her family living in a rural Midwestern state. Data for this idiographic study would be collected from a range of sources, including interviews with family members, observations of family interactions at home and in the community, a focus group with the youth and her friend group, another focus group with the mother and her social network, etc. The aim would be to gain a very holistic picture of this family's experiences.
On the other hand, nomothetic research is invested in trying to uncover what is ‘true’ for many. It seeks to develop a general understanding of a very specific relationship between variables. The aim is to produce generalizable findings, or findings that apply to a large group of people. This is done by gathering a large sample and looking at a limited or restricted number of aspects. A nomothetic study might involve a national survey of heath care providers in which thousands of providers are surveyed regarding their current knowledge and competence in treating transgender individuals. It would gather data from a very large number of people, and attempt to highlight some general findings across this population on a very focused topic.
Idiographic and nomothetic research represent two different research categories existing at opposite extremes on a continuum. Qualitative research generally exists on the idiographic end of this continuum. We are most often seeking to obtain a rich, deep, detailed understanding from a relatively small group of people.
Non-probability sampling
Non-probability sampling refers to sampling techniques for which a person’s (or event’s) likelihood of being selected for membership in the sample is unknown. Because we don’t know the likelihood of selection, we don’t know whether a sample represents a larger population or not. But that’s okay, because representing the population is not the goal of nonprobability samples. That said, the fact that nonprobability samples do not represent a larger population does not mean that they are drawn arbitrarily or without any specific purpose in mind. We typically use nonprobability samples in research projects that are qualitative in nature. We will examine several types of nonprobability samples. These include purposive samples, snowball samples, quota samples, and convenience samples.
Convenience or availability
Convenience sampling, also known as availability sampling, is a nonprobability sampling strategy that is employed by both qualitative and quantitative researchers. To draw a convenience sample, we would simply collect data from those people or other relevant elements to which we have the most convenient access. While convenience samples offer one major benefit—convenience—we should be cautious about generalizing from research that relies on convenience samples because we have no confidence that the sample is representative of a broader population. If you are a social work student who needs to conduct a research project at your field placement setting and you decide to conduct a focus group with the staff at your agency, you are using a convenience sampling approach – you are recruiting participants that are easily accessible to you. In addition, if you elect to analyze existing data that your social work program has collected as part of their graduation exit surveys, you are using data that you readily have access to for your project; again, you have a convenience sample. The vast majority of students I work with on their proposal design rely on convenience data due to time constraints and limited resources.
Purposive
To draw a purposive sample, we begin with specific perspectives or purposive criteria in mind that we want to examine. We would then seek out research participants who cover that full range of perspectives. For example, if you are studying mental health supports on your campus, you may want to be sure to include not only students, but mental health practitioners and student affairs administrators as well. You might also select students who currently use mental health supports, those who dropped out of supports, and those who are waiting to receive supports. The "purposive" part of purposive sampling comes from selecting specific participants on purpose because you already know they have certain characteristics—being an administrator, dropping out of mental health supports, for example—that you need in your sample.
Note that these differ from inclusion criteria, which are more general requirements a person must possess to be a part of your sample; to be a potential participant that may or may not be sampled. For example, one of the inclusion criteria for a study of your campus’ mental health supports might be that participants had to have visited the mental health center in the past year. That differs from purposive sampling. In purposive sampling, you know characteristics of individuals and recruit them because of those characteristics. For example, I might recruit Jane because she stopped seeking supports this month, because she has worked at the center for many years, and so forth.
Also, it’s important to recognize that purposive sampling requires you to have prior information about your participants before recruiting them because you need to know their perspectives or experiences before you know whether you want them in your sample. This is a common mistake that many students make. What I often hear is, “I’m using purposive sampling because I’m recruiting people from the health center,” or something like that. That’s not purposive sampling. In most instances they really mean they are going to use convenience sampling-taking whoever they can recruit that fit the inclusion criteria (i.e. have attended the mental health center). Purposive sampling is recruiting specific people because of the various characteristics and perspectives they bring to your sample. Imagine we were creating a focus group. A purposive sample might gather clinicians, patients, administrators, staff, and former patients together so they can talk as a group. Purposive sampling would seek out people that have each of those attributes.
If you are considering using a purposive sampling approach for your research proposal, you will need to determine what your purposive criteria involves. There are a range of different purposive strategies that might be employed, including: maximum variation, typical case, extreme case, or political case, and you want to be thoughtful in thinking about which one(s) you select and why.
Purposive Strategy | Description | Student Example |
Maximum Variation | Case(s) selected to represent a range of very different perspectives on a topic | You interview student leaders from the schools of social work, business, the arts, math & science, education, history & anthropology and health studies to ensure that you have the perspective of a variety of disciplines |
Typical Case | Case(s) selected to reflect a commonly held perspective. | You interview a child welfare worker specifically because many of their characteristics fit the state statistical profile for providers in that service area. |
Extreme Case | Case(s) selected to represent extreme or underrepresented perspectives. | You examine websites devoted to rare cancer survivor support. |
Political Case | Case(s) selected to represent a contemporary politicized issue | You analyze media interviews with Planned Parenthood providers, employees, and clients from 2010 to present. |
Expert Case | Case(s) selected based on specialized content knowledge or expertise | You are interested in studying resilience in trauma providers, so you research and reach out to a handful of authorities in this area. |
Theory-Based Case | Case(s) selected based on their representation of a specific theoretical orientation or some aspect of a given theory | You are interested in studying how training methods vary by practitioner according to their theoretical orientation. You specifically reach out to a clinician who identifies as a Cognitive Behavioral clinician, one who identifies as Bowenian, and one who identifies as Structural Family. |
Critical Case | Case(s) selected based on the likelihood that the case will yield the desired information | You examine a public gaming network forum on social media to see how participants offer support to one another. |
It can be a bit tricky determining how to approach or formulate your purposive cases. Below are a couple additional resources to explore this strategy further.
For more information on purposive sampling consult this webpage from Laerd Statistics on purposive sampling and this webpage from the University of Connecticut on education research.
Snowball
When using snowball sampling, we might know one or two people we’d like to include in our study but then we have to rely on those initial participants to help identify additional participants. Thus, our sample builds and grows as the study continues, much as a snowball builds and becomes larger as it rolls through the snow. Snowball sampling is an especially useful strategy when you wish to study a stigmatized group or behavior. These groups may have limited visibility and accessibility for a variety of reasons, including safety.
Malebranche and colleagues (2010)[12] were interested in studying sexual health behaviors of Black, bisexual men. Anticipating that this may be a challenging group to recruit, they utilized a snowball sampling approach. They recruited initial contacts through activities such as advertising on websites and distributing fliers strategically (e.g. barbershops, nightclubs). These initial recruits were compensated with $50 and received study information sheets and five contact cards to distribute to people in their social network that fit the study criteria. Eventually the research team was able to recruit a sample of 38 men who fit the study criteria.
Snowball sampling may present some ethical quandaries for us. Since we are essentially relying on others to help advertise for us, we are giving up some of our control over the process of recruitment. We may be worried about coercion, or having people put undue pressure to have others' they know participate in your study. To help mitigate this, we would want to make sure that any participant we recruit understands that participation is completely voluntary and if they tell others about the studies, they should also make them aware that it is voluntary, too. In addition to coercion, we also want to make sure that people's privacy is not violated when we take this approach. For this reason, it is good practice when using a snowball approach to provide people with our contact information as the researchers and ask that they get in touch with us, rather than the other way around. This may also help to protect again potential feelings of exploitation or feeling taken advantage of. Because we often turn to snowball sampling when our population is difficult to reach or engage, we need to be especially sensitive to why this is. It is often because they have been exploited in the past and participating in research may feel like an extension of this. To address this, we need to have a very clear and transparent informed consent process and to also think about how we can use or research to benefit the people we work in the most meaningful and tangible ways.
Quota
Quota sampling is another nonprobability sampling strategy. This type of sampling is actually employed by both qualitative and quantitative researchers, but because it is a nonprobability method, we’ll discuss it in this section. When conducting quota sampling, we identify categories that are important to our study and for which there is likely to be some variation. Subgroups are created based on each category and the researcher decides how many people (or whatever element happens to be the focus of the research) to include from each subgroup and collects data from that number for each subgroup. To demonstrate, perhaps we are interested in studying support needs for children in the foster care system. We decide that we want to examine equal numbers (seven each) of children placed in a kinship placement, a non-kinship foster placement, group home, and residential placements. We expect that the experiences and needs across these settings may differ significantly, so we want to have good representation of each one, thus setting a quota of seven for each type of placement.
Sampling Type | Brief Description |
Convenience/ Availability | You gather data from whatever cases/people/documents happen to be convenient |
Purposive | You seek out elements that meet specific criteria, representing different perspectives |
Snowball | You rely on participant referrals to recruit new participants |
Quota | You select a designated number of cases from specified subgroups |
As you continue to plan for your proposal, below you will find some of the strengths and challenges presented by each of these types of sampling.
Sampling Type | Strengths | Challenges |
Convenience/ Availability | Allows us to draw sample from participants who are most readily available/accessible | Sample may be biased and may represent limited or skewed diversity in characteristics of participants |
Purposive | Ensures that specific expertise, positions, or experiences are represented in sample participants | It may be challenging to define purposive criteria or to locate cases that represent these criteria; restricts our potential sampling pool |
Snowball | Accesses participant social network and community knowledge
Can be helpful in gaining access to difficult to reach populations |
May be hard to locate initial small group of participants, concerns over privacy—people might not want to share contacts, process may be slow or drawn-out |
Quota | Helps to ensure specific characteristics are represented and defines quantity of each | Can be challenging to fill quotas, especially for subgroups that might be more difficult to locate or reluctant to participate |
Wait a minute, we need a plan!
Both qualitative and quantitative research should be planful and systematic. We've actually covered a lot of ground already and before we get any further, we need to start thinking about what the plan for your qualitative research proposal will look like. This means that as you develop your research proposal, you need to consider what you will be doing each step of the way: how you will find data, how you will capture it, how you will organize it, and how you will store it. If you have multiple types of data, you need to have a plan in place for each type. The plan that you develop is your data collection protocol. If you have a team of researchers (or are part of a research team), the data collection protocol is an important communication tool, making sure that everyone is clear what is going on as the research proceeds. This plan is important to help keep you and others involved in your research consistent and accountable. Throughout this chapter and the next (Chapter 18—qualitative data gathering) we will walk through points you will want to include in your data collection protocol. While I've spent a fair amount of time talking about the importance of having a plan here, qualitative design often does embrace some degree of flexibility. This flexibility is related to the concept of emergent design that we find in qualitative studies. Emergent design is the idea that some decision in our design will be dynamic and fluid as our understanding of the research question evolves. The more we learn about the topic, the more we want to understand it thoroughly.
Exercises
A research protocol is a document that not only defines your research project and its aims, but also comprehensively plans how you will carry it out. If this sounds like the function of a research proposal, you are right, they are similar. What differentiates a protocol from a proposal is the level of detail. A proposal is more conceptual; a protocol is more practical (right down to the dollars and cents!). A protocol offers explicit instructions for you and your research team, any funders that may be involved in your research, and any oversight bodies that might be responsible for overseeing your study. Not every study requires a research protocol, but what I'm suggesting here is that you consider constructing at least a limited one to help though the decisions you will need to make to construct your qualitative study.
Al-Jundi and Sakka (2016)[13] provide the following elements for a research protocol:
- What is the question? (Hypothesis) What is to be investigated?
- Why is the study important (Significance)
- Where and when will it take place?
- What is the methodology? (Procedures and methods to be used).
- How are you going to implement it? (Research design)
- What is the proposed time table and budget?
- What are the resources required (technical, scientific, and financial)?
While your research proposal in its entirety will focus on many of these areas, our attention for developing your qualitative research protocol will hone in on the two highlighted above. As we go through these next couple chapters, there will be a number of exercises that walk you though decision points that will form your qualitative research protocol.
To begin developing your qualitative research protocol:
- Select the question you have decided is the best to frame your research proposal.
- Write a brief paragraph about the aim of your study, ending it with the research question you have selected.
Here are a few additional resources on developing a research protocol:
Cameli et al., (2018) How to write a research protocol: Tips and tricks.
Ohio State University, Institutional Review Board (n.d.). Research protocol.
World Health Organization (n.d.). Recommended format for a research protocol.
Exercises
Decision Point: What types of data will you be using?
- After having considered the different types of data that have been reviewed here, what type(s) are you planning on using?
- Why is this a good choice, given your research question?
- Are you thinking about using more than one type?
- If so, provide support for this decision.
Exercises
Decision Point: Which non-probability sampling strategy will you employ?
- Thinking about the four non-probability sampling strategies we reviewed, which one makes the most sense for your proposal?
- Why is this is a good fit?
- What challenges or limitations does this present for your study?
- What steps might your take to address these challenges?
Recruiting strategies
Much like quantitative research, recruitment for qualitative studies can take many different approaches. When considering how to draw your qualitative sample, it may be helpful to first consider which of these three general strategies will best fit your research question and general study design: public, targeted, or membership-based. While all will lead to a sample, the process for getting you there will look very different, depending on the strategy you select.
Public
Taking a public approach to recruitment offers you access to the broadest swath of potential participants. With this approach, you are taking advantage of public spaces in an attempt to gain the attention of the general population of people that frequent that space so that they can learn about your study. These spaces can be in-person (e.g. libraries, coffee shops, grocery stores, health care settings, parks) or virtual (e.g. open chat forums, e-bulletin boards, news feeds). Furthermore, a public approach can be static (such as hanging a flier), or dynamic (such as talking to people and directly making requests to participate). While a public approach may offer broad coverage in that it attempts to appeal to an array of people, it may be perceived as impersonal or easily able to be overlooked, due to the potential presence of other announcements that may be featured in public spaces. Public recruitment is most likely to be associated with convenience or quota sampling and is unlikely to be used with purposive or snowball sampling, where we would need some advance knowledge of people and the characteristics they possess.
Targeted
As an alternative, you may elect to take a targeted approach to recruitment. By targeting a select group, you are restricting your sampling frame to those individuals or groups who are potentially most well-suited to answer your research question. Additionally, you may be targeting specific people to help craft a diverse sample, particularly with respect to personal characteristics and/or opinions.
You can target your recruitment through the use of different strategies. First, you might consider the use of knowledgeable and well-connected community members. These are people who may possess a good amount of social capital in their community, which can aid in recruitment efforts. If you are considering the use of community members in this role, make sure to be thoughtful in your approach, as you are essentially asking them to share some of their social capital with you. This means learning about the community or group, approaching community members with a sense of humility, and making sure to demonstrate transparency and authenticity in your interactions. These community members may also be champions for the topic you are researching. A champion is someone who helps to draw the interest of a particular group of people. The champion often comes from within the group itself. As an example, let's say you're interested in studying the experiences of family members who have a loved one struggling with substance use. To aid in your recruitment for this study, you enlist the help of a local person who does a lot of work with Al-Anon, an organization facilitating mutual support groups for individuals and families affected by alcoholism.
A targeted approach can certainly help ensure that we are talking to people who are knowledgeable about the topic we are interested in, however, we still need to be aware of the potential for bias. If we target our recruitment based on connection to a particular person, event, or passion for the topic, these folks may share information that they think is viewed as favorable or that disproportionately reflects a particular perspective. This phenomenon is due to the fact that we often spend time with people who are like-minded or share many of our views. A targeted approach may be helpful for any type of non-probability sampling, but can be especially useful for purposive, quota, or snowball sampling, where we are trying to access people or groups of people with specific characteristics or expertise.
Membership-based
Finally, you might consider a membership-based approach. This approach is really a form of targeted recruitment, but may benefit from some individual attention. When using a membership-based approach, your sampling frame is the membership list of a particular organization or group. As you might have guessed, this organization or group must be well-suited for helping to answer your research question. You will need permission to access membership, and the identity of the person authorized to grant permission will depend on the organizational structure. When contacting members regarding recruitment, you may consider using directories, newsletters, listservs or membership meetings. When utilizing a membership-based approach, we often know that members possess specific inclusion criteria we need, however, because they are all associated with that particular group or organization, they may be homogenous or like-minded in other ways. This may limit the diversity in our sample and is something to be mindful of when interpreting our findings. Membership-based recruiting can be helpful when we have a membership group that fulfills our inclusion criteria. For instance, if you want to conduct research with social workers, you might attempt to recruit through the NASW membership distribution list (but this access will come with stipulations and a price tag). Membership-based recruitment may be helpful for any non-probability sampling approach, given that the membership criteria and study inclusion criteria are a close fit. Table 17.5 offers some additional considerations for each of these strategies with examples to help demonstrate sources that might correspond with them.
Recruitment Strategy | Strengths/Challenges | Example of Source for Recruitment |
Public | Strengths: Easier to gain access; Exposure to large numbers of people
Challenges: Can be impersonal, Difficult to cultivate interest |
Advertising in public events & spaces
Accessing materials in local libraries or museums Finding public web-based resources and sources of data (websites, blogs, open forums)
|
Targeted | Strengths: Prior knowledge of potential audience, More focused use of resources
Challenges: May be hard to locate/access target group(s), Groups may be suspicious of/or resistant to being targeted |
Working with advocacy group for issue you are studying to aid recruitment
Contacting local expert (historian) to help you locate relevant documents Advertising in places that your population may frequent
|
Membership-Based | Strengths: Shared interest (through common membership), Potentially existing infrastructure for outreach
Challenges: Organization may be highly sensitive to protecting members, Members may be similar in perspectives and limit diversity of ideas |
Membership newsletters
Listserv or Facebook groups Advertising at membership meetings or events |
Key Takeaways
- Qualitative research predominately relies on non-probability sampling techniques. There are a number of these techniques to choose from (convenience/availability, purposive, snowball, quota), each with advantages and limitations to consider. As we consider these, we need to reflect on both our research question and the resources we have available to us in developing a sampling strategy.
- As we consider where and how we will recruit our sample, there are a range of general approaches, including public, targeted, and membership-based.
Exercises
Decision Point: How will you recruit or gain access to your sample?
- Now you've decided your data type and sampling strategy...
- If you are recruiting people, how will you identify them? If necessary (and it often is), how will gain permission to do this?
- If you are using documents or other artifacts for your study, how will you gain access to these? If necessary (and it often is), how will gain permission to do this?
17.5 What should my sample look like?
Learning Objectives
Learners will be able to...
- Explain key factors that influence the makeup of a qualitative sample
- Develop and critique a sampling strategy to support their qualitative proposal
Once you have started your recruitment, you also need to know when to stop. Knowing when to stop recruiting for a qualitative research study generally involves a dynamic and reflective process. This means that you will actively be involved in a process of recruiting, collecting data, beginning to review your preliminary data, and conducting more recruitment to gather more data. You will continue this process until you have gathered enough data and included sufficient perspectives to answer your research question in rich and meaningful way.
The sample size of qualitative studies can vary significantly. For instance, case studies may involve only one participant or event, while some studies may involve hundreds of interviews or even thousands of documents. Generally speaking, when compared to quantitative research, qualitative studies have a considerably smaller sample. Your decision regarding sample size should be guided by a few considerations, described below.
Amount of data
When gathering quantitative data, the amount of data we are gathering is often specified at the start (e.g. a fixed number of questions on a survey or a set number of indicators on a tracking form). However, when gathering qualitative data, we are often asking people to expand on and explore their thoughts and reactions to certain things. This can produce A LOT of data. If you have ever had to transcribe an interview (type out the conversation while listening to an audio recorded interview), you quickly learn that a 15-minute discussion turns into many pages of dialogue. As such, each interview or focus group you conduct represents multi-page transcripts, all of which becomes your data. If you are conducting interviews or focus groups, you will know you have collected enough data from each interaction when you have covered all your questions and allowed the participant(s) to share any and all ideas they have related to the topic. If you are using observational data, you need to spend sufficient time making observations and capturing data to offer a genuine and holistic representation of the thing you are observing (at least to the best of your ability). When using documents and other sources of media, again, you want to ensure that diverse perspectives are represented through your artifact choices so that your data reflects a well-rounded representation of the issue you are studying. For any of these data sources, this involves a judgment call on the researcher's part. Your judgment should be informed by what you have read in the existing literature and consultation with your professor.
As part of your analysis, you will likely eventually break these larger hunks of data apart into words or small phrases, giving you potentially thousands of pieces of data. If you are relying on documents or other artifacts, the amount of data contained in each of these pieces is determined in advance, as they already exist. However, you will need to determine how many to include. With interviews, focus groups, or other forms of data generation (e.g. taking pictures for a photovoice project), we don’t necessarily know how much data will be generated with each encounter, as it will depend on the questions that are asked, the information that is shared, and how well we capture it.
Type of study
A variety of types of qualitative studies will be discussed in greater detail in Chapter 22. While you don't necessarily need to have an extensive understanding of them all at this point in time, it is important that you understand which of the different design types are best for answering certain research questions. For instance, if our question involves understanding some type of experience, that is often best answered by a phenomenological design. Or, if we want to better understand some process, a grounded theory study may be best suited. While there are no hard and fast rules regarding qualitative sample size, each of these different types of designs has different guidelines for what is considered an acceptable or reasonable number to include in your sample. So drawing on the previous examples, your grounded theory study might include 45 participants because you need more people to gain a clearer picture of each step of the process, while your phenomenological study includes 20 because that provides a good representation of the experience you are interested in. Both would be reasonable targets based on the respective study design type. So as you consider your research question and which specific type of qualitative design this leads you to, you will need to do some investigation to see what size samples are recommended for that particular type of qualitative design.
Diversity of perspectives
As you consider your research question, you also may want to think about the potential variation in how your study population might view this topic. If you are conducting a case study of one person, this obviously isn’t a concern, but if you are interested in exploring a range of experiences, you want to plan to intentionally recruit so this level of diversity is reflected in your sample. The level of variation you seek will have direct implications for how big your sample might be. In the example provided above in the section on quota sampling, we wanted to ensure we had equal representation across a host of placement dispositions for children in foster care. This helped us define our target sample size: (4) settings a quota of (7) participants from each type of setting = a target sample size of (28).
In Chapter 18, we will be talking about different approaches to data gathering, which may help to dictate the range of perspectives you want to represent. For instance, if you conduct a focus group, you want all of your participants to have some experience with the thing that you are studying, but you hope that their perspectives differ from one another. Furthermore, you may want to avoid groups of participants who know each other well in the same focus group (if possible), as this may lead to groupthink or level of familiarity that doesn't really encourage differences being expressed. Ideally, we want to encourage a discussion where a variety of ideas are shared, offering a more complete understanding of how the topic is experienced. This is true in all forms of qualitative data, in that your findings are likely to be more well-rounded and offer a broader understanding fo the issue if you recruit a sample with diverse perspectives.
Saturation
Finally, the concept of saturation has important implications for both qualitative sample size and data analysis. To understand the idea of saturation, it is first important to understand that unlike most quantitative research, with qualitative research we often at least begin the process of data analysis while we are still actively collecting data. This is called an iterative approach to data analysis. So, if you are a qualitative researcher conducting interviews, you may be aiming to complete 30 interviews. After you have completed your first five interviews, you may begin reviewing and coding (a term that refers to labeling the different ideas found in your transcripts) these interviews while you are still conducting more interviews. You go on to review each new interview that you conduct and code it for the ideas that are reflected there. Eventually, you will reach a point where conducting more interviews isn’t producing any new ideas, and this is the point of saturation. Reaching saturation is an indication that we can stop data collection. This may come before or after you hit 30, but as you can see, it is driven by the presence of new ideas or concepts in your interviews, not a specific number.
This chapter represents our transition in the text to a focus on qualitative methods in research. Throughout this chapter we have explored a number of topics including various types of qualitative data, approaches to qualitative sampling, and some considerations for recruitment and sample composition. It bears repeating that your plan for sampling should be driven by a number of things: your research question, what is feasible for you, especially as a student researcher, best practices in qualitative research. Finally, in subsequent chapters, we will continue the discussion about reflexivity as it relates to the qualitative research process that we began here.
Key Takeaways
- The composition of our qualitative sample comes with some important decisions to consider, including how large should our sample be and what level and type of diversity it should reflect. These decisions are guided by the purposes or aims of our study, as well as access to resources and our population.
- The concept of saturation is important for qualitative research. It helps us to determine when we have sufficiently collected a range of perspectives on the topic we are studying.
Exercises
Decision Point(s): What should your sample look like (sample composition)?
- How will you determine you have gathered enough data?
- Will you start in advance with a set a number of data sources (people or artifacts)?
- If so, how many?
- How was this number determined?
- OR will you use the concept of saturation to determine when to stop?
- Will you start in advance with a set a number of data sources (people or artifacts)?
- How diverse should your sample be and in what ways?
- What supports your decision in regards to the previous question?
Exercises
This isn't so much a decision point, but a chance for you to reflect on the choices you've made thus far in your protocol with regards to your: (1) ethical responsibility, (2) commitment to cultural humility, and (3) respect for empowerment of individuals and groups as a social work researcher. Think about each of the decisions you've made thus far and work across this grid to identify any important considerations that you need to take into account.
Decision Point | Ethical Responsibility | Cultural Humility | Empowerment |
Research Question | |||
Type of Data | |||
Sampling Approach | |||
Recruitment/ Access | |||
Sample Composition |
Exercises
Reflexive Journal Entry Prompt
You have been prompted to make a number of choices regarding how you will proceed with gathering your qualitative sample. Based on what you have learned and what you are planning, respond to the following questions below.
- What are the strengths of your sampling plan in respect to being able to answer your qualitative research question?
- How feasible is it for you, as a student researcher, to be able to carry out your sampling plan?
- What reservations or questions do you still need to have answered to adequately plan for your sample?
- What excites you about your proposal thus far?
- What worries you about your proposal thus far?