4.1 Evaluating sources

In the previous section, we discussed how to formulate search queries to get the most relevant results. At this point, you should not be staring at a Google Scholar window with 1,000,000 search results. If you haven’t played around in multiple databases and refined your queries using the suggestions in section 3.2, pause here and spend some time working on your search queries. Hopefully, you can find at least a few different queries that provide relevant resources to help you answer your working question or introduce new ideas that might revise or update your working question. Remember that your working question should be revised and updated as you learn more about your topic by searching in the literature.

Skim abstracts

All databases will give you access to an article’s abstract. The abstract is a summary of the main points of an article. It will provide the purpose of the article and summarize the author’s conclusions. Once you have a few good search queries, start skimming through abstracts and find the articles that are most relevant to your working question. Soon enough, you will find articles that are so relevant that you may decide to read the full text.

It is a good idea to cast a wide net at this point, since your project is just beginning. If you like the article, make sure to download the full text PDF to your computer so you can read it later. Save it in a new folder using a descriptive title. I like to include the first author’s last name, year, and the first few words of the title since the file names assigned by journals are often unclear.

What do you do with all of those PDFs? I usually keep mine in folders on my cloud storage drive, arranged by topic. For those who are more ambitious, you may want to use a reference manager like Mendeley or Zotero, which can help keep your sources and notes organized. Both them can also help you build a proper bibliography and organize your notes on each article. At the very least, write some notes on paper or in a word processor reflecting on the articles you skim and how they might be of use in your study.

Getting through paywalls to the article

When students try to download the full text of an article, they will often hit an obstacle: the paywall. A paywall is when a publisher charges you to access a publication. Academic journal articles are expensive, but good news! Part of the tuition and fees your university charges you goes toward paying major publishers of academic journals for the privilege of accessing their articles. You should be able to access the full text for most of the journal articles you need using your library’s website. Do not limit your literature search to the free articles or random PDFs you find on the internet!

Because journal publishers charge a lot of money for access to journals, your school likely does not pay for all the journals in the world. If your university does not already pay for access to the article, you still do not have to pay for it! You will need to use the interlibrary loan feature on your library’s website. It is often listed under Services. Just enter the information for your article (e.g. author, publication year, title, DOI), and a librarian will get you the PDF of the article that you need from a school that does pay for access to that article.

After you no longer have a university library login, getting a PDF of a journal article becomes more challenging. You can subscribe to journals yourself, and many practitioners do that. However, you can always ask an author for a copy of their article. They will usually send it to you. Some journal articles are available completely for free. If you followed the links in section 3.1 to examples of different types of articles, all of these pointed to articles that were free to access, as the authors of this text cannot be sure that all readers of this textbook will have access to paywalled articles. Open access journal articles do not have a paywall so are open to the public, but less than 13% of journal articles are open access (Chimmer, Geschuhn, & Vogler, 2015).^[1]

Another alternative way to access journal articles is if they are available in for-profit repositories like Social Science Research Network, Academia.edu, and ResearchGate. You will likely encounter results from these websites if you search for articles in a search engine like Google or Bing, as well as on Google Scholar. Sometimes, these websites require users to create an account in order to view content, and students should consider using a throwaway email to do so. ResearchGate has been sued by scholarly publishers for illegally hosting copyrighted and paywalled content (Mackenzie, 2018).^[2] By contrast, non-profit and community-owned repositories like SocArXiv and university repositories (like Virginia Commonwealth University’s Scholars Compass) allow authors to legally share a final draft of their work either prior to publication or after publication within the boundaries set out by publishers. This practice is called self-archiving or green open access—as contrasted with gold open access, in which the publisher’s version of the journal article is freely available rather than just the author’s draft. Self-archived papers look a lot like the papers you turn in for class. They are formatted according to the style guide predominant in the discipline and intended to be reviewed by experts as part of peer review.

We will address the social equity implications of paywalls and open access at the end of this chapter when we talk about information privilege. For now, we will pass along this advice adapted from the brilliant librarians at Virginia Tech on getting through paywalls. While you are in school, your library will likely pay for access to some of the articles you want. Search your library’s collection, and if it does not have access to it right away, request it via inter-library loan, and a copy will come within a few days thanks to some dedicated librarians at your school. If you need the article right away or are not affiliated with a university library, the steps below might be helpful in finding your article. 

1. Check Google Scholar for alternative versions of the article you are trying to access. On the bottom right of each source in Google Scholar results is a link that says “all # versions.” If the author has uploaded an open access version of their work, you may need to click this link to get to a free copy rather than the paywalled copy at the journal.
2. Conduct a search for the article in a normal Google search (possibly adding “.pdf” to the query) to see if you can find a copy of the article without a paywall. Often times, you will find links to a paywalled journal article on a for-profit repository like Social Science Research Network, Acacdemia.edu, or ResearchGate. These are okay to download from, but make sure to keep a keen eye on suspicious websites that promise access to journal articles but actually try to infect your computer with malware. If your browser gives you a warning when you go to a page or when you try to download a PDF, don’t risk it. Find another way to get the article.
3. Go to unpaywall.org and install the Unpaywall extension for Firefox and Chrome. Once installed, a green icon of an open lock will appear on any scholarly resource for which there is a publicly accessible version without a paywall.
4. Go to openaccessbutton.org and either install the Chrome extension or use the web portal to request that the author of an article send you a free copy. The purpose of this project is to make seeking free copies of scholarship as easy as possible, though it still relies on the author to send their article when you request it using this service.
5. Go to oahelper.org and install their desktop or mobile app to search for unpaywalled and open access versions of publications.

Everyone can access the abstracts of research articles for free, as long as they have an internet connection. Abstracts are a great place to start, but you need to access the full text of articles for your student projects. I’ve often seen student papers where they do a simple Google Scholar search and only read articles for which there is free access, with the PDF link in the search results. This is a mistake because few journal articles are free to access (Chimmer, Geschuhn, & Vogler, 2015).^[3] Use your library to gain access to the full text of each journal article you need for your search. While you are studying at your university, your login is the key to unlock the paywall and grant you access to the information you need.

Which articles do I download?

Keeping your working question in mind, you should look at your potential sources and evaluate whether they are relevant to your inquiry. To assess the relevance of a source, ask yourself if the source will help you answer or think more deeply about your working question. Does the information help you answer this question, challenge your assumptions, or connect your question with another topic? Does the information present an opposing point of view, so you can show that you have addressed all sides of the argument in your paper? Does the article just provide a broad overview of your topic or does it have a specific focus on what you want to study? If the article isn’t helpful to you, it’s okay not to read it. No matter how good your searching skills, some articles won’t be relevant. You don’t need to read and include everything you find!

You may also want to check the relevance of the source with your professor or course syllabus. In my class, I have specific questions I will ask students to address in their literature reviews. You may want to find sources that help you answer the questions in your professor’s prompt for a literature review. For example, my prompt asks “how many people experience this problem or issue?” My students then seek out a source that helps them understand the prevalence or incidence of an issue.

In addition to relevance, you should ask yourself about the quality of the source you’ve found. Is the information outdated? Is the source more than 5-10 years old? If so, it will not provide what we currently know about the topic–just what we used to know. Older sources are helpful for historical information, such as how our understanding of a topic has changed over time or how the prevalence of an issue has increase or decreased. However, unless historical analysis is the focus of your literature review, try to limit your sources to those that are current. Doing so will also narrow down your list of results considerably in a database. Also, you may want to consult with your professor to see if they have additional guidelines on which articles are “too old” to include in a literature review.

Older sources may be of some importance, however, if they are seminal articles. As we learned in section 3.1, seminal articles are cited often in the literature. They are clearly important to a lot of scholars in the field. While not all articles are seminal, you can get a quick sense of how important an article is to the broader literature on a topic by looking at how many other sources cited it. If you search for the article on Google Scholar (see Figure 3.1 for an example of a search result from Google Scholar), you can see how many other sources cited this information. Generally, the higher the number of citations, the more important the article. Of course, articles that were recently published will likely have fewer citations than older articles, and the citation count is only one indicator of an article’s importance. 

What evidence is in each type of article?

Literature searching is about finding evidence to inform your ideas, supporting or refuting what you think about a topic. Each type of article (we reviewed some of them in section 3.1) provides different kinds of evidence. As we talked about in section 3.2, a good place to start your literature search is with review articles—meta-analyses, meta-syntheses, and systematic reviews. These types of articles give you a birds-eye view of the literature in a topic area, and with meta-analyses and meta-syntheses, conduct empirical analyses on enormous datasets comprised of the raw data from multiple studies. As a result, their conclusions represent what is broadly true about the topic area. They also have comprehensive reference lists that you can browse for sources relevant to your topic.

In my experience, students are often tempted to read short articles because they can complete assignments more quickly that way. This is a trap for students starting a literature review. Short articles take less time to read, sure. But this isn’t about reading a certain number of articles, but finding the information you need to write your literature review as efficiently as possible. You’ll save time by reading more relevant articles with lengthier and more comprehensive literature reviews—in particular, review articles.

Review articles are the best place to start for any literature search, but you should also look for specific types of articles based on your working question. If your working question asks about an intervention, like a therapeutic technique or program, randomized controlled trials (RCTs) provide the strongest evidence if a meta-analysis or systematic review of relevant RCTs is unavailable. Quasi-experimental designs are considered inferior because researchers have less control over the research process. You will want to avoid relying heavily on articles that use non-experimental designs and include words like “pilot study,” “convenience sample,” or “exploratory study” in their methods section. These are preliminary studies that are done prior to a more rigorous experiment like an RCT, and their conclusions are tentative and collected for the purpose of informing future inquiry, not establishing what is true for broader populations. We will discuss experimental design in Chapter 13, but for now, it’s important to know that the purpose is often to establish the efficacy of an intervention and the truth value of the evidence contained in them varies based on the design of the experiment, with RCTs being the gold standard.

Experiments are one of two quantitative designs explored in this book. The other design is survey research. Looking at survey research is a good idea in any project, as it provides evidence about broader populations by generalizing from a smaller sample of people. For example, surveys can tell you about the risk and protective factors for a social problem by querying the people who are likely to experience that problem over time. Longitudinal surveys are often the most helpful in understanding causality because you have a record of how things have changed over time. Cross-sectional surveys are more limited in establishing causal relationships, as they only query people at one point in time. We discuss the differences between these types of surveys in Chapter 12. Both cross-sectional and longitudinal surveys are very commonly cited types of sources in social work literature reviews because their results are often applicable across broad populations. However, they are limited in the degree to which they can establish causality, as they lack the controlled environment of an experiment. As with experiments, students should be very cautious about using survey results that are “exploratory” or a “pilot study,” as the purpose of those studies is to inform future research rather than understand what is true about broader populations.

The previous few paragraphs can be summarized in this hierarchy of evidence, as described by McNeese & Thyer (2004).^[4] The higher a type of article on this list, the better they can reliably and directly inform evidence-based practice in social work.

1. Systematic reviews and meta-analyses
2. Randomized controlled trials
3. Quasi-experimental studies
4. Case-control and cohort studies
5. Pre-experimental (or non-experimental) group studies
6. Surveys
7. Qualitative studies

The hierarchy of evidence is a useful heuristic, and it is based on sound reasoning. A systematic review or meta-analysis will provide you with a better picture of what is generally true for most people about a given topic than a qualitative study. However, generalizable objective truth is not the only thing researchers want to know. For example, if you wanted to research the impact of gentrification on a community, systematic reviews would not provide you the depth you need to understand the stories of people impacted, displaced, and discriminated against in housing policy. That’s not what systematic reviews are designed to do.

To find evidence like that, the most relevant evidence is in qualitative studies, at the bottom of the hierarchy. For this reason, it is better to think of the hierarchy of evidence within the broader context of your working question and the knowledge you need to investigate it. Different types of sources are useful depending on what your question is. Table 4.1 contains a suggested starting point for evaluating what types of literature will provide the most relevant evidence for your project.

Qualitative studies are not designed to provide information about a broader population. As a result, you should treat their results as related to the specific time and place in which they occurred. If a study’s context is similar to the one you plan to research, then you might expect similar results to emerge in your research project. Qualitative studies provide the lived experience and personal reflections of people knowledgeable about your topic. These subjective truths provide evidence that is just as important as other studies in the hierarchy of evidence.

4.2 Organizing information

After working through the exercises Chapter 3, you should have a folder full of articles. For me, searching the literature is the fun part of a literature review. I have grand ideas about reading this article and that article and every article! Once I’m satisfied I have enough literature to speak intelligently on my topic, a creeping sense of dread kicks in. How will I ever find the time to read 60 journal articles?! This chapter is all about how to pick the most relevant articles, write notes about them, and incorporate the information relevant to your working question in your literature review.

Read literature reviews

At the beginning of a research project, you don’t know a lot about your research topic. You don’t even know what you don’t know! That’s why it’s a good idea to get a broad, birds-eye view of the literature by looking at other literature reviews. Someone has gone through the trouble of reading a few dozen sources and telling you what’s important about them. Get a broad sense of the literature and follow up on subtopics that interest you. As we discussed in Chapter 3, review articles are useful because they synthesize all the information on a given topic. By reading what another researcher thinks about the literature, you can get a more wide-ranging sense of it than reading the results of only one study.

As we discussed in the last chapter, review articles will often have “literature review,” “systematic review,” or other similar terms in the title. These articles are 100% literature review. The author’s primary goal is to present a comprehensive and authoritative view of the important research in a particular topic area. Think of these as a way to engage with dozens of articles at the same time, and you can find a lot of relevant references from reading the article. For the same reason, meta-analyses and meta-syntheses are also excellent sources as you are starting a literature review.

Unfortunately, review articles do not exist for every topic. If you are unable to find a review article, try to find an empirical article that has a lengthy literature review. You are mostly reading to see what the author says about the literature on your topic in the introduction and discussion sections. In this way, you can use the author’s literature search to inform your own. You will likely find new ideas and get a sense of the broader scientific literature just by reading what other researchers think about the literature.

This section will introduce you to three tools scholars use to organize and synthesize (i.e., weave together) information from multiple sources. First, we will discuss how to build a summary table containing information from empirical articles that are highly relevant—from literature review, to methods and results—to your entire research proposal. These are articles you will need to know the details of back-to-front because they are so highly related to your proposed study.

Second, we’ll discuss what to do with the other articles you’ve downloaded. As we’ve discussed previously, you’re not going to read most of the sources you download from start-to-finish. Instead, you’ll look at the author’s literature review, key ideas, and skim for any relevant passages for your project. As you do so, you should create a topical outline that organizes all relevant facts you might use in your literature that you’ve collected from the abstract, literature review, and conclusion of the articles you’ve found. Of course, it is important to note the original source of the information you are citing.

Finally, we will revisit concept mapping as a technique for visualizing the concepts in your study. Altogether, these techniques should help you create intermediary products—documents you are not likely to show to anyone or turn in for a grade—but that are vital steps to a final research proposal.

Organizing empirical articles using a summary table

Your research proposal is an empirical project. You will collect raw data and analyze it to answer your question. Over the next few weeks, identify about 10 articles that are empirically similar to the study you want to conduct. If you plan on conducting surveys of practitioners, it’s a good idea for you to read in detail other articles that have used similar methods (sampling, measures, data analysis) and asked similar questions to your proposal. A summary table can help you organize these Top 10 articles: empirical articles that are highly relevant to your proposal and working question.

Using the annotations in Section 4.2 as a guide, create a spreadsheet or Word table with your annotation categories as columns and each source as new row. For example, I was searching for articles on using a specific educational technique in the literature. I wanted to know whether other researchers found positive results, how big their samples were, and whether they were conducted at a single school or across multiple schools. I looked through each empirical article on the topic and filled in a summary table. At the end, I could do an easy visual analysis and state that most studies revealed no significant results and that there were few multi-site studies. These arguments were then included in my literature review. These tables are similar to those you will find in a systematic review article.

A basic summary table is provided in Figure 5.1. A more detailed example is available from Elaine Gregersen’s blog, and you can download an Excel template from Raul Pacheco-Vega’s blog. Remember, although “the means of summarizing can vary, the key at this point is to make sure you understand what you’ve found and how it relates to your topic and research question” (Bennard et al., 2014, para. 10).^[5] As you revisit and revise your working question over the next few weeks, think about those sources that are so relevant you need to understand every detail about them.

A good summary table will also ensure that when you cite these articles in your literature review, you are able to provide the necessary detail and context for readers to properly understand the results. For example, one of the common errors I see in student literature reviews is using a small, exploratory study to represent the truth about a larger population. You will also notice important differences in how variables are measured or how people are sampled, for instance, and these details are often the source of a good critical review of the literature.

Synthesizing facts using a topical outline

If we’re only reading 10 articles in detail, what do we do with the others? Raul Pacheco-Vega recommends using the AIC approach: read the abstract, introduction, and conclusion (and the discussion section, in empirical articles). For non-empirical articles, it’s a little less clear but the first few pages and last few pages of an article usually contain the author’s reading of the relevant literature and their principal conclusions. You may also want to skim the first and last sentence of each paragraph. Only read paragraphs in which you are likely to find information relevant to your working question. Skimming like this gives you the general point of the article, though you should read in detail the most valuable resource of all—another author’s literature review.

It’s impossible to read all of the literature about your topic. You will read about 10 articles in detail. For a few dozen more (there is no magic number), you will read the abstract, introduction, and conclusion, skim the rest of the article, but ultimately never read everything. Make the most out of the articles you do read by extracting as many facts as possible from each. You are starting your research project without a lot of knowledge of the topic you want to study, and by using the literature reviews provided in academic journal articles, you can gain a lot of knowledge about a topic in a short period of time. This way, by reading only a small number of articles, you are also reading their citations and synthesis of dozens of other articles as well.

As you read an article in detail, we suggest copying any facts you find relevant in a separate word processing document. Another idea is to copy anything you’ve annotated as background information in Section 4.2 into an outline. Copying and pasting from PDF to Word can be difficult because PDFs are image files, not documents. To make that easier, use the HTML version of the article, convert the PDF to Word in Adobe Acrobat or another PDF reader, or use the “paste special” command to paste the content into Word without formatting. If it’s an old PDF, you may have to simply type out the information you need. It can be a messy job, but having all of your facts in one place is very helpful when drafting your literature review.

You should copy and paste any fact or argument you consider important. Some good examples include definitions of concepts, statistics about the size of the social problem, and empirical evidence about the key variables in the research question, among countless others. It’s a good idea to consult with your professor and the course syllabus to understand what they are looking for when reading your literature review. Facts for your literature review are principally found in the introduction, results, and discussion section of an empirical article or at any point in a non-empirical article. Copy and paste into your notes anything you may want to use in your literature review.

Importantly, you must make sure you note the original source of each bit of information you copy. Nothing is worse than needing to track down a source for fact you read who-knows-where. If you found a statistic that the author used in the introduction, it almost certainly came from another source that the author cited in a footnote or internal citation. You will want to check the original source to make sure the author represented the information correctly. Moreover, you may want to read the original study to learn more about your topic and discover other sources relevant to your inquiry.

Assuming you have pulled all of the facts out of multiple articles, it’s time to start thinking about how these pieces of information relate to each other. Start grouping each fact into categories and subcategories as shown in Table 5.2. For example, a statistic stating that single adults who are homeless are more likely to be male may fit into a category of gender and homelessness. For each topic or subtopic you identify during your critical analysis of each paper, determine what those papers have in common. Likewise, determine which differ. If there are contradictory findings, you may be able to identify methodological or theoretical differences that could account for these contradictions. For example, one study may sample only high-income earners or those living in a rural area. Determine what general conclusions you can report about the topic or subtopic, based on all of the information you’ve found.

Create a separate document containing a topical outline that combines your facts from each source and organizes them by topic or category. As you include more facts and more sources in your topical outline, you will begin to see how each fact fits into a category and how categories are related to one another. Keep in mind that your category names may change over time, as may their definitions. This is a natural reflection of the learning you are doing.

A complete topical outline is a long list of facts arranged by category. As you step back from the outline, you should assess which topic areas for which you have enough research support to allow you to draw strong conclusions. You should also assess which areas you need to do more research in before you can write a robust literature review. The topical outline should serve as a transitional document between the notes you write on each source and the literature review you submit to your professor. It is important to note that they contain plagiarized information that is copied and pasted directly from the primary sources. In this case, it is not problematic because these are just notes and are not meant to be turned in as your own ideas. For your final literature review, you must paraphrase these sources to avoid plagiarism. More importantly, you should keep your voice and ideas front-and-center in what you write as this is your analysis of the literature. Make strong claims and support them thoroughly using facts you found in the literature. We will pick up the task of writing your literature review in section 5.3.

Putting the pieces together: Building a concept map

Developing a concept map or mind map around your topic can be helpful in figuring out how the facts fit together. We talked about concept mapping briefly in Chapter 2, when we were first thinking about your topic and sketching out what you already know about it. Concept mapping during the literature review stage of a research project builds on this foundation of knowledge and aims to improve the “description of the breadth and depth of literature in a domain of inquiry. It also facilitates identification of the number and nature of studies underpinning mapped relationships among concepts, thus laying the groundwork for systematic research reviews and meta-analyses” (Lesley, Floyd, & Oermann, 2002, p. 229).^[6] Its purpose, like other question refinement methods, is to help you organize, prioritize, and integrate material into a workable research area—one that is interesting, answerable, feasible, objective, scholarly, original, and clear.

Think about the topics you created in your topic outline. How do they relate to one another? Within each topic, how do facts relate to one another? As you write down what you have, think about what you already know. What other related concepts do you not yet have information about? What relationships do you need to investigate further? Building a conceptual map should help you understand what you already know, what you need to learn next, and how you can organize a literature review.

This technique is illustrated in this YouTube video about concept mapping. You may want to indicate which concepts and relationships you’ve already found in your review and which ones you think might be true but haven’t found evidence of yet. Once you get a sense of how your concepts are related and which relationships are important to you, it’s time to revise your working question.

Revising your working question

You should be revisiting your working question throughout the literature review process. As you continue to learn more about your topic, your question will become more specific and clearly worded. This is normal, and there is no way to shorten this process. Keep revising your question in order to ensure it will contribute something new to the literature on your topic, is relevant to your target population, and is feasible for you to conduct as a student project.

For example, perhaps your initial idea or interest is how to prevent obesity. After an initial search of the relevant literature, you realize the topic of obesity is too broad to adequately cover in the time you have to do your project. You decide to narrow your focus to causes of childhood obesity. After reading some articles on childhood obesity, you further narrow your search to the influence of family risk factors on overweight children. A potential research question might then be, “What maternal factors are associated with toddler obesity in the United States?” You would then need to return to the literature to find more specific studies related to the variables in this question (e.g. maternal factors, toddler, obesity, toddler obesity).

Similarly, after an initial literature search for a broad topic such as school performance or grades, examples of a narrow research question might be:

• “To what extent does parental involvement in children’s education relate to school performance over the course of the early grades?”
• “Do parental involvement levels differ by family social, demographic, and contextual characteristics?”
• “What forms of parent involvement are most highly correlated with children’s outcomes? What factors might influence the extent of parental involvement?” (Early Childhood Longitudinal Program, 2011).^[7]

In either case, your literature search, working question, and understanding of the topic are constantly changing as your knowledge of the topic deepens. A literature review is an iterative process, one that stops, starts, and loops back on itself multiple times before completion. As research is a practice behavior of social workers, you should apply the same type of critical reflection to your inquiry as you would to your clinical or macro practice.

4.3 Critical information literacy

Ultimately, searching and reviewing the literature will be one of the most transferable skills from your research methods classes. All social workers have to consume research information as part of their jobs. In this closing section, we hope to ground this orientation toward scientific literature in your identity as a social worker, scholar, and social scientist. In one sense, to be a critical consumer of research is to be discerning about which sources of information you read and why. If you have followed along with the exercises in this chapter, you hopefully have about 30, 50, or even 100 PDFs of articles that are relevant to your project in a folder on your computer. You do not have to read every article or source that discusses your topic.

At some point, reading another article won’t add anything new to your literature review. We’ve discussed in this chapter how to read sources strategically, getting the most out of each one without wasting your time. Ultimately, the number of sources you need should be guided, at a minimum, by your professor’s expectations and what you need to provide a comprehensive review of the literature on your topic. The purpose of the literature review is to give an overview of the research relevant to your project—everything a reader would need to understand the importance, purpose, and thought behind your project—not for you to restate everything that has ever been studied about your topic.

Research is a human right

But when we talk about being critical in social work, we mean more than critical thinking. The core skill you are developing in Chapters 3-5 of this textbook is information literacy, or “a set of abilities requiring individuals to ‘recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information” (American Library Association, 2020).^[8] Information literacy is key to the lifelong learning process of social work practice, as you will have to continue to absorb new information after you leave school and progress as a social work practitioner. In the evidence-based practice process, information literacy is a key component in helping clients and communities achieve their goals.

However, social work researchers and librarians dedicated to social change embrace a more progressive conceptualization of this skill called critical information literacy. It is not enough to simply know how to use the tools we use to share knowledge. Instead, social work researchers should critically examine “the social, political, economic, and corporate systems that have power and influence over information production, dissemination, access, and consumption” (Gregory & Higgins, 2013, p. ix).^[9] Just like all other social structures, those we use to create and share scientific knowledge are subject to the same oppressive forces we encounter in everyday life—racism, sexism, classism, and so forth. Critical information literacy combines both fine-tuned skills for finding and evaluating literature as well as an understanding of the cultural forces of oppression and domination that structure the availability of information. As Tewell (2016)^[10] argues, “critical information literacy is an attempt to render visible the complex workings of information so that we may identify and act upon the power structures that shape our lives” (para. 6).

From critical information literacy, we get the term information privilege which is defined by Booth (2017)^[11] as:

the accumulation of special rights and advantages not available to others in the area of information access. Individuals with the resources to access the information they need, are affiliated with research or academic institutions and libraries, or live close to a public library with access to resources and services such as free interlibrary loan are examples of those with information privilege. Those who are unable to access the information they need are information underprivileged or impoverished [emphasis in original]; this includes people who are incarcerated, poor, unaffiliated with a university or research institution, or live in rural areas distant from a public library.

How can you recognize information privilege? The video from the Steely Library at Northern Kentucky University below will help you.

It is important to recognize your own information privilege and use it to help enact social change on behalf of those who are oppressed within the system of scholarly knowledge production and sharing we currently have. We previously discussed the issue of paywalls in this chapter. Paywalls lock away access to knowledge to only those who can afford to pay, shutting out those with fewer resources. This effect of publishing only in journals that require a paywall is endemic to social work scholarship, as Pendell (2020, 2018)^[12] describes, there are few social work open access journals and social work researchers do not post open access copies of their articles even when explicitly allowed by publishers to do so. The reasons for this are not financial, as authors do not profit from the sales of journal articles. Indeed, faculty who edit, review, and author articles are not paid for their work under the current model of commercial journal publication. Scholars do not have a meaningful share in the 31.3% profit margins of Elsevier, the largest commercial publisher of scholarly journals (RELX, 2019).^[13] Libraries are struggling to keep up with annual price increases. This topsy-turvy world is best encapsulated in Figure 4.1, which summarizes the problems in scholarly publication succinctly.

As we discussed at the beginning of this chapter, paywalls effectively shut out social work students from accessing journal articles after they graduate.^[14] However, paywalls also shut out community members, clients, and self-advocates from accessing the research information they need to effectively advocate for themselves. This is because paywalls are an obstruction to the basic human right to access and interact with the knowledge humanity creates.

As Willinsky (2006) states, “access to knowledge is a human right that is closely associated with the ability to defend, as well as to advocate for, other rights.” The open access movement is a human rights movement that seeks to secure the universal right to freely access information in order to produce social change. Information is a resource, and the current approach to sharing that resource—the key to human development—excludes many oppressed groups from accessing what they need to address matters of social justice. Appadurai (2006)^[15] conceptualizes this as a “right to research…the right to the tools through which any citizen can systematically increase that stock of knowledge which they consider most vital to their survival as human beings and to their claims as citizens” (p. 168). From a human rights perspective, research is not something confined to professional researchers but “the capacity to systematically increase the horizons of one’s current knowledge, in relation to some task, goal or aspiration” (p. 173).

In Chapter 2, we discussed action research which includes community members as part of the research team. Action research addresses information privilege by addressing the power imbalance between academic researchers and community members, ensuring that the voice of the community is represented throughout the process of creating scientific knowledge. This equity-enhancing process provides community members with access to scientific knowledge in many ways: (1) access via academic to scholarly databases, (2) access to the training and human capital of the researcher, who embodies years of education in how to conduct social science research, and (3) access to the specialized resources like statistical or qualitative data analysis software required to conduct scientific research. Moreover, it highlights that while open access is important, it is only one half of the equation. Open access only addresses how people receive scholarly information. But what about how social science knowledge is created? Action research underscores that equity is not just about accessing scientific knowledge, but co-creating scientific knowledge as partners with community members. Critical information literacy critiques the current practices of scientific inquiry in social work and other disciplines as exclusionary, as they reinforce existing sources of privilege and oppression.

This imbalance between academic researchers and community members should be considered within an English-speaking, Western context. The widespread adoption of open access and open science in the developing world underscores the extreme imbalances that international researchers face in accessing scholarly information (Arunachalam, 2017).^[16] Paywalls, barriers to international travel, and the tradition of in-person physical conferences structure the information sharing practices in social science in a manner that excludes participation from researchers who lack the financial and practical resources to access these exclusionary spaces (Bali & Caines, 2016; Eaves, 2020).^[17]

In the United States, the federal government has increasingly adopted policies that facilitate open access, including the National Institute of Health’s requirement that all publications from projects it funds must be open access within a year after publication in a commercial journal as well as President Obama’s open data initiative. At the time of this writing, the Scholarly Publishing and Academic Resources Coalition (SPARC) is working with the Trump administration on a formal proposal to make all research articles published using federal grant funds accessible to the public as open access, though at the time of this writing there is no final agreement on the policy design or implementation. In Europe, Plan S is a multi-stakeholder social change coalition that is seeking structural change to scholarly publication and open access to research. Latin America and South America are far ahead of their Western counterparts in adopting open access, providing a view of what is possible when community and equity are centered in scholarly publication (Aguado-Lopez & Becerril-Garcia, 2019).^[18]

The slow progress of social movement organizations to ensure free access to scientific knowledge has inspired radical action. Out of growing frustration with journal paywalls, Sci-Hub was created by a computer programmer from Kazakhstan, Alexandra Elbakyan. This resource is an illegal, pirate repository of about 85% of all paywalled journal articles in existence up to 2020, when it stopped uploading papers in compliance with an injunction from an Indian court (Himmelstein et al., 2018).^[19] Sci-Hub is not legal under international copyright law,^[20] yet researchers unaffiliated with a well-funded university in the developed world cannot conduct research without it. The site has moved multiple times as their domains are shut down commercial publishers whose content is illegally stored by Sci-Hub, and its founder hopes that by successfully arguing her case before and Indian court, she can continue to operate Sci-Hub legally under Indian law, and in effect, guarantee the human right to access research for free within India (Cummins, 2021).^[21] Sci-Hub is a symptom of a broken system of scholarly publication that locks away scientific knowledge behind impossibly high textbook and journal paywalls. These are impossible for nearly all people to pay for in the course of a normal literature review, and thus, our broken system of scholarly publication necessitates that researchers use shadow libraries like Sci-Hub, Z-Library, and Library Genesis. It should not require mass-scale copyright infringement for the world to access scientific information.

The current model of scientific publishing privileges those already advantaged and raises important obstacles for those in oppressed groups to access and contribute to scientific knowledge or research-informed social change initiatives. For a more thorough, Black feminist perspective on scholarly publication see Eaves (2020).^[22] As social workers, it is your responsibility to use your information privilege—the access you have right now to the world’s knowledge and the skills you gain during your graduate education and post-graduate professional development—to fight for social change.

Being a responsible consumer of research

Social work often involves accessing, creating, and otherwise engaging with social science knowledge to create change. On the micro-level, critical information literacy is needed to inform evidence-based decision-making in clinical practice. On the meso- and macro-level, social workers can use information literacy skills to give a voice for community concerns, help communities access and create the knowledge they need to foster change, or evaluate how well existing programs serving the community achieve their goals. Now that you are familiar with how to conduct ethical and responsible research and how to read the results of others’ research, you have an obligation to use your information literacy skills to create social change. This is part of critical information literacy: using your information privilege to address social injustices.

Collecting, sharing, and creating research findings for social change requires you to take seriously your identity as a social scientist. Doing so is in part a matter of being able to distinguish what you do know based on research findings from what you do not know. It is also a matter of having some awareness about what you can and cannot reasonably know.

When assessing social scientific findings, think about the information provided to you. Social scientists should be transparent with how they collected their data, ran their analyses, and formed their conclusions. These methodological details can help you evaluate the researcher’s claims. If, on the other hand, you come across some discussion of social scientific research in a popular magazine or newspaper, chances are that you will not find the same level of detailed information that you would find in a scholarly journal article. With secondary sources like news articles, it is hard to know enough about the study to be an informed consumer of information. Always read the primary source when possible.

Additionally, take into account whatever information is provided about a study’s funding source. Most funders want, and in fact require, that recipients acknowledge them in publications. Keep in mind that some sources may not disclose who funds them. If this information is not provided in the source from which you learned about a study, it might behoove you to do a quick search on the internet to see if you can learn more about the funding. Findings that seem to support a particular political agenda, for example, might have more or less influence on your thinking about your topic once you know if the funding posed a potential conflict of interest.

There is some information that even the most responsible consumer of research cannot know. Because researchers are ethically bound to keep the identity of people in our study confidential, for example, we will never know exactly who participated in a given study or sometimes even in what location it was conducted. Awareness that you may never know everything about a study should provide some humility in terms of what we can “take away” from a given report of findings.

Unfortunately, researchers also sometimes engage in unethical behavior and do not disclose things that they should. For example, a recent study on COVID-19 (Bendavid et al., 2020)^[23] did not disclose that it was funded by the chief executive of JetBlue, an airline company losing money from COVID-19 travel restrictions. It is alleged that this study was paid for by airline executives in order to provide scientific support for increasing the use of air travel and lifting travel restrictions. These conflicts of interest demonstrate that science is not distinct from other areas of social life that can be abused by those with more resources in order to benefit themselves.

COVID-19 is a particularly instructive case in spotting bad science. In a rapidly evolving information context, social workers and others were forced to make decisions quickly using imperfect information. The Lancet and the New England Journal of Medicine, two of the most prestigious journals in medicine, had to retract two studies due to irregularities missed by pre-publication peer review despite the fact that the results had already been used to inform policy changes (Joseph, 2020).^[24] At the same time, President Trump’s lies and misinformation about COVID-19 were a grave reminder of how information can be corrupted by those with power in the social and political arena (Paz, 2020).^[25]

Figure 4.2 presents a rough guide to spotting bad science that is particularly useful when science has not had enough time for peer review and scholarly debate to thoroughly and systematically investigate a topic, as in the COVID-19 crisis. It is also a useful quick-reference guide for social work practitioners who encounter new information about their topics from the news media, social media, or other informal sources of information like friends, family, and colleagues.

Feminist perspectives on science generally support the framework in Figure 4.2, but consider them to be incomplete, as they do not critique the scientific process itself. By drawing a line between science vs. pseudoscience, scientific inquiry can further social justice by using data to battle misinformation and uncover social inequities. However, feminist perspectives also draw our attention to historically overlooked aspects of the scientific process. D’ignazio and Klein (2020)^[26] instead offer seven intersectional feminist principles for equitable and actionable COVID-19 data, as visualized in Figure 4.3.

Information literacy beyond scholarly literature

The research process we described in this chapter will help you arrive at an understanding of the scientific literature. However, that is not the only literature of value for researchers. For those conducting action research that engages more with communities and target populations, researchers are responsible not only for reviewing the scientific literature on their topic but also the literature that communities find important. If there are local newspapers, television shows, religious services or events, community meetings, or other sources of knowledge that your target population finds important, you should engage with those resources as well. Understanding these information sources builds empathetic understanding of participating groups and can help inform your research study. Moreover, they are likely to contain knowledge that is not a part of the scientific literature but is nevertheless crucial for conducting scientific research appropriately and effectively in a community.