Thinking Critically about Research

A Step by Step Approach

Jane Ogden

• Publisher: Routledge
• Year: 2018
• Print ISBN: 9780367000196
• eBook ISBN: 9780429816673
• Edition: 1
• Topic: Medicine
• Subtopic: History & Theory in Psychology

We live in an age of unprecedented access to information. The last decade has seen an exponential growth in data and material available, often at the touch of a button. However, this has also made it harder to discern between fact and fiction. What is real and what is fake? What should we believe and what should we reject?

In an environment of information overload, a distrust of experts, the circulation of misinformation and false facts, and public debates based upon poor evidence, Thinking Critically About Research comes at a vital juncture. The book is designed to help readers develop a critical understanding of evidence and the ways in which evidence is presented, and to challenge the information they receive in both academic and non-academic sources. The author presents a step-by-step approach with a focus on knowing methods, culminating in a bespoke 'critical tool kit' which offers a practical checklist designed to be used when carrying out research.

Also containing learning features including tasks and worked examples, drawing on real research studies, this is an essential resource for students and researchers, and those putting research into practice, who want to have better critical thinking skills.

1
Why do we need to be critical?

OVERVIEW

Over the past 30 years, there has been a proliferation of information through an increase in published research, access to the internet, and the explosion of social media. We also gather our information from a vast array of sources including films, the radio, magazines, friends, and colleagues. This chapter will explore the problem of information overload and why we need to develop skills to think critically about research. It then describes what it is reasonable to expect from research and outlines the step-by-step approach for thinking critically about research used in this book.

INFORMATION OVERLOAD

The first academic journal called Philosophical Transactions of the Royal Society was published in 1665. Between then and 2010 it is estimated that about 50 million papers have been published (Jinha, 2010). Further, in 2006 Björk, Annikki, and Lauri estimated that 1.346 million articles were published in 23,750 journals and in 2009 it was estimated that there were about 25,400 peer-reviewed journals publishing about 1.5 million articles a year. These numbers are still increasing at about 2.5% per year (The National Science Board) although this may have increased even further within the past few years since the Open Access drive and the explosion of online journals. Currently, PubMed, the online data base for medically related research contains 19 million papers, whereas Scopus and the ISI Web of Knowledge which also include the humanities have about 40 million papers. There has also been a parallel growth in the number of researchers of about 3% per year with there now being about 5–10 million researchers depending on the definition used. Of these about 20% publish more than one paper (Jinha, 2010). That is a lot of research to read.

Our information, however, also comes from a range of other sources. In 2017, the ONS reported that 90% of households in Great Britain have internet access, which has increased from only 57% in 2006. Of these, 73% access the internet on their phone ‘on the go’ which is twice the rate of 36% in 2011. Across the world, the percentage of the population that uses the internet is as follows: Africa 35.2%; Asia 48.1%; Europe 85.2%; Latin America/Caribbean 67%; Middle East 64.5%; North America 95%; Oceania/Australia 68.9%. The world total is 54.5%. The ONS survey in Great Britain also showed that the largest rise between 2007 and 2014 was for using the internet to read or download newspapers or magazines (from 20% to 55%).

Social media is now also a common source of information. Twitter was set up in 2006 when the first tweet was sent by its creator Jack Dorsey. By 2007, there were on average 5,000 tweets per day and it took 3 years until 2009 to reach the billionth tweet. Now there are about 500 million tweets per day and about 200 billion per year, which works out as 350,000 per minute. Similarly, Facebook currently has 1,754,000,000 monthly active users, which has increased by 13.2% between 2014 and 2016. These users spend an average of 18 minutes per Facebook visit and 48% log in on any given day.

Task 1 Where do you get your information from?

Yet, we don’t just get information from research articles, the internet, or social media. Whenever we watch a film, turn on the TV, listen to the radio, read a newspaper or magazine, or speak to a friend, nuggets of information come our way. I have learned all about shark attacks from watching ‘Jaws’ and a lot about how pirates have guidelines not rules from ‘The Pirates of the Caribbean’. I also feel pretty qualified as an emergency doctor from watching ‘24 hours in A and E’ on the TV and ‘Call the Midwife’ has definitely taught me about medicine in London in the 1960s. Task 1 illustrates where I get my information. Have a look at Task 1 and think where you get yours.

THE PROBLEM

This process of information overload is fantastic in many ways as the days of standing in musty libraries, queuing at the photocopier, or starring at a micro-fiche are over (if you know what these are). It is so much easier to find information on anything that you want simply with a quick search and a press of the mouse. However, it also means that there is just too much information and with this comes the problem of deciding what to believe and what to dismiss. At times, this problem might simply be a problem of source. It is therefore easier to reject information that looks like Task 3 and appears in sources that feel less credible.

It may also be easier to accept information that looks like Task 2 as these are academic journals which appear far more convincing.

Task 2 The role of source: do these sources make information seem more credible?

Task 3 The role of source: do these sources make information look less credible?

But source is by no means the best way to sort information into the accept and reject piles as it is quite likely that the information in less credible sources (Task 3) contains elements of truth whereas the information in academic journals (Task 2) is flawed.

The skill is therefore to think critically about all information, not only taking into account the source but also the content of the information as well as how it is presented. For thinking critically about research, this process involves understanding what to expect from research. It then involves asking two questions that are the focus of this book ‘what evidence is there’? and ‘how is the evidence being presented’?

WHAT TO EXPECT FROM RESEARCH

From the outside, research can seem like a simple search for truth with each study offering an answer to a specific research question. Therefore, if we want to know ‘what is the best way to teach students’ we do a study, analyse the results, and know the answer. Likewise, if we want to find out the best way to treat cancer, we carry out a research study and treat patients in line with the findings. In reality, research is more complex and often far more disappointing than we would like it to be. This is due to the following reasons:

• Research is always flawed: As this book will show, all studies are flawed in some way whether it be due to their sample, design, measures, or analysis. All research findings need to be understood in the context of these flaws.
• The role of uncertainty: All research involves an element of risk, probability, and uncertainty. Consequently, the results from most studies are never true for all of the people all of the time, but only true for some of the people some of the time. This is manageable within the academic world when researchers can acknowledge this uncertainty and incorporate it into their next research study. It becomes more problematic when trying to communicate the findings to a lay audience who want a more definite answer than ‘it depends’. It is also a problem when translating research into practice whether it be to inform education, health care, or even the law. We might know that an intervention works better than usual practice, but we can never know whether it works for this one person standing in front of us.
• Real-life limitations: There are many real-life limitations which impact upon what research can be done. These may be financial limitations that can reduce the sample size, limit the accessibility to equipment, influence the choice of measurement tools (cheap self-report measures rather than expensive blood tests or scans), or reduce the number of collaborators and the geographical reach of a study. It might also be time limitations with researchers having busy jobs and only being able to do so much. These real-life factors can also limit the level of follow up for any study, as most funding bodies rarely fund studies with follow ups for more than one (sometimes two) years and researchers may not want to devote more than a few years on a single study (when a 30 year follow up would be ideal for the research question).
• Ethical considerations: Research is also limited by ethical considerations. The perfect design might be to randomly allocate half the sample to smoke 20 cigarettes per day for 30 years and then see who dies. This isn’t ethical. It might also be to pump out carbon dioxide into the atmosphere for 50 years and see if the climate changes. Again – not ethical.
• What is feasible: Feasibility also limits research. We might want people to eat a low fat diet for 40 years to see if this prevents heart disease. But people will not stick to it and it’s just not feasible to ask them to do this. We also might want half the world to run 5 km every weekend for 20 years to see if they develop stronger bones than the other half of the world, but again this isn’t going to happen. Feasibility is a huge block for the ideal research study.

We may have high expectations from research but in reality, all research is flawed and limited by cost, time, ethics, and what is feasible. It is therefore key to accept that research may not be perfect but it can be good enough. This doesn’t mean that research should be accepted without criticism. Thinking critically about research involves acknowledging that no research is perfect, then critically analysing why it isn’t perfect and what the implications of these imperfections are for any conclusions made. This book shows how this can be done.

THINKING CRITICALLY ABOUT RESEARCH: A STEP-BY-STEP GUIDE

This book offers a step-by-step guide to thinking critically about research and asks two questions ‘what evidence is there’? and ‘how is the evidence presented’? The steps are as follows:

Step 1: Knowing Methods

Before you can critically analyse research you need to know your methods. This first step offers a clear understanding of the basics of research including research questions, variables and the sample, describes the different types of research designs and their strengths and weaknesses, illustrates the different approaches to measurement, and covers qualitative and quantitative data analysis.

Step 2: Thinking Critically about what Evidence there is

Once you know your methods, the next step is to think critically about all the stages involved in research including sampling, design, measurement, data analysis, and theory. This can be done through being aware of a wide range of issues such as researcher and responder bias, reverse causality and the third factor problem, poor operationalisation of measures, and a recognition of the assumptions underlying data analysis and theory.

Step 3: Thinking Critically about how Evidence is Presented

Research is presented in research articles in respected journals and through a wide range of different media outlets. However, regardless of where it is presented, this process of dissemination always involves a number of persuasive strategies to convince the reader of the credibility of the work. This third step describes these different persuasive strategies and shows how they are apparent in all sorts of writing and impact upon what we believe to be true and what we reject.

Step 4: Using the Critical Tool Kit

This fourth step involves pulling together all the key ideas and terms from the previous three steps into one Critical Tool Kit that can be used to think critically about research whether presented as a research paper in an academic journal, downloaded from the internet, seen on social media, or read from the newspaper as you are eating your fish and chips. It also covers how to use critical thinking in everyday life in terms of dealing with facts, making logical decisions, and dealing with uncertainty.

Step 5: Being Extra Critical

The previous four steps are sufficient to become a good critical thinker as they cover the questions ‘what evidence is there’? and ‘how is it presented’? This final chapter explores how to take thinking critically about research to a higher level in terms of notions of truth, the assumptions inherent in any discipline, and thinking outside of the box.

In Summary

Together with all the films we watch, newspapers we read, and conversations we have, the proliferation of research papers, the development of the internet, and the use of social media has resulted in information overload. This problem creates the need to be able to think critically so we can decide what to believe and what to reject. In terms of research, thinking critically involves accepting that research can only be good enough as it is always flawed in some way and limited by cost, time, feasibility, or ethics. However, thinking critically also involves understanding why and how research is flawed and how this affects any conclusions that can be made. It is also a c...

Table of contents

• Cover
• Half Title
• Title
• Copyright
• CONTENTS
• Acknowledgements
• Overview
• About this book
• About the author
• CHAPTER 1 Why do we need to be critical?
• STEP 1 KNOWING METHODS
• STEP 2 THINKING CRITICALLY ABOUT METHOD: WHAT EVIDENCE IS THERE?
• STEP 3 THINKING CRITICALLY ABOUT PRESENTATION: HOW IS EVIDENCE PRESENTED?
• STEP 4 PULLING THIS ALL TOGETHER IN THE CRITICAL TOOL KIT
• STEP 5 BEING EXTRA CRITICAL
• Concluding remarks
• Further reading
• References
• Index