If you're studying education much of what you do is informed by educational research but how do you interpret it effectively? How do you judge whether results are valid or relevant?
This is the perfect guide to engaging with educational research. It explores how to read journal articles critically; what key academic terms really mean; different approaches to educational research, how they are used and what they aim to uncover, and how high quality findings can be meaningful for teaching and learning.
Supported by examples that demonstrate the use (and misuse) of research in education, this is youressential guide to understanding and effectively using research.
Where To Find Good Research and How To Reference It Properly
Organising Your Research Reading and Avoiding Bias
1 The Hierarchy of Research Publications
Chapter aims
Understand how research is reported
Look at the variety of publications research may appear within
Assess the rigour of publications in presenting research/information
Comprehend what is meant by impact with respect to journals and articles
Understand the basics of the peer-review system
Introduction
We are bombarded by information to an unprecedented degree. The internet and social media have fundamentally changed how we view information. Research is, at its most basic, information. How research is communicated, which information channels are used and how the message about research evolves or alters each time it is reported will affect how we view research.
Research does not have the privilege of being seen by everyone in its original form. It would be unrealistic to say that all research should be read in its original form. Summaries of research, or the reporting of findings, can be substantially altered according to the mode of communication and who is communicating it. Where original research is published can also impact the level of confidence we should place on the work and its findings. In this chapter, we will examine different levels of publications which form a hierarchy from academic publishing down to popular media such as newspapers and individual blogs online.
Where is research published?
The key question to ask ourselves here is how reliable is the report of the research? This is the point at which we enter the murky, confusing and complex world of academic articles, peer review, impact, professional and everyday writing and publishing.
Before the invention of the internet and the birth of the blog, getting your ideas into print required either an ability to satisfy a publisher that your writing was worth publishing or the ability to self-publish (a process sometimes called âvanityâ publishing). Blogs and websites can be created by anyone and allow almost any opinion to be widely available, with little to no checks, balances or review. The internet provides a platform for a cheap and easy form of vanity publishing.
Broadly, distinguishing between professionally produced online material that is, for want of a better term, âquality assuredâ and the writings of a lone âexpertâ who may not even have any training in the field that he or she writes about is not difficult. The quality of the writing, spelling, grammar (or lack thereof) and poorly constructed arguments are signs that something has not undergone any form of professional editing process. That said, well-written material could be superficially persuasive, but may not be rigorous or stand up to scrutiny.
Tables 1.1â1.3 list a variety of forms of published output that may contain research or the outcomes of research. Each form has its advantages and disadvantages. For busy professional teachers, directly accessing first-hand research can be time-consuming and potentially difficult as many education research journals are not open access, i.e. the articles are not freely available. The Chartered College of Teaching (CCoT) has made access to first-hand research easier for its members by negotiating their access to a wide variety of education research journals, which should improve the situation for teachers interested in looking at research.
One important yet contentious measure of the value of different forms of publishing is the âimpact factorâ that a journal can have. Impact factors are nearly always applied to academic and professional journals (see Box 1.1). We must also differentiate between the âimpact factorâ a journal may have and the âimpactâ a piece of research may have. The impact that research may have could affect government policy, the practice of teachers or how educational institutions are run. This happens independently from the journal within which research is published.
Table 1.1
Table 1.2
Table 1.3
Box 1.1 Impact factor
Many journals advertise their impact factor and use this to attract researchers wishing to publish their research. Impact factors can be calculated in a variety of ways but generally it is a measure of the number of citations articles receive, related to the actual number of articles a journal prints.
The formula for calculating an impact factor is relatively simple:
IF = Impact Factor; y = year
This formula will work out the average number of citations a published paper receives in a given period â in this case, two years. The higher the number of citations per paper, the higher the impact factor. Journals will also use a five-year period for calculating impact factors, which due to the increased time is more accurate, but impact factors can be distorted by new journals where there is less than two years of data/publishing or where the frequency of publishing is very low, e.g. one journal per year.
The reliability of impact factors is an issue, especially when comparing journals across different disciplines. Some journals have very high citation rates; e.g. in the sciences, the journal Nature is very highly regarded and commands many high-quality and ground-breaking articles whose importance leads to high citation rates. In other circumstances, the type of publication can distort the impact factor. For example, review articles are often very highly cited and a journal that publishes many review articles may have a higher impact factor rating as a result. As Amin and Mabe (2000 p.6) state, âthey [impact factors] are not a direct measure of quality and must be used with considerable careâ. More recently, Thelwall and Fairclough (2015 p.263) have warned that journal impact factors (JIFs) are âwidely used and promoted but have important limitations. In particular, JIFs can be unduly influenced by individual highly cited articles and hence are inherently unstableâ.
A further issue with impact factors concerns journals that are known within the academic world as âpredatoryâ journals. These try to give the impression that they are bona fide academic journals, but exist only to extract money from unwary and inexperienced researchers who are charged high fees for publishing in them. They do not have any credibility, often use fake names and university attributions for their âeditorial boardsâ, and will publish just about anything that is sent to them. They then work hard to extract high fees from the authors. Academics are routinely spammed with emails inviting them to submit an article, even though the journal mentioned has nothing to do with the discipline in which the researcher works. Such âjournalsâ also routinely promote a fake impact factor to make it seem as if they are a legitimate publishing operation.
Publication types
Academic publications
Academic journals are at the top of any hierarchy for research output. For researchers working in education departments in universities (or any academic discipline) it is often a contractual obligation to produce material for publication in academic journals. Universities in the UK are judged on their academic output via a system called the Research Excellence Framework (REF). This is a huge exercise that judges the output of all lecturers on research contracts and grades this output on a scale from 1* (lowest) to 4* (highest). Not only do journals have a hierarchy, whatâs published within journals can be rated as well, and articles produced by academics are also rated on the 1*â4* scale. A criticism of this exercise is that it is too cumbersome and that the judgements made may not be reliable. University research funding in the UK is dependent on a good REF outcome, so there is pressure on researchers to publish high-quality research that has wide impact. While this is not the place to debate the pros and cons of the REF, there is no doubt that publication in the academic journals is still seen as the highest quality form of research output.
The structure of a journal article will vary from journal to journal, but generally it will have many if not all of the sections outlined in Box 1.2. Chapter 6 goes into more detail about the writing process for research.
Box 1.2 A typical research articleâs structure
Abstract
This is a brief summary of the paper and its main findings â usually no more than 200â300 words long.
Introduction
This should set out the problem being addressed and why it is important. There is usually a short statement about what is lacking in our current knowledge and it should state the objectives of the study or the research question(s).
Literature review
Key literature and previous research that is relevant to the study are discussed, critiqued and analysed so that what is currently known about the topic or concept is clearly stated. The literature review will also form the basis for...
