Pilot Studies and the Aims of Piloting

7 Key excerpts on "Pilot Studies and the Aims of Piloting"

• 

  eBook - ePub

  A Practical Guide to Cluster Randomised Trials in Health Services Research

  • Sandra Eldridge, Sally Kerry(Authors)
  • 2012(Publication Date)
  • Wiley

    (Publisher)

  However, it is likely that there will be some aspects of a trial that would benefit from empirical testing prior to the full trial. A pilot study can be defined as a small scale study conducted in advance of a main research study with the aim of refining or improving the design of the main study or assessing its feasibility. There has been a tendency to describe small trials with limited resources and/or time scales or even small student research projects as ‘pilot’ studies in order to justify the small sample size or lack of formal power calculations. This is not an appropriate use of the term (Thabane et al., 2010); investigators should always embark on a pilot study with an expectation that a main trial will follow. Nevertheless, because one of the aims of a pilot study is to assess feasibility of a main trial, not all pilot studies result in a main trial. An example of pilot work which resulted in the full trial being abandoned is that reported by Eldridge et al. (2005); this pilot study is described in more detail in Section 4.2.2. The publication of the pilot study results allowed the information to be accessible to other researchers who might wish to carry out similar interventions. A general introduction to good practice in pilot studies is provided by Lancaster, Dodd and Williamson (2004) and Thabane et al. (2010). Sometimes there may be a series of feasibility studies testing different aspects of a main trial. In developing the intervention for the Diabetes Manual trial (Sturt et al., 2005), the researchers carried out a series of small feasibility studies. Firstly, they conducted a needs assessment, using focus groups to find out what diabetic patients would want from an educational intervention package. This was followed by semi-structured interviews with eight healthcare professionals to ascertain their views on self-management interventions, followed by a questionnaire to 300 patients

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Writing Dissertations and Theses in Psychology

  A Student’s Guide for Success

  • Stephen Haynes, John Hunsley, Stephen N. Haynes, John D. Hunsley(Authors)
  • 2020(Publication Date)
  • Routledge

    (Publisher)

  8    Moving Your Research Along Pilot Studies, Research Teams, Initial Data Collection, Funding, and Getting Unstuck

  By now you are familiar with the complexity of a dissertation and the many challenges you could encounter during its completion. In the previous chapter we suggested ways to help you organize your work so that you can conduct it efficiently. In this chapter we discuss five additional aspects of dissertation work: (a) whether you should conduct a pilot study of your methods prior to, or as part of, your dissertation, (b) how to develop and manage a dissertation research team, (c) the pros and cons of collecting data prior to your proposal meeting, (d) possibilities for funding your research, and (e) what to do if you get stuck along the way.

  Should You Conduct a Pilot Study or Pilot Test?

  A pilot study is a small-scale preliminary study conducted in order to evaluate some or all aspects of your research methods prior to beginning the full study. A pilot study can facilitate the conduct of your thesis/dissertation by examining important aspects of it. The goals of a pilot study are to (a) determine if the full-scale study is feasible and (b) inform the design of the study and identify aspects of the methods that should be modified or enhanced before the full study is implemented.

  The pilot study might focus, for example, on the validity of participants’ scores on measures, the applicability and acceptance of your methods with your selected population, or the duration or feasibility of parts of your study. A pilot test usually refers to a less formal evaluation of some limited aspect of your dissertation methods. For example, you might want to recruit a few volunteers to evaluate a new website, a daily behavior monitoring system, or a set of instructions to participants.

  If conducted appropriately, a pilot study or pilot test reduces the chance that errors will occur in your research protocol that could impair inferences that can be drawn from your results

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Routledge International Handbook of Advanced Quantitative Methods in Nursing Research

  • Susan J Henly, Susan J Henly, Susan J. Henly(Authors)
  • 2015(Publication Date)
  • Routledge

    (Publisher)

  National Institutes of Health clinical trial phases adapted for nursing intervention development and testing (Whittemore & Grey, 2002) highlight the importance of testing the protocol of an RCT before proceeding to an evaluation of intervention effects. Thus, the primary purpose of a pilot study is to refine the protocol for the full-scale study by shedding light on strengths, inadequacies, or omissions of the preliminary plan (Conn et al., 2010 ; Feeley et al., 2009 ; Loscalzo, 2009 ; Polit & Beck, 2012 ; Shanyinde, Pickering, & Weatherall, 2011 ; Thabane et al., 2010). Findings from the pilot study are utilized to optimize the protocol for the full-scale study, so that it can be successfully and efficiently executed to generate the highest quality evidence for nursing practice. Pilot study data are not used to estimate intervention efficacy (Polit & Beck, 2012). Reviews of published pilot studies suggest that this is often misunderstood. For example, 81% of “pilot” studies published in seven major medical journals from 2007–2008 inappropriately included hypothesis-testing (Arain, Campbell, Cooper, & Lancaster, 2010). Another review of published pilot study findings indexed in MEDLINE and EMBASE over the past decade found that only 56% addressed methodological issues (Shanyinde et al., 2011). These findings suggest that there is a need to clarify and emphasize what are the appropriate objectives of pilot RCTs. Aims The aims of this chapter are to: (a) define feasibility and acceptability, (b) identify design elements that should be assessed for feasibility and acceptability prior to conducting a full-scale trial, (c) describe the decisions that can be taken based on the findings, and (d) summarize current controversies in the use of pilot studies. The focus is on feasibility and acceptability questions emanating from study design

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Writing Dissertation and Grant Proposals

  Epidemiology, Preventive Medicine and Biostatistics

  • Lisa Chasan-Taber(Author)
  • 2014(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  This small sample does not typically have adequate power to decisively answer your proposed hypotheses but ideally suggests an association, therefore motivating the rationale for the larger pro-posed study. Pilot studies may also be conducted in an existing dataset (i.e., secondary 134 Writing Dissertation and Grant Proposals data analysis). However, this dataset will likely differ from your proposed study in impor-tant ways—via characteristics of the study population (e.g., age and race/ethnicity) or via study design (e.g., cross sectional) or via study methods utilized (e.g., less precise measurement tools). All of these reasons would, again, help to motivate the rationale for your proposed study. Regardless of the type of pilot study, the goal of the pilot is to pro-vide findings that in some way support your proposed aims and justify the need for your proposed study (e.g., improved measurement device, stronger study design, different study population). In terms of the second example above, feasibility studies demonstrate to reviewers that you can logistically pull off your proposed study. Feasibility studies can provide key data on a number of factors. They can provide evidence that you, as a PI, are able to recruit subjects and collect data and that the proposed study is feasible in the planned study population. Such preliminary data have the added benefit of providing key figures necessary for calculating power and sample size calculations for your current proposal. Participant satisfaction surveys administered in a feasibility study can also provide data on the acceptability of your methods. Validation studies of your proposed methods (as described in Chapter 14, Reproducibility and Validity Studies ) can provide assurance that a study based upon these methods will work. In summary, ideally, the goal is to show proof of principal and demonstrate to the reviewers that you can pull it off .

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  The Psychologist's Companion for Undergraduates

  A Guide to Success for College Students

  • Robert J. Sternberg, Karin Sternberg(Authors)
  • 2017(Publication Date)
  • Cambridge University Press

    (Publisher)

  Although such approval sometimes can delay the execution of the study, it must be sought, nevertheless. 6.1.14 Conducting a Pilot Study Before starting final data collection, you should consider testing pilot participants. Pilot testing enables you to spot flaws in the experiment before you actually conduct it. You may find that your directions are unclear, that you have not allowed enough time for your participants to complete the task, or that the task is too difficult for your par- ticipants. The list of possible flaws is endless. Pilot testing is like an insurance policy. By making a small investment in advance, you can potentially save yourself much greater costs later. The more careful the pilot testing (the larger the insurance premium), the less likely it is that you will end up with disastrous results (i.e., the greater the insurance coverage). We have found that there is almost always some potential problem uncovered during pilot testing. 6.2 EXECUTING EXPERIMENTAL RESEARCH If you have planned your experiment carefully, then its execution should be straightforward. You should make sure, as much as you possibly can, that extraneous variables are kept constant from session 109 Writing the Experimental Research Paper to session. Thus, things like lighting, ventilation, and seating arrange- ment should not be varied. Outside noise should be minimized. If the experiment involves a number of separate parts, you may want to keep a list so that you do not forget any of them. Experimenters, like participants, sometimes get distracted; and once data are lost, they are difficult or impossible to replace. Use the experimental sessions as an informal opportunity to gain insights into how participants perform the experimental tasks. Par- ticipants occasionally make comments about what they are doing or how they are doing it.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Survey Research and Sampling

  • Jan Eichhorn, Author(Authors)
  • 2022(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  It may not always be possible, for example, due to budget constraints, to undertake extensive qualitative testing of questions in advance; however, researchers should always strive to perform some form of pretesting that provides feedback on the questions before piloting the actual survey. If cognitive interviewing techniques aren’t feasible, at least expert evaluations from people not directly involved in the development of the survey may be obtained (Bowden et al., 2002, p. 327). A proper pilot will not typically permit researchers to gain the qualitative insights discussed above, so it is ideal to triangulate feedback from different insights. When conducting a pilot, the main goal is to investigate how the actual administration works in the field. So the sample selected for the pilot should ideally already approximate the one that will ultimately be selected, following the same methods of sampling and data collection. For large surveys, Dillmann (2000) suggests that a good pilot sample would consist of 100–200 respondents to permit meaningful analyses; however, typically, standard population surveys often generate smaller samples of closer to 50 respondents, in particular when data collection modes are more expensive, such as in face-to-face collection methods. In practice, survey developers typically distinguish between two approaches: straightforward piloting and full dress rehearsals (a term used fairly commonly by practitioners; see, e.g. Moser & Kalton, 1992). While a pilot will be administered using the same methods selected for the overall project, the focus is largely on an analysis of the results. This means that once the data are available, distributions of responses are checked – in particular to identify problems of non-response, high levels of ‘don’t know’ or ‘refused’ kind of responses or dropout rates, for example. In a dress rehearsal, the scope of the pilot is much wider

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Survey Methods in Social Investigation

  • C.A. Moser, G. Kalton(Authors)
  • 2017(Publication Date)
  • Routledge

    (Publisher)

  Common sense suggests the necessity of doing a few test interviews or sending out trial forms by way of preparing for the main survey, and such informal trial and error is as much part of the preliminary study as are talks with experts and study of the literature. But it is necessary to go further, and to try out systematically all the various features of the main enquiry. This may take the form, first, of a series of small 'pre-tests' on isolated problems of the design; and then, when the broad plan of the enquiry is established, of a pilot survey, a small-scale replica of the main survey.

  The pilot survey is the dress rehearsal and, like a theatrical dress rehearsal, it will have been preceded by a series of preliminary tests and trials. Pre-tests and pilot surveys are standard practice with professional survey bodies and are widely used in research surveys. Specifically,1 they provide guidance on:

  (i) The adequacy of the sampling frame from which it is proposed to select the sample. The features of some much-used frames—e.g. the Register of Electors—are well known but, if a less familiar one is to be used, its completeness, accuracy, adequacy, up-to-dateness and convenience should be tested.

  One might be planning to use the pay-roll of workers in a factory as the basis for drawing a sample. It is just as well then to give it a try beforehand—it may not be conveniently arranged or it may exclude some workers (e.g. those temporarily on leave) whom the survey is to include, or cards may be temporarily removed from it when required for some other purpose. Whether or not such defects can be overcome is another matter; what is vital is to be aware of them before starting on the survey.

  (ii) The variability (with regard to the subject under investigation) within the population to be surveyed. This is of importance in determining an efficient sample design, and we shall see later that the very decision on sample size requires some knowledge of the variability of the population. Suppose that a survey were being designed to find out the average weekly rent paid by workers in a factory. The size of the sample needed to achieve the required precision would depend on how much variation there was in rent expenditure. Putting the point in extreme simplicity: if all the workers spent exactly £5 a week (i.e. variability between them were zero) a sample of 1 would suffice to give all the required precision; if, on the other hand, the variability of expenditures were high (i.e. rents ranging from, say, £3 to £10 per week) a relatively large sample would be required to provide a sufficiently precise estimate. Thus it is clear that in order to decide on a sampling plan it is helpful, to say the least, to possess some prior knowledge of the population to be sampled. Pilot studies, however, are rarely of much value in providing estimates of variability, because they are too small to yield estimates with any worth-while precision. Usually the researcher has available some statistics for the same or a related area, and careful analysis of these will normally lead to a more useful estimate than the pilot study can produce. Still, the pilot study does provide valuable supporting evidence, and especially so when no closely related statistics can be found.

  Sign up to read

  Learn more about book