Quality Criteria

7 Key excerpts on "Quality Criteria"

• 

  No longer available |Learn more

  Applied Psychology in Talent Management

  • Wayne F. Cascio, Herman Aguinis(Authors)
  • 2018(Publication Date)
  • SAGE Publications, Inc

    (Publisher)

  The challenge is to develop theories, concepts, and measurements that will achieve the twin objectives of enhancing the utility of available procedures and programs and deepening our understanding of the psychological and behavioral processes involved in job performance. Ultimately, our goal is to develop a comprehensive theory of the performance of men and women at work (Campbell & Wiernik, 2015; Viswesvaran & Ones, 2000).

  In the early days of applied psychology, according to Jenkins (1946), most researchers and practitioners tended to accept the tacit assumption that criteria were either given by God or just to be found lying about. It is regrettable that even today we often resort to the most readily available or most expedient criteria when, with a little more effort and thought, we could probably develop much better ones. Nevertheless, progress has been made as the field has come to recognize that criterion measures are samples of a larger performance universe and that as much effort should be devoted to understanding and validating criteria as is devoted to identifying predictors of them (Campbell & Wiernik, 2015). Wallace (1965) expressed the matter aptly when he said that the answer to the question “Criteria for what?” must certainly include “for understanding” (p. 417). Let’s begin by defining our terms.

  Definition

  Criteria has been defined from more than one point of view. From one perspective, criteria are standards that can be used as yardsticks for measuring employees’ degree of success on the job (Bass & Barrett, 1981; Guion, 1965; Landy & Conte, 2016). This definition is quite adequate within the context of personnel selection, placement, promotion, succession planning, and performance management. It is useful when prediction is involved—that is, in the establishment of a functional relationship between one variable, the predictor, and another variable, the criterion. However, there are times when we simply wish to evaluate without necessarily predicting. Suppose, for example, that the HR department is concerned with evaluating the effectiveness of a recruitment campaign aimed at attracting members of underrepresented groups (e.g., women for science, technology, engineering, and mathematics—STEM—positions). Various criteria must be used to evaluate the program adequately. The goal in this case is not prediction but rather evaluation. Fundamentally, one distinction between predictors and criteria is time (Mullins & Ratliff, 1979). For example, if evaluative standards such as written or performance tests are administered before an employment decision is made (i.e., to hire or to promote), the standards are labeled predictors. If evaluative standards are administered after

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Analysis in Qualitative Research

  • Hennie R Boeije(Author)
  • 2009(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  This perspective has received criticism from different angles. By posing alternative demands on qualitative research, it is said to disengage itself from mainstream social science, with all the expected debates on the (non-)existence of a different kind of social science ensuing. A second point of criticism is that the newly formed criteria are, in their nature, derived from the original ones. In other words: it is old wine in new barrels. Confirmability parallels objectivity, dependability resembles reliability, credibility resembles internal validity, and transferability resembles external validity that we come to speak of later on. A third point of criticism is made by those who feel that political or humanistic criteria should not have such a large place in social scientific research.

  In the third perspective on quality, the criteria validity and reliability are maintained as worthy aims (Seale, 1999; Kirk & Miller, 1986). Although the terms are retained, the procedures to realize reliability and validity are adjusted to the specific nature of qualitative research. In doing so, qualitative research does not alienate itself from conventional social science, and it does not need to be treated in isolation. Researchers using these procedures should reflect on them by addressing how the research process – the analytical methods, the data collection procedures and the researcher’s presence – affects the results. ‘It is research that looks at itself’ (Alexander, 2008: 355). That way, readers can judge what influence the researchers’ presence and research activities may have had on the findings.

  Within this perspective on quality in qualitative research, the relationship between reliability and validity is sometimes viewed differently from the way it is viewed in quantitative research. In quantitative research reliability is commonly seen as a necessary (but in itself not sufficient) condition for validity. It is assumed that researchers simply cannot observe what they want to observe (validity) when the measurements are unreliable. In qualitative research, it is difficult to establish reliability by repeating the measurement since it often does not involve measurement with standard instruments. Instead the measures have to be developed specifically for a particular study, and they are often further adapted during use in the field (Jorgensen, 1989). As a consequence it might be argued that reliability must be doubted when using a human being as a measuring instrument.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Using Qualitative Methods in Psychology

  • Mary Kopala, Lisa A. Suzuki, Mary Kopala, Lisa A. Suzuki(Authors)
  • 1999(Publication Date)
  • SAGE Publications, Inc

    (Publisher)

  I have chosen the first two—trustworthiness and reflex-ivity—because of the consensus about these as hallmarks of quality work, and the third—representation—because it seems to be a crucial, next issue in the field. It is important to note at the start that whether the terms reliability and validity belong in considerations of qualitative research is debatable. (For 25 26 T H E P H I L O S O P H I C A L F O U N D A T I O N S examples of divergent views, see Becker [1996], Lather [1993], and Wolcott, [1990].) After all, these criteria have traditionally been used to assess the quality of quantitative research. Traditionally, reliability is described as the extent to which a research endeavor and findings can be replicated; validity refers to the extent to which findings can be considered true (Stiles, 1993). As these terms have been defined and used in discussions of quantitative work, they are not truly appropriate for discussing qualitative research. Never-theless, I choose to begin my consideration of quality in qualitative research with an exploration of these terms partly because most psychologists are familiar with, and have been trained to evaluate, research using these criteria. In addition, my choice is rooted in the belief that reliability and validity have been appropriated by quantitative researchers for too long. My hope, in a vein similar to Lather (1993), is that qualitative researchers may reclaim and redefine the terms needed to discuss qualitative work. Believing that the research we conduct is both reliable and valid, I discuss it as such. Thoughtful use of these terms—not as a defense or an appeal to the positivist paradigm— creates space to consider what is important in qualitative research endeavors. Acknowledging the many divergent opinions about evaluation criteria for qualitative research, I rely heavily on Denzin and Lincoln (1994) to summarize four positions: 1.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Managing Quality in Qualitative Research

  • Uwe Flick, Author(Authors)
  • 2018(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  Chapter Three Criteria in Qualitative Research

  Contents

  • Transferring the discussion about criteria to qualitative research 26
  • Traditional or new criteria to answer the question of quality in qualitative research? 27
  • Reformulation of traditional criteria 28
  • Alternative, method-appropriate criteria 35
  • Quality of qualitative research – an issue for technology? 38

  Chapter objectives

  After reading this chapter, you should:

  • see the problems and limitations in the attempt to apply traditional criteria to qualitative research;
  • know more about suggestions for reformulating these criteria for qualitative research; and
  • be familiar with discussions about specific criteria for qualitative research.

  As the preceding chapter should have shown, the discussion of standards of or for qualitative research is complicated first by the fact that standards are often imported from a generalist view of empirical research or from a quantitative background. A second complication results from the question of whether these standards should be relevant for all qualitative research approaches or just for one specific understanding of qualitative research. The third source of complication comes from whether standards should be used for defining research programmes as good (and better than other ones) or for assessing the particular project done with a specific approach or with qualitative research in general. This last complication relates to how to nail down the general methodological question of research standards or quality to the concrete level of doing research. Instead of using the term ‘standards’ a bigger part of the literature on quality is based on the notion of criteria for judging the quality of qualitative research.

  Transferring the discussion about criteria to qualitative research

  When the discussion about criteria is transferred to qualitative research, we can distinguish two streams of discussion. Either, first, a narrowly focused and at the same time generalist concept is pursued. It is narrowly focused, because the (wider) question of what is qualitative is reduced to formulating and applying criteria (e.g. for the validity of interview statements). It is generalist, because – often at least – the hope in the background is that the criteria are the same or similar to those in quantitative research and that they at least will have the same function in both. Or, second, a rather widely conceived but at the same time specific concept

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Qualitative research methods in psychology

  • Aparna Raghvan(Author)
  • 2023(Publication Date)
  • Society Publishing

    (Publisher)

  Green and Thorogood (2004) bring up the investigation and translations made by scientists ought to be dependable and furthermore straightforward, with clear connections between the information and any clarifications gave. In quantitative information examination the factors of interest are normally predefined and determined in the exploration plan. Likewise, the coding outlines used to order and examine information are additionally frequently predefined and connect with outside reference focuses. For instance, dietary admission information is much of the time connected with normalized dietary reference values to evaluate sufficiency or hazard of a specific infection or condition creating. Accentuation is put on measurable strategies to control information or factors and to decide ‘importance’ levels. The point of investigation is as a rule to relate information back to an underlying speculation or exploration Qualitative Research Methods in Psychology 216 question, for example is a BMI of specific level related with expanded chance of illness X in a populace Y? As noted, before, the sorts of clarification delivered will generally be of two sorts, causal proclamations or backing of specific cases. 7.8. THE QUALITY OF QUALITATIVE RESEARCH Quality is an indispensable issue. Notwithstanding the developing acknowledgment of the worth of subjective exploration, there is as yet an issue in detailing will in general make the feeling that is some way or another delicate and, as referenced prior, that its discoveries express minimal more than the specialists’ emotional impressions. It is, hence, critical to lay out the quality and meticulousness of subjective examination, which is maybe especially important inside applied wellbeing research, where the significance and utilization of discoveries outside the specific setting of the exploration setting or populace should be laid out.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  An Introduction to Qualitative Research

  • Uwe Flick(Author)
  • 2022(Publication Date)
  • SAGE Publications Ltd

    (Publisher)

  One starting point is to reflect upon the construction of social realities in the field under study and in the research process. The decisive question, however, is whose constructions were addressed and were successful in the process of knowledge production and in the formulation of the results – those of the researcher, or those met in the studied field? The problem of grounding qualitative research is made concrete with three questions: How far are the researchers’ findings based on constructions in the field? How are the translation and documentation of these constructions in the field made into texts, which are the empirical material? How did the researcher proceed from the case study to the developed theory or to the general patterns found? Grounding qualitative research becomes a question of analysing the research as process. After discussing the alternatives mentioned, the impression remains that both strategies – the application of traditional criteria and the development of alternative, specific criteria – have featured in recent discussions and that neither has yet given a wholly satisfactory answer to the problem of grounding qualitative research.

  The equation or connection of alternative and traditional criteria by Miles and Huberman (1994, p. 278) outlines an interesting perspective for structuring this field. They have structured the major traditional criteria together with the most prominent alternative criteria in the following five groups:

  • Objectivity/confirmability
  • Reliability/dependability/auditability
  • Internal validity/credibility/authenticity
  • External validity/transferability/fittingness
  • Utilization/application/action orientation.

  But at the same time, this equation makes clear that attempts to reformulate criteria for qualitative research did not really lead to new solutions. Rather, the problems with traditional criteria derived from different backgrounds have to be discussed in the case of alternative criteria, too.

  Quality Assessment as a Challenge for Qualitative Research

  The question of how to assess the quality of qualitative research currently arises in three respects. First, by the researchers who want to check and secure their proceedings and their results. Second, by the consumers of qualitative research, the readers of publications or the funding agencies, who want to assess what has been presented to them. And finally in the evaluation of research in reviewing research proposals and in peer reviews of manuscripts submitted to journals. In the last context, you will find a growing number of guidelines for evaluating research papers (articles, proposals, etc.). Seale (1999, pp. 189–92) presents a criteria catalogue of the British Sociological Association’s Medical Sociology Group, which includes a set of questions referring to 20 areas from research questions on sampling, collection and analysis of data, to presentation and ethics. The guiding questions are helpful, but when you answer them, you will find that you are drawn back to your own – maybe implicit – criteria: for example, when you want an answer in area 19 (‘Are the results credible and appropriate?’), the question ‘Do they address the research question(s)?’ (1999, p. 192) is suggested.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Qualitative Research Methods in Mental Health and Psychotherapy

  A Guide for Students and Practitioners

  • David Harper, Andrew R. Thompson, David Harper, Andrew R. Thompson(Authors)
  • 2011(Publication Date)
  • Wiley

    (Publisher)

  A key concern relates to whether or not quality involves the same constructs in qualitative and quantitative research. There is much disquiet about a simple transfer of accepted ‘quantitative’ criteria, given the wide range of epistemological and ontological positions adopted by qualitative researchers and the flexible non-standardized nature of their methods. Nevertheless, some argue for retaining concepts such as validity and reliability although the terms may be defined and applied in more or less traditional ways (Beck, 1993; Kirk & Miller, 1986; Le Compte & Goetz, 1982).

  Guba and Lincoln have famously developed a parallel set of ‘naturalistic’ criteria to replace validity and reliability: credibility , transferability , dependability and confirmability (Guba & Lincoln 1981; Lincoln & Guba, 1985). Other researchers have retained the term validity, for example, but re-defined it to mean the impact of a study in promoting greater understanding or social change (Stiles, 1993). Finally, there are some who advocate abandoning all reference to notions of validity and reliability in favour of qualities such as intensive personal involvement, improvisation and flexibility (Agar, 1986).

  Another major consideration is whether there are any generic qualitative indicators or whether each method requires its own criteria. While transparency about research decisions and analysis procedures might be accepted by some researchers as applicable across a number of different methods, more detailed aspects of quality may well be method-specific. For example, in conversation analysis, quality might be assessed in terms of the way a study makes evident the taken-for-granted competences people use in everyday conversations (see Chapter 13), whereas this would hold little relevance for an interview-based study. In document analysis, the question of whether or not an author directly witnessed an event might be seen as a way of establishing the ‘authenticity’ of a document (Platt, 1981), whereas treating a participant as a witness or informant in an interview-based study is only one of several different readings that can be made (Kvale & Brinkman, 2009).

  Sign up to read

  Learn more about book