Hypothesis-testing Behaviour

Fenna H. Poletiek

• Publisher: Psychology Press
• Year: 2013
• Topic: Psychology
• eBook ISBN:
  9781134951659
• Subtopic: Cognitive Psychology & Cognition
• Index: Psychology

How do people search evidence for a hypothesis? A well documented answer in cognitive psychology is that they search for confirming evidence. However, the rational strategy is to try to falsify the hypothesis. This book critically evaluates this contradiction. Experimental research is discussed against the background of philosophical and formal theories of hypothesis testing with striking results: Falsificationism and verificationism - the two main rival philosophies of testing - come down to one and the same principle for concrete testing behaviour, eluding the contrast between rational falsification and confirmation bias. In this book, the author proposes a new perspective for describing hypothesis testing behaviour - the probability-value model - which unifies the contrasting views. According to this model, hypothesis testers pragmatically consider what evidence and how much evidence will convince them to reject or accept the hypothesis. They might either require highly probative evidence for its acceptance, at the risk of its rejection, or protect it against rejection and go for minor confirming observations. Interestingly, the model refines the classical opposition between rationality and pragmaticity because pragmatic considerations are a legitimate aspect of 'rational' hypothesis testing. Possible future research and applications of the ideas advanced are discussed, such as the modelling of expert hypothesis testing.

CHAPTER ONE

Theories of testing in the philosophy of science

INTRODUCTION

This chapter provides a survey of theories of testing in the philosophy of science. These theories are frequently used as points of departure in psychological research on testing behaviour, with the purpose of describing and explaining the conduct of human testers. The central questions governing the discussion of the theories are: What is regarded as a rational manner of searching and using evidence when judging a theory or hypothesis, and what is seen as irrational or biased? The crucial concepts borrowed from the philosophy of science by cognitive psychology in order to study human testing behaviour are those of “confirmation” and “falsification”. These concepts play a central role in the philosophies of the science of logical positivism (Carnap, 1936–1937) and Popper's critical rationalism (1959/1974, 1963/1978). Most psychologists have adopted the principle of falsification rather than that of confirmation as a standard for human hypothesis testing.

Borrowing this standard from the philosophy of science, psychologists assume that the scientific situation, for which these standards have been developed, is relevant to the general situation of humans trying to acquire knowledge about their environment. On the other hand, philosophers (e.g., Popper, 1963/1978) have emphasised that scientific knowledge acquisition is a systematised version of knowledge acquisition in ordinary life. Because of the great impact of the philosophy of science, and especially of the concepts of confirmation and falsification in the psychological study of human testing behaviour, the background of these concepts will be thoroughly discussed. What makes, in the authors' views, a principle into an adequate guideline to test scientific theories, and why is falsification preferred to confirmation as a normative principle in cognitive psychology? How do confirmation and falsification relate to one another? Is the contrast between the two principles, which is much emphasised in the psychology of testing, justified? Criticism of the two standard theories of science is also discussed, since some authors in psychology make use of these critical philosophies in their study of human hypothesis testing.

The theories of science are dealt with in three sections, moving from philosophical views about theory and evidence to more concrete theories concerning the way tests should be performed. The first section introduces the principles of confirmation and falsification, starting with the classical theories of the logical positivists and Popper, and reviewing criticisms of both. The next section deals with these principles at the methodological level of testing individual hypotheses. The term “methodological level” is used to indicate the concrete level at which the philosophical principles are applied in science. It also refers to the formal elaboration of the two philosophies. This section ends with an integration of the two principles. Whereas, at a philosophical level, the contrasts between the two key theories of science will be emphasised, at a methodological level, their concordance will be demonstrated. Subsequently, some remarks are made on the implications of the previous philosophical analyses for the psychological study of human testing behaviour. The chapter finishes with a short concluding section.

PHILOSOPHICAL PERSPECTIVES ON THEORY AND EVIDENCE

One of the major topics in the philosophy of science is the relationship between theories and empirical data. The work of Popper and Carnap deals with how this relation should be justified in scientific work. Other philosophies do not aim at a rational justification but merely describe how, in science, theories are related to data. In this review, logical positivism and falsificationism will be presented, followed by a survey of the critical reactions generated by these main theories.

Logical positivism

Logical positivism is the philosophy of science developed by the members of the Viennese Circle in the first decades of the 20th century. The most important motive of these philosophers was to rid science of metaphysics and “meaningless” statements. These meaningless statements were literally statements without significance. In order to distinguish between meaningful and meaningless statements, the logical positivists applied the criterion of “capable of being verified” (Carnap, 1936–1937). Thus, statements only have meaning if they are capable of being verified. How can theoretical statements, or in short, theories, be verified?

The philosophy of logical positivism consists of a “reconstruction” of science. The concept of “reconstruction” means that scientific developments are described by logical positivists as being a set of logical theories that have an empirical foundation and are not merely historical processes. Accordingly, it is a rational reconstruction. This reconstruction, later called the “received view”, is a logical system consisting of two languages: a theoretical language and an observational language. Theoretical terms can be reduced to terms from the observational language. Terms in the observational language refer directly to perceivable objects, or perceivable qualities of objects. The logical system of positivism can be regarded as a pyramid where the base is a collection of statements concerning perceivable objects and qualities, and where the peak consists of abstract theoretical statements that are linked to the base. The statements at the base (the “protocol sentences”) in the observational language are directly verifiable by an observer. But how are the observational statements (which, after all, are mini-theories in themselves) actually verified?

Developing a theory of verification was an important goal for the logical positivists because they wished to ensure that theories could be “anchored” in “true” observations. Carnap twice produced a theory concerning the verification of observational statements. The first theory of verification (Carnap, 1928) posits that the statements at the base (the protocol sentences) have the character of statements about experience. This is the so-called “phenomenalistic” approach to verification. The kernel of this vision is that the truth of the protocol sentences can be established immediately and indisputably. Accordingly, the statement “I have the perception of red” can be immediately confirmed or refuted by each individual.

The weakness of this vision of basic statements is that their verification varies from individual to individual. In other words, verification is indeed indisputable but it is not guaranteed intersubjectively. After all, different people may have different perceptions of the same object. The second theory regards the protocol sentences as statements about the qualities of objects, instead of about visual perceptions (Carnap 1936–1937). These statements are considered to be intersubjectively verifiable. But one is less certain that the statement is true, for instance, whether the colour really is red, since the judgement could be incompatible with a subjective perception. Thus, in the second theory, one observes and decides whether the statement has been verified or falsified. Consequently, verification here contains an element of convention. These are the two theories produced in the logical positivist framework which aimed at linking theoretical terms to observations. Eventually, by means of these theories, meaningful (scientific) statements could, in principle, be distinguished from meaningless statements.

Strictly speaking, the verification criterion is not a criterion for testing a theory. That is, it is not a set of rules which prescribes how, in a deductive process, a given theory can be rationally tested. It tells how a theory should be induced from observations in the first place, before being tested. Verificationism is mainly concerned with this inductive process. The fact that the theory is induced from verified observations gives it the status of scientific knowledge. Verification, as used in logical positivism, belongs to an inductive view of science. Suppe (1977, p. 15) emphasised the inductive nature of the received view as:

Thus science proceeds “upwards” from particular facts to theoretical generalisations about phenomena, this upward process proceeding in an essentially Baconian fashion.

Thus, an essential aspect of logical positivism is that it dictates searching, to the greatest extent possible, for verifiable observations which “confirm” a theory. Carnap's confirmation theory will also be discussed when dealing with the consequences of the verification principle at a methodological level. It provides guidelines as to which observations (protocol sentences) should be pursued in order to confirm a theory as being a scientifically true theory.

Before doing so, the most important opponent of logical positivism will be introduced: Karl Popper. He substituted the inductive representation favoured by the logical positivists with an explicitly deductive view of science. Popper shifted the focal point from the induction of hypotheses based on observation, to the testing of any given hypothesis.

Popper's critical rationalism

While the logical positivists were attempting to exclude metaphysics from science, Popper placed the accent on the growth of scientific knowledge. According to Popper (1963/1978, p. 215), scientific knowledge can grow when one repeatedly endeavours to refute theories and to replace them with better ones:

It is the way of its growth which makes science rational and empirical; the way, that is, in which scientists discriminate between available theories and choose the better one …

Popper's philosophy of science is referred to as being “critical” because of the importance of theory selection. Theories must always be evaluated critically so that one can improve or replace them. Accordingly, Popper posits the falsification principle instead of the verification principle.

A great deal of Popper's work contains criticism of the verification principle. In this criticism, the verification principle is not merely regarded as a method of evaluating basic statements with respect to their truth or falsehood. It has many more negative consequences (Popper, 1963/1978). The verification principle is regarded as one that leads to stagnation and dogma when pursued by researchers. This criticism by Popper has an explicit psychological nature in addition to the philosophical one, in the sense that verificationism seems to be condemned as an attitude (1963/1978, p. 229).

… the rationality of science lies not in its habit of appealing to empirical evidence in support of its dogmas—astrologers do so too—but solely in its critical approach: In an attitude which, of course, involves the critical use, among other arguments, of empirical evidence (especially in refutations).

The psychological and normative tone of this criticism has contributed to the formation of a rich breeding ground for its application in the psychology of reasoning. Popper's theory became prescriptive for most psychological research on test behaviour. Peter Wason, at that time a student in psychology inspired by the theory of Popper, adopted his normative view. Wason published the first psychological experiment concerning human testing of hypotheses. In this publication, he wrote (1960, p. 139):

The task simulates a miniature scientific problem …. The kind of attitude this task demands is that implicit in the formal analysis of scientific procedure proposed by Popper (1959). It consists in a willingness to attempt to falsify hypotheses, and thus to test those intuitive ideas which so often carry the feeling of certitude.

From Popper's and Wason's standpoint, the falsification principle is depicted as an open-minded strategy, in contrast to the verification principle which is a conservative strategy directed towards repeatedly confirming that which is already established. But it is debatable whether this interpretation corresponds to the intentions of the positivists. They too wished to remove metaphysical theories, such as astrology, from science and in this respect certainly did not wish to propagate any conservative attitude.

The philosophical part of Popper's argument against verificationism, as opposed to the psychological part, runs as follows: A universal statement cannot be verified purely logically by a limited number of observations, but it can be falsified. One single contrary example is sufficient to refute the statement, according to the modus tollens reasoning. The famous white swans problem can serve as an example. The universal statement “all swans are white” is not definitely verified when one has observed a number of white swans. Even a great many white swans are not enough to make it true. However, one black swan makes the statement definitely false. Thus, according to Popper, verification fails on grounds of principle because the demands of verifiability can never be satisfied in cases of universal statements. And the main aim of science is to produce true universal statements.

Popper's recommendation to attempt to refute theories thus essentially consists of two kinds of arguments, each of which is common in psychological studies of hypothesis testing: a logical and a psychological (or moral) argument. However, the arguments have quite different impacts. Interestingly, they do not necessarily lead to the same prediction about testing behaviour. For example, the logical argument for falsifica...

Table of contents

• Cover
• Half Title
• Full Title
• Copyright
• Dedication
• Contents
• Foreword
• Introduction
• 1. Theories of testing in the philosophy of science
• 2. Formal theories of testing
• 3. Wason's rule discovery task
• 4. Wason's selection task
• 5. Hypothesis testing under uncertainty
• 6. The probability value perspective
• References
• Appendix 1
• Author Index
• Subject Index