Bayes' Theorem

10 Key excerpts on "Bayes' Theorem"

• 

  No longer available |Learn more

  Statistical Paradoxes and Statistical Theorems

  • (Author)
  • 2014(Publication Date)
  • Orange Apple

    (Publisher)

  ________________________ WORLD TECHNOLOGIES ________________________ Chapter- 4 Bayes' Theorem and Central Limit Theorem Bayes' Theorem In probability theory and applications, Bayes' Theorem shows the relation between two conditional probabilities which are the reverse of each other. This theorem is named for Thomas Bayes and often called Bayes' law or Bayes' rule . Bayes' Theorem expresses the conditional probability, or posterior probability, of a hypothesis H (i.e. its probability after evidence E is observed) in terms of the prior probability of H, the prior probability of E, and the conditional probability of E given H. It implies that evidence has a stronger confirming effect if it was more unlikely before being observed. Bayes' Theorem is valid in all common interpretations of probability, and it is commonly applied in science and engineering. However, there is disagreement among statisticians regarding its proper implementation. The key idea is that the probability of an event A given an event B (e.g., the probability that one has breast cancer given that one has tested positive in a mammogram) depends not only on the relationship between events A and B (i.e., the accuracy of mammograms) but on the marginal probability (or simple probability) of occurrence of each event. For instance, if mammograms are known to be 95% accurate, this could be due to 5.0% false positives, 5.0% false negatives (missed cases), or a mix of false positives and false negatives. Bayes' Theorem allows one to calculate the conditional probability of having breast cancer, given a positive mammogram, for any of these three cases. The probability of a positive mammogram will be different for each of these cases.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Medical Decision Making

  • Harold C. Sox, Michael C. Higgins, Douglas K. Owens(Authors)
  • 2013(Publication Date)
  • Wiley-Blackwell

    (Publisher)

  Chapter 4

  Understanding new information: Bayes' Theorem

  Clinicians must take decisive action despite uncertainty about the patient's diagnosis or the outcome of treatment. To help them navigate this wilderness, Chapter 3 introduced probability as a language for expressing uncertainty and showed how to “translate” one's uncertainty into this language. Using probability theory does not eliminate uncertainty about the patient. Rather, it reduces uncertainty about uncertainty. This chapter builds upon the preceding chapter to show how to interpret new information that imperfectly reflects the patient's true state. This chapter has eight parts:

  4.1 Introduction 4.2 Conditional probability defined 4.3 Bayes' Theorem 4.4 The odds ratio form of Bayes' Theorem 4.5 Lessons to be learned from Bayes' Theorem 4.6 The assumptions of Bayes' Theorem 4.7 Using Bayes' Theorem to interpret a sequence of tests 4.8 Using Bayes' Theorem when many diseases are under consideration  

  4.1 Introduction

  Why is it important to understand the effect of new information on uncertainty? Clinicians want to minimize their margin of error, in effect bringing probability estimates as close as possible to 1.0 or 0. Without knowing how new information has affected—or will affect—probability, the clinician may acquire too much information, too little, or the wrong information.

  How does a clinician monitor progress toward understanding the patient's true state? Let us represent the process of diagnosis by a straight line (Figure 4.1

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Philosophy of Statistics

  • Dov M. Gabbay, Paul Thagard, John Woods(Authors)
  • 2011(Publication Date)
  • North Holland

    (Publisher)

  E), are both meaningful and have well-defined values. Classical statistics holds that these prior probabilities are generally not available to guide statistical inference. In contrast, Bayesians hold that these probabilities, properly construed, are available. Hence, the controversy concerns applications of Bayes's theorem and the interpretation of the probabilities it regulates rather than the theorem itself.

  A statistical test of a hypothesis collects statistical data to evaluate the hypothesis. The Bayesian decision-theoretic approach to statistics uses a statistical test along with prior information to evaluate a hypothesis. Bayesians methods generate the hypothesis's probability at the conclusion of the test. First, they use judgment to assign prior probabilities to the hypothesis and to the test's possible evidential outcomes. Then, in light of the test's actual outcome E, they employ Bayes's theorem to update the probability of the hypothesis from its prior value, P(H), to a new value equal to P(H|E). A last step may be to ground a decision on the results of the statistical test. For example, a pharmaceutical company may use a statistical test to decide whether to market a new drug. The success of marketing the drug depends on the drug's effectiveness. So statistical tests providing information about the drug's effectiveness may direct the company's decision about marketing the drug.

  Some statisticians have employed decision methods directly to statistical inference. Lindley [1971] presents an account of decision making that analyzes gathering information and applies Bayes's Theorem to update probabilities. Lindley's approach includes assessment of the value of gathering new information for a decision.

  DeGroot [1970] treats as a decision problem selection of an experiment to perform. According to DeGroot, “A statistician attempts to acquire information about the value of some parameter. His reward is the amount of information, appropriately defined, that he obtains about this value through experimentation. He must select an experiment from some available class of experiments, but the information that he will obtain from any particular experiment is random” (p. 88 ). DeGroot maintains that a statistician should choose experiments that minimize the risk of being mistaken about the truth of a hypothesis. He derives this policy from Wald [1950]

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Philosophy of Science

  A Contemporary Introduction

  • Alex Rosenberg, Lee McIntyre(Authors)
  • 2019(Publication Date)
  • Routledge

    (Publisher)

  prior probability of e must be low, so let’s let p(e) be low, say, 0.2; and the prior probability of such unexpected data, given Newton’s laws plus auxiliary hypotheses, will also be quite low, say, p(e/h) is 0.1. If p(h) for Newton’s laws plus auxiliaries is 0.95, then Bayes’ theorem tells us that for the new e, the precession data for Mercury, the p(h/e) = (0.1) × (0.95)/(0.2) = 0.475, a significant drop from 0.95. Naturally, recalling the earlier success of Newton’s laws in uncovering the existence of Neptune and Uranus, the initial blame for the drop was placed on the auxiliary hypotheses. Bayes’ theorem can even show us why. Though the numbers in our example are made up, in this case, the auxiliary assumptions were eventually vindicated, and the data about the much greater than expected precession of the perihelion of Mercury undermined Newton’s theory, and (as another application of Bayes’ theorem would show), increased the probability of Einstein’s alternative theory of relativity.

  Philosophers and many statisticians hold that the reasoning scientists use to test their hypotheses can be reconstructed as inferences in accordance with Bayes’ theorem. These theorists are called Bayesians, and they seek to show that the history of acceptance and rejection of theories in science honors Bayes’ theorem, thus showing that in fact, theory testing has been on firm footing all along. Other philosophers and statistical theorists attempt to apply Bayes’ theorem in order to determine the probability of scientific hypotheses when the data are hard to get, sometimes unreliable, or only indirectly relevant to the hypothesis under test. For example, they seek to determine the probabilities of various hypotheses about evolutionary events, like the splitting of ancestral species from one another, by applying Bayes’ theorem to data about differences in the polynucleotide sequences of the genes of currently living species.

  How Much Can Bayes’ Theorem Really Help?

  How much understanding of the nature of empirical testing does Bayesianism

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Machine Learning and Visual Perception

  • Baochang Zhang(Author)
  • 2020(Publication Date)
  • De Gruyter

    (Publisher)

  4 Bayesian learning Introduction Bayesian decision theory is a basic method of statistical inference for pattern classi-fication, in which the Bayes theorem is used to update the probability for a hypoth-esis as more evidence or information becomes available (Choudhuri, 2005) (Li, 2012; Albert, 2009). The basic principle lies in finding a compromise between the different classification decisions of probabilities and the corresponding decision costs. It is assumed that the decision problem can be described in terms of probabil-ity and that all relevant probability structures are known. Before making the actual observations, it is assumed that judgments are made on the categories to be presented the next time, also assuming that any wrong deci-sion will pay the same price and produce the same result. The only information that can be used is prior probability, which is ω 1 if P ð ω 1 Þ > P ð ω 2 Þ , and ω 2 otherwise. If only one decision is required, then the decision rule discussed previously is reasonable. However, for multiple sentences, it might not be suitable to reapply such a rule since the same result will always be achieved. For situations requiring multiple decisions, very little information is used to make the judgment. The example described next is used as an illustration to deter-mine the male and female sex. The observed height and weight information can be used to improve the performance of the classifier since different sexes will produce different height and weight distributions and then represent them as probabilistic variables. 4.1 Bayesian learning 4.1.1 Bayesian formula Suppose that x is a continuous random variable whose distribution depends on the state of the class and is expressed in the form of P ð x j ω Þ .

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Analytical Network and System Administration

  Managing Human-Computer Networks

  • Mark Burgess(Author)
  • 2012(Publication Date)
  • Wiley

    (Publisher)

  6 lymphocytes, which is several orders of magnitude greater than any computer system. What one lacks in numbers must therefore be made up in specificity or intelligence. The search problem is made more efficient by making identifications at many scales. Indeed, even in the body, proteins are complicated folded structures with a hierarchy of folds, which exhibit a structure at several different scales. These make a lock and key fit with receptors, which amount to keys with sub-keys and sub-sub-keys and so on. By breaking up a program structurally over the scale of procedure calls, loops and high-level statements, one stands a much greater chance of finding a pattern combination that signals danger.

  17.3 Bayes’ theorem

  The basic formula normally used in learning is Bayes’ theorem for conditional probability. This prescribes a well-defined method for a learning procedure, but it is not the only method (see section 17.6). We have already seen how conditional probability allows us to attach a causal arrow to the development of system information (see section 9.7). We now take advantage of this to develop a method of increasing certainty, or refined approximation by including the effect of observed evidence.

  Bayes’ formula is an expression of conditional probability. The probability of two separate events, A and B, occurring together may be written

  (17.3)

  If the events are independent of the order in which they occur, that is, they occur simultaneously by coincidence, then this simplifies to

  (17.4)

  The symmetry between A and B in eqn. (17.3) tells us that

  (17.5)

  This trivial re-expression is the basis for system learning. If we rewrite it for more than two kinds of events (see fig. 17.1 ), using fixed classes ci , for i = 1… C, and an unknown event E

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  TORUS 1 - Toward an Open Resource Using Services

  Cloud Computing for Environmental Data

  • Dominique Laffly(Author)
  • 2020(Publication Date)
  • Wiley-ISTE

    (Publisher)

  7.2. Bayesian modeling

  In this chapter, we will mainly focus on Bayesian methods. Bayesian probability theory is distinguished by defining probabilities as degrees of certainty, which provides a system for reasoning with uncertainty. In the following, we will introduce basic probability theory, the Bayes rule, along with the notions of parameters estimation, these being the basic notions for statistical machine learning.

  7.2.1. Basic probability theory

  This section gives an overview of probability theory. The basic rules of probability theory are as follows [BER 03]:

  • – the probability P(A) of a statement A is a real number ∈ [0, 1]. P(A) = 1 indicates absolute certainty that A is true, P(A) = 0 indicates absolute certainty that A is false and values between 0 and 1 correspond to varying degrees of certainty;
  • – the joint probability P(A, B) of two statements A and B is the probability that both statements are true (clearly, P(A, B) = P(B, A));
  • – the conditional probability P(A|B) of A given B is the probability that we would assign A as true, if we knew B to be true. The conditional probability is defined as P(A|B) = P(A, B)/P(B);
  • – the product rule: the probability that A and B are both true is given by the probability that B is true, multiplied by the probability we would assign to A if we knew B to be true:

  [7.2]

  Similarly, P(A, B) = P(B|A)P(A). This rule follows directly from the definition of conditional probability;

  • – the sum rule: the probability of a statement being true and the probability that it is false must sum to 1:

  [7.3]

  As a result, given a set of mutually exclusive statements A

  i

  , we have

  [7.4]

  • – marginalization:
    [7.5]

  where A

  i

  are mutually exclusive statements, of which exactly one must be true. In the simplest case where the statement A may be true or false we can derive:

  [7.6]

  • – independence: two statements are independent if and only if P(A, B) = P(A)P(B). If A and B are independent, then it follows that P

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Risk Assessment and Decision Analysis with Bayesian Networks

  • Norman Fenton, Martin Neil(Authors)
  • 2018(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  H , E ) = P (E | H )P (H )

  Substituting this value of P (H , E ) into the first equation gives us

  P ( H | E ) =

  P ( E | H ) P ( H )

  P ( E )

  which is Bayes’ theorem.

  In fact, the pure Bayesian approach to probability actually uses Bayes’ theorem as the fourth axiom. This is how conditional probability is defined rather than being defined in terms of the joint probability P (H, E ).

  Another immediate result of Bayes’ theorem is that it demonstrates the difference between P (H | E ) (which is 0.08 in the earlier example) and P (E | H ) (which is 0.8 in the example). Many common fallacies in probabilistic reasoning arise from mistakenly assuming that P (H | E ) and P (E | H ) are equal. We shall see further important examples of this.

  Box 6.2 provides a justification of why Bayes’ theorem produces the “right” answer by using frequency counts for the same example.

  Box 6.2Justification of Bayes’ Theorem Using Frequencies Suppose the chest clinic has 1000 patients. Of these

  ■ 50 have cancer (i.e., 5%)

  ■ 500 are smokers (i.e., 50%)

  ■ 40 of the 50 cancer patients are smokers (i.e., 80%)

  When we asked the question (in Bayes’ theorem) “What is the probability a patient has cancer given that they smoke?” this is the same, using frequency counts, as asking “What proportion of the smokers have cancer?”

  Solution : Clearly 40 of the 500 smokers have cancer. That is 8%, which is exactly the same result we got when applying Bayes’ theorem.

  We will see further intuitive justifications of Bayes’ theorem later in the chapter. But it is also worth noting that, using frequencies, Bayes is equivalent to the following formula in this case:

  =

  40 50

  ×

  50 1000

  500 1000

  =

  40 500

  The reason for using Bayes’ theorem, as opposed to Axiom 5.4 , as the basis for conditional probability was that in many situations it is more natural to have information about P (E | H ) (the likelihood of the evidence) rather than information about P (H,E ) (the joint probability). However, in Bayes (as well as Axiom 5.4 ) we still need to have information about P (E

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Digital Communication Systems

  • Simon Haykin(Author)
  • 2013(Publication Date)
  • Wiley

    (Publisher)

  Bayes’ Rule Suppose we are confronted with a situation where the conditional probability [A | B ] and the individual probabilities [A] and [B] are all easily determined directly, but the conditional probability [B | A ] is desired. To deal with this situation, we first rewrite (3.12) in the form Clearly, we may equally write The left-hand parts of these two relations are identical; we therefore have Provided that [A] is nonzero, we may determine the desired conditional probability [B | A ] by using the relation (3.14) This relation is known as Bayes’ rule. As simple as it looks, Bayes’ rule provides the correct language for describing inference, the formulation of which cannot be done without making assumptions. 2 The following example illustrates an application of Bayes’ rule. EXAMPLE 1 Radar Detection Radar, a remote sensing system, operates by transmitting a sequence of pulses and has its receiver listen to echoes produced by a target (e.g., aircraft) that could be present in its surveillance area.  A 1 A 2 B     A 1 B    A 2 B        B   ------------------------------------------------------------- = A 1 B  A 2 B   A 1 A 2 B     A 1 B     A 2 B    +  B   ------------------------------------------------------------ =  A 1 B     B   --------------------------  A 2 B     B   -------------------------- + =  A B     A B    B   =  A B     B A    A   =  A B    B    B A    A   =  B A    A B    B    A   -------------------------------- = 96 Chapter 3 Probability Theory and Bayesian Inference Let the events A and B be defined as follows: A = {a target is present in the area under surveillance} A c = {there is no target in the area} B = {the radar receiver detects a target} In the radar detection problem, there are three probabilities of particular interest: [A] probability that a target is present in the area; this probability is called the prior probability .

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Elementary Bayesian Biostatistics

  • Lemuel A. Moyé, Lemuel A. Moye(Authors)
  • 2016(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  29 1 Basic Probability and Bayes Theorem 1.1 Probability’s Role Questions propel us through life. Instinctually driven tens of thousands of years ago, the search for their solution may now be more cerebral, but their irresistible tug is unchanged. Be they questions of personal survival in primitive cultures (“Where will my next meal come from? Will I be attacked? Will my baby sur-vive?”), questions of agriculture (“Will our crops be raided by the enemy? Will there be enough rain?”), modern national security (“Will our enemy attack? When will that be? Will we be ready?”), society is besought with questions. Personal questions are no less demanding (Will I graduate? Can I find a good job? Will I survive this car accident? Can I pay for the damages?). The consuming capability of emotional questions (“Will he ever leave me? Does she really love me? Will my death be long and painful?”) speaks to their own personal power. Our survival seems destined to depend on questions, and our life’s character is shaped by the search for their answers. From thousands of years ago to the present day, man believed the answers to questions were found wholly in the supernatural. Later, society learned to appre-ciate its own role in shaping its own history (“the answers lie not in the stars, Brutus, but in ourselves,” Shakespeare’s character Cassius reminded 16 th century audiences in Julius Caesar ). Yet, whatever the source of the final truth, we are unwilling (and many times cannot afford) to wait until the answer is self-evident. We seek an early view to the future so that we can influence that future (e.g., abandon the development of a new drug that will not be very effective and cause dangerous adverse events). The goal is accurate predictions. Accurate predictions generate precise redirections of our actions, changing the future and perhaps ourselves.

  Sign up to read

  Learn more about book