Hybrid Frequentist/Bayesian Power and Bayesian Power in Planning Clinical Trials

Andrew P. Grieve

• Publisher: Chapman and Hall/CRC
• Year: 2022
• Print ISBN:
  9781032111292
• eBook ISBN:
  9781000590234
• Topic: Medicina
• Subtopic: Probabilidad y estadística

Hybrid Frequentist/Bayesian Power and Bayesian Power in Planning Clinical Trials provides a practical introduction to unconditional approaches to planning randomised clinical trials, particularly aimed at drug development in the pharmaceutical industry. This book is aimed at providing guidance to practitioners in using average power, assurance and related concepts. This book brings together recent research and sets them in a consistent framework and provides a fresh insight into how such methods can be used.

Features:

• A focus on normal theory linking average power, expected power, predictive power, assurance, conditional Bayesian power and Bayesian power.
• Extensions of the concepts to binomial, and time-to-event outcomes and non-inferiority trials
• An investigation into the upper bound on average power, assurance and Bayesian power based on the prior probability of a positive treatment effect
• Application of assurance to a series of trials in a development program and an introduction of the assurance of an individual trial conditional on the positive outcome of an earlier trial in the program, or to the successful outcome of an interim analysis
• Prior distribution of power and sample size
• Extension of the basic approach to proof-of-concept trials with dual success criteria
• Investigation of the connection between conditional and predictive power at an interim analysis and power and assurance
• Introduction of the idea of surety in sample sizing of clinical trials based on the width of the confidence intervals for the treatment effect, and an unconditional version.

1 Introduction

DOI: 10.1201/9781003218531-1

The power of a test is both a conditional and predictive concept. Conditional on the assumed statistical model, the alternative hypothesis and other parameters of the model, which may or may not be nuisance parameters, the outcome of an experiment is predicted and the proportion of predictions giving a “significant result” at a predetermined level is the power. The power of an experiment will therefore only achieve its nominal level if the conditionally assumed model and alternative hypothesis are true.

In this book, it is my intention to review unconditional (absolute) approaches, essentially Bayesian in construct, which can be used both in the planning phase of a clinical trial and during an interim analysis to adjust the sample size or in the extreme case to halt the study for futility. These approaches account for the current uncertainty in our knowledge of the treatment effect and have variously been called the strength, the expected power, the average power (AP), the predictive power (PP) and the assurance of the test.

In the experimental sciences, received wisdom has it that there are four factors that drive a statistical design, some, but not all, of which are in the direct control of the experimenter. These are:

1. An appropriate null hypothesis of no treatment effect;
2. The probability of committing a type I error (rejecting the null hypothesis when true), α, the significance level;
3. An alternative hypothesis representing a treatment effect magnitude that is of interest or one that is expected to be achieved – variously termed the clinically relevant difference (Lachin, 1977) or minimally clinically important difference (MCID) (Chuang-Stein et al., 2011a);
4. A sample size per experimental arm to give the probability of committing a type II error (not rejecting the null hypothesis when the alternative hypothesis is true), β.

For illustrative purposes, throughout this book, we will assume a simple two treatment experiment in which n₁ patients are randomised to each arm. The difference in sample means, $\widehat{\delta }\text{,}$ is assumed to have a normal distribution

(1.1)$$\widehat{\delta }~N\left(\delta ,2{\sigma }^2/n_1\right)$$

in which δ is the true difference in population means and we assume that the variance σ² is known. With these assumptions, the sample size, n₁, to test the null hypothesis H_o : δ = 0 against the alternative hypothesis H_A : δ = δ₀ is usually given by the formula

(1.2)$$n_1=\frac{2{\sigma }^2{\left(Z_{1-\alpha }+Z_{1-\beta }\right)}^2}{{\delta }_0^2}\text{.}$$

In all cases, we will be considering a one-sided test, but this is not a restrictive assumption.

Although this approach is not appropriate in all circumstances, the central limit theorem will often allow it to be used, at least following a suitable transformation. In clinical trials, Spiegelhalter et al. (2004) suggest it is reasonable to assume in many cases that after m “effective observations” relevant to a treatment effect ϕ on a suitable scale, a summary statistic y_m exists with approximately a normal likelihood

$$N\left(\varphi ,\frac{{\tau }^2}{m}\right)\text{.}$$

Table 1.1 illustrates four examples and shows how the definition of m differs from case to case. We will show how this approach can be used in simple scenarios and that more complex models can also be dealt with.

Table 1.1 Normal Approximations to Standard Likelihoods

Distribution	Treatment Effect	Variance - V(ym)	Definition of m	τ^2
Normal	ym = difference in means	$\frac{2{\sigma }^2}{m}$	Sample size per group	2σ^2
Binary	ym = log (odds ratio)	$\cong \frac{4}{m}$	Total # events	4
Poisson count	ym = log (rate ratio)	$\cong \frac{4}{m}$	Total count	4
Survival	ym = log (hazard ratio)	$\cong \frac{4}{m}$	Total # events	4

For many, the concept of the power of a test was introduced by Neyman and Pearson (1933a) both to define tests of specific hypotheses and as a measure for comparing different tests of the same hypothesis, for a fixed type I error. However, in truth, the basic idea had appeared four years earlier in work by Pearson and Adyanthāya (1929) on the robustness of Student’s t-test to non-normal symmetrical distributions or skew distributions and independently considered by Dodge and Romig (1929) in their introduction of consumer risks and producer risks in acceptance sampling. The type-I error corresponds to the producer’s risk that consumers reject a good product or service indicated by the null hypothesis. In other words, a producer introduces a good product and in doing so, he/she takes a risk that consumers will reject it. In contrast, the type II error corresponds to the consumer’s risk for not rejecting a possibly worthless product or service indicated by the null hypothesis.

Neyman and Pearson’s proposal had little immediate impact on the design of individual clinical trials. As Campbell (2013) has commented, “even exemplary clinical trials done in the early 1940s did not use statistical arguments to justify their Sample sizes”, including the Medical Research Council trial of streptomycin in the treatment of tuberculosis (Marshall et al., 1948). It was not until the early 1960s that power calculations began to appear in reports of clinical trials, one of the earliest being Zubrod et al. (1960).

In psychological research, Cohen (1962) was the first to seriously study the power of studies. He found amongst 70 studies from the Journal of Abnormal and Social Psychology that the median power of a medium effect size (ES), defined as the treatment difference divided by the pooled standard deviation of 0.5, was 0.48. This led to the recommendation that “investigators use larger sample sizes than they customarily do”, not least because “the chance of obtaining a significant result was about that of tossing a head with a fair coin” (Cohen, 1992). Later, Cohen produced a “power handbook” for psychologists “to solve the problem” (Cohen, 1969). One aspect of his work was the identification of the ratio of the type II to type I error rates as a measure of the relative importance of a type I error compared to a type II error. For example, in many studies, the type I error is set at 5% and the type II error at 20% which Cohen says implies that “mistaken rejection of the null hypothesis is considered four times as serious as mistaken acceptance”.

Recently, there has been increased interest in investigating optimal type I and type II errors when the objective is to minimise the weighted sum of errors in which the weights represent the relative importance of the errors (Mudge et al., 2012; Grieve, 2015). Grieve showed that for a fixed sample size, the optimal type I and type II error rates depend upon the non-centrality parameter (NCP), which is related to Cohen’s ES. Figure 1.1 displays the optimal error rates as a function of the NCP of a normal, two-arm comparative study, with known variance when type I errors are 4 times more important than type II errors. To illustrate, suppose that the NCP, ${\delta }_0\sqrt{n_1/2}/\sigma \text{,}$ takes the value 2 and we believe that type I errors are 4 times more important than type II errors. From Figure 1.1 we can read off that the optimal type I (two-sided) error is 0.090 (2×0.045) and the corresponding optimal type II error is 0.380 and their ratio is not 4:1. The optimal values are dependent on the value of the NCP and change as the NCP changes. For example, if the NCP takes the value 1.5 the optimal two-sided type I error is 0.094 and the optimal type II error is 0.569. Going further, Grieve also considered the inverse problem. For what relative importance are type I (two-sided) and type II error rates of 0.05 and 0.2, respectively, optimal? He showed that the solution is independent of the NCP. A two-sided type I error of 0.05 and a type II error of 0.2 are optimal for a relative importance of 4.79, 20% larger than the nominal ratio of 4. Walley and Grieve (2021) extend the analytic results developed by Grieve (2015) to studies for which the primary analysis is planned to be Bayesian. In the context of this book in which we concentrate on two-arm trials, Walley and Grieve (2021) consider examples in which a prior is available for the treatment effect and when an informative prior is available for the response mean in the control arm with a vague prior on the treatment effect which is related to Walley et al. (2015) and Lim et al. (2018).

Figure 1.1 Optimised type I/type II errors as a function of the NCP, if type I errors are 4 times more important than type II errors.

In 1978, Freiman et al. (1978) investigated 71 “negative” trials taken mainly from the New England Journal of Medicine, The Lancet and the Journal of the American Medical Association restricting their attention to studies with a binar...

Table of contents

• Cover Page
• Half Title page
• Series Page
• Title Page
• Copyright Page
• Dedication
• Contents
• List of Figures
• List of Tables
• Preface
• Acknowledgements
• Author
• List of Acronyms
• 1 Introduction
• 2 All Power Is Conditional Unless It's Absolute
• 3 Assurance
• 4 Average Power in Non-Normal Settings
• 5 Bayesian Power
• 6 Prior Distributions of Power and Sample Size
• 7 Interim Predictions
• 8 Case Studies in Simulation
• 9 Decision Criteria in Proof-of-Concept Trials
• 10 Surety and Assurance in Estimation
• References
• Appendix 1 Evaluation of a Double Normal Integral
• Appendix 2 Besag's Candidate Formula
• Index