![]()
1
A simple comparative experiment
1.1 Key concepts
1. Good experimental designs allow for precise estimation of one or more unknown quantities of interest. An example of such a quantity, or parameter, is the difference in the means of two treatments. One parameter estimate is more precise than another if it has a smaller variance.
2. Balanced designs are sometimes optimal, but this is not always the case.
3. If two design problems have different characteristics, they generally require the use of different designs.
4. The best way to allocate a new experimental test is at the treatment combination with the highest prediction variance. This may seem counterintuitive but it is an important principle.
5. The best allocation of experimental resources can depend on the relative cost of runs at one treatment combination versus the cost of runs at a different combination.
Is A different from B? Is A better than B? This chapter shows that doing the same number of tests on A and on B in a simple comparative experiment, while seemingly sensible, is not always the best thing to do. This chapter also defines what we mean by the best or optimal test plan.
1.2 The setup of a comparative experiment
Peter and Brad are drinking Belgian beer in the business lounge of Brussels Airport. They have plenty of time as their flight to the United States is severely delayed due to sudden heavy snowfall. Brad has just launched the idea of writing a textbook on tailor-made design of experiments.
[Brad] I have been playing with the idea for quite a while. My feeling is that design of experiments courses and textbooks overemphasize standard experimental plans such as full factorial designs, regular fractional factorial designs, other orthogonal designs, and central composite designs. More often than not, these designs are not feasible due to all kinds of practical considerations. Also, there are many situations where the standard designs are not the best choice.
[Peter] You don’t need to convince me. What would you do instead of the classical approach?
[Brad] I would like to use a case-study approach. Every chapter could be built around one realistic experimental design problem. A key feature of most of the cases would be that none of the textbook designs yields satisfactory answers and that a flexible approach to design the experiment is required. I would then show that modern, computer-based experimental design techniques can handle real-world problems better than standard designs.
[Peter] So, you would attempt to promote optimal experimental design as a flexible approach that can solve any design of experiments problem.
[Brad] More or less.
[Peter] Do you think there is a market for that?
[Brad] I am convinced there is. It seems strange to me that, even in 2011, there aren’t any books that show how to use optimal or computer-based experimental design to solve realistic problems without too much mathematics. I’d try to focus on how easy it is to generate those designs and on why they are often a better choice than standard designs.
[Peter] Do you have case studies in mind already?
[Brad] The robustness experiment done at Lone Star Snack Foods would be a good candidate. In that experiment, we had three quantitative experimental variables and one categorical. That is a typical example where the textbooks do not give very satisfying answers.
[Peter] Yes, that is an interesting case. Perhaps the pastry dough experiment is a good candidate as well. That was a case where a response surface design was run in blocks, and where it was not obvious how to use a central composite design.
[Brad] Right. I am sure we can find several other interesting case studies when we scan our list of recent consulting jobs.
[Peter] Certainly.
[Brad] Yesterday evening, I tried to come up with a good example for the introductory chapter of the book I have in mind.
[Peter] Did you find something interesting?
[Brad] I think so. My idea is to start with a simple example. An experiment to compare two population means. For example, to compare the average thickness of cables produced on two different machines.
[Peter] So, you’d go back to the simplest possible comparative experiment?
[Brad] Yep. I’d d...
