Polynomial Regression

6 Key excerpts on "Polynomial Regression"

• 

  eBook - PDF

  Random Phenomena

  Fundamentals of Probability and Statistics for Engineers

  • Babatunde A. Ogunnaike(Author)
  • 2009(Publication Date)
  • CRC Press

    (Publisher)

  This class of regression models is important because, in engineering, many unknown functional relationships, y ( x ), can be approximated by polynomials. Because Eq (16.191) is a special case of Eq (16.133), all the results obtained earlier for the more general problem transfer directly, and there is not much to add for this restricted class of problems. However, in terms of practical appli-cation, there are some peculiarities unique to Polynomial Regression analysis. In many practical problems, the starting point in Polynomial Regression is often a low order linear model; when residual analysis indicates that the simple model is inadequate, the model complexity is then increased, typically by adding the next higher power of x , until the model is deemed “adequate.” But one must be careful: fitting an m th order polynomial to m +1 data points (e.g., fitting a straight line to 2 points) will produce a perfect R 2 = 1 but the parameter estimates will be unreliable. The primary pitfall to avoid in such an exercise is therefore “overfitting,” whereby the polynomial model is of an order higher than can be realistically supported by the data. Under such circumstances, the improvement in R 2 must be cross-checked against the corresponding R 2 adj value. Regression Analysis 697 The next examples illustrate the application of Polynomial Regression. Example 16.9: BOILING POINT OF HYDROCARBONS: REVISITED In Example 16.6, a linear two-parameter model was postulated for the relationship between the number of carbon atoms in the hydrocarbon compounds listed in Table 16.1 and the respective boiling points. Upon evaluation, however, the model was found to be inadequate; specifically, the residuals indicated the potential for a “left over” quadratic compo-nent. Postulate the following quadratic model, Y = θ 0 + θ 1 x + θ 2 x 2 + ǫ (16.192) and evaluate a least-squares fit of this model to the data. Compare this model fit to the simple linear model obtained in Example 16.6.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Handbook of Regression and Modeling

  Applications for the Clinical and Pharmaceutical Industries

  • Daryl S. Paulson(Author)
  • 2006(Publication Date)
  • Chapman and Hall/CRC

    (Publisher)

  7 Polynomial Regression Polynomial Regression models are useful in situations in which the curvilinear response function is too complex to linearize by means of a transformation, and an estimated response function fits the data adequately. Generally, if the modeled polynomial is not too complex to be generalized to a wide variety of similar studies, it is useful. On the other hand, if a modeled polynomial ‘‘overfits’’ the data of one experiment, then, for each experiment, a new polynomial must be built. This is generally ineffective, as the same type of experiment must use the same model if any iterative comparisons are required. Figure 7.1 presents a dataset that can be modeled by a polynomial function, or that can be set up as a piecewise regression. It is impossible to linearize this function by a simple scale transformation. For a dataset like this, it is important to follow two steps: 1. Collect sufficient data that are replicated at each x i predictor variable. 2. Perform true replication, not just repeated measurements in the same experiment. True replication requires actually repeating the experiment n times. Although this sounds like a lot of effort, it will save hours of frustration and interpret-ation in determining the true data pattern to be modeled. Figure 7.2 shows another problem—that of inadequate sample points within the x i s. The large ‘‘gaps’’ between the x i s represent unknown data points. If the model were fit via a polynomial or piecewise regression with both replication and repeated measurements, the model would still be inadequate. This is because the need for sufficient data, specified in step 1, was ignored. Another type of problem occurs when repeated measurements are taken, but the study was not replicated (Figure 7.3). The figure depicts a study that was replicated five times, and each average repeated measurement plotted.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Regression Analysis and Linear Models

  Concepts, Applications, and Implementation

  • Richard B. Darlington, Andrew F. Hayes(Authors)
  • 2016(Publication Date)
  • The Guilford Press

    (Publisher)

  Polynomials can also be nice ways of dealing with nonlinearity in co- variates. Even if the relationship between an independent variable X and a dependent variable Y is linear, when those variables relate nonlinearly to a covariate C, it is important to allow for that nonlinearity in order to properly visualize and estimate the partial association between X and Y. We wouldn’t typically care if the polynomial is a substantively or theoret- ically meaningful representation of the nonlinear relationship between a covariate and independent and dependent variables if it does a good job at capturing that nonlinearity and thereby affords a better adjustment for constructing measures of partial association between key variables in your analysis. Polynomial Regression is often used as a means of testing for nonlinear- ity in the relationship between X and Y. Because polynomials can describe such a wide range of curves, a test of nonlinearity can be conducted by determining if adding successive powers or sets of powers of X improves the fit of the model to a statistically significant degree. The test described in section 5.3.3 can be used for this purpose. We will see an example of this in section 12.2.2. When a variable X is included as a regressor along with various powers of that variable, we usually think of that set of variables as a compound variable representing X. So, for example, if you think that age is nonlinearly related to something like attitudes toward gun control, you could use age as well as age 2 and perhaps even age 3 as regressors in the model. Any test involving age would involve all three of these. For instance, you could test whether gun control is related to age while controlling for sex 350 Regression Analysis and Linear Models and income by adding age, age 2 , and age 3 to a model of gun control that already contains income and sex.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Lean Six Sigma Black Belt. Certification manual

  • Socconini, Luis(Authors)
  • 2024(Publication Date)
  • Marge Books

    (Publisher)

  153 Learning objectives 1. Apply multiple regression analysis to improve business decision-making. 2. Analyze and interpret the results of statistical programs for One-Factor, Multiple, and Polynomial Regression models. 3. Evaluate the significance of independent variables in a regression model. Content > Regression Analysis > One-Factor Regression Model > Multiple Linear Regression > Polynomial Regression > Examples and exercises Regression Analysis 8 154 Uses for Regression: 1. Description: Represent the behavior of a process. 2. Prediction and estimation:  Prediction is based on a known x value.  Estimation is based on an unknown x value. 3. Control: Obtain a certain desired response from the process. Regression Analysis is a technique used to model the relationship between one or more independent variables and a dependent variable (response variable).  y = Dependent variable to model (response).  x = Independent variable (y predictor).  e = Error component (measurement + natural error). Random variable.  b 0 = Intersection. If data includes zero, then it represents y’s distribution mean when all x = 0. The intercept has no meaning if x never equals zero.  b 1 = Slope. This is the change in y for every incremental change in x. ε β β 1 0 + + = x y Where: Regression Analysis Regression Analysis One-Factor Regression Model 155 One-Factor Regression Model Regression Analysis We can calculate the parameters using the least squares method, which consists of minimizing the error of the model.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Linear Regression Analysis

  • George A. F. Seber, Alan J. Lee(Authors)
  • 2012(Publication Date)
  • Wiley

    (Publisher)

  7.3 Polynomial Regression IN SEVERAL VARIABLES 7.3.1 Response Surfaces An important application of Polynomial Regression in several variables is in the study of response surfaces. We illustrate some of the basic features of response surface methodology by considering the simple case of just two regressors. Suppose that the response (yield) r from a given experiment is an un-known function of two variables, x (temperature) and x-i (concentration), namely, 77 = g{x,X2). It is assumed that this three-dimensional surface is well-behaved, in particular is smooth with a single well-defined peak. The response 77 is measured with error so that we actually observe V = r) + e, where E[e] = 0 and var[e] = a 2 . One basic problem of response theory then is to estimate the coordinates, (xoi^oai^o) sav > of the summit. One method of doing this is to use a sequence of experiments and a steepest ascent technique to climb up the surface. Typically, experimental data points are expensive to obtain, so we need to choose a design with a small number of data points and locate them in an optimal fashion so as to maximise the efficiency of estimation. For points away from the summit the surface is relatively linear in a small region, so that it can be represented locally by a Polynomial Regression IN SEVERAL VARIABLES 181 plane, namely, EY] = ßo + ßixi + fax 3 . (7.23) To estimate the coefficients ßi, we can, for example, use a very simple design such as the 2 2 design; we shall see later that that such a design has certain optimal properties. In this design we observe Y at the four vertices of a small rectangle, with center Pi, in the (xi,X2) plane (Figure 7.1). Suppose that Y ra is the Y observed at (x r i,x„2), where x T i (r = 1,2) are the two chosen values of i i and x S 2 (s = 1,2) are the two chosen values of X2-Then we can fit the model Y T8 = ßo + ßixn + ß 2 x, 2 + e rs , (7.24) where r = 1,2 and s = 1,2, and obtain the fitted plane Y =

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Statistics II For Dummies

  • Deborah J. Rumsey(Author)
  • 2021(Publication Date)
  • For Dummies

    (Publisher)

  The doctor finds that patients with very low or very high blood pressure had a higher occurrence of problems, while patients whose blood pressure fell in the middle, constituting the normal range, had fewer problems. This pattern of data has a U-shape, and a parabola would fit this data well. In this section, you see what a Polynomial Regression model is, how you can search for a good-fitting polynomial for your data, and how you can assess polynomial models. Bringing back polynomials You may recall from algebra that a polynomial is a sum of x terms raised to a vari-ety of powers, and each x is preceded by a constant called the coefficient of that term. For example, the model y x x x 2 3 6 2 3 is a polynomial. The general form for a Polynomial Regression model is y x x x x k k 0 1 1 2 2 3 3  . Here, k represents the total number of terms in the model. The ε represents the error that occurs simply due to chance. (Not a bad kind of error, just random fluctua -tions from a perfect model.) Here are a few of the more common polynomials you run across when analyzing data and fitting models. Remember, the simplest model that fits is the one you use (don’t try to be a hero in statistics — save that for Batman and Robin). The models I discuss in this book are some of your old favorites from algebra: second-, third-, and fourth-degree polynomials. » Second-degree (or quadratic) polynomial: This model is called a second-degree (or quadratic ) polynomial, because the largest exponent is 2. An example model is y x x 2 3 2 . A second-degree polynomial forms a parabola shape — either an upside-down or right-side-up bowl; it changes direction one time (see Figure 8-3). » Third-degree polynomial: This model has 3 as the highest power of x . It typically has a sideways S-shape, changing directions two times (see Figure 8-4). » Fourth-degree polynomial: Fourth-degree polynomials involve x 4 .

  Sign up to read

  Learn more about book