Residuals

8 Key excerpts on "Residuals"

• 

  eBook - ePub

  Essential Numerical Computer Methods

  • Michael L. Johnson(Author)
  • 2010(Publication Date)
  • Academic Press

    (Publisher)

  The significance of subtle behavior in Residuals may suggest the presence of a significant system property that is overlooked by the current mathematical model. But a more fundamental role served by examination of Residuals is in providing information on which to base a judgment about the appropriateness of a particular mathematical description of system behavior as a function of some independent, experimental variable(s). If an examination of the Residuals obtained from a parameter-estimation procedure on some experimental data yields the conclusion that the data are reliably characterized by the mathematical model (i.e., that a good fit to the data is obtained possessing no unaccounted for residual systematic behavior in the data), this is not to say that this represents justification for necessarily accepting the model as correct. Rather, it indicates that the model employed is sufficient to characterize the behavior of the experimental data. This is the same as saying that the data considered provide no reason to reject the current model as unacceptable. The Residuals for a case in which a “good fit” is obtained then, in principle, represent the experimental uncertainty distribution for the data set. However, if an examination of the Residuals indicates inconsistencies between the data and the behavior predicted by the analysis, then the current model may correctly be rejected and considered unacceptable (unless some other source for the residual systematic behavior is identified).

  When considering Residuals, a qualitative approach is often the most revealing and informative. For example, generating plots to represent visually trends and correlations provides a direct and often unambiguous basis for a judgment on the validity of a fit. Of course, quantitative methods to test more rigorously particular properties of Residuals sometimes must be considered in order to quantitate the statistical significance of conclusions drawn as a result of data analysis. Some of the available methods for considering Residuals will be discussed below with the aid of illustrative examples.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Comparative Analysis Of Nations

  Quantitative Approaches

  • Robert Perry(Author)
  • 2019(Publication Date)
  • Routledge

    (Publisher)

  If a pattern emerges, as we have briefly noted in Chapter 15 with respect to Figure 15.11, the power of our theory to generalize may be in jeopardy. Residual analysis assumes an important role, therefore, within regression analysis in particular, and cross-national analysis in general. Analyzing the variance components and parameter assessments of the regression model alone (including the confidence in-terval of the regression coefficient) cannot allow such a direct examination of the er-ror terms in our model. For that we must use the parameter estimates to compute the respective residual errors and attach to each the name of the respective country. It is to this important aspect of applied regression that we will now direct our attention. Residual Error in Regression Analysis The residual, as we have seen, is literally the gap between our expected value of Y and the observed value of Y. We denote the predicted value of Y as Y'. As residual error in-creases across a sample of scores for Y, the explained variance declines and the capacity of X to accurately predict Y' is reduced accordingly. Because all scores within a regres-sion analysis will, of course, deviate to some degree from the predicted value of the score, we are not interested in simply measuring the absolute value of the deviation. The Role if Residual Ana!Jsis in Cross-National Research Rather, we gauge the degree of deviation of a score relative to the sample distribution of deviations. By exploring not merely the broad patterns of residual error in regression analysis, but also the residual error associated with specific countries within our cross-national sample, we greatly enhance our ability to refine and sharpen our comparative assess-ment of political behavior and/ or institutional authority. across countries. In this sec-tion, we will carefully consider the practical application and utility of residual analysis drawn from regression parameter estimates within a cross-national research strategy.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Introductory Regression Analysis

  with Computer Application for Business and Economics

  • Allen Webster(Author)
  • 2013(Publication Date)
  • Routledge

    (Publisher)

  Chapter 5

  RESIDUAL ANALYSIS AND MODEL SPECIFICATION

     

  Introduction

  5.1

  Using Residuals to Evaluate the Model

  • A Test for Randomness

  • Testing the Assumption of a Constant Variance

  • The Presence of Autocorrelation

  • Checking for Linearity

  • Test for Normality

  5.2

  Standardized Regression Coefficients

  5.3

  Proper Model Specification: Getting it Right

  • Consequences of an Omitted Variable

  • All Combinations

  • Backward Elimination, Forward Selection, and Stepwise Regression

  5.4

  Rescaling the Variables

  • Rescaling the Dependent Variable

  • Rescaling an Independent Variable

  5.5

  The Lagrange Multiplier Test for Significant Variables

  Chapter Problems

  Conceptual Problems

  Computer Problems

  INTRODUCTION

  The regression model is based on several assumptions detailed in Chapter 2 . Any violation of these assumptions threatens the integrity of the model. It is essential that after the model is estimated certain tests be conducted to determine if these assumptions hold. If it is decided not all assumptions are valid, some corrective action must be taken to avoid basing any policy decisions on a flawed model. These assumptions must be authenticated if the model is to command any integrity or validity.

  A battery of checks on these requisite conditions can be performed using residual plots. Once the model is estimated the Residuals are calculated and examined to verify whether the assumptions prevail. The residual, you recall, is the difference between the actual Y -value and the estimated or fitted Y -value, (Y i − Ŷ ). It reflects that portion of the variation in Y not explained by the model.

  This chapter provides a thorough discussion of the base assumptions and details the manner in which Residuals can be used to test for their veracity. Numerous statistical tests can be carried out to determine if the conditions necessary to conduct OLS procedures are present. Based on the results of these diagnostics the researcher is able to proceed using the proper analytical tools with reasonable certainty that the final results offer valid conclusions.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Regression Graphics

  Ideas for Studying Regressions Through Graphics

  • R. Dennis Cook(Author)
  • 2009(Publication Date)
  • Wiley-Interscience

    (Publisher)

  Graphics for this type of inquiry are developed in Section 15.2. 303 304 GRAPHICS FOR MODEL ASSESSMENT 15.1. RESIDUAL PLOTS Many standard graphical methods for detecting model weaknesses rely on plots of sample Residuals i versus linear combinations of the predictors a'x. When the model is correct or nearly so, we can often expect such plots to appear as if F ll d x , except perhaps for dependences in small samples caused by substituting estimates for unknown parameters. Historically, the choice of the linear combinations has been mostly a matter of taste and prior information. Plots of Residuals versus individual predictors or fitted values are common preferences. 15.1.1. Rationale To explore the rationale for residual plots, consider a regression in which it is conjectured that YAxlE,(Y 1x1 (15.2) where EM denotes expectation under the postulated model M.*Here, we imag- ine that the value of the parameter B is equal to the limit of 6 -a ~ n -+ 00. For use in practice M would be replaced by the estimated model M. The methods of Chapter 4 could be used as a basis for investigating (15.2) when there are p 5 2 predictors. Different methods are required when p > 2, however, and this is where Residuals may be useful. Condition (15.2) is equivalent to which in turn is equivalent to where (r,E,,(y 1 x ) ) is any one-to-one function of (y,E,(y 1 x ) ) . As implied by the notation, r is intended as a residual computed under M. However, there are not yet any constraints on r, except for the one-to-one requirement just stated. Residuals are distinguished by the additional requirement that they be independent of the regression function when the model is true; that is, when y l l x 1 EM(y 1 x). In sum, under the location regression (15.2), a population residual r is defined to be any function of (y,E,(y I x)) so that (r,EM@ I x ) ) is a one-to-one function of (V,EM(y I x)), and rlLEM(y 1 x ) when the model is true.

  Sign up to read

  Learn more about book
• 

  No longer available |Learn more

  Applying the Rasch Model in Social Sciences Using R

  • Iasonas Lamprianou(Author)
  • 2019(Publication Date)
  • Routledge

    (Publisher)

  3 Expectations and Residuals In this chapter, we discuss how we can use data to make predictions about the future. However, no matter how well calculated our predictions may be, we will often find that there is a discrepancy between expectations and reality. The magnitude of the residual between expectations and observa-tions will help us evaluate how successfully our model aligns with our data. This chapter presents a mathematical formulation of the Rasch model, building on concepts such as logarithms and exponents. The reader is encouraged to study the chapter, even if he or she is not mathematically savvy. Those who are more algebraically oriented have the opportunity to engage with the formulas and the numerical examples, although they do not really have to. Those who are less algebraically savvy may ignore the formulas and focus on the verbal explanations of the Rasch model instead. A GENTLE, BUT FORMAL, PRESENTATION OF THE RASCH MODEL Having a magic black box at our disposal has been really convenient up to now because we did not have to engage with the mathematical aspects of the Rasch model. However, it is now the time to present the Rasch model more formally. To make the discussion more practical, let us consider the application of the Rasch model in the context of the example discussed in the previous chapter. To refresh our memory, a test of 13 mathematics items was administered to a sample of 424 primary school pupils. Responses to each of the 13 items were scored as incorrect (0 marks awarded) or correct (1 mark awarded). expectations and Residuals 80 As most people who like betting already know, the odds of an event are related to the probability that the event will happen, over the probability that it will not happen.

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Theory of Linear Models

  • Bent Jorgensen(Author)
  • 2019(Publication Date)
  • Routledge

    (Publisher)

  CHAPTER 5

  Analysis of Residuals

  We now present the basic theory of analysis of Residuals and its use in model checking.

  5.1 Residuals and the hat matrix

  5.1.1 The raw Residuals and their distribution

  The general idea of analysis of Residuals is to study various plots of Residuals in order to evaluate the fit of a given model to the data. We hence need to study the distribution of the Residuals, in order to know how they are supposed to behave in the plots. We first study their null distribution (i.e. their distribution under the model being studied). The non-null distribution of Residuals will be studied in Section 5.3 .

  Consider a linear model with Y i ∼N (μ i , σ 2 ), i =1, …, n , independent and μ εL 1 =span {X }, where the design matrix X has linearly independent columns x 1 , …, x k . Define the raw Residuals by

  R = Y -

  μ ^

  (1.1)

  where the terminology “raw” is used to distinguish definition (1.1 ) from other types of modified Residuals to be introduced later on in this chapter.

  Recall from Section 2.2.1 that the estimator for μ , called the vector of fitted values , may be written as

  μ = HY,

  where

  H =X(XT X)-1 XT

  denotes the hat matrix. The vector of Residuals for the model is hence

  R = Y -

  μ ^

  = Y - H Y = ( I - H ) Y . (1.2)

  Recall that H is symmetric and idempotent,

  H H = H   and  

  H T

  = H ,

  (1.3)

  H being a projection matrix. Since I − H is the projection matrix for the space

  L 1 ⊥

  the matrix I − H is also symmetric and idempotent,

  ( I - H

  ) T

  = I - H , ( I - H ) ( I - H ) = I - H . (1.4)

  Of course, (1.4 ) easily follows from (1.3 ) by direct calculation.

  By (1.2 ), R is a linear combination of the elements of Y , and hence normally distributed. We have, for μ ε L 1 ,

  E ( R ) = ( I - H ) μ = μ - H μ = 0 ,

  where we have used that H μ = μ for μ ε L 1 . The variance matrix of R is, using (1.4

  Sign up to read

  Learn more about book
• 

  eBook - ePub

  Regression Analysis and its Application

  A Data-Oriented Approach

  • Richard F. Gunst, Robert L. Mason(Authors)
  • 2018(Publication Date)
  • CRC Press

    (Publisher)

  Figure 7.6 are evidence of violations of the assumption of normality for the error terms.

  Figure 7.7 Normal probability plot for reaction time Residuals with fourteen predictor variables.

  7.3 MODEL MISSPECIFICATION

  Correct model specification in a regression analysis involves two important aspects. All relevant variables must be contained in the data base and the proper functional form of each predictor must be defined in the prediction equation. Both of these requirements have been discussed and examined repeatedly in each of the prior chapters of this text. Apart from the lack of fit outlined in Section 6.3.3 , however, little mention has been made of the problems of detecting misspecification in a regression model. It is the purpose of this section to present some useful graphical techniques associated with residual analysis that aid the data analyst in correctly specifying a model.

  Two important types of residual plots will be examined: plots of Residuals against each predictor variable and partial residual plots. Each of these graphs provides a different view of model specification and, when the two are combined, they yield a useful evaluation of the predictor variables and their correct functional expressions. The question of whether to use raw Residuals or scaled Residuals in these plots is always present. While scaled Residuals are preferable due to their standardization properties, we will use the simple raw Residuals since they are the most common output of computer regression programs and, hence, will be seen most frequently.

  7.3.1 Plots of Residuals Versus Predictor Variables

  Plots of Residuals against each predictor variable indicate the distribution of the Residuals as a function of the predictor variables. The patterns provided by the plots are helpful in choosing the correct functional form of the predictors and determining the need for additional terms in the prediction equation.

  Sign up to read

  Learn more about book
• 

  eBook - PDF

  Sensitivity Analysis in Linear Regression

  • Ali S. Hadi, Samprit Chatterjee(Authors)
  • 2009(Publication Date)
  • Wiley

    (Publisher)

  volume of confidenceellipsoids, 5. likelihood function, 6. subsets of regression coefficients, and 7. eigenstructure of X. 4.2.1. Measures Based on Residuals Residuals play an important role in regression diagnostics; no analysis is complete without a thorough examination of the Residuals. The standard analysis of regression results is based on certain assumptions (see Section 1.4). For the analysis to be valid, it is necessary to ensure that these assumptions hold. 0.2.1. MEASURES BASED ON Residuals 73 The problem here is that the E~ can neither be observed nor can they be estimated. However, the ith residual ei measures, in some sense, the random errors E? By (2.5) the ordinary least squares Residuals e can be written as e = (I - P)E. This iden- tity indicates that for e to be a reasonable substitute for E, the off-diagonal elements of P must be sufficiently small. Furthermore, even if the elements of E are independent and have the same variance, identity (2.5) indicates that the ordinary Residuals are not independent (unless P is diagonal) and they do not have the same variance (unless the diagonal elements of P are equal). Consequently, the Residuals can be regarded as a reasonable substitute for the Ei if the rows of X are homogeneous (hence the diago- nal elements of the prediction matrix are approximately equal) and the off-diagonal elements of P are sufficiently small. For these reasons, it is preferable to use a transformed version of the ordinary Residuals for diagnostic purposes. That is, instead of ei one may use where CTi is the standard deviation of the ith residual. Four special cases of (4.2) are the normalized residual, the standardized residual, the internally Studentized residual, and the externally Studentized residual. The ith normalized residual is obtained by replacing oi in (4.2) by (eTe)l/* and obtaining The i!h standardized residual is found by substituting n- for Oi in (4.2) and obtaining

  Sign up to read

  Learn more about book