![]()
1
INTRODUCTION TO REGRESSION ANALYSIS
1.1 INTRODUCTION
The general purpose of regression analysis is to study the relationship between one or more dependent variable(s) and one or more independent variable(s). The most basic form of a regression model is where there is one independent variable and one dependent variable. For instance, a model relating the log of wage of married women to their experience in the work force is a simple linear regression model given by log(wage) = β0 + β1exper + ε, where β0 and β1 are unknown coefficients ande is random error. One objective here is to determine what effect (if any) the variable exper has on wage. In practice, most studies involve cases where there is more than one independent variable. As an example, we can extend the simple model relating log(wage) to exper by including the square of the experience (exper2) in the work force, along with years of education (educ). The objective here may be to determine what effect (if any) the explanatory variables (exper, exper2, educ) have on the response variable log(wage). The extended model can be written as
where β0, β1, β2, and β3 are the unknown coefficients that need to be estimated, and ε is random error.
An extension of the multiple regression model (with one dependent variable) is the multivariate regression model where there is more than one dependent variable. For instance, the well-known Grunfeld investment model deals with the relationship between investment (Iit) with the true market value of a firm (Fit) and the value of capital (Cit) (Greene, 2003). Here, i indexes the firms and t indexes time. The model is given by Iit = β0i + β1iFit + β2iCit + εit. As before, β0i, βi, and β2i are unknown coefficients that need to be estimated and εit is random error. The objective here is to determine if the disturbance terms are involved in cross-equation correlation. Equation by equation ordinary least squares is used to estimate the model parameters if the disturbances are not involved in cross-equation correlations. A feasible generalized least squares method is used if there is evidence of cross-equation correlation. We will look at this model in more detail in our discussion of seemingly unrelated regression models (SUR) in Chapter 8.
Dependent variables can be continuous or discrete. In the Grunfeld investment model, the variable Iit is continuous. However, discrete responses are also very common. Consider an example where a credit card company solicits potential customers via mail. The response of the consumer can be classified as being equal to 1 or 0 depending on whether the consumer chooses to respond to the mail or not. Clearly, the outcome of the study (a consumer responds or not) isa discrete random variable. In this example, the response is a binary random variable. We will look at modeling discrete responses when we discuss discrete choice models in Chapter 10.
In general, a multiple regression model can be expressed as
where
y is the dependent variable,
β0,…,
βk are the
k + 1 unknown coefficients that need to be estimated,
x1,…,
xk are the
k independent or explanatory variables, and
ε is random error. Notice that the model is linear in parameters
β0,…,
βk and is therefore called a linear model. Linearity refers to how the parameters enter the model. For instance, the model
is also a linear model. However, the exponential model
y =
β0 exp(–
xβ1) is a nonlinear model since the parameter
β1 enters the model in a nonlinear fashion through the exponential function.
1.1.1 Interpretation of the Parameters
One of the assumptions (to be discussed later) for the linear model is that the conditional expectation
E(
ε|
x1, …,
xk) equals zero. Under this assumption, the expectation,
E(
y|
x1,…,
xk) can be written as
That is, the regression model can be interpreted as the conditional expectation of y for given values of the explanatory variables
x1,
…, xk. In the Grunfeld example, we could discuss the expected investment for a given firm for known values of the firm's true market value and value of its capital. The intercept term,
β0, gives the expected value of y when all the explanatory variables are set at zero. In practice, this rarely makes sense since it is very uncommon to observe values of all the explanatory variables equal to zero. Furthermore, the expected value of y under such a case will often yield impossible results. The coefficient
βk is interpreted as the ...
