Regression analysis is a method for testing hypotheses using interval or ratio level variables. The power of regression analysis lies in the fact that we can easily test multiple hypotheses with a number of independent variables simultaneously.
Essentially, regression analysis attempts to decribe the nature of the relationship between a dependent variable and independent variable(s) in the form of an equation (known as a regression model):
=bx+a,
Where: = the predicted value of the dependent variable;
x = the value of the independent variable; a = the intercept, or constant; and b = the slope
We call the predicted value of the dependent variable because what we are trying to do with regression analysis is predict the value of the dependent variable for each of the cases in our study using the independent variable(s) as an aid in predicting.
In this way, regression analysis is very similar to the "guessing game" analogy we used in calculating Lambda and in discussing P.R.E. interpretations of measures of association.
Key Pieces of Information in Regression Output
Rē or R-Square
Rough equivalent of PRE interpretation of measures of association. It is interpreted as a percentage, and it tells you how much of the variation in the dependent variable you have "explained" with the independent variable(s) in your model.
b, or Unstandardized Regression Coefficient
This is used in generating the predicted values for the dependent variable. Essentially, it indicates the strength and direction of the relationship between the specific independent variable and the dependent variable. However, the "strength" is not interpreted in the same way that we do nominal or ordinal measures of association. In reality, what it tells us is the expected increase (or decrease) in the value of the dependent variable associated with a 1 unit increase in the independent variable.
BETA, or Standardized Regression Coefficient
Like b, but has a much more straightforward interpretation. Essentially, it measures the strength and direction of the relationship between the independent and dependent variable (very similar to Pearson's r). It also has the additional feature of allowing us to compare the relative importance of each of the independent variables in the model in "explaining" the dependent variable.
t and/or significance (or probability) value
t is the functional equivalent of Chi-Square. It's function is to generate the probability value that we use to determine if the relationship between the independent and dependent variable is statistically significant (i.e. can we reject the null hypothesis?)
=bx+a,