Heteroscedasticity

In statistics, a collection of random variables is heteroscedastic, or heteroskedastic, if there are sub-populations that have different variabilities than others. Here "variability" could be quantified by the variance or any other measure of statistical dispersion. Thus heteroscedasticity is the absence of homoscedasticity.

The possible existence of heteroscedasticity is a major concern in the application of regression analysis, including the analysis of variance, because the presence of heteroscedasticity can invalidate statistical tests of significance that assume the effect and residual (error) variances are uncorrelated and normally distributed.

The term means "differing variance" and comes from the Greek "hetero" ('different') and "skedasis" ('dispersion').

Contents 1 Definition 2 Consequences 3 Detection 4 Fixes 5 Examples 6 References 7 Further reading

Definition

Suppose there is a sequence of random variables {Y_t}_t=1ⁿ and a sequence of vectors of random variables, {X_t}_t=1ⁿ. In dealing with conditional expectations of Y_t given X_t, the sequence {Y_t}_t=1ⁿ is said to be heteroskedastic if the conditional variance of Y_t given X_t, changes with t. Some authors refer to this as conditional heteroscedasticity to emphasize the fact that it is the sequence of conditional variance that changes and not the unconditional variance. In fact it is possible to observe conditional heteroscedasticity even when dealing with a sequence of unconditional homoscedastic random variables, however, the opposite does not hold.

When using some statistical techniques, such as ordinary least squares (OLS), a number of assumptions are typically made. One of these is that the error term has a constant variance. This might not be true even if the error term is assumed to be drawn from identical distributions.

For example, the error term could vary or increase with each observation, something that is often the case with cross-sectional or time series measurements. Heteroscedasticity is often studied as part of econometrics, which frequently deals with data exhibiting it. White's influential paper^[1] used "heteroskedasticity" instead of "heteroscedasticity" whereas the latter has been used in later works.^[2]

Consequences

Heteroscedasticity does not cause ordinary least squares coefficient estimates to be biased, although it can cause ordinary least squares estimates of the variance (and, thus, standard errors) of the coefficients to be biased, possibly above or below the true or population variance. Thus, regression analysis using heteroscedastic data will still provide an unbiased estimate for the relationship between the predictor variable and the outcome, but standard errors and therefore inferences obtained from data analysis are suspect. Biased standard errors lead to biased inference, so results of hypothesis tests are possibly wrong. As an example of the consequence of biased standard error estimation which OLS will produce if heteroskedasticity is present, a researcher might find compelling results against the rejection of a null hypothesis at a given significance level as statistically significant, when that null hypothesis was actually uncharacteristic of the actual population (i.e., make a type II error).

It is widely known that, under certain assumptions, the OLS estimator has a normal asymptotic distribution when properly normalized and centered (even when the data does not come from a normal distribution). This result is used to justify using a normal distribution, or a chi square distribution (depending on how the test statistic is calculated), when conducting a hypothesis test. This holds even under heteroscedasticity. More precisely, the OLS estimator in the presence of heteroscedasticity is asymptotically normal, when properly normalized and centered, with a variance-covariance matrix that differs from the case of homoscedasticity. In 1980, White^[1] proposed a consistent estimator for the variance-covariance matrix of the asymptotic distribution of the OLS estimator. This validates the use of hypothesis testing using OLS estimators and White's variance-covariance estimator under heteroscedasticity.

Heteroscedasticity is also a major practical issue encountered in ANOVA problems.^[3] The F test can still be used in some circumstances.^[4]

However, it has been said that students in econometrics should not overreact to heteroskedasticity.^[2] One author wrote, "unequal error variance is worth correcting only when the problem is severe."^[5] And another word of caution was in the form, "heteroscedasticity has never been a reason to throw out an otherwise good model."^[6][2]

With the advent of heteroscedasticity-consistent standard errors allowing for inference without specifying the conditional second moment of error term, testing conditional homoscedasticity is not as important as in the past.

The econometrician Robert Engle won the 2003 Nobel Memorial Prize for Economics for his studies on regression analysis in the presence of heteroscedasticity, which led to his formulation of the autoregressive conditional heteroscedasticity (ARCH) modeling technique.

Detection

There are several methods to test for the presence of heteroscedasticity:

• Park test (1966)^[7]
• Glejser test (1969)^[8][9]
• White test (1980)
• Breusch–Pagan test
• Goldfeld–Quandt test
• Cook–Weisberg test
• Harrison–McCabe test
• Brown–Forsythe test
• Levene test

These methods consist, in general, in performing hypothesis tests. These tests consist of a statistic (a mathematical expression), a hypothesis that is going to be tested (the null hypothesis), an alternative hypothesis, and a distributional statement about the statistic (the mathematical expression).

Many introductory statistics and econometrics books, for pedagogical reasons, present these tests under the assumption that the data set in hand comes from a normal distribution. A great misconception is the thought that this assumption is necessary. Most of the methods of detecting heteroscedasticity outlined above can be used even when the data do not come from a normal distribution. In most cases, this assumption can be relaxed by using asymptotic distributions which can be obtained from asymptotic theory.

Fixes

There are three common corrections for heteroscedasticity:

• View Logged data. Unlogged series that are growing exponentially often appear to have increasing variability as the series rises over time. The variability in percentage terms may, however, be rather stable.
• Use a different specification for the model (different X variables, or perhaps non-linear transformations of the X variables).
• Apply a weighted least squares estimation method, in which OLS is applied to transformed or weighted values of X and Y. The weights vary over observations, usually depending on the changing error variances. In one variation the weights are directly related to the magnitude of the dependent variable, and this corresponds to least squares percentage regression. ^[10]
• Heteroscedasticity-consistent standard errors (HCSE), while still biased, improve upon OLS estimates (White 1980). HCSE is a consistent estimator of standard errors in regression models with heteroskedasticity. The White method corrects for heteroscedasticity without altering the values of the coefficients. This method may be superior to regular OLS because if heteroscedasticity is present it corrects for it, however, if the data is homoscedastistic, the standard errors are equivalent to conventional standard errors estimated by ols. Several modifications of the White method of computing heteroscedasticity-consistent standard errors have been proposed as corrections with superior finite sample properties.

Examples

Heteroscedasticity often occurs when there is a large difference among the sizes of the observations.

• A classic example of heteroscedasticity is that of income versus expenditure on meals. As one's income increases, the variability of food consumption will increase. A poorer person will spend a rather constant amount by always eating less expensive food; a wealthier person may occasionally buy inexpensive food and at other times eat expensive meals. Those with higher incomes display a greater variability of food consumption.
• Imagine you are watching a rocket take off nearby and measuring the distance it has traveled once each second. In the first couple of seconds your measurements may be accurate to the nearest centimeter, say. However, 5 minutes later as the rocket recedes into space, the accuracy of your measurements may only be good to 100 m, because of the increased distance, atmospheric distortion and a variety of other factors. The data you collect would exhibit heteroscedasticity.

References

Further reading

Most statistics textbooks will include at least some material on heteroscedasticity. Some examples are:

• Studenmund AH. Using Econometrics (2nd ed.). ISBN 0-673-52125-7.  (devotes a chapter to heteroscedasticity)
• Verbeek, Marno (2004) A Guide to Modern Econometrics, 2. ed., Chichester: John Wiley & Sons
• Greene, W.H. (1993) Econometric Analysis, Prentice–Hall, ISBN 0-13-013297-7, an introductory but thorough general text, considered the standard for a pre-doctorate university Econometrics course;
• Hamilton, J.D. (1994), Time Series Analysis, Princeton University Press ISBN 0-691-04289-6, the text of reference for historical series analysis; it contains an introduction to ARCH models.
• Vinod, H.D. (2008) Hands On Intermediate Econometrics Using R: Templates for Extending Dozens of Practical Examples.World Scientific Publishers: Hackensack, NJ. ISBN 978-9812818850 (Section 2.8 provides R snippets)

Statistics   Descriptive statistics Continuous data Location Mean (Arithmetic, Geometric, Harmonic) Median Mode Dispersion Range Standard deviation Coefficient of variation Percentile Interquartile range Shape Variance Skewness Kurtosis Moments L-moments Count data Index of dispersion Summary tables Grouped data Frequency distribution Contingency table Dependence Pearson product-moment correlation Rank correlation (Spearman's rho, Kendall's tau) Partial correlation Scatter plot Statistical graphics Bar chart Biplot Box plot Control chart Correlogram Forest plot Histogram Q-Q plot Run chart Scatter plot Stemplot Radar chart   Data collection Designing studies Effect size Standard error Statistical power Sample size determination Survey methodology Sampling Stratified sampling Opinion poll Questionnaire Controlled experiment Design of experiments Randomized experiment Random assignment Replication Blocking Factorial experiment Optimal design Uncontrolled studies Natural experiment Quasi-experiment Observational study   Statistical inference Statistical theory Sampling distribution Sufficient statistic Meta-analysis Bayesian inference Bayesian probability Prior Posterior Credible interval Bayes factor Bayesian estimator Maximum posterior estimator Frequentist inference Confidence interval Hypothesis testing Likelihood-ratio Specific tests Z-test (normal) Student's t-test F-test Pearson's chi-squared test Wald test Mann–Whitney U Shapiro–Wilk Signed-rank Kolmogorov–Smirnov test General estimation Bias Robustness Efficiency Maximum likelihood Method of moments Minimum distance Density estimation   Correlation and regression analysis Correlation Pearson product-moment correlation Partial correlation Confounding variable Coefficient of determination Regression analysis Errors and residuals Regression model validation Mixed effects models Simultaneous equations models Linear regression Simple linear regression Ordinary least squares General linear model Bayesian regression Non-standard predictors Nonlinear regression Nonparametric Semiparametric Isotonic Robust Generalized linear model Exponential families Logistic (Bernoulli) Binomial Poisson Partition of variance Analysis of variance (ANOVA) Analysis of covariance Multivariate ANOVA Degrees of freedom   Categorical, multivariate, time-series, or survival analysis Categorical data Cohen's kappa Contingency table Graphical model Log-linear model McNemar's test Multivariate statistics Multivariate regression Principal components Factor analysis Cluster analysis Copulas Time series analysis Decomposition (Trend, Stationary process) ARMA model ARIMA model Vector autoregression Spectral density estimation Survival analysis Survival function Kaplan–Meier Logrank test Failure rate Proportional hazards models Accelerated failure time model   Applications Biostatistics Bioinformatics Biometrics Clinical trials & studies Epidemiology Medical statistics Engineering statistics Chemometrics Methods engineering Probabilistic design Process & Quality control Reliability System identification Social statistics Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population Psychometrics Spatial statistics Cartography Environmental statistics Geographic information system Geostatistics Kriging Category Portal Outline Index