Stationary process

In mathematics and statistics, a stationary process (a.k.a. a strict(ly) stationary process or strong(ly) stationary process) is a stochastic process whose joint probability distribution does not change when shifted in time. Consequently, parameters such as mean and variance, if they are present, also do not change over time.

Since stationarity is an assumption underlying many statistical procedures used in time series analysis, non-stationary data is often transformed to become stationary. The most common cause of violation of stationarity are trends in mean, which can be due either to the presence of a unit root or of a deterministic trend. In the former case of a unit root, stochastic shocks have permanent effects and the process is not mean-reverting. In the latter case of a deterministic trend, the process is called a trend stationary process, and stochastic shocks have only transitory effects which are mean-reverting (i.e., the mean returns to its long-term average, which changes deterministically over time according to the trend).

A trend stationary process is not strictly stationary, but can easily be transformed into a stationary process by removing the underlying trend, which is solely a function of time. Similarly, processes with one or more unit roots can be made stationary through differencing. An important type of non-stationary process that does not include a trend-like behavior is a cyclostationary process, which is a stochastic process that varies cyclically with time.

A "stationary process" is not the same thing as a "process with a stationary distribution". Indeed, there are further possibilities for confusion with the use of the term "stationary" in the context of stochastic processes; for example a "time-homogeneous" Markov chain is sometimes said to have "stationary transition probabilities". Besides, all stationary Markov random processes are time-homogeneous.

Definition

Formally, let $\left\{X_{t}\right\}$ be a stochastic process and let $F_{{X}}(x_{{t_{1}+\tau }},\ldots ,x_{{t_{k}+\tau }})$ represent the cumulative distribution function of the joint distribution of $\left\{X_{t}\right\}$ at times $t_{1}+\tau ,\ldots ,t_{k}+\tau$ . Then, $\left\{X_{t}\right\}$ is said to be strictly (or strongly) stationary if, for all ${k}$ , for all ${\tau }$ , and for all $t_{1},\ldots ,t_{k}$ ,

F_{{X}}(x_{{t_{1}+\tau }},\ldots ,x_{{t_{k}+\tau }})=F_{{X}}(x_{{t_{1}}},\ldots ,x_{{t_{k}}}).

Since $\tau$ does not affect $F_{X}(\cdot )$ , $F_{{X}}$ is not a function of time.

Examples

Two simulated time series processes, one stationary and the other non-stationary, are shown above. The augmented Dickey–Fuller (ADF) test statistic is reported for each process; non-stationarity cannot be rejected for the second process at a 5% significance level.

As an example, white noise is stationary. The sound of a cymbal clashing, if hit only once, is not stationary because the acoustic power of the clash (and hence its variance) diminishes with time. However, it would be possible to invent a stochastic process describing when the cymbal is hit, such that the overall response would form a stationary process. For example, if the cymbal were hit at moments in time corresponding to a homogeneous Poisson process, the overall response would be stationary.

An example of a discrete-time stationary process where the sample space is also discrete (so that the random variable may take one of N possible values) is a Bernoulli scheme. Other examples of a discrete-time stationary process with continuous sample space include some autoregressive and moving average processes which are both subsets of the autoregressive moving average model. Models with a non-trivial autoregressive component may be either stationary or non-stationary, depending on the parameter values, and important non-stationary special cases are where unit roots exist in the model.

Let Y be any scalar random variable, and define a time-series { X_t }, by

X_{t}=Y\qquad {\text{ for all }}t.

Then { X_t } is a stationary time series, for which realisations consist of a series of constant values, with a different constant value for each realisation. A law of large numbers does not apply on this case, as the limiting value of an average from a single realisation takes the random value determined by Y, rather than taking the expected value of Y.

As a further example of a stationary process for which any single realisation has an apparently noise-free structure, let Y have a uniform distribution on (0,2π] and define the time series { X_t } by

X_{t}=\cos(t+Y)\quad {\text{ for }}t\in {\mathbb {R}}.

Then { X_t } is strictly stationary.

Weaker forms of stationarity

Weak or wide-sense stationarity

A weaker form of stationarity commonly employed in signal processing is known as weak-sense stationarity, wide-sense stationarity (WSS), covariance stationarity, or second-order stationarity. WSS random processes only require that 1st moment (i.e. the mean) and autocovariance do not vary with respect to time. Any strictly stationary process which has a mean and a covariance is also WSS.

So, a continuous time random process x(t) which is WSS has the following restrictions on its mean function

{\mathbb {E}}[x(t)]=m_{x}(t)=m_{x}(t+\tau )\,\,{\text{ for all }}\,\tau \in {\mathbb {R}}

and autocovariance function

{\mathbb {E}}[(x(t_{1})-m_{x}(t_{1}))(x(t_{2})-m_{x}(t_{2}))]=C_{x}(t_{1},t_{2})=C_{x}(t_{1}+(-t_{2}),t_{2}+(-t_{2}))=C_{x}(t_{1}-t_{2},0).

The first property implies that the mean function m_x(t) must be constant. The second property implies that the covariance function depends only on the difference between $t_{1}$ and $t_{2}$ and only needs to be indexed by one variable rather than two variables. Thus, instead of writing,

\,\!C_{x}(t_{1}-t_{2},0)\,

the notation is often abbreviated and written as:

C_{x}(\tau )\,\!{\mbox{ where }}\tau =t_{1}-t_{2}.

This also implies that the autocorrelation depends only on $\tau =t_{1}-t_{2}$ , that is

\,\!R_{x}(t_{1},t_{2})=R_{x}(t_{1}-t_{2}).

The main advantage of wide-sense stationarity is that it places the time-series in the context of Hilbert spaces. Let H be the Hilbert space generated by {x(t)} (that is, the closure of the set of all linear combinations of these random variables in the Hilbert space of all square-integrable random variables on the given probability space). By the positive definiteness of the autocovariance function, it follows from Bochner's theorem that there exists a positive measure μ on the real line such that H is isomorphic to the Hilbert subspace of L²(μ) generated by {e^{−2πiξ⋅t}}. This then gives the following Fourier-type decomposition for continuous time stationary stochastic process: there exists a stochastic process ω_ξ with orthogonal increments such that, for all t

x(t)=\int e^{{-2\pi i\lambda \cdot t}}d\omega _{{\lambda }},

where the integral on the right hand side is interpreted in a suitable (Riemann) sense. Same result holds for a discrete-time stationary process, with the spectral measure now defined on the unit circle.

When processing WSS random signals with linear, time-invariant (LTI) filters, it is helpful to think of the correlation function as a linear operator. Since it is a circulant operator (depends only on the difference between the two arguments), its eigenfunctions are the Fourier complex exponentials. Additionally, since the eigenfunctions of LTI operators are also complex exponentials, LTI processing of WSS random signals is highly tractable—all computations can be performed in the frequency domain. Thus, the WSS assumption is widely employed in signal processing algorithms.

Other terminology

The terminology used for types of stationarity other than strict stationarity can be rather mixed. Some examples follow.

Priestley uses stationary up to order m if conditions similar to those given here for wide sense stationarity apply relating to moments up to order m.^[1]^[2] Thus wide sense stationarity would be equivalent to "stationary to order 2", which is different from the definition of second-order stationarity given here.

Honarkhah and Caers also use the assumption of stationarity in the context of multiple-point geostatistics, where higher n-point statistics are assumed to be stationary in the spatial domain.^[3]

Tahmasebi and Sahimi have presented an adaptive Shannon-based methodology that can be used for modeling of any non-stationary systems.^[4]

Differencing

One way to make a time series stationary is to compute the differences between consecutive observations. This is known as differencing.

Transformations such as logarithms can help to stabilize the variance of a time series. Differencing can help stabilize the mean of a time series by removing changes in the level of a time series, and so eliminating trend and seasonality.

One of the ways for identifying non-stationary times series is the ACF plot. For a stationary time series, the ACF will drop to zero relatively quickly, while the ACF of non-stationary data decreases slowly.^[5]

References

↑ Priestley, M. B. (1981). Spectral Analysis and Time Series. Academic Press. ISBN 0-12-564922-3.
↑ Priestley, M. B. (1988). Non-linear and Non-stationary Time Series Analysis. Academic Press. ISBN 0-12-564911-8.
↑ Honarkhah, M.; Caers, J. (2010). "Stochastic Simulation of Patterns Using Distance-Based Pattern Modeling". Mathematical Geosciences. 42 (5): 487–517. doi:10.1007/s11004-010-9276-7.
↑ Tahmasebi, P.; Sahimi, M. (2015). "Reconstruction of nonstationary disordered materials and media: Watershed transform and cross-correlation function" (PDF). Physical Review E. 91 (3). doi:10.1103/PhysRevE.91.032401.
↑ "8.1 Stationarity and differencing | OTexts". www.otexts.org. Retrieved 2016-05-18.

External links

Spectral decomposition of a random function (Springer)

Statistics

Descriptive statistics

Continuous data

Center	Mean arithmetic geometric harmonic Median Mode
Dispersion	Variance Standard deviation Coefficient of variation Percentile Range Interquartile range
Shape	Moments Skewness Kurtosis L-moments

Count data

Index of dispersion

Summary tables

Dependence

Graphics

Data collection

Study design	Population Statistic Effect size Statistical power Sample size determination Missing data
Survey methodology	Sampling stratified cluster Standard error Opinion poll Questionnaire
Controlled experiments	Design control optimal Controlled trial Randomized Random assignment Replication Blocking Interaction Factorial experiment
Uncontrolled studies	Observational study Natural experiment Quasi-experiment

Statistical inference

Statistical theory

Frequentist inference

Point estimation	Estimating equations Maximum likelihood Method of moments M-estimator Minimum distance Unbiased estimators Mean-unbiased minimum-variance Rao–Blackwellization Lehmann–Scheffé theorem Median unbiased Plug-in
Interval estimation	Confidence interval Pivot Likelihood interval Prediction interval Tolerance interval Resampling Bootstrap Jackknife
Testing hypotheses	1- & 2-tails Power Uniformly most powerful test Permutation test Randomization test Multiple comparisons
Parametric tests	Likelihood-ratio Wald Score

Specific tests

Z (normal) Student's t-test F
Goodness of fit	Chi-squared Kolmogorov–Smirnov Anderson–Darling Lilliefors Jarque–Bera Normality (Shapiro–Wilk) Likelihood-ratio test Model selection Cross validation AIC BIC
Rank statistics	Sign Sample median Signed rank (Wilcoxon) Hodges–Lehmann estimator Rank sum (Mann–Whitney) Nonparametric anova 1-way (Kruskal–Wallis) 2-way (Friedman) Ordered alternative (Jonckheere–Terpstra)

Bayesian inference

Correlation	Pearson product-moment Partial correlation Confounding variable Coefficient of determination
Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models Multivariate adaptive regression splines (MARS)
Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression
Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust Heteroscedasticity Homoscedasticity
Generalized linear model	Exponential families Logistic (Bernoulli) / Binomial / Poisson regressions
Partition of variance	Analysis of variance (ANOVA, anova) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

Categorical

Multivariate

Time-series

General	Decomposition Trend Stationarity Seasonal adjustment Exponential smoothing Cointegration Structural break Granger causality
Specific tests	Dickey–Fuller Johansen Q-statistic (Ljung–Box) Durbin–Watson Breusch–Godfrey
Time domain	Autocorrelation (ACF) partial (PACF) Cross-correlation (XCF) ARMA model ARIMA model (Box–Jenkins) Autoregressive conditional heteroskedasticity (ARCH) Vector autoregression (VAR)
Frequency domain	Spectral density estimation Fourier analysis Wavelet

Survival

Survival function	Kaplan–Meier estimator (product limit) Proportional hazards models Accelerated failure time (AFT) model First hitting time
Hazard function	Nelson–Aalen estimator
Test	Log-rank test

Applications

Biostatistics	Bioinformatics Clinical trials / studies Epidemiology Medical statistics
Engineering statistics	Chemometrics Methods engineering Probabilistic design Process / quality control Reliability System identification
Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population statistics Psychometrics
Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Category
Portal
Commons
WikiProject

This article is issued from Wikipedia. The text is licensed under Creative Commons - Attribution - Sharealike. Additional terms may apply for the media files.