Spectral density estimation

For the statistical concept, see probability density estimation.

For a broader coverage related to this topic, see Spectral density.

In statistical signal processing, the goal of spectral density estimation (SDE) is to estimate the spectral density (also known as the power spectral density) of a random signal from a sequence of time samples of the signal. Intuitively speaking, the spectral density characterizes the frequency content of the signal. One purpose of estimating the spectral density is to detect any periodicities in the data, by observing peaks at the frequencies corresponding to these periodicities.

SDE should be distinguished from the field of frequency estimation, which assumes that a signal is composed of a limited (usually small) number of generating frequencies plus noise and seeks to find the location and intensity of the generated frequencies. SDE makes no assumption on the number of components and seeks to estimate the whole generating spectrum.

Overview

Example of voice waveform and its frequency spectrum

A triangle wave pictured in the time domain (top) and frequency domain (bottom). The fundamental frequency component is at 220 Hz (A2).

Spectrum analysis, also referred to as frequency domain analysis or spectral density estimation, is the technical process of decomposing a complex signal into simpler parts. As described above, many physical processes are best described as a sum of many individual frequency components. Any process that quantifies the various amounts (e.g. amplitudes, powers, intensities, or phases), versus frequency can be called spectrum analysis.

Spectrum analysis can be performed on the entire signal. Alternatively, a signal can be broken into short segments (sometimes called frames), and spectrum analysis may be applied to these individual segments. Periodic functions (such as $\sin (t)$ ) are particularly well-suited for this sub-division. General mathematical techniques for analyzing non-periodic functions fall into the category of Fourier analysis.

The Fourier transform of a function produces a frequency spectrum which contains all of the information about the original signal, but in a different form. This means that the original function can be completely reconstructed (synthesized) by an inverse Fourier transform. For perfect reconstruction, the spectrum analyzer must preserve both the amplitude and phase of each frequency component. These two pieces of information can be represented as a 2-dimensional vector, as a complex number, or as magnitude (amplitude) and phase in polar coordinates (i.e., as a phasor). A common technique in signal processing is to consider the squared amplitude, or power; in this case the resulting plot is referred to as a power spectrum.

In practice, nearly all software and electronic devices that generate frequency spectra apply a fast Fourier transform (FFT), which is a specific mathematical approximation to the full integral solution. Formally stated, the FFT is a method for computing the discrete Fourier transform of a sampled signal.

Because of reversibility, the Fourier transform is called a representation of the function, in terms of frequency instead of time; thus, it is a frequency domain representation. Linear operations that could be performed in the time domain have counterparts that can often be performed more easily in the frequency domain. Frequency analysis also simplifies the understanding and interpretation of the effects of various time-domain operations, both linear and non-linear. For instance, only non-linear or time-variant operations can create new frequencies in the frequency spectrum.

The Fourier transform of a stochastic (random) waveform (noise) is also random. Some kind of averaging is required in order to create a clear picture of the underlying frequency content (frequency distribution). Typically, the data is divided into time-segments of a chosen duration, and transforms are performed on each one. Then the magnitude or (usually) squared-magnitude components of the transforms are summed into an average transform. This is a very common operation performed on digitally sampled time-domain data, using the discrete Fourier transform. This type of processing is called Welch's method. When the result is flat, it is commonly referred to as white noise. However, such processing techniques often reveal spectral content even among data which appears noisy in the time domain.

Periodogram

Suppose that a signal is sampled at $N$ different times, with the samples uniformly spaced by $\Delta t$ , giving values $x_n$ . Since the power spectral density of a continuous function defined on the entire real line is the modulus squared of its Fourier transform, the simplest technique to estimate the spectrum is the periodogram, given by the modulus squared of the discrete Fourier transform,

S(f)=\frac{\Delta t}{N} \left| \sum_{n=0}^{N-1} x_n e^{-i2\pi n f} \right|^2, \qquad -\frac{1}{2\Delta t} < f \le \frac{1}{2\Delta t}

where $1/(2\Delta t)$ is the Nyquist frequency. The name "periodogram" was coined by Arthur Schuster in 1898.^[1]

Despite the simplicity of the periodogram, the method suffers from severe deficiencies. It is an inconsistent estimator because it does not converge to the true spectral density as $N\rightarrow\infty$ . It exhibits very high spectral leakage although this can be reduced by multiplying $x_n$ by a window function. In the presence of additive noise, the estimate has a positive bias.

Techniques

Many different techniques for spectral estimation have been developed to overcome the problems of the naive periodogram. These techniques can generally be divided into non-parametric and parametric methods. The non-parametric approaches explicitly estimate the covariance or the spectrum of the process without assuming that the process has any particular structure. The periodogram itself is a non-parametric approach, and is essentially equivalent to the Fourier transform of the biased autocovariance convolved with a Fejér kernel. Some of the most common estimators in use for basic applications (e.g. Welch's method) are non-parametric estimators closely related to the periodogram. By contrast, the parametric approaches assume that the underlying stationary stochastic process has a certain structure which can be described using a small number of parameters (for example, using an auto-regressive or moving average model). In these approaches, the task is to estimate the parameters of the model that describes the stochastic process.

Following is a partial list of non-parametric spectral density estimation techniques:

Periodogram, the basic modulus-squared of the discrete Fourier transform
Bartlett's method is the average of the periodograms taken of multiple segments of the signal to reduce variance of the spectral density estimate
Welch's method a windowed version of Bartlett's method that uses overlapping segments
Multitaper is a periodogram-based method that uses multiple tapers, or windows, to form independent estimates of the spectral density to reduce variance of the spectral density estimate
Least-squares spectral analysis, based on least squares fitting to known frequencies
Non-uniform discrete Fourier transform is used when the signal samples are unevenly spaced in time
Singular spectrum analysis is a nonparametric method that uses a singular value decomposition of the covariance matrix to estimate the spectral density
Short-time Fourier transform

Below is a partial list of parametric techniques:

Autoregressive model (AR) estimation, which assumes that the nth sample is correlated with the previous p samples.
Moving-average model (MA) estimation, which assumes that the nth sample is correlated with noise terms in the previous p samples.
Autoregressive moving average (ARMA) estimation, which generalizes the AR and MA models.
Maximum entropy spectral estimation is an all-poles method useful for SDE when singular spectral features, such as sharp peaks, are expected.

Parametric estimation

In parametric spectral estimation, one assumes that the signal is modeled by a stationary process which has a spectral density function (SDF) $S(f;a_1,\ldots,a_p)$ that is a function of the frequency $f$ and $p$ parameters $a_1,\ldots,a_p$ .^[2] The estimation problem then becomes one of estimating these parameters.

The most common form of parametric SDF estimate uses as a model an autoregressive model $AR(p)$ of order $p$ .^[2]^:392 A signal sequence $\{Y_t\}$ obeying a zero mean $AR(p)$ process satisfies the equation

Y_t = \phi_1Y_{t-1} + \phi_2Y_{t-2} + \cdots + \phi_pY_{t-p} + \epsilon_t,

where the $\phi_1,\ldots,\phi_p$ are fixed coefficients and $\epsilon_t$ is a white noise process with zero mean and innovation variance $\sigma^2_p$ . The SDF for this process is

S(f;\phi_1,\ldots,\phi_p,\sigma^2_p) = \frac{\sigma^2_p\Delta t}{\left| 1 - \sum_{k=1}^p \phi_k e^{-2i\pi f k \Delta t}\right|^2} \qquad |f| < f_N,

with $\Delta t$ the sampling time interval and $f_N$ the Nyquist frequency.

There are a number of approaches to estimating the parameters $\phi_1,\ldots,\phi_p,\sigma^2_p$ of the $AR(p)$ process and thus the spectral density:^[2]^:452-453

The Yule-Walker estimators are found by recursively solving the Yule-Walker equations for an $AR(p)$ process
The Burg estimators are found by treating the Yule-Walker equations as a form of ordinary least squares problem. The Burg estimators are generally considered superior to the Yule-Walker estimators.^[2]^:452 Burg associated these with maximum entropy spectral estimation.^[3]
The forward-backward least-squares estimators treat the $AR(p)$ process as a regression problem and solves that problem using forward-backward method. They are competitive with the Burg estimators.
The maximum likelihood estimators assume the white noise is a Gaussian process and estimates the parameters using a maximum likelihood approach. This involves a nonlinear optimization and is more complex than the first three.

Alternative parametric methods include fitting to a moving average model (MA) and to a full autoregressive moving average model (ARMA).

Frequency estimation

Frequency estimation is the process of estimating the complex frequency components of a signal in the presence of noise given assumptions about the number of the components.^[4] This contrasts with the general methods above, which do not make prior assumptions about the components.

Finite number of tones

A typical model for a signal $x(n)$ consists of a sum of $p$ complex exponentials in the presence of white noise, $w(n)$

x(n) = \sum_{i=1}^p A_i e^{j n \omega_i} + w(n)

The power spectral density of $x(n)$ is composed of $p$ impulse functions in addition to the spectral density function due to noise.

The most common methods for frequency estimation involve identifying the noise subspace to extract these components. These methods are based on eigen decomposition of the autocorrelation matrix into a signal subspace and a noise subspace. After these subspaces are identified, a frequency estimation function is used to find the component frequencies from the noise subspace. The most popular methods of noise subspace based frequency estimation are Pisarenko's method, the multiple signal classification (MUSIC) method, the eigenvector method, and the minimum norm method.

Pisarenko's method

\hat P_{PHD}(e^{j \omega}) = \frac{1}{|\mathbf{e}^{H}\mathbf{v}_{min}|^2}

MUSIC

\hat P_{MU}(e^{j \omega}) = \frac{1}{\sum_{i=p+1}^{M} |\mathbf{e}^{H} \mathbf{v}_i|^2}

Eigenvector method

\hat P_{EV}(e^{j \omega}) = \frac{1}{\sum_{i=p+1}^{M}\frac{1}{\lambda_i} |\mathbf{e}^H \mathbf{v}_i|^2}

Minimum norm method

\hat P_{MN}(e^{j \omega}) = \frac{1}{|\mathbf{e}^H \mathbf{a}|^2} ; \mathbf{a} = \lambda \mathbf{P}_n \mathbf{u}_1

Single tone

If one only wants to estimate the single loudest frequency, one can use a pitch detection algorithm. If the dominant frequency changes over time, then the problem becomes the estimation of the instantaneous frequency as defined in the time–frequency representation. Methods for instantaneous frequency estimation include those based on the Wigner-Ville distribution and higher order ambiguity functions.^[5]

If one wants to know all the (possibly complex) frequency components of a received signal (including transmitted signal and noise), one uses a discrete Fourier transform or some other Fourier-related transform.

References

↑ Schuster, A., "On the investigation of hidden periodicities with application to a supposed 26 day period of meteorological phenomena," Terrestrial Magnetism, 3, 13-41, 1898.
1 2 3 4 Donald B. Percival and Andrew T. Walden (1992). Spectral Analysis for Physical Applications. Cambridge University Press. ISBN 9780521435413.
↑ Burg, J.P. (1967) "Maximum Entropy Spectral Analysis", Proceedings of the 37th Meeting of the Society of Exploration Geophysicists, Oklahoma City, Oklahoma.
↑ Hayes, Monson H., Statistical Digital Signal Processing and Modeling, John Wiley & Sons, Inc., 1996. ISBN 0-471-59431-8.
↑ Lerga, Jonatan. "Overview of Signal Instantaneous Frequency Estimation Methods" (PDF). University of Rijeka. Retrieved 22 March 2014.

Study design	Effect size Standard error Statistical power Sample size determination

Survey methodology	Sampling stratified cluster Opinion poll Questionnaire

Controlled experiments	Design control optimal Controlled trial Randomized Random assignment Replication Blocking Factorial experiment

Uncontrolled studies	Observational study Natural experiment Quasi-experiment

Statistical inference

Statistical theory

Frequentist inference

Confidence interval Testing hypotheses Power

Unbiased estimators	Mean unbiased minimum-variance Median unbiased

Biased estimators	Maximum likelihood Method of moments Minimum distance Density estimation

Parametric tests	Likelihood-ratio Wald Score

Specific tests

Z (normal) Student's t-test F Shapiro–Wilk Kolmogorov–Smirnov

Goodness of fit	Chi-squared G Sample source (Anderson–Darling) Sample normality (Shapiro–Wilk) Skewness / kurtosis normality (Jarque-Bera) Model comparison (Likelihood-ratio) Model quality (Akaike criterion)

Signed-rank	1-sample (Wilcoxon) 2-sample (Mann–Whitney U) 1-way anova (Kruskal–Wallis)

Bayesian inference

Correlation	Pearson product–moment Partial correlation Confounding variable Coefficient of determination

Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models Multivariate adaptive regression splines (MARS)

Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression

Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust Heteroscedasticity Homoscedasticity

Generalized linear model	Exponential families Logistic (Bernoulli) / Binomial / Poisson regressions

Partition of variance	Analysis of variance (ANOVA, anova) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical / Multivariate / Time-series / Survival analysis

Categorical

Multivariate

Time-series

General	Decomposition Trend Stationarity Seasonal adjustment Exponential smoothing Cointegration Structural break Granger causality

Specific tests	Dickey–Fuller Johansen Q-statistic (Ljung–Box) Durbin–Watson Breusch–Godfrey

Time domain	Autocorrelation (ACF) partial (PACF) Cross-correlation (XCF) ARMA model ARIMA model (Box–Jenkins) Autoregressive conditional heteroskedasticity (ARCH) Vector autoregression (VAR)

Frequency domain	Spectral density estimation Fourier analysis Wavelet

Survival

Survival function	Kaplan–Meier estimator (product limit) Proportional hazards models Accelerated failure time (AFT) model First hitting time

Hazard function	Nelson–Aalen estimator

Test	Log-rank test

Applications

Biostatistics	Bioinformatics Clinical trials / studies Epidemiology Medical statistics

Engineering statistics	Chemometrics Methods engineering Probabilistic design Process / quality control Reliability System identification

Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population Psychometrics

Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Category
Portal
Commons
WikiProject

This article is issued from Wikipedia - version of the Tuesday, November 17, 2015. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.