Mean square quantization error

From Wikipedia, the free encyclopedia

Mean square quantization error (MSQE) is a figure of merit for the process of analog to digital conversion.

As the input is varied, the input's value is recorded when the digital output changes. For each digital output, the input's difference from ideal is normalized to the value of the least significant bit, then squared, summed, and normalized to the number of samples.