Plackett–Burman design

Plackett–Burman designs are experimental designs presented in 1946 by Robin L. Plackett and J. P. Burman while working in the British Ministry of Supply.^[1] Their goal was to find experimental designs for investigating the dependence of some measured quantity on a number of independent variables (factors), each taking L levels, in such a way as to minimize the variance of the estimates of these dependencies using a limited number of experiments. Interactions between the factors were considered negligible. The solution to this problem is to find an experimental design where each combination of levels for any pair of factors appears the same number of times, throughout all the experimental runs (refer to table). A complete factorial design would satisfy this criterion, but the idea was to find smaller designs.

For the case of two levels (L = 2), Plackett and Burman used the method found in 1933 by Raymond Paley for generating orthogonal matrices whose elements are all either 1 or −1 (Hadamard matrices). Paley's method could be used to find such matrices of size N for most N equal to a multiple of 4. In particular, it worked for all such N up to 100 except N = 92. If N is a power of 2, however, the resulting design is identical to a fractional factorial design, so Plackett–Burman designs are mostly used when N is a multiple of 4 but not a power of 2 (i.e. N = 12, 20, 24, 28, 36 …).^[3] If one is trying to estimate less than N parameters (including the overall average), then one simply uses a subset of the columns of the matrix.

For the case of more than two levels, Plackett and Burman rediscovered designs that had previously been given by Raj Chandra Bose and K. Kishen at the Indian Statistical Institute.^[4] Plackett and Burman give specifics for designs having a number of experiments equal to the number of levels L to some integer power, for L = 3, 4, 5, or 7.

When interactions between factors are not negligible, they are often confounded in Plackett–Burman designs with the main effects, meaning that the designs do not permit one to distinguish between certain main effects and certain interactions. This is called aliasing or confounding.

Extended uses

In 1993, Dennis Lin described a construction method via half-fractions of Plackett–Burman designs, using one column to take half of the rest of the columns.^[5] The resulting matrix, minus that column, is a "supersaturated design"^[6] for finding significant first order effects, under the assumption that few exist.

Box–Behnken designs can be made smaller, or very large ones constructed, by replacing the fractional factorials and incomplete blocks traditionally used for plan and seed matrices, respectively, with Plackett–Burmans. For example, a quadratic design for 30 variables requires a 30 column PB plan matrix of zeroes and ones, replacing the ones in each line using PB seed matrices of −1s and +1s (for 15 or 16 variables) wherever a one appears in the plan matrix, creating a 557 runs design with values, −1, 0, +1, to estimate the 496 parameters of a full quadratic model.

By equivocating certain columns with parameters to be estimated, Plackett–Burmans can also be used to construct mixed categorical and numerical designs, with interactions or high order effects, requiring no more than 4 runs more than the number of model parameters to be estimated. Sort on columns assigned to categorical variable A, defined as A = 1 + int(a·i /(max(i) + 0.00001)) where i is row number and a is A's number of values. Next sort on columns assigned to any other categorical variables and repeat as needed. Such designs, if large, may otherwise be incomputable by standard search techniques like D-optimality. For example, 13 variables averaging 3 values each could have well over a million combinations to search. To estimate the 105 parameters in a quadratic model of 13 variables, one must formally exclude from consideration or compute |X'X| for well over 10⁶C10², i.e. 3¹³C105, or roughly 10⁴⁸⁴ matrices.

4 to 48 runs, sorted to show half-fractions

P.B.4
+ + + 
+ – – 
– + – 
– – + 
P.B.8
+ + + + + + + 
+ + – – – – + 
+ – + + – – – 
+ – – – + + – 
– + + – + – – 
– + – + – + – 
– – + – – + + 
– – – + + – + 
P.B.12
+ + + + + + + + + + + 
+ + + + – – – + – – – 
+ + – – – + – – + – + 
+ – + – + + + – – – – 
+ – – + – – + – + + – 
+ – – – + – – + – + + 
– + + – – – + – – + + 
– + – + + + – – – + – 
– + – – + – + + + – – 
– – + + + – – – + – + 
– – + – – + – + + + – 
– – – + – + + + – – + 
P.B.16
+ + + + + + + + + + + + + + + 
+ + + – – – – – – – – + + + + 
+ + – + – – – – + + + – – – + 
+ + – – + + + + – – – – – – + 
+ – + + + – – + + – – + – – – 
+ – + – – + + – – + + + – – – 
+ – – + – + + – + – – – + + – 
+ – – – + – – + – + + – + + – 
– + + + – + – + – + – – + – – 
– + + – + – + – + – + – + – – 
– + – + + – + – – + – + – + – 
– + – – – + – + + – + + – + – 
– – + + – – + + – – + – – + + 
– – + – + + – – + + – – – + + 
– – – + + + – – – – + + + – + 
– – – – – – + + + + – + + – + 
P.B.20
+ + + + + + + + + + + + + + + + + + + 
+ + + – + – + – – – – + + – – + – – + 
+ + – + – + – – – – + + – – + – – + + 
+ + – + – – – – + + – – + – – + + + – 
+ + – – – – + + – – + – – + + + + – – 
+ – + + + + – + – + – – – – + + – – – 
+ – + – + – – – – + + – – + – – + + + 
+ – + – – + + + + – + – + – – – – + – 
+ – – + – – + + + + – + – + – – – – + 
+ – – – + + – – + – – + + + + – + – – 
– + + + + – + – + – – – – + + – – + – 
– + + + – + – + – – – – + + – – + – + 
– + + – – + – – + + + + – + – + – – – 
– + – – + + + + – + – + – – – – + + – 
– + – – + – – + + + + – + – + – – – + 
– – + + – – + – – + + + + – + – + – – 
– – + – – – – + + – – + – – + + + + + 
– – – + + + + – + – + – – – – + + – + 
– – – + + – – + – – + + + + – + – + – 
– – – – – + + – – + – – + + + + – + + 
P.B.24
+ + + + + + + + + + + + + + + + + + + + + + + 
+ + + – + – + + – – + + – – + – + – – – – – + 
+ + + – – + + – – + – + – – – – – + + + + – – 
+ + – + + – – + + – – + – + – – – – – + + + – 
+ + – + – + + – – + + – – + – + – – – – – + + 
+ + – – – – – + + + + – + – + + – – + + – – – 
+ – + + – – + – + – – – – – + + + + – + – + – 
+ – + – + + – – + + – – + – + – – – – – + + + 
+ – + – + – – – – – + + + + – + – + + – – + – 
+ – – + + – – + – + – – – – – + + + + – + – + 
+ – – + – + – – – – – + + + + – + – + + – – + 
+ – – – – + + + + – + – + + – – + + – – + – – 
– + + + + – + – + + – – + + – – + – + – – – – 
– + + + – + – + + – – + + – – + – + – – – – + 
– + + – – + – + – – – – – + + + + – + – + + – 
– + – + – – – – – + + + + – + – + + – – + + – 
– + – – + + – – + – + – – – – – + + + + – + + 
– + – – + – + – – – – – + + + + – + – + + – + 
– – + + + + – + – + + – – + + – – + – + – – – 
– – + + – – + + – – + – + – – – – – + + + + + 
– – + – – – – – + + + + – + – + + – – + + – + 
– – – + + + + – + – + + – – + + – – + – + – – 
– – – – + + + + – + – + + – – + + – – + – + – 
– – – – – – + + + + – + – + + – – + + – – + + 
P.B.28
+ + + + + + + + + + + + + + + + + + + + + + + + + + – 
+ + + – – + + + – + + – – + + + + – – – – – – – – + + 
+ + + – – – – – – – – + + + + – – + + + – + + – – + + 
+ + – + + – – + + + + – – – – – – – – + + + + – – + + 
+ + – + – – + + – – – + – – + + – + – – + – + – + – – 
+ + – + – – + – + – + – + + – + – – + + – – – + – – – 
+ + – – – + – – + + – + – – + – + – + – + + – + – – – 
+ – + + – + – – + + – – – + – – + + – + – – + – + – – 
+ – + – + + – + – – + + – – – + – – + + – + – – + – – 
+ – + – + – + + – + – – + + – – – + – – + + – + – – – 
+ – + – + – + – + – + – + – + – + – + – + – + – + – + 
+ – – + + + + – – – – – – – – + + + + – – + + + – + + 
+ – – + + + – + + – – + + + + – – – – – – – – + + + + 
+ – – – – – – – – + + + + – – + + + – + + – – + + + + 
– + + + + – – + + + – + + – – + + + + – – – – – – – + 
– + + + + – – – – – – – – + + + + – – + + + – + + – + 
– + + + – + + – – + + + + – – – – – – – – + + + + – + 
– + + – – + + + + – – – – – – – – + + + + – – + + + + 
– + – – + + – + – – + – + – + – + + – + – – + + – – – 
– + – – + + – – – + – – + + – + – – + – + – + – + + – 
– + – – + – + – + – + + – + – – + + – – – + – – + + – 
– – + + – + – – + – + – + – + + – + – – + + – – – + – 
– – + + – – – + – – + + – + – – + – + – + – + + – + – 
– – + – + – + – + + – + – – + + – – – + – – + + – + – 
– – – + + + + – – + + + – + + – – + + + + – – – – – + 
– – – + – – + + – + – – + – + – + – + + – + – – + + – 
– – – – – + + + + – – + + + – + + – – + + + + – – – + 
– – – – – – – + + + + – – + + + – + + – – + + + + – + 
P.B.32
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 
+ + + + – – – – – – – – – – – – – – – – + + + + + + + + + + + 
+ + + – + – – – – – – – – + + + + + + + – – – – – – – + + + + 
+ + + – – + + + + + + + + – – – – – – – – – – – – – – + + + + 
+ + – + + – – – – + + + + – – – – + + + – – – – + + + – – – + 
+ + – + – + + + + – – – – + + + + – – – – – – – + + + – – – + 
+ + – – + + + + + – – – – – – – – + + + + + + + – – – – – – + 
+ + – – – – – – – + + + + + + + + – – – + + + + – – – – – – + 
+ – + + + + – – + + – – + + – – + + – – + – – + + – – + – – – 
+ – + + – – + + – – + + – – + + – – + + + – – + + – – + – – – 
+ – + – + – + + – – + + – + – – + + – – – + + – – + + + – – – 
+ – + – – + – – + + – – + – + + – – + + – + + – – + + + – – – 
+ – – + + – + + – + – – + – + + – + – – – + + – + – – – + + – 
+ – – + – + – – + – + + – + – – + – + + – + + – + – – – + + – 
+ – – – + + – – + – + + – – + + – + – – + – – + – + + – + + – 
+ – – – – – + + – + – – + + – – + – + + + – – + – + + – + + – 
– + + + + – + – + – + – + – + – + – + – – + – + – + – – + – – 
– + + + – + – + – + – + – + – + – + – + – + – + – + – – + – – 
– + + – + + – + – + – + – – + – + – + – + – + – + – + – + – – 
– + + – – – + – + – + – + + – + – + – + + – + – + – + – + – – 
– + – + + + – + – – + – + + – + – – + – + – + – – + – + – + – 
– + – + – – + – + + – + – – + – + + – + + – + – – + – + – + – 
– + – – + – + – + + – + – + – + – – + – – + – + + – + + – + – 
– + – – – + – + – – + – + – + – + + – + – + – + + – + + – + – 
– – + + + – – + + – – + + – – + + – – + – – + + – – + – – + + 
– – + + – + + – – + + – – + + – – + + – – – + + – – + – – + + 
– – + – + + + – – + + – – – – + + – – + + + – – + + – – – + + 
– – + – – – – + + – – + + + + – – + + – + + – – + + – – – + + 
– – – + + + + – – – – + + + + – – – – + + + – – – – + + + – + 
– – – + – – – + + + + – – – – + + + + – + + – – – – + + + – + 
– – – – + – – + + + + – – + + – – – – + – – + + + + – + + – + 
– – – – – + + – – – – + + – – + + + + – – – + + + + – + + – + 
P.B.36
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – 
+ + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + 
+ + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + 
+ + + – – – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + 
+ + – + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + 
+ + – + – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – – 
+ + – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – – 
+ + – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – + – – 
+ + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – – 
+ – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – + – – 
+ – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – – 
+ – + – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – – 
+ – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – – 
+ – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + 
+ – – + + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + 
+ – – + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + 
+ – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + 
+ – – – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + 
– + + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + – + 
– + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + + 
– + + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – – – + 
– + + – – + + + + + – + + + + – – + + – – – – – – + + + + – – – – – + 
– + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + – + 
– + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – 
– + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – 
– + – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – 
– + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – 
– – + + – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – 
– – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – 
– – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – 
– – + – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – 
– – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – + 
– – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – – – + 
– – – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – 
– – – – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + – + 
– – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – + 
P.B.40
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – 
+ + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + – 
+ + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + – 
+ + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + – 
+ + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + – 
+ + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + 
+ + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + 
+ + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + 
+ + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + 
+ + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + 
+ – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + 
+ – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + 
+ – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + 
+ – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + 
+ – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + 
+ – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + – 
+ – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + – 
+ – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + – 
+ – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + – 
+ – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + – 
– + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – 
– + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – 
– + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – 
– + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – 
– + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – 
– + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + + 
– + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + + 
– + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + + 
– + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + + 
– + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + + 
– – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + + 
– – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + + 
– – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + + 
– – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + + 
– – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + + 
– – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – 
– – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – 
– – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – 
– – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – 
– – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – 
P.B.44
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 
+ + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + – 
+ + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + 
+ + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – – 
+ + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – – 
+ + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + 
+ + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + 
+ + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + – 
+ + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – – 
+ + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + 
+ + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + 
+ – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + 
+ – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – – 
+ – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + 
+ – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – – 
+ – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + – 
+ – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – – 
+ – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + – 
+ – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + 
+ – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + 
+ – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + – 
+ – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + 
– + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + + 
– + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – 
– + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – + 
– + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – 
– + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + + 
– + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – 
– + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – + 
– + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – 
– + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – 
– + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + + 
– + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – 
– – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – + 
– – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – 
– – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + + 
– – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – 
– – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – + 
– – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – 
– – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + + 
– – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – + 
– – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – 
– – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – + 
– – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – 
P.B.48
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + 
+ + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + – 
+ + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – – 
+ + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + 
+ + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + 
+ + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + – 
+ + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – – 
+ + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + 
+ + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – – 
+ + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – – 
+ + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + 
+ + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + 
+ – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + 
+ – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – – 
+ – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + 
+ – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + – 
+ – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + – 
+ – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + 
+ – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + – 
+ – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + 
+ – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + 
+ – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + – 
+ – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – – 
+ – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + 
– + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – 
– + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – + 
– + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + + 
– + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – 
– + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + + 
– + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – + 
– + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – 
– + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – 
– + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – 
– + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + + 
– + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + + 
– + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – 
– – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – 
– – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + + 
– – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + + 
– – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – + 
– – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – + 
– – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – 
– – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – 
– – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – 
– – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – + 
– – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – 
– – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – 
– – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – + 

References

 This article incorporates public domain material from the National Institute of Standards and Technology website http://www.nist.gov.