Measurements Course 4600:483-001
Measurement Errors and Uncertainty Analysis Information about this course can be found at http://gozips.uakron.edu/~dorfi/
Introduction In the previous lecture we defined probabilistic measures such as the true mean and sample mean and confidence intervals based on statistical distributions. We derived equations, which can be used to bracket the data within a certain interval for a given level of confidence (always less than 100%). These concepts will now be applied to uncertainty analysis. Uncertainty is the probable error, which can occur during the measurement. It is not the true error, which is the difference between the true value and the measured value, but it is an error range, which is most likely not exceeded. The likelihood is usually chosen to be 95% for uncertainty analysis. In other words we are willing to accept a 5% chance (1 in 20) that the real error exceeds the probable error as estimated from the uncertainty analysis. When talking about measurement errors we do not include things such as: • • •
misreading instruments failing to apply known correction or calibration factors recording incorrect data or misinterpreting the data
These are simply mistakes, which can and must be avoided.
Bias (Systematic) and Precision (Random) Errors An error may contain either a bias (systematic) or precision (random) errors or both. Recall the interpretation of bias, precision and accuracy using the dart board example. Precision is equivalent to good repeatability (small standard deviation for repeated measurements).
Precise but off-center (biased)
Not Precise but not off-center (no bias)
Precise and on-center (small bias) = ACCURATE
Dart Board Example: Precision, Bias and Accuracy Accuracy requires both good precision and small bias. An alternate terminology to describe precision and bias is • precision error = random error • bias error = systematic error An error is considered a precision or random error, if it is obtained from a statistical estimation only. By holding controlled inputs constant, the random error can be estimated through repeated measurements. If the error cannot be estimated through statistical means it is considered a bias or systematic error. Bias errors cannot be found through repeated measurements. To estimate a bias error, we need a reference for comparison such as a calibration process, comparison of tests between laboratories, multiple methodologies or experience. With uncertainty analysis we establish error bands based on the precision and bias of the instruments and transducers used in a particular experiment. This lecture gives an overview of uncertainty analysis. Further examples of uncertainty analysis will be given throughout the semester.
Dr. Hans R. Dorfi
02/25/07 Page 1of 8
Measurements Course 4600:483-001
Measurements Errors Envision an experiment, which is repeated several times. The errors can be identified as follows (see figure below): • The total error in any single Measured measurement is the sum of the bias and value True Value (Mean) x' Systematic the precision error in that single measurement. • The total error in a set of Bias Error measurements can be described by an average bias error and a statistical Sample x Mean estimate of the precision errors. • The average bias error shifts the sample Probability Error Band mean away from the true mean by a Distribution (= Precision error ) fixed amount. # of • The precision or random errors create a measurements normal distribution or scatter around Distribution of Errors upon Repeated Measurements the sample mean. Since the precision error is associated with the random variation during repeated measurements, it can be determined from a statistical analysis of the variation in the repeat measurements. On the other hand, since the bias error is a systematic error, it can only be determined or estimated through comparison to a calibrated reference. Since we desire to estimate the true value of the measurand, we make use of the statistical description given in the previous lecture, which estimates the true mean based on the sample mean and a precision range. This precision range is now the uncertainty ux,: The uncertainty ux can be interpreted as an error band, which contains the true value with a certain probability (usually 95% for uncertainty analysis). x′ = x ± u x (P % probability) Uncertainty analysis is the method used to quantify the ux term. In the above example ux consists of both a contribution from the precision error and the bias error. Therefore the next question that arises is how to add these two errors.
Combining Errors: Root Sum Squares (RSS) Method Any experiment is composed of multiple instruments and sensors, all of which introduce errors and thus uncertainties. A strategy to combine errors or uncertainties is the RSS method. It combines the errors by adding the squares of each error and taking the square root of the sum. The RSS method is based on the assumption that the errors are not correlated (errors combine randomly).
u x = ± u12 + u 22 + ... + u K2
( P % probability )
In the above equation it is important that all error terms are based on the same units and with respect to the same reference. A common reference is to convert all errors to a percentage of the output signal.
Dr. Hans R. Dorfi
02/25/07 Page 2of 8
Measurements Course 4600:483-001
Example: Combining Errors A 26lbm calibrated weight (accurate to within 26.4 0.1%) is placed on a scale. 20 repeated 26.2 measurements are performed and the data is shown on the right. 26 A statistical analysis of the data gives a mean value x of 25.72 and a sample standard deviation Sx = 0.34. Estimate the uncertainty of the mass measurement um.
Nominal Cal Weight
25.8
Sample Mean of data
25.6 25.4
0
5
10
15
20
Data Sample (N=20)
1) The mass of the calibration weight itself has some uncertainty. It is given as 0.1%. It can be assumed that this percentage has a confidence level of 95%, in other words 95% of the calibration weights will be within a 0.1% range of the nominal value (26lbm ). Therefore
mcal = mcal − nominal ± ucal = 26 ± 0.026 lbm (Prob. = 95%) and
ucal = 0.026 lbm From the data we recognize that there is an offset between the measured mean value and the nominal cal weight. This is a bias or systematic error in the measurement.
ubias = 26 − 25.72 = 0.28 lbm Finally, the data indicates that there we have limited precision when the measurement is repeated due to the scatter of the data around the mean. This is the precision or random error. We can also assign an uncertainty to it based on the student-t distribution for 20 data points and a 95% probability.
u Pr ecision = tv , P S x = t19,95% S x = 2.093 *0.34 = 0.71 lbm Combining all the uncertainties based on the RSS method we get the total uncertainty in the mass measurement 2 2 um = ucal + ubias + uPr2 ecision = .026 2 + .282 + .712 = 0.77 lbm
The analysis shows that for the given measurement system we can only estimate the mass with an uncertainty of 0.77lbm. The result also shows the major contributors to the uncertainty, the bias and the precision error. The error in the cal weight is much smaller (as it should be for a good calibration reference). Using the calibration process, one can identify and thus remove the systematic bias error ubias from the data by subtracting the average bias error. In that case the uncertainty in the mass measurement can be reduced to 2 um = ucal + 0 + uPr2 ecision = .026 2 + 0 + .712 = 0.71 lbm
Note that the total uncertainty has not been significantly reduced due to the relative large precision error.
Dr. Hans R. Dorfi
02/25/07 Page 3of 8
Measurements Course 4600:483-001
Design Stage Uncertainty Analysis Uncertainty analysis can be performed in different stages of the experiment (experimental design phase, data collection phase, data processing phase). One important stage is the design stage, where the experiment, the instrumentation and the data analysis are being defined. Uncertainty Analysis of the design stage is particularly important when certain accuracy is critical to obtaining meaningful results. Design state uncertainty analysis is also useful in selecting instrumentation and measurement techniques and to identify potential sources of errors. Different types of uncertainty are discussed below. Zero-order Uncertainty: u0 (Instrument Resolution Error) Even if all errors are otherwise zero, a value of uncertainty will be provided by the resolution of each instrument. This uncertainty associated with the instrument resolution is called zero-order uncertainty and denoted u0. Typically we assign the error u0 to be ½ of the resolution of the instrument at 95% probability:
1 u0 = ± resolution ( P = 95% probability ) 2 Example: Resolution Error A 12 bit data acquisition board is configured for a Voltage range of 0-10V. What is the uncertainty u0 associated with the board resolution? Recall the resolution of a DAQ board as
Res =
VRange 2
Nbits
=
10V = 2.44mV 212
The uncertainty due to the DAQ board resolution is therefore
u0 = ±
2.44mV Res =± = ±1.22mV 2 2
Uncertainty associated with Instrument Error: uc In addition to the error associated with the resolution, there are likely to be errors associated with the linearity and repeatability of the instrument and possibly other error sources. The specification sheet for each instrument usually gives estimates for these errors. Typically the uncertainty associated with the instrument errors is established during a calibration process. The calibration process may be performed by the instrument manufacturer and a calibration sheet is provided with each instrument. The uncertainty associated with the total instrument error is denoted by uc . As mentioned before it may be composed of multiple errors. The RSS method can again be used to combine these errors into a single instrument uncertainty uc. Example: Instrument Error The specification for the 12-bit data acquisition board configured for a range of 0-10V lists the following errors: linearity error : 0.01% of full range repeatability error : 3 LSB(Least Significant Bits, digitzed number can be off by up to 3 bits) offset error : 1mV
Dr. Hans R. Dorfi
02/25/07 Page 4of 8
Measurements Course 4600:483-001
What is the uncertainty uc associated with this instrument? Note that the errors have different units. In order to combine them, we have to convert to common units and to the same reference. The most appropriate reference to convert to would be the signal level going into the DAQ board in Volts. The linearity error u1 is given as a percentage at full range, conversion to Volts yields
u1 = 0.01% *10V = 1mV The repeatability error is given as a bit (LSB) error, convert to Volts using the board resolution, which relates a bit change to a voltage change:
u 2 = 3bits
10V = 7.3mV 4096bits
The offset error is already given in Volts, it is
u3 = 1mV The total instrument error uc is found by combining the three errors with the RSS method:
uc = u12 + u22 + u32 = 7.32 + 12 + 12 mV = 7.4mV For this instrument, the repeatability error clearly dominates the instrument and offset error.
Total Design Stage Uncertainty ud The design stage uncertainty for an instrument is the combination of the zero-order uncertainty (the resolution error) and the instrument error. If we make again use of the RSS method we get
ud = u02 + uc2 The design stage uncertainty can be used as a guide to select equipment. It is not the uncertainty associated with an experiment, because at this point we have not accounted for any uncertainty in test conditions or measurement procedure but simply for uncertainty in the instrument itself. For example, temperature or supply power fluctuations during the experiment would increase the uncertainty associated with the experiment. However, the design state uncertainty ud is the lowest level of uncertainty we can achieve under perfect test conditions with this particular instrument.
Example: Total Design Stage Uncertainty Calculate the total design state uncertainty for the 12-bit data acquisition board configured for a range of 010V using the zero-order uncertainty and the instrument error from the earlier examples:
ud = u02 + uc2 = 7.4 2 + 1.22 2 mV = 7.5mV The total design stage uncertainty for this data acquisition board is 7.5mV. It is primarily due to the 3-bit repeatability error of the board.
Sources of Errors In general error sources can be divided into the following three groups: • Calibration Errors: • Bias and precision errors in the calibration standard (the calibration is only as good as the standard used to calibrate the instrument) • Application of the standard to the measurement system (how is the calibration performed) • Data Acquisition Errors: • Instrument errors (signal conditioning, digitization, see also earlier examples) Dr. Hans R. Dorfi
02/25/07 Page 5of 8
Measurements Course 4600:483-001
• Uncontrolled variables such as environmental changes, power fluctuations, spatial variation,… Data Reduction Errors: • Truncation error of calculations • Curve fit errors When trying to identify errors, it is useful to first try to identify possible error sources based on the above groupings. •
Propagation of Error Quite often we need to know, how an error in one instrument propagates throughout the measurement system. Suppose we have a relationship y=y(x) between a dependent (output) variable y and a measured variable x. How will the uncertainty ux associated with x be reflected in the uncertainty uy of y? In equation form we can write the relationship between a nominal input x and its uncertainty ux and output y and the uncertainty uy due to the propagation of ux as
y = f (x) y ± u y = f (x ± ux )
Our objective is to determine uy, the uncertainty associated with the dependent variable y (see figure). Expansion of the second equation into a Taylor series yields Output y y=f(x) ∂y uy y ± u ≈ f (x) ± u y
∂x
x
∂y ∂x
x= x
y = f (x ) yields the uncertainty uy as ∂y uy = ux ∂x x = x
y
Substitution of
x= x
ux
The uncertainty uy of the dependent variable y is therefore the uncertainty ux associated with the measured variable x multiplied by the sensitivity (slope) of y with respect x at the operating point x .
Input x
x Error Propagation
If the dependent variable y depends on several inputs xi use the RSS method to combine the uncertainties
∂y u y = ∂x1
2
u 2 + ∂y x1 ∂x2 x1 = x1
u 2 + ... x2 x2 = x2 2
1 2
To estimate the relative uncertainty (percentage) associated with the operating point the above equation with respect to
∂y = y ∂x1
uy
y normalize (divide)
y
2
u x2 ∂y 1 + y 2 ∂x2 x1 = x1
u x2 2 + ... y2 x2 = x2 2
1 2
The relative uncertainty, which is propagated from each independent variable xi to y is defined as ei:
ei =
1 ∂y y ∂xi
u xi xi = xi
therefore the total relative uncertainty ey is
e y = e12 + e22 + ... + e i2 Dr. Hans R. Dorfi
02/25/07 Page 6of 8
Measurements Course 4600:483-001
Uncertainty for dependent variables y, which are proportional to powers of xi For functions, which are powers of the independent variables xi, the relative uncertainty can be quickly calculated. Assume that y can be written as
y = f ( x1 ,..., xi ,...) = g ( x1 ,...) x ni The relative uncertainty was defined earlier as
ei =
1 ∂y y ∂xi
u xi xi = xi
Differentiate y with respect to the variable xi .
∂y ∂ ( g () xin ) = = g ()nx in −1 and substitute the result into the equation for ei ∂xi ∂xi nx in−1 ux 1 ∂y 1 n −1 ei = u xi = g ()nx i u xi = n u xi = n i n y ∂xi g () x i xi xi In summary we find that if the variable xi is a power of n, its relative uncertainty ei is simply
ei = n
u xi xi
Example: A ruler is used to measure the dimensions of a cylinder. The volume is then calculated from the cylinder dimensions. What is the uncertainty in the volume? Measured cylinder dimensions are: Diameter d= 2”, Height h=10”. The resolution of the ruler is 1/16”.
π
d 2h
The cylinder volume is given by
V=
The total relative uncertainty in the volume can be written as
eV = ed2 + eh2
4
Cylinder Volume ?
But since d and h are simple powers of V(d,h) , we can use the equation
ei = n
u xi xi
,
where n=2 for d (d2) and n=1 for h. Thus
ed = 2
ud d
and
eh = 1
uh h 2
The total relative uncertainty in the volume is thus
u u eV = 2 d + h d h
To go from relative to total uncertainty
uV = VeV
Dr. Hans R. Dorfi
2
02/25/07 Page 7of 8
Measurements Course 4600:483-001
The uncertainty in the length measurement of d and h is the zero order uncertainty (instrument resolution error), which is simply ½ the instrument resolution:
ud =
1 1 1 1 in = in and u h = in 2 16 32 32 2
2
1 / 32 1 / 32 Thus eV = 2 + = 0.031 = 3.1% 2 10 The volume uncertainty can now be determined by multiplying the relative uncertainty with the measured cylinder volume.
uV = VeV =
π 4
* 4in 2 *10in * 0.031 = 0.97in 3
With the given ruler resolution and the cylinder dimensions the uncertainty in the volume was found as about 1 cubic inch or 3% of the measured volume.
Strategies for Uncertainty Calculations When combining uncertainties from different sources, it is helpful to use the following strategy: 1. 2. 3. 4. 5. 6. 7.
Draw a diagram of the test system and label the different signals and variables in the test system. Determine the nominal values of the signals and variables in the test system. Nominal may be the max input or the nominal input. Determine uncertainties for the test variables. If appropriate, write an equation that determines the test result from the test variables. Determine, how uncertainties propagate from the test variables to the test result. Apply the known uncertainties to the propagation equation to calculate the propagation of the uncertainties. Add all uncertainties using the RSS equation into a total uncertainty value (can only be done, if all uncertainties are referred to the same reference)
Dr. Hans R. Dorfi
02/25/07 Page 8of 8
