Manual of Petroleum Measurement Standards Chapter 13 Statistical Aspects of Measuring and Sampling

Transcription

1 Manual of Petroleum Measurement Standards Chapter 13 Statistical Aspects of Measuring and Sampling Section 2 Methods of Evaluating Meter Proving Data SECOND EDITION, DRAFT 2017 This document is not an API Standard; it is under consideration within an API technical committee but has not received all approvals required to become an API Standard. It shall not be reproduced or circulated or quoted, in whole or in part, outside of API committee activities except with the approval of the Chairman of the committee having jurisdiction and staff of the API Standards Dept. Copyright API. All rights reserved. 1

2 Introduction Minimizing petroleum measurement errors, estimating remaining errors and informing affected parties of errors is important to the petroleum industry. A consistent basis for estimating the size and significance of errors is essential for communication among affected parties. A fundamental source of measurement error is the inaccuracy of the meter factor which is a number that represents the performance of a meter throughout a batch. Meter factors are monitored to gain confidence in the reported volumes and to detect trends or sudden deviations as indications of when to perform maintenance on or calibration of measurement equipment. The purpose of this standard is to suggest procedures for monitoring variations in meter factors so that uncertainties are understood and consistent with the objectives of parties affected by the measurement operations. The procedures for monitoring meter performance, limits on meter factor variations and actions to take because of these variations are left to the agreement among the parties affected by the measurement operations which can depend on company policy, contract agreements, regulations and laws. 2

3 1 Scope This standard establishes the basic concepts and procedures to estimate and report meter performance uncertainty in consistent and comparable ways. 2 Normative References The following referenced documents are indispensable for the application of this document. For dated references, only the edition cited applies. For undated references, the latest edition of the reference document (including any amendments) applies. API Manual of Petroleum Measurement Standards (MPMS) Chapter 1, Online Terms and Definitions Database API Manual of Petroleum Measurement Standards (MPMS) Chapter 4, Proving Systems API Manual of Petroleum Measurement Standards (MPMS) Chapter 5, Metering API Manual of Petroleum Measurement Standards (MPMS) Chapter 13.3, Measurement Uncertainty 3 Terms, Definitions and Acronyms For the purposes of this document, the following terms and definitions apply. Terms of more general used may be found in the API MPMS Chapter 1 Online Terms and Definitions Database. Acronyms are defined in the text at first use. 3.1 standard deviation limit A control limit equal to 3 standard deviations from the arithmetic mean of the set. Note: For a set of normal data this limit is also known as the 3 sigma limit. 3.2 action limits Horizontal lines found on a run chart or control chart which, if exceeded, suggests that some sort of action is necessary to modify the underlying process or to modify the reports generated from the process. Related terms: tolerance limit, warning limit. 3.3 central line A line on a control chart or run chart that represents the central value. 3.4 central value The mean value of the statistic under consideration. Synonym: central tendency. 3.5 control chart A graphical method for evaluating whether a process is in or out of statistical control. 3.6 control limits Horizontal lines on a control chart used in the evaluation of whether or not a process is in statistical control. 3.7 control log A tabular method for evaluating whether a process is in or out of statistical control. 3.8 datetime A number with integer and decimal parts used to designate a particular point in time in which the integer represents the day and the decimal represents the fraction of the day. 3.9 measurement resolution The smallest change in a quantity being measured that causes a perceptible change in the corresponding indication. Synonym: resolution 3

4 3.10 meter curve A graph of meter factors or K factors as a function of flow rate meter factor log A table for recording meter factors and associated data in chronological order normal The designation for a set of data that exhibits a Gaussian distribution. Note: In this standard all references to normal and associated word forms (normality and normally) are meant in the Gaussian sense and not in the typical sense rational subgroups Subsets of a set of data selected in a way to maximize the variation between the subsets and to minimalize the variation within the subsets routine variation The variation inherent in a process. Synonyms: noise, common cause variation 3.15 run chart A graphical method to display process data in chronological order special cause variation Variation in a process that is not routine variation. Synonyms: exceptional variation, signal, assignable cause variation statistical control Predictable within a confidence interval student s t A statistical function that varies in magnitude with degrees of freedom tolerance maximum permissible error in a series of measurements tolerance limit Horizontal lines found on a run chart or control chart which, if exceeded, strongly suggests that special cause variation in the process has occurred requiring some action to mitigate or explain. Synonyms (control charts only): natural process limit, control limit. Related terms: action limit, warning limit warning limit A horizontal line found on a run chart or control chart which, if exceeded, suggests that the underlying process should be monitored more closely. Related terms: action limit, tolerance limit. 4 Significance and Use API MPMS Chapters 4, Proving Systems and Chapter 5, Metering shall be consulted for the installation, operation and proving of meters. These standards shall govern should any conflict exist between this standard and these standards. Additionally, this standard assumes knowledge of the content of API MPMS Chapter 13.3, Measurement Uncertainty. Caution: Statistics derived from a set of data cannot be used to prove a point about the data; statistics can only, to some degree, suggest that point. All statements in this standard should be interpreted with this in mind. 4

5 5 Elements that Influence Meter Factors The performance of a flow meter will vary in response to changes in flow rate, mechanical condition of the meter and changes in fluid properties. A meter factor, a number that represents the performance of a meter in situ, is established through calibration against a standard. The standard is commonly known as a meter prover and the calibration process is commonly referred to as a meter proof. However, since the meter factor is determined from a proof, a change in a meter factor can be due to a change in meter performance OR a change in the proving system OR a change in other peripheral equipment OR a change in fluid properties. For example: Consider a turbine meter and an associated bi directional ball prover with all signals flowing to a flow computer. The mechanical state of the meter such as sharpness of the leading edge of the turbine blades and bearing wear will directly affect the meter s performance. Fluid properties that directly affect the flow meter s performance are viscosity and lubricity with density (API gravity), temperature, pressure and concentration of contaminants having indirect effects. The proving system for a ball displacement prover consists of interchange valves, detector switches, prover sphere and prover coating along with proving procedures and the operator performing the proof. Peripheral equipment includes piping components (basket strainers, valves, thermowells, etc.), secondary instrumentation (pressure, temperature, density, etc.), pulse generators, wiring from the pulse generators to the flow computer, the flow computer and the programming in the flow computer. Other types of meters and provers can have different influences. While the meter factor is the current representation of the performance of a meter during a batch, the meter factor is actually the performance of the metering system and the proving system during the time the meter is on proof. An operator of a metering skid has to consider these other variables and develop metering and proving procedures designed to filter out the noise from variation of influences external to the meter in order to discover meter problems. 6 General Suggestions to Improve the Monitoring Process Meter factors should be determined as often as practical as the more frequent the proving, the better the statistics derived from the data and the better the chance of early discovery of performance changes. In addition to routine, planned proofs, proofs should be initiated whenever the process has changed too much since the last proof. Too much of each process component is best determined from proving data and the tolerance for error. Once the proof is completed, the data should be analyzed soon thereafter in conjunction with previous proving data in order to discover any problems in a timely manner. Frequent proving can indicate how much a particular meter in a particular metering system is influenced by the various elements mentioned in Section 5. Note: A meter factor needs to be judged in relationship to a series of other meter factors obtained from the same meter in order to draw meaningful conclusions regarding the meter s performance. For example, a meter factor of may be perfectly fine but that wouldn t be known without comparison to other contemporaneous meter factors from this meter. The ideal monitoring program would result in never performing maintenance when it is not needed (no false positives) and always performing maintenance when it is needed (no false negatives). When action is suggested, successful monitoring will indicate what action needs to occur. 7 Data Signals All processes return variable results. The variability inherent in a process is called routine variation. Another type of variability is special cause which is an indication of a problem with the inputs into a process or a problem with the execution of the process in other words, trouble. If the process is predictable, then routine variation is the best that can be expected of the process; to reduce the 5

6 common cause variability, the process must be improved. Special causes need to be identified and dealt with because they are not typically part of the process. The idea behind monitoring is to separate the signals from the noise and act upon signals. Monitoring meter factors will generate data that show variation in meter factors over time and analysis of the data might return signals to the operator of a metering skid to take some kind of action. The signals are known as limits with various modifiers such as warning, action and tolerance corresponding to increasing degree of variance or discovered trends and corresponding increasing levels of response. Exceeding 3 standard deviation limits is nearly always due to special cause regardless of the underlying distribution*. Accordingly, it is suggested that the tolerance limit be at 3 standard deviations. See Table 1 below for a primitive example of confidence intervals and the approximately equivalent standard deviations that could be associated with the three limits mentioned above. As stated above, this standard explicitly does not mandate any of the criteria shown in Table 1; it is shown to help the user craft company guidelines for monitoring the performance of meters. * See Wheeler, Donald J., Probability Limits, Quality Digest, column/probability limits.html, March 2, 2015 Limit that is Exceeded Confidence Interval Equivalent Standard Deviations Warning 90 95% Slightly < 2 sd Action 95 99% Between 2 and 3 sd Response(s) to consider (Responses at any part in the table include responses listed above in the table) Reprove the meter Verify temperature, pressure and gravity inputs. Verify meter strainer condition. Verify valve seals. Verify the proving system. Depending on what is found, correct the custody transfer ticket. Tolerance 99% + > 3 sd Repair or replace the meter. Table 1 Responses to consider when Limits are exceeded The limits can be determined from statistics derived from the data (See section 8.4 for details) but the limits can depend on company policy, contract agreements, regulations or laws. The latter limits are known as fixed as opposed to limits that change based on changes in the data. Fixed limits may be thought of as specifications. 8 Ways to Monitor Meter Factors 8.1 General Following are four ways to monitor meter factors presented in order of increasing complexity. Not all are applicable to all installations due to differences in process stability and in the ability to gather data. Logging Run Charting One dimensional Control Charting Multi dimensional Control Charting 6

7 8.2 Logging The most basic form of monitoring meter factors is maintaining a meter factor log which is a table for recording each meter factor and associated data in chronological order. This is applicable to all metering installations. An example of a meter log is shown in Table 3 which shows data from running a 12 point meter curve followed by data from 36 daily proofs. The column titles for Table 3 are as shown in Table 2 immediately below: DateTime Date and Time of the proof Pm Average pressure at the meter during the proof Pp Average pressure in the prover during the proof SG Average Relative Density, (60/60) during the proof Tm Average temperature at the meter during the proof Tp Average temperature in the prover during the proof %S&W Average S&W percentage during the proof Rate Average Flow rate during the proof r/r % dev Range % deviation of the trial factors # runs Number of runs needed to get 5 consecutive runs meeting deviation limits MF Meter factor % crv Meter factor % deviation from curve % prev Meter factor % deviation from previous meter factor % baseline Meter factor % deviation from baseline Table 2 Meaning of Column Titles in Table 3 Logs can be maintained as a list on a piece of paper but are more useful in an electronic spreadsheet. An electronic spreadsheet can be used to easily create statistics or charts and, through the use of conditional formatting, can highlight data that exceed fixed limits. Note that a reported meter factor is actually the arithmetic mean of some number of trial meter factors. Additional information would be gained by logging the trial runs instead of or in addition to the meter factors themselves. Another way to identify changes in meter factors is to calculate changes from a baseline meter factor. Baseline can be defined in many ways and what is appropriate for one installation can be inappropriate for another. The baseline could be defined as the first meter factor obtained after installation or major maintenance, but a more representative baseline might be the arithmetic mean of the first few meter factors obtained few suggested as being at least five to get a good representation of standard operating conditions. Yet another reasonable baseline might be a rolling average of the previous five meter factors. All of these can be easily calculated and displayed in the meter factor log. The baseline factor in Table 3 is It was selected because it was the curve factor determined at the rate that is closest to the expected average flowrate of future proofs. 7

8 DateTime Pm Pp RD Tm Tp %S&W Rate r/r % dev # runs MF % crv % prev % baseline 06/29 13: #N/A 06/29 13: /29 13: /29 14: /29 14: /29 15: /29 15: /29 15: /29 16: /29 16: /29 16: /29 17: /30 22: /01 10: /02 09: /03 10: /04 09: /05 09: /06 09: /07 08: /09 03: /09 10: /10 09: /11 09: /12 10: /13 11: /14 09: /15 09: /16 15: /17 10: /18 09: /19 09: /20 02: /21 09: /22 09: /23 09: /24 10: /25 08: /26 12: /27 12: /28 14: /29 10: /30 08: /31 13: /01 08: /02 11: /03 00: /03 10: Table 3 Meter Factor Log 8

9 8.3 Run Charting A run chart is, in its basic form, a line graph of data with the variable of interest on the y axis plotted in chronological order and the associated ordinal numbers (first, second, third, etc.) on the x axis. Fixed limits can also be shown on a run chart on the y axis or the secondary y axis which provides visual indication of whether or not the process meets specification. The basis for a run chart used to monitor meter factors would be the data found on a meter factor log. As such, this method of monitoring is applicable to all operations. A run chart allows for easy identification of obvious trends or patterns but does not indicate whether or not the process is in control. Figure 1 below shows three of many possible examples of patterns on run charts of meter factors that suggest further investigation into the metering/proving system. Figure 1 Three patterns that suggest further investigation CAUTION: Care must be given to the selection of the chart type to prevent inadvertent visual distortion of the data. A line chart assumes equal increments on the x axis which make sense for data that is gathered at equally spaced time intervals (which is what ordinal numbers imply). See Figure 2 below for 9

10 a basic run chart of the first twenty meter factors from Table 3 as an example of this distortion. Note from the meter factor log that the first twelve factors were determined while running a meter curve (proof intervals of minutes) and the next eight meter factors were from scheduled proofs (proof intervals of days). The top plot in Figure 2 gives a distorted view of meter performance because the reader is induced to assume equal time intervals which would lead to the interpretation that the meter is steadily changing performance upward under typical circumstances for the first twelve proofs. This distortion can be corrected by substituting datetime for ordinal numbers on the x axis and using a scatter plot instead of a line chart. The bottom plot in Figure 2 shows the results of doing this using the same meter factors displayed in the top plot MF (Line Chart) MF MF (Scatter Plot) MF /29 06/30 07/01 07/02 07/03 07/04 07/05 07/06 07/07 07/08 Figure 2 Basic Run charts Because data can be easily compared to fixed limits on a run chart, it can be useful to plot deviations from previous proof, deviations from baseline or deviations from meter curve on a run chart. Many operators have fixed limits for these statistics and it can be useful to create a run chart for each specification. Other data from the log can be plotted on the secondary y axis to help with interpreting the process. The analyst should experiment with chart formatting and what data to plot to derive the clearest exposition of meter performance including creating many different charts that can be viewed simultaneously. 10

11 Figure 3 shows a scatter plot example of a run chart that displays percent deviation from curve with fixed deviation limits (company policy) of ±0.15% on the primary y axis. Relative density is plotted on the secondary y axis. The data is from Table 3. % Deviation Recent Meter xxxx Proof Deviations /29 07/04 07/09 07/14 07/19 07/24 07/29 08/03 % crv LL UL RD Relative Density Figure 3 Run Chart of deviations from the curve with fixed limits of ±0.15% deviation from the curve on the primary y axis and relative density on the secondary y axis Note that other related plots can be created by substituting meter temperature, S&W % or flow rate for relative density in Figure 3 and be using the underlying meter factor instead deviation from curve. 8.4 Univariate Control Charting A univariate (one dimensional) control chart is the next step in monitoring meter factors. This method of monitoring is useful only to operations in which the underlying data is available and in which the underlying process is judged to be controllable. Control charting, when correctly done, will indicate whether or not this process is in control. For reasons explained in this section, control charting may or may not be useful for a given meter installation. If control charting is not indicated, run charting will yield the same information as control charting with less effort. Control charting can be easily done using an electronic spreadsheet. The term univariate is used because only one variable, meter factor, is charted. See the next section for multivariate control charts. Figures 4 7 show four univariate control charts developed from the same data in Table 3. They are presented in order of fewest assumptions (shortcuts) to most assumptions made. There are many other univariate control charts that can be developed and most assume normal data. Crucial assumptions in control charting are that all the data is representative of an unchanging process AND the next data point will be a random sample from the same process. The techniques used to create Figures 4 7 assume the data are normally distributed but they will work for data that are nearly normal. See Annex A for details on normality and tests for normality. Annex B shows a univariate control charting technique that doesn t require normal data plus other charts that could be created from the Table 3 data. Univariate control charts were developed by Walter Shewhart and explained in his 1931 book, Economic Control of Quality of Manufactured Product. Despite his careful exposition, univariate 11

12 control charts are often created or used improperly by people and computer programs. Part of the problem is his antiquated language and the use of the word control which has unfortunate connotations. Using a control chart for process data will not enable the analyst to control a process; a control chart indicates whether or not a process is in control within limits. A modern interpretation of control is predictable. Accordingly, a process that is in control within limits is predictable within the same limits; that is, the next data point gathered will probably be within those limits. Note: Henceforth in this standard, the words in control and predictable will be used to mean in control within limits and predictable within limits respectively. Shewhart outlined creating univariate control charts as follows: 1. Divide the data into rational subgroups. 2. Compute measures of location and measures of dispersion for these data which, if the quality is not predictable, will be most likely to indicate the presence of trouble. 3. Construct univariate control charts with limits of the form: Average ± 3 sigma 4. If an observed point falls outside the limits of this chart, take this as an indication of trouble or a lack of control. The forming of subgroups so that the variation within the subgroup is minimized and the variation between the subgroups is maximized is what Shewhart meant by rational. In other words, a rational subgroup is a set of measurements produced under repeatability conditions. Rational subgroups are meant to represent a snapshot of the process. The variation within the subgroups represents the common cause variation in the process at any given time. Consider a set of meter factors which are calculated as the arithmetic mean of 5 consecutive trial factors that agree to within 0.05%. The data is the set of all trial factors. The rational subgroups would be the reported meter factors, which are, by design, produced under repeatability conditions. Two important points are that a process can be predictable but not meet specifications and a process can meet specifications and not be predictable. The latter is often the case for meter factors from meters in crude oil service because of the wide variation of many aspects of the crude oil the next meter factor will not be predictable even if the metering equipment and procedures are perfect. The difference between a run chart and a control chart is that the limits displayed on a control chart come from statistics performed on the underlying data while the limits for a run chart come from company policy, manufacturer specifications or government regulations. Because of this, some of the discussion in the literature refers to run charts as control charts with fixed limits. To avoid confusion, this standard doesn t use that nomenclature due to Shewhart s definition specifications cannot be used to determine limits on a control chart because it would not be a control chart any more. This standard calls such charts run charts. This is mentioned to aid the user in understanding the discussion in other sources. Note that even though a control chart gives more information than the run chart, the utility of both types of charts is anticipated to be the same for the typical user of this standard. Below are some univariate control charts in order of fewer assumptions to more assumptions. The data for each chart and how the values are derived are found in the control logs in Annex C. In all of these charts and tables, avg means arithmetic mean of the meter factors and sd means sample standard deviation. Figure 4 was created directly from Shewhart s definition. 12

13 Limits from sd of Trial Factors MF avg (MF) UL LL Figure 4 Control Chart of Meter Factors using sd of Trial Factors Assume that the trial factors are not available but the range % deviation of the trial factors is available for each meter factor. The range % can be used to set the limits Limits from Reported Run to Run Deviation MF avg (MF) UL LL Figure 5 Control Chart of Meter Factors using range % of the Trial Factors. Suppose an analyst does not have any information concerning the trial runs that were used to derive the meter factors in Table 3 and has only meter factors in the order that they were attained. The analyst could use an XmR chart which is very similar to the standard deviation approach but uses a moving range of two to calculate the control limits rather than the standard deviations of the trial factors. This is shown in Figure 6. 13

14 Figure 6 Control Chart of Meter Factors using moving range (mr) of the meter factors. The XmR chart has two parts. The top part is where the individual measurements are analyzed and the bottom part is where the moving ranges are analyzed. The control limits are determined using factors that are rooted in well established theory that deals with the distribution of the average range. This technique uses the idea that the moving range is a subgroup of 2 that accurately reflects the routine variation in the run of measurements in this example meter factors. For a non technical explanation, see Wheeler, Donald J., Making Sense of Data. Note: Figure 6 is technically incorrect because the limits were determined assuming that meter factors are individual values when, in fact, meter factors are actually averages. Figure 6 tells essentially the same story as Figures 4 and 5 because, for the data used for these particular charts, the variability of the minute to minute trial factors is much less than the variability of the day to day meter factors. Thus far in this section, control charts have been created assuming a priori the existence of 36 meter factors to create the limits. This paragraph discusses how to start out ; that is, how to monitor a meter that is newly installed or maintained. With no history, it is impossible to determine meaningful limits but as proofs are performed, the limits become more meaningful. This chart shows the common practice of re calculating the limits at each data point until seven points have gathered; fixing the limits 14

15 using the seven points; and then, after twenty points, using the twenty points to re calculate a new set of unchanging limits that will be useful until the process changes. These limits were determined using the same technique used to create Figure 5. Figure 7 Control Chart of Meter Factors Using Limits Calculated from Moving Averages This standard cannot suggest when a re evaluation should occur because each installation is different but once limits are set they should be reevaluated from time to time. Below are three examples of some situations that that would require new limits for any of Figures 4 through 7. Example 1: If the meter factors are from a PD meter, then the analyst is assuming the meter has not worn appreciably over the data period only the analyst knows if that is a good assumption. Example 2: If the 37 th proof will be run on diesel when the first 36 proofs were run on gasoline, then the process will be different and the charts will require newly calculated limits. Example 3: If the instrumentation for meter temperature was recalibrated after the 36 th proof, then the process is different and the charts will require newly calculated limits. Figures 4 through 7 show that the four different ways to create a control chart are equivalently valid. These figures also all indicate this process is out of control because the rate of out of control points far exceeds the 3 points in 1000 rate which is the expected rate to occur by chance alone. However, each of these figures also shows 36 consecutive meter factors all between and (± from the average). How to interpret this depends on knowledge outside of the data in Table 3. For instance, if this was a meter used to measure the volume of product from a refinery tower to a storage tank, then considering only the run of meter factors, this performance would be troubling considering the supposed stability of the process essentially the same fluid, same temperature, same flow rate, same concentration of contaminants, etc.; however, if this meter was in crude pipeline service where any of the crude characteristics can change significantly at any time, then this performance would be considered very good. It would make sense to use control charting in the refinery case because the process is expected be in control and an analyst would be very interested in the times that it wasn t in control so as to focus attention to improve measurement. In the crude pipeline case it wouldn t make sense to use control charting because there is no expectation that the process would ever be in control. Run charts would be indicated for use in the crude pipeline case. 15

16 Once a process is predictable, analysts are often interested in increasing the sensitivity of their control charts by analyzing patterns observed in the data. See Annex D for more information. To summarize section 8.4: 1. Gather meter factor data in chronological order as shown in Table Create a control chart using one of the techniques discussed. 3. If the control chart has no out of control points, then the metering process can be considered in control and the control chart can be used to monitor the metering system. See step 5 for how. 4. If the control chart has out of control points then the metering process is out of control. The user of the standard should investigate each out of control point to determine each cause. a. If the cause is something that cannot be addressed, then using a control chart to monitor the process is not indicated. The best tool to monitor such a process is the run chart technique discussed in section 8.3. b. If all the causes are addressed, go back to step 2 and create a control chart with the new data. 5. If the control chart created in step 2 has no out of control points, plot new meter factors in this control chart as they are obtained without recalculating the mean or limits. The metering system should be considered in control if: a. Subsequent meter factors are within the limits. b. Subsequent meter factors obey the runs rules detailed in Annex D. c. If subsequent meter factors violate 5a (out of control) or 5b (maybe out of control), then the user should investigate. Depending on what is found, either continue using the control chart, or if the process has changed but is still in control, create a new control chart with the new data. Note: Statistically speaking, a run of meter factors in control means any of one of them is just as accurate as any other. The control chart mean is the best estimate of the true meter factor and should be used in the volume calculations. The purpose of subsequent meter factors is to demonstrate the process is still in control. Before implementing this statistical interpretation, ensure this interpretation does not violate company policy, commercial agreements or regulations. Note: It is good practice to recalculate a new control chart every approximately every twenty meter factors by including all previous factors plus the new factors. The expectation is that the limits will narrow and the mean will stay virtually the same. If the new limits widen, this is cause to investigate even if the process is still in control. 8.5 Multivariate Control Charts for Meter Factors The last statistical technique discussed in this standard that can be used to monitor meter factors is multivariate control charting. The power of multivariate control charting comes from the ability to monitor the relationships between multiple variables simultaneously. Although univariate control charts can be made for each variable, drawing conclusions on the process control status without considering their combined influence can be misleading. A multivariate control chart will account for the correlation of each variable which provides a more accurate indication of whether or not the process is in control. All variables of interest should be adequately modeled by a normal distribution. There are techniques for stepping away from normality, but they require customized construction for the process in question. 16

17 Multivariate statistical control charts are developed using matrix algebra to calculate the Hotelling s T Squared statistic, which involves calculating the variance covariance matrix and mean vector of the variables of interest. If the T Squared statistic is in control, then the process is in control. Calculating this is intricate and involved; using commercially available software and expert statistical advice is strongly advised. The mathematics for this exists for any number of variables but the technique is more easily explained using two variables. Using the Table 3 data, consider two variables: meter factor and relative density. In Figure 8 below the horizontal lines represent the control limits for meter factor and the vertical lines represent the control limits for relative density. The red ellipse represents the area of both variables being in control at the same time. The interpretation of this chart would be all relative density points and meter factors are in control when considered separately, but, when considered together, the fifteen points outside the red ellipse would be considered out of control. Figure 8 Generalized Multivariate Chart Multivariate charts and univariate charts should be used in tandem. Multivariate charts will tell you only that the process as a whole is out of control, but not necessarily what variable caused it. Looking at the data in Table 3, the univariate and multivariate charts on the following page are constructed considering the four factors of meter factor, S&W%, flow rate and Relative Density. The software used to create the multivariate control chart was set to show the relative contribution of the four factors in the process to the T Squared statistic for a given point. Here the multivariate control chart would suggest checking the relative density at point 17 before it went out of control at point 18 in the univariate case. Noticing the huge change at point 27 in the Multivariate Control Chart due to meter factor shift might have caused a change to be made so the next 3 meter factors would have been in control. Notice these charts all work together to allow for a proper decision to be made. 17

18 Observation T-Squared MF %S&W Rate RD Table 4 Relative contribution of each component to the T Squared Statistic 18

19 Rate T Squared Hotelling's T 2 Largest Contributor Labeled Rate Control Chart UCL Rate RD MF %S&W H. T2 Rate UCL LCL %S&W Relative Density Meter Factor Meter Factor Control Chart Relative Density Control Chart %S&W Control Chart MF UCL LCL RD UCL LCL %S&W LCL UCL Figure 9 Multivariate Chart for Table 3 data and the associated Four Univariate Charts 19

20 ANNEX A Tests for Normality The control charts discussed so far are made with the assumption that the observations follow a normal distribution. If there is a departure from normality, however, then the control limits given can be inappropriate and a run chart with fixed limits or a control chart created using the percentile technique discussed in Annex B would be suggested. It is always important to check the normality assumption when using a control chart of individual variables. A simple way to perform this check is with the normal probability plot (aka QQ Plot) easily created using an electronic spreadsheet. Figure A.1 shows the normal probability plot for the Table 3 data. The line represents the standard normal with mean=0 and standard deviation=1. Since the observed points do not fall exactly on the line, the data is not normal; but, since they fall close to this line the data are said to be approximately normally distributed. The QQ plot is a visual test and interpretation of Figure A.1 shown below is that the data is close enough to normal for them for the use of a control chart to be appropriate. Non normality in a QQ Plot will often show the tails drifting away for the line or the plot data will have a curve like in Figure A.2. Figure A.2 shows data that is distributed from an exponential distribution (left) and a log normal distribution (right). Notice how the plot data does not fall on the line, therefore we cannot assume that a normal distribution would model either of the datasets pictured in Figure A.2. Note: It is suggested that the QQ plot should be used together a numerical test such as one of the two tests explained later in this Annex as part of the decision to use control charts. All charts and calculations in Annex A were created using data from the last 36 proofs in Table 3. Caution: No test for normality proves that a set of data is normally distributed it only suggests it. These tests can prove that a set of data is not normally distributed. What these tests will probably show is that the set of data is not perfectly normal as it would be rare for any data set to be perfectly normal; judgment is required in each case as to whether or not the data is close enough to normal to make control charting (or any other statistical technique requiring normality) to be useful. The set of data for Figure A.1 is on the next page. Std Deviation QQ Plot Meter Factor Figure A.1 Normal Probability Plot for Table 3 data 20

21 Count 36 sd Mean correlation 0.99 Ordinal # MF Std MF Normal #NUM! Table A.1 Data for the QQ Plot in Figure A.1 Where: Ordinal # is the rank in the list MF is the list of meter factors sorted minimum to maximum Std MF is the Standardized MF. The Standardization or Normalization function uses inputs of the meter factor, data set mean and data set standard deviation to return the standard deviations from the mean for the given meter factor Normal is the data that forms a line representing perfectly normal. The function used is the inverse of the standard normal cumulative distribution with input of (Ordinal #/Count) 21

22 Figure A.2 Normal Probability plots for non normal data Statistical tests such as the Anderson Darling or Shapiro Wilk tests discussed below are considered more rigorous tests for normality than the QQ Plot. Other tests for normality exist and all should be considered equally applicable. These calculations are found in commercially available software and should be performed on data sets prior to using control charts that assume normality. For detailed information, consult any standard statistics textbook. Anderson Darling Test The Anderson Darling statistic is used to objectively test for normality, data independence, and adequacy of measurement resolution relative to the overall variation in the dataset. Two Anderson Darling (A D) statistics, A D rms /A^2 and A D MR /A^2 Modified, can be calculated via a software package. The A D rms uses a numerical estimate of the sample standard deviation(s) as per the RMS (root mean square) and the A D MR uses a moving range of two to perform the necessary calculations in order to get a test statistic. Many software packages produce both test statistics as the interpretation of the test relies on both. There are three cases that occur when analyzing data. CASE 1: Both A D rms and A D MR are <<1.0 (very much less than 1.0). This is to be interpreted as no compelling evidence to reject the hypotheses that the data is normal, independent, with adequate measurement resolution. Proceed to construct a control chart. CASE 2: Both A D rms and A D MR are >>1.0 (very much greater than 1.0) and the normal probability plot shows two to three staircases, which means that the data is stacking or is clustered into two or three distinct values. This is evidence that there is not enough variation in the data usually due to inadequate numerical resolution. (This is where a run chart becomes a useful tool with constant monitoring.) This case usually occurs when a process variable could be considered more of a constant rather than a variable. This can also occur if the variable experiences frequent periods of stability but has step changes between these periods. CASE 3: A D rms is <<1.0, but A D MR >1.0. This indicates that the results are serially correlated, or not independent. This is usually caused by a cyclic effect due to the environment of the process, or moderate trending of data due to normal degradation of equipment. If these types of behavior are normal for the process, proceed to construct a control chart using standard deviations rather than the moving range. If the software has the capability, a P value can be calculated based on comparing quantile values from the normal distribution through simulation and Monte Carlo methods. To interpret the P value, a comparison is made to the level of significance (usually being α=0.05 for 95% confidence or α=0.01 for 22

23 99% confidence). If the P value is greater than α, the data can be considered normal; if the P value is less than or equal to α then the data can be considered not to be normal. The Anderson Darling test results from the Table 3 data are shown below. Statistic Value A2/ A-D rms Modified Form/ A-D MR P-Value Table A.1 Anderson Darling statistics for Table 3 data The data show both statistics are <<1.0 so the data fall into CASE 1 stating the data are adequately modeled by a normal distribution. This conclusion is supported by the P value being greater than α=0.05. Shapiro Wilk Test (from ASTM D6299 A1.4.2) Shapiro Wilk Test is another procedure that can be used to test for normality. Commercially available software is used to calculate the statistic. For the data in Table 3 the Shapiro Wilk test results can been seen in Table A.2 below. Statistic Value W P-Value Table A.2 Shapiro Wilk statistics for Table 3 Data The interpretation of W depends on the underlying data; that is, there is no general threshold for W that suggests normality or non normality. In general, W = 1 means the data is perfectly normally distributed and W < 1 means the data is not perfectly normal. The further from 1 W is, the further the departure from normality. The software uses the W and P Value statistics together to assess normality. The P value is the same statistic discussed in the section on the Anderson Darling test above and is interpreted the same way. The software that generated Table A.2 indicated that the data was adequately modeled by a normal distribution. 23

24 Annex B Other Charts All charts and calculations in Annex B were created using data from the last 36 proofs in Table Limits Using Policy R/R Deviation = 0.05% MF avg (MF) UL LL Figure B.1 Run Chart using Policy Run to Run Limit of 0.05% Figure B.1 shows a run chart where the limits are derived from a company policy that states that meter factors shall be averaged from 5 consecutive trial factors which agree to within 0.05%. This chart is built exactly like the one in Figure 5 except 0.05% was used in place of the reported r/r%. The interpretation of this chart would be the meter performance is well within company specification. One use for this chart would be to monitor performance of a new or newly repaired meter until enough data is gathered to perform an analysis (compare to Figure 7). Another use would be if the analyst did not have run to run deviation data (compare to Figure 5). 24