Tutorial for the Total Vertical Uncertainty Analysis Tool in NaviModel3 May, 2011
1. Introduction The Total Vertical Uncertainty Analysis Tool in NaviModel3 has been designed to facilitate a determination of the quality of a hydrographic survey. The part of the quality that is investigated is the Total Vertical Uncertainty (TVU). The hydrographic data to be tested could either be acquired by means of single-beam or by multi-beam techniques. 2. Utilizing the Total Vertical Uncertainty Analysis Tool The basis of the analysis is a base model that must be superior, in terms of TVU, to the survey-spread that is to be tested. In the present context, the base model is termed the Reference Model. The survey-spread to be tested is termed the Test Survey. 2.1. Creating a Reference Model Bearing in mind that the TVU analysis tool is based on comparing a survey line against a reference model, it is of the outmost importance to attain a good, superior reference model. This can be achieved in different ways: By utilizing a superior survey configuration (calibration, acquisition method, instrumentation etc.) By surveying the area with multiple lines and in a variety of directions By thoroughly cleaning and editing the data acquired The ideal method would probably be to do a combination of the three. As a minimum, however, it is recommended to perform the survey several times over the same area, in multiple directions with a substantial overlap. Before the TVU analysis is done the complete survey system must however be completely calibrated. This includes validation/calibration of attitude sensors as well as performing a patch test through which the mount angles of the multibeam transducer are determined. The acquisition of the data could be accomplished as depicted in figure 1, below. Observe how the data have been acquired in directions perpendicular to one another. In an ideal situation, all survey lines must be run twice, in opposite direction. A line-spacing that allows for full coverage in all directions must furthermore be employed. Figure 1 Survey Example for Reference Model
The reference model must be generated in NaviModel3. Normally this would employ loading of the data via NaviEdit, either by linking NaviModel3 to the NaviEdit database or by performing an export to binary NED-files from NaviEdit and subsequently loading these files into NaviModel3. Prior to performing the TVU analysis, cleaning of the data must be performed. It has been established that the most efficient cleaning method, in terms of accuracy and speed, is the automatic S-can cleaning method that can only be performed within the framework of NaviModel3. Figure 2 Reference Survey generated and loaded into NaviModel3 2.2. Creating a Test Survey The reference model has to be tested up against a test survey. It is recommended that this survey is not executed parallel to any of the reference runlines. A good example of such a survey is visualised in Figure 3 below.
Figure 3 Survey Example for Test Survey One of the more important parts of editing the test survey data, is to perform cleaning on it. This can be done directly in NaviEdit, via the PlanView/PoinEdit editor (see Figure 4 below). Figure 4 PlanView/PointEdit editor Alternatively it is possible to link to the NaviEdit database from NaviModel3 and perform the cleaning from there, via the S-can automatic cleaning, and then subsequently sending the cleaned data back to the database. Once cleaned, the test survey data must be exported to an ASCII file containing X,Y,Z, beam angle and beam quality. The format is named Ascii XYZ,Angle,Quality and is exported from NaviEdit as shown below in Figure 5. Alternatively, it is possible to execute the Total Vertical Uncertainty analysis against a singlebeam observation set. The information required for this must be exported from NaviEdit as an ASCII XYZ file. Figure 5 Exporting the ASCII XYZ (Angle Quality) file from NaviEdit Figure 6 below shows an example of an ASCII XYZ (Angle Quality) file.
Figure 6 Example from ASCII XYZ (Angle Quality) 2.3. Performing the TVU Analysis Once the reference model is generated and adequately cleaned in NaviModel3 and the test survey has been created on the basis of the edited and cleaned data, the Total Vertical Uncertainty analysis can be performed. With the reference model residing in NaviModel3, the functionality can be invoked by right-clicking on the Model entry of the Project Tree window and choosing the menu-item Vertical Uncertainty Analysis as shown below in Figure 7. Once this is done, the appropriate test survey file must be chosen. This can either be in the form of an Ascii (XYZ) file, in case of single-beam data, or it could be in the form of an Ascii XYZ (angle, quality), in case of multi-beam data. Figure 7 Invoking the Total Vertical Uncertainty analysis in NaviModel3
Either way however, the Performance Test window will open as shown below in Figure 8. At the same time a Vertical Uncertainty Analysis entry will appear in the Project Tree window. When this entry is selected, the properties of the TVU will show in the Properties window (see Figure 9). Figure 8 Total Vertical Uncertainty analysis window Figure 9 TVU entry in the Project Tree window (left) and properties (right) The result of the TVU analysis is in other words given in an alphanumerical window that contains various, statistical information all associated with the TVU of the test survey, as well as in the three diagrams of the Total Vertical Uncertainty analysis window: Confidence and mean error versus beam angle
Histogram, probability versus error Histogram, probability vs. beams in reference cell 2.3.1. Confidence and Mean Error versus Beam Angle Figure 10 Confidence versus Beam Angle (left), Angle limit and histogram bins (right) This diagram in Figure 10 above shows the quality of beams as a function of the beam angle. The X-axis is the beam angle (in degrees), whereas the Y-axis depicts the associated error in meters. The blue line is the mean error (in meters). The grey area is the depth of 95% of all beams. The Angle Limit option from the properties window, shown to the right in the figure can be used to limit or to expand the angle span (number of beams per scan) and thereby to include or exclude the outer beams from the comparison (the Total Vertical Uncertainty analysis). In an ideal situation, all 95% values are even and close to 0. This is actually the case in the above histogram. Furthermore the magnitude is (more than) acceptable, with an average magnitude between 20 and 25 mm cm and with maximum values around 40 mm.
2.3.2. Histogram, probability versus error Figure 11 Probability versus error, Angle limit and histogram bins (right) The histogram shows the error distribution of all beams below the selected angle limit. The X-axis shows the error in meters, whereas the Y-axis and shows probability in percent. The number of bins can be changed by altering the Histogram Bins as shown to the right in the figure. The example dataset shows an error distribution that is close to the expected 0-value. This is as expected, bearing in mind the conclusion from the Confidence versus Beam Angle comparison above.
2.3.3. Histogram, probability vs. beams in reference cell Figure 12 Probability versus Beams in the Reference model This histogram shows the probability distribution of a given number of beams in the reference model. The X-axis shows number of beams, whereas the Y-axis gives the probability in percent of any given number. Basically this depicts the validity of the analysis. The more beams/observations, the more statistically significant the TVU analysis is. On the one side we would like models with as small cell-size as possible, in order for these generalised representations of the raw observations to be considered as illustrative of the raw data as possible. The danger on the other hand is, that the models and thereby the analysis becomes statistically weak when decreasing the cell size to a value that will result in a low observation population in the cells.
2.3.4. Statistics Figure 13 The Properties window with the Alphanumerical Statistical Information The Alphanumerical, Statistical information, shown in Figure 13 above, refer in general terms to requirements to survey as well as to Total Vertical Uncertainty as these are specified in IHO and in USACE standard compliance. The Settings entry is used to define various parameters for the TVU analysis, as follows: Angle Limit: the option can be used to limit or to expand the angle span (number of beams per scan) and thereby to include or exclude the outer beams from the analysis Histogram Bins: defines the number of bins used in Error Probability and the Beam Count histograms Limit Depth: defines the depth value used to calculate the TVU on the basis of the formula given above (the d-value in the formula). The limit depth can be defined on the basis of the data (minimum, maximum, average of the test survey data) or it can be user defined. The latter is the case in the example above, where it has been set to 20 m
Used Limit Depth: shows the depth value use (d-value), either on the basis of the test survey based entry (greyed out) or on the basis of the user defined input. In the latter case, the user will have to input the desired value IHO Test: True (include IHO Test) or False (exclude IHO Test) USACE Test: True (include USACE Test) or False (exclude USACE Test) Userdefined Test: only available when IHO Test is True. True (include Userdefined Test) or False (exclude Userdefined Test) Userdefined a: only available when Userdefined Test is True. Defines the a-value for the test Userdefined b: only available when Userdefined Test is True. Defines the b-value for the test The General statistics comprise the following information, in sequence: Mean Difference: gives the mean difference between test survey and reference model for the area investigated Standard Deviation: gives the standard deviation of the difference between the test survey and the reference model Minimum Difference: expresses the minimum difference between the test survey and the reference model Maximum Difference: expresses the maximum difference between the test survey and reference model Mean + 1.96 * standard deviation: adds together the mean difference between test survey and reference model and two times the standard deviation of the difference between the test survey and reference model Data Mean Z: expresses the average depth value of the test survey data Reference Mean Z: expresses the average depth value of the reference model Reference Min: expresses the minimum depth value within the reference model Reference Max: expresses the maximum depth value within the reference model The IHO-related statistics are associated with the Special Publication No. 44, IHO STANDARDS FOR HYDROGRAPHIC SURVEYS, 5th Edition, February 2008. It comprise the following information, in sequence: Special Limit: specified in the publication as TVU (Total Vertical Uncertainty) and computed at the 95% confidence level as 2 2 a ( b* d). For special order a is 0.25 m, b is 0.0075 and d is the depth. Consequently the 95% confidence limit for Special Order can be calculated to 0.292 m (for a depth value of 20 m) in the example above in Figure 13 Special Test: the maximum allowable value of 0.292 m is compared to the achieved value of Mean + 1.96 * standard deviation of 0.033 m (see above) Order 1 Limit: uses the same formula as above, only with a being 0.5 m and b being 0.013. Consequently the 95% confidence limit can be calculated to 0.564 m (20 m waterdepth) for Order1 in the example above in Figure 13
Order 1 Test: the maximum allowable value of 0.564 m for the order is compared to the achieved value of Mean + 1.96 * standard deviation of 0.033 m (see above) Order 2 Limit: uses the same formula as in connection with Special Order above, only with a being 1 m and b being 0.023. Consequently the 95% confidence limit can be calculated to 1.101 m for Order 2 in the example above Order 2 Test: the maximum allowable value of 1.101 m for the order is compared to the achieved value of Mean + 1.96 * standard deviation of 0.033 m (see above) User Defined Limit: this item will only appear when the Userdefined Test has been set to True under Settings above. It uses the same formula as in connection with Special Order above, only with a and b being identical to the user defined value (0.05 m and 0.002 in the example above). The 95% confidence limit can consequently be calculated to 0.064 with the depth of 20 m in the example above Order 2 Test: the maximum allowable value of 0.064 m for the order is compared to the achieved value of Mean + 1.96 * standard deviation of 0.033 m (see above) The USACE-related statistics item will only appear when the USACE Test has been set to True under Settings above. The test values are associated with the Engineering and Design, for Hydrographic Surveying, dated April, 1, 2004 from US Army Corps of Engineers. The standards state, that the resultant elevation depth 95% confidence accuracy (1.96 times the standard deviation gives the 95% confidence level, which in turn means that 95 percent of the difference population is less than this value) must meet the following limits for navigation channels and dredging support surveys (note that all limits are given in US Survey feet (1 m = 3.280833 US Survey feet)): System Depth Hard Bottom Soft Bottom Mechanical systems < 15 ft. ± 0.25 ft. ± 0.25 ft. Acoustic systems < 15 ft. ± 0.5 ft. ± 0.50 ft. Acoustic systems 15 ft < 40 ft. ± 1.0 ft. ± 1.0 ft. Acoustic systems > 40 ft. ± 1.0 ft. ± 2.0 ft. Maximum Outlier 1.0 ft. 1.0 ft. Mean Difference (Reference Test) ± 0.1 ft. ± 0.2 ft. Associated with these limits, the USACE test comprises the following information, in sequence: Max Outlier: gives the maximum observed difference between the test survey and the reference model Mean Difference: gives the mean observed difference between the test survey and the reference model (in US Survey feet) Depth Confidence: Expresses the 95% confidence level for the test survey Hard Difference: comparing the value to the limit of ± 0.1 ft. The test is FAILED in the example above in Figure 13 Hard Outlier: comparing the value to the limit of 1.0 ft. The test is FAILED in the example above in Figure 13
Hard Accuracy: this compares the 95% depth confidence value of the test survey with the limit for hard bottom of 1 ft (depth > 40 ft.). The test is ACCEPTED in the example Hard Bottom Test: Compiles the tests in the above for hard bottom. Is passed only if all tests are passed. This is not the case in the example above, since the test is FAILED Soft Difference: comparing the value to the limit of ± 0.2 ft.. The test is FAILED in the example above Soft Outlier: comparing the value to the limit of 1.0 ft for soft bottom. The test is FAILED in the example above Soft Accuracy: this compares the 95% depth confidence value of the test survey with the limit for soft bottom of 2 ft. (depth > 40 ft.). The test is ACCEPTED in the example above Soft Bottom Test: Compiles the tests in the above for hard bottom. Is passed only if all tests are passed. This is not the case in the example above, therefore the test is FAILED 3. Generating The Report By right-clicking in the Vertical Uncertainty Analysis entry in the Project Tree window and choosing the menu-item Generate PDF, an overall report of the TVU analysis can be generated. The settings defined in the properties window are used to generate the report (see Figure 14 below.
Figure 14 Example of Performance Test Reports (2 * 45 degrees opening angle (left) and 2 * 60 degrees opening (right)) Dedicated, graphical plots can also be generated from the histogram window by rightclicking on one of the three histograms. Examples of this is shown below in figures 15-17. Figure 15 The Confidence Plot generated from the histogram window
Figure 16 The Error Plot generated from the histogram window Figure 17 The Beam Count Plot generated from the histogram window