Developing a Method to Validate the Navigational Bathymetric Database

Developing a Method to Validate the Navigational Bathymetric Database Katrina Wyllie 1*, Marcus Cole 2, Grant Froelich 1, Matthew Wilson 1, Kurt Nelson 2, CAPT Richard Brennan, NOAA 1, and Travis Newman 3 1 Hydrographic Surveys Division, NOAA/NOS, Office of Coast Survey 2 Coast Survey Development Lab, NOAA/NOS, Office of Coast Survey 3 Marine Chart Division, NOAA/NOS, Office of Coast Survey * Presenter Biography: Katrina Wyllie has a B.S. in Marine Biology from College of Charleston and a M.S. in Ocean Mapping from University of New Hampshire. Katrina has worked with NOAA's Office of Coast Survey for eight years and is presently working for Operations Branch in Silver Spring, MD. Abstract NOAA s Office of Coast Survey is implementing a high-resolution navigational bathymetric database (NBD) with the goal of providing authoritative bathymetry data for current and future nautical chart products and services. A prototype NBD was created for the Puget Sound area in 2016 so initial testing could begin. This paper will discuss the methods used to validate the diverse source suppression rules, traceable metadata attribution, and extracted sounding and contour output. Where and why the NBD output differs from current nautical chart products will be discussed. Lessons learned from the validation methods used in the Puget Sound prototype NBD will help to refine and develop automated validation tools needed as larger regions are incorporated into this system.

Introduction For the past 210 years, NOAA Coast Survey has been supporting safe and efficient maritime commerce with the creation of nautical chart products. The science behind making nautical chart products continues to improve with advancements in technology yet the chart products themselves have not advanced as rapidly. The definition of hydrographic services Coast Survey is mandated to provide by the Coast and Geodetic Survey Act of 1947 and the Hydrographic Service Improvement Act (2008) include not only nautical charts but also production of nautical information databases. To support current and future nautical chart products, including the vision presented in NOAA Marine Chart Division s National Charting Plan, Coast Survey is implementing a navigational bathymetric database (NBD). This is a database of all available bathymetry sources, smartly combined to create an authoritative navigational surface of the ocean from which to derive the soundings and contours found on nautical chart products. An initial prototype NBD was created for the Puget Sound region of Washington State (Figure 1). This area will directly benefit from future rescheming and standardization of nautical chart products due to current inconsistencies between existing products and a need for additional contours to support deep draft vessels. Teledyne CARIS Bathy DataBASE Server and BASE Manager were used to create the NBD for Puget Sound. Utilizing the database server capability and the available software allows for data accessibility and data stewardship. This paper will discuss the methods used to validate bathymetric source suppression rules, traceable metadata attribution, and extracted sounding and contour output. The scope of this project does not include feature objects beyond soundings and contours. Lessons learned from the validation methods used in the Puget Sound prototype NBD will help to refine and develop automated validation tools needed as larger regions are incorporated into this system. Figure 1: Puget Sound region of Washington State 2

Bathymetric Sources Tacoma Harbor, located in South Puget Sound, was selected as the case study for this paper. Tacoma has three U.S. Army Corps of Engineers (USACE) waterways and various NOAA surveys ranging from 1935-2010. The USACE waterway data was downloaded in xyz format directly from the Seattle District website. The data was projected and gridded using basic weighted mean to the finest resolution supported (1 and 2 meters). To support the Source Authority custom attribute used for the combine rules described in the following sections, the USACE grids were clipped to the maintained channel limits (Figure 2). Figure 2: Left: USACE Seattle District webpage of Tacoma Harbor waterways (Thea Foss Waterway, Blair Waterway and Hylebos Waterway). Right: Example in Blair Waterway where USACE data was clipped to maintained channel extents. Red represents data in the USACE waterway and blue represents data outside of the USACE waterway. NOAA surveys in Tacoma are available as Bathymetric Attributed Grids (BAGs) and Bathy Point Store point clouds (Figure 3). The BAGs were downloaded from NOAA National Centers for Environmental Information (NCEI). Bathy Point Store surveys pre-date BAG archival and are point clouds of various survey sounding density based on age of survey and technique used to collect the bathymetry. The point clouds were projected and from the points, a triangulated irregular network (TIN) was created. The TIN allows for interpolation between points so a grid may be generated. Ultimately, grids of 16m resolution were created form the original point clouds. In the newest release of software, Teledyne CARIS introduced an option to grid data at variable resolution based on density, which would be more appropriate than a user-defined 16m resolution. 3

Figure 3: Tacoma data in BAG format (left) and Bathy Point Store point cloud format (right) Many bathymetric sources, besides NOAA and USACE surveys, are responsible for existing charted soundings. Examples include other Federal Agencies, Power Squadron, Port Authority, satellite-derived, and International Organizations. External and crowd-sourced datasets acquired for reasons other than hydrography also exist and may support and improve current and future nautical chart products. One of the goals of the NBD is to ingest all sources of available bathymetry. The available bathymetric sources for the Tacoma case study were limited to NOAA and USACE datasets. Database Catalogue With the diverse sources of bathymetry and the goal of the NBD to end up with an authoritative combined surface to support nautical chart products, it was necessary to create custom metadata attributes in the database catalogue. It is challenging to add attributes to an existing database so being generic yet flexible in the customization is essential. The customized attributes created for Tacoma NBD are listed in Table 1. These attributes were created to support the combine rules and they are described in the next section. Table 1: Custom attributes created for the Tacoma NBD Attribute Description Values dgrton Danger to Navigation 0=true 1=false autrty Source Authority 0=true 1=false supscr Suppression Score Float (0-100) dcyscr Decay Score Float (0-100) uniqid Unique survey ID String (e.g. H12345) While the database catalogue was edited to include customized attributes, it was also edited to set mandatory and conditional flags for certain metadata attributes. For example, Category Zone of Confidence (CATZOC) and Survey End Date (SUREND) need to be populated for the combine rules and were therefore flagged as mandatory. This means these attributes have to be populated 4

in order to load a survey into the database. With the intent on fully using the Python API available in Teledyne CARIS BASE Manager to populate fields from source metadata, the attributes listed in Table 1 were all set to conditional. Figure 4 is an example of a survey s attribution once loaded into the Tacoma NBD using the custom catalogue. Figure 4: Custom and standard survey attributes populated for a BAG Combine Rules Danger to Navigation The customized database catalogue allows for scripted combine rules that support an authoritative, navigational surface. The rules are applied hierarchically so that if there is a winner on the first rule, the remaining rules are ignored. While there are many ways to combine data, it is imperative based on the scope of this navigational database that mariner safety is at the forefront of every decision. For this reason, the first rule is Danger to Navigation (dgrton) = true (0) wins. Regardless of the source of the Danger to Navigation and what water it is located in (i.e. USACE channel or not), a reported Danger is critical to mariner safety and must win until it is shown to not exist or 5

newer, confident data replaces it. Moreover, in the rare case there are two Dangers at the same node, Dangers to Navigation are assigned a Suppression Score (supscr) of 100 which is then decayed (dcyscr) by CATZOC and time so the report from the more confident source would win and if the confidence is the same, the newest date would win. For the Tacoma NBD, a fake Danger was created as a single point cloud point for testing purposes. This would represent the same type of dangers to navigation that are found on nautical charts in other regions that will need to be ingested into the NBD. More consideration will be required to ensure Dangers that are features other than soundings (e.g. wrecks and obstructions) are also adequately represented. It is possible to automatically decay a Danger to dgrton = false once the same unique survey ID (uniqid) is imported into the NBD but due to the safety concerns surrounding Dangers to Navigation, turning the flag from True to False is currently a manual operation. Authority Ruleset The second combine rule is Source Authority (autrty) = True (0) wins. A bathymetric provider may have charting authority over all other sources in an area. The best example of this is USACE data within maintained channels but there may be other situations when this rule applies. When authority = 0, the suppression score (supscr) is set to 90 and decays with time. As demonstrated in Figure 5 with sample data, the newest data wins regardless of CATZOC. As the combine rules are applied in hierarchical order, Dangers to Navigation will have an initial supscr of 100 and an example is shown in Figure 5 as a single, green data point. Score Years Elapsed Since Survey Figure 5: Sample data modeling USACE data collected in a maintained channel (i.e. autrty = 0) 6

The USACE data for Tacoma was clipped to the channel limits and autrty attribute set to true for the data inside the channel. Only two of the three waterways had corresponding dredge areas on the Electronic Navigational Chart (ENC) so the Thea Foss Waterway has autrty attribute set to false. Figure 6 display the result of this source authority rule for Tacoma. Figure 6: Result of the Source Authority Rule for Tacoma NBD. The red data is USACE and blue is NOAA. The maintained channels are represented in the final combine surface because autrty attribute = true. Decay Score (CATZOC, SUREND, and α) Determining if a source survey is a Danger to Navigation or in an area where authority wins is straightforward. The third, and most complicated, combine rule is that the largest Decay Score (dcyscr) wins. However, this rule is mostly ranking incoming source by the CATZOC value, which is a generalized proxy for uncertainty. CATZOC values are defined by depth accuracy, position accuracy, and seafloor coverage (Table 2). As mentioned earlier, CATZOC is a mandatory attribute and must be assessed before a survey is imported into the database. 7

Table 2: Categories of ZOC and their associated definitions For Bathy Point Store data that did not have a CATZOC value assessed, a CATZOC value was assigned using a Python script based on NOAA s CATZOC Descriptions shown in Table 3. Table 3: NOAA CATZOC Descriptions for the ZOCs The suppression score custom attribute (supscr) is assigned directly from the initial CATZOC value: (Danger = 100), A1=99, A2=79, B=59, C=39 and D=19. The years elapsed since survey is captured from the mandatory attribute SUREND which has the end of survey date formatted in YYYYMMDD. The third variable for the decay score custom attribute is the decay coefficient, α. We used the following equation to determine the Decay Score and the greatest value (0-100) wins this combine rule. The decay score allows for newer surveys of a lesser CATZOC to potentially supersede surveys of a superior CATZOC that are significantly older (Figure 7). 8

Score Years Elapsed Since Survey Figure 7: Left: Sample data demonstrating the decay score equation with α = -0.022. Right: The effect of different values of α on the score. NOAA s Hydrographic Health Team is currently modeling changeability with the same idea of CATZOC decaying based on an area s changeability. The 0.022 alpha value used in Tacoma NBD came directly from that project. A constant alpha assumes water depths change at the same rate, which is false. The NOAA Hydrographic Health team defined three change agents (currents, storms and debris) and multiplied those change agent values by alpha = 0.022 to get a more realistic, geographic decay coefficient (Gonsalves et al, 2016). The NBD can directly support their efforts by providing accessible data for the Team to ground truth parts of the model. It is important to reference the CATZOC definitions with the decay model results as some ZOC bands decay by depth accuracy but others decay by possible uncharted features. It is the intent of the NBD to read in the decay coefficient from the Hydrographic Health Team and depending on the resulting raster resolution, it may be necessary to adjust the combine rules so decay is applied on a node-to-node basis instead of a survey metadata attribute. Resolution and Depth The final two combine rules are finest resolution wins and then shoalest depth wins. Where the BAG resolutions overlap from a single survey, the lowest resolution BAG surface is preferred. The resolution rule is followed by the final combine rule: primary z value (depth) is least. This will ensure that if there is a tie, the combine process will consider the navigational purpose of the project and pick the more shoal depth. Figure 8 is an image of the combine rule dialog from CARIS BASE Manager. 9

Figure 8: Order of the combine rules for NBD Traceable Metadata While the product of the NBD is an authoritative navigational surface, the purpose of the NBD is to support current and future nautical chart products. All soundings on the nautical chart must be clearly marked with their source. This is currently achieved by using the attributes Source Indication (SORIND) and Source Date (SORDAT). Any nautical chart products, like soundings, derived from the NBD need to have traceable metadata. Metadata is stored in the Contributor layer created during the combine process (Figure 9). Any of the metadata attributes can be added to the contributor layer but its main function is to feed the SORIND and SORDAT information for derived soundings that will be represented on nautical chart products. The Contributor layer is also helpful in validating the NBD combined surface. It immediately illuminated TIN edge effects from the Bathy Point Store data that was unintentionally masking areas from having derived soundings. It also highlighted survey edge effects from the combine process, as seen in Figure 10. Current NOAA nautical chart products have tabulated channels with no depths or contours. The NBD generated depths and contours over all waters covered by the clipped combined surface. The Contributor layer highlights one of the major ways a NBD can support future nautical chart products in Figure 11 by illustrating how USACE and NOAA data can be charted together with high-resolution contours that will directly support safe and efficient maritime commerce. The National Charting Plan states this as a way to improve chart production efficiency because Coast Survey believes that mariners would be best served by having USACE data distributed seamlessly and simultaneously with NOAA ENCs. 10

Figure 9: Left: The combined depth grid of Tacoma. Right: The Contributor layer of Tacoma showing each of the sources that won from the combine ruleset Figure 10: The top left are NBD-derived soundings shown with the combined depth layer. Below that are the same soundings shown with the contributor layer. The top right are the NBD-derived 1m contours shown with the combined depth layer. Below that are the same contours shown with the contributor layer. Some survey edge effects are noticeable. 11

Figure 11: Contributor layer from the combined Tacoma NBD. The USACE data is shown in blue and contours at 1m intervals are generated for the area, including inside the maintained channel. Validation Approach Sounding Selection Preparation The product of the NBD at this point is a combined surface with traceable metadata that needs to be validated in order to be deemed authoritative and appropriate to supply soundings and contours to nautical chart products. As soundings do not belong on land, the combined Tacoma surface was clipped to the most recent shoreline (Figure 12). Figure 12: Before clipping Tacoma to the recent shoreline, bathymetric data from 1930 s is displayed over land that has been developed in recent years. 12

Vertical Validation: Triangle Test The main validation approach is to find existing charted soundings that are shoal of the database output and then understand why they are more shoal. This could be a missing source or something wrong with the combine ruleset. Finding the vertical disagreements can be accomplished rapidly using the Triangle Tool, a function that will identify new data that is shoal of the existing charted data (i.e. Danger finder); similarly, it can be used to check a prospective cartographic sounding selection to ensure no shoal soundings failed to be selected (Wilson et al, 2016). The output of the Triangle Tool is an image and GIS layers that indicate the location and magnitude of the discrepancies (Figure 13). For the NBD validation process, the opposite math will find the existing charted soundings that are shoal of the database-derived soundings. This extended functionality is currently in development. While locating sounding discrepancies is an automated process, it may be challenging to track down the source of a current ENC sounding if its SORIND attribute has not been updated since the ENC was created from the Raster Navigational Chart (RNC). When ENCs were created, all soundings from the RNC chart had the SORIND attribute mapped to the RNC chart number. The source of these older soundings can be found by manually investigating NOAA s RNC Standards archive. Because many of the documents in the Standards are scanned copies of letters or diagrams with hand-written annotations, this is not a task that can be automated. As the NBD expands to larger areas, there will be more examples of sounding discrepancies, which may help improve the NBD by locating a missing bathymetric source or identifying a custom attribute, or combine rule that is not currently in use. Figure 14: Left: Image result of the Triangle Tool test where NBD soundings are shoal of the chart. Right: Digital output of the Triangle Tool indicating where and by how many chart units the NBD data is shoal of the chart. Horizontal Validation: Radius The other half of validating the sounding output of the NBD is to understand the allowed horizontal migration of a currently charted sounding and assigning a radius from which to look for a vertical match. Maximum cartographic horizontal migration of charted soundings is defined by the NOAA Nautical Chart Manual to be ½ the height of the charted whole number (NOAA, 2013). While radii can be implemented based on this defined RNC-based scale for initial confirmation of the NBD 13

sounding output, it may be more appropriate in the future to define the horizontal radius by the existing source CATZOC position accuracy with a minimum value of the combined grid resolution. Conclusion Coast Survey s effort to rescheme and standardize navigational chart products will rely heavily on a database of all available bathymetric sources smartly combined into an authoritative, navigational surface from which soundings and contours can be derived and traced back to their original source. This is the primary reason Coast Survey is implementing a navigational bathymetric database. This paper described the lessons learned from the initial prototype NBD created in the Puget Sound region. Customized survey metadata attributes were created and added into the database catalogue. These customized attributes were populated with terms that supported navigationally based suppression rules. The process of combining the various bathymetric sources produced a Contributor layer which included all needed metadata to populate the SORIND and SORDAT for derived soundings and contours. This initial approach as described can be used to validate the soundings derived from the database as compared to the existing ENC soundings. The lessons learned from this prototype NBD will help to refine and develop automated validation tools needed as larger regions are incorporated. References Gonsalves, M. et al. (2016), Developing a Spatial Risk-based Model of Hydrographic Priorities in the United States: A Modern Approach to Realizing Survey Efficiencies, in Proceedings Canadian Hydrographic Conference, Halifax, Nova Scotia, Canada. Hydrographic Services Improvement Act. Pub. L. 110-386. 122 Stat. 4106. 10 Oct. 2008. Web. NOAA. (2013). Nautical Chart Manual Volume 1, Version 2013. 1 January 2013. Office of Coast Survey. Marine Chart Division. National Charting Plan. A Strategy to Transform Nautical Charting. 28 Feb. 2017. Web. 10 Mar. 2017. "Hydrographic Surveys" Surveying & Mapping. USACE, Web. 22 Feb. 2017. <http://navigation.usace.army.mil/survey/hydro>. 33 USC. Sec 883a. Chapter 17. 2007. Web. Wilson, M., Masetti, G., and Calder, B. (2016), "NOAA QC Tools: Origin, Development, and Future", in Proceedings Canadian Hydrographic Conference, Halifax, Nova Scotia, Canada. 14