Abstract
1. Introduction
For the last few decades, hydrographic surveyors have adopted a variety of partial, manufacturer-specific, and hand-made solutions to prepare the sound speed profiles for use in acquisition and post-processing applications. Far from optimal, this approach carries a number of disadvantages like being error prone and inconsistent among different sound speed profile systems. Practically, the requirements for a modern survey using acoustic means cannot overlook the accurate environmental characterization of the water column. Approximate or erroneous characterization heavily affects all survey products from the bathymetry and backscatter mosaic through the water column imagery. The sound speed profile is critical for ray tracing, whilst the temperature and the salinity variability through the water column is crucial in the calculation of realistic absorption coefficients.
NOAA Coast Survey has developed over the years, for internal use, a series of applications (the latest one named Pydro Velocipy) to perform the sound speed profiling task and other agency-specific quality control functionalities. In 2012, UNH’s Center for Coastal and Ocean Mapping (CCOM), through the NSF-founded MAC (Multibeam Advisory Committee) project, developed SVP Editor. This application was officially adopted by the University-National Oceanographic Laboratory System (UNOLS) vessels to homogenize the use of sound speed for multibeam acquisition across the fleet (Beaudoin et al., 2011; Johnson et al., 2012). In 2014, the original MAC application evolved into HydrOffice SSP Manager that, among other improvements, enriched the SVP Editor functionalities with a relational database to provide data persistency, and to track the changes applied to the imported raw values.
In 2016, NOAA Coast Survey and CCOM decided to collaborate on the joint development of an open-source application to provide a long-term and reliable solution for sound speed profile processing and management known as Sound Speed Manager. Among the many advantages of such a move, the merge of the many overlapping functionalities between Velocipy and SSP Manager provides a simplified, single, feature-rich application with a larger community of users, a long-term support plan, and accommodation of the operational and scientific needs of both NOAA and UNOLS. Since inception, the main aim of the joint development has been to create an application that fulfills the accuracy and validity requirements of a modern survey workflow. In January 2017, after a few months of testing, Sound Speed Manager was officially deployed to be used as an official tool for the 2017 field season by NOAA vessels (Gallagher et al., 2017), and is available for use by the UNOLS fleet.
2. Main application functionalities
Sound Speed Manager reads data collected in various formats from CTDs, velocimeters, Expendable Bathythermographs (XBT), Moving Vessel Profilers (MVP) etc. The user can enhance the profiles (e.g., extension from oceanographic atlases) and export the processed data in formats that are ready to be used by commonly used hydrographic (acquisition and processing) applications. As shown in Figure 1, when a new cast is imported, Sound Speed Manager presents an intuitive, but feature-rich interface that guides the user through the processing steps required to deliver an enhanced profile to the acquisition system. In this specific example, an XBT cast is imported. In the right pane, the single salinity value stored in the cast (in solid blue) is shown, but is expected to be substituted by the model-retrieved salinity (in dashed orange). This operation also triggers the recalculation of the sound speed values.
3. Input data sources
Sound Speed Manager currently supports raw data formats used by the most common sound speed, XBT and CTD probes (e.g., Castaway, Digibar, Seabird, Sippican, Sonardyne, Turo and Valeport). It also supports network reception of data currently from Sippican and MVP systems. Figure 2 shows the “Import file” section listing the file formats that Sound Speed Manager recognizes. The “ Retrieve from” section provides buttons to retrieve the profile data from databases and through a network connection from Kongsberg SIS. Note that in the Figure, the user has the option to retrieve a synthetic profile from both the WOA 2013 database and the from the older, but smaller WOA 2009 database (~500 MB vs. 17 GB for WOA13).
In addition, the application can retrieve ‘synthetic’ profiles from oceanographic databases, such as the NOAA World Ocean Atlas (WOA) (Levitus et al. 2013), or modeling forecasts (e.g. the NCEP Global Real-Time Forecast System) (Spindler et al. 2006; Mehra et al. 2011). These profiles can be used to extend to a deeper depth, samples collected by devices, complement the values of environmental variables that are not directly measured, such as the salinity for XBTs, or model profile information when no profile sampling effort is made (e.g., transit surveys).
Customization is one of the key points of Sound Speed Manager. Straightforw ard inheritance models have been implemented to ease the creation of new format drivers (for both input and output files) and data listeners (network based, using UDP packets) as well as the support of institute-specific oceanographic databases. These oceanographic databases are usually built by oceanographic data centers through analyzing and synthesizing data sets with a large number of profiles.
The adoption of an open source paradigm provides several advantages from a user perspective. First, the user has direct access to the source code, allowing them to learn what the application is actually doing to the data (https://github.com/hydroffice/hyo_soundspeed). Many details cannot be integrated in the documentation without making it lengthy, verbose, and too difficult to track development changes. Access to the code ‘recipe’ can provide the interested user with all the required levels of information. Further, access to the source code makes it possible for users to directly modify and extend the available functionalities. Sound Speed Manager is implemented in Python (Python Software Foundation, https://www.python.org/). Python is a very popular language, and widely used in the scientific community, so the decision to implement Sound Speed Manager in Python was specifically taken to make it easy to extend. The same level of flexibility is not available in proprietary applications, where the user only has the option of asking the developers to fix a bug, or to add functionality – a process which is subject to the timelines and priorities of the developers, rather than the users.
4. Profile Processing
Once raw data are imported, three plots (sound speed, temperature, and salinity) are presented to the user, with an overview of all of the imported data, for visual inspection. When active and available, data closest to the profile from oceanographic databases/models are also displayed to assist in evaluation of the quality of the imported profile. In case of multiple available options, the user can select which salinity, temperature, and sound speed ‘synthetic’ data to use to enhance the original raw data and to extend the depth profile as required by the output format (e.g. to 12,000 meters for Kongsberg systems) to ensure correct ray-tracing of the soundings.
The application provides an intuitive way to perform interactive graphical inspection (e.g., outlier removal, point additions) of sound speed, temperature, and salinity profiles. Augmentation of the sound speed profile surface layer with measured sound speed values is provided both manually, or by retrieving this information over a network.
The user can visualize and edit the data in a tabular form, and edit the profile metadata. To avoid the execution of repetitive operations, it is possible to define a set of default metadata (e.g., the institution, the survey name, the vessel) to be applied to each new imported profile. The graphic interface allows easy and intuitive validation of any user input.
5. Project Database
Although it is possible to use Sound Speed Manager in a memory-less mode (i.e., loading, processing, and transmitting each profile individually without keeping any record of the data), the storage of the processed profile data in a per-project local SQLite database is recommended (Figure 3). Once stored in the database, the system provides analysis functions and tools to manage the collected profiles.
Database storage empowers the users with a large number of functionalities. This includes re-loading previous profiles so as to evaluate the evolution of the sampled water column environment (Figure 4) through comparison plots. In particular, the database is structured so that each profile may contain three types of stored data: the raw data (enabling reprocessing of the profile from scratch); the processed samples (with flags to identify the various different source of data); and an optional SIS profile (which represents the result of the data reduction, or thinning, process required by Kongsberg SIS).
The adoption of a SQLite-based technology makes the databases both robust and cross-platform. Functionality was also incorporated to ease the transfer of profiles between databases. A possible use case of such functionality is the collection of profiles from multiple survey launches and their subsequent transfer to the mother vessel at the end of the survey day.
6. Output Creation
Given the limitation in the number of samples accepted in a single profile by some echo sounders, after being enhanced and extended, a profile may also need to be sub-sampled before use. This task is performed using a variant of the Douglas-Peucker algorithm (Douglas and Peucker, 1973). It is also possible to preview the thinned profile resulting from of this operation so that the user can verify that the generalized profile adequately represents the original data.
Sound Speed Manager can export the resulting processed data to disk in a large variety of data formats (e.g., multiple cast CARIS HIPS V2, Elac, Hypack, Kongsberg, Sonardyne) or transmit them to well known hydrographic data acquisition systems (Hypack, Reson PDS2000, QPS QINSy, and Kongsberg SIS), or any application that supports network transport of profiles using UDP data packets. Of particular scientific relevance in this context, is the format used by the NOAA National Center for Environmental Information (NCEI). Sound Speed Manager offers an easy way to prepare a set of profiles with all the metadata required by this net CDF- based format so that new data can be submitted to NCEI to feed the creation of future oceanographic models (see https://www.nodc.noaa.gov/submit/index.html for submission details).
It is also possible to visualize the georeferenced metadata (Figure 5), or to export them in several popular formats (Figure 6).
7. Synthetic-profile Server Mode
Currently, Sound Speed Manager can operate in two mutually exclusive modes: Operator mode, and Synthetic-profile Server mode. In most cases, Operator mode as described in the previous sections, is used.
To enable better results from opportunistic mapping when no profiles are acquired, the Synthetic-profile Server mode was developed to deliver WOA/RTOFS-derived synthetic sound speed profiles to one or more network clients in a continuous manner. Given the potential for biases inherent in such an approach, this mode is expected to only be used in transit, capturing the current position from network broadcast packets, and using it as input to lookup reference data from the selected oceanographic model.
8. Plugin Architecture
Sound Speed Manager provides functionalities to help bridge the gap between sound speed profiling instrumentation and acquisition systems. As such, it currently provides a large number of drivers to read file formats and to listen to network UDP packets. However, these two kinds of data feeds are not the only ones available, so a plugin architecture has also been developed so that additional custom data sources can be integrated.
The pilot plugin used to test the architecture was to directly interact with Sea-Bird SeaCAT CTD profilers via an RS-232 serial interface. Other plugins are currently in development, and advanced users are encouraged to contribute back to the project any newly developed plugins.
9. Technical requirements and documentation
Sound Speed Manager is publicly available as both a stand-alone application and a Python package (https://github.com/hydroffice/ hyo_soundspeed).In this latter form, it is already integrated in Pydro, a NOAA-specific Python application set and software distribution.
The stand-alone application is made available as a portable (that is, not requiring an installer) zipped archive on the official webpage (http://www.hydroffice.org/soundspeed/main). This solution is ideal for users that simply want to use the application, without any interest in extending it with custom functionality.
Thanks to the selection of Python as the development language, Sound Speed Manager can also be installed on a wide variety of platforms. It has been tested on various Windows versions (XP, 7, 10), Linux (Ubuntu, Mint) and macOS (Yosemite), using both 32- and 64-bit binary models. Being open source, it can be installed in development mode so that an advanced user, or an oceanographic institute, can modify the source code to specific requirements. A graphical user interface “look-and- feel” that is consistent with the host operating system is made possible by the use of PySide, an LGPL Python binding of the popular Qt graphic library (https://github.com/pyside/PySide).
Sound Speed Manager is documented in a PDF manual (embedded with the library) and online as ‘live’ html-based documentation set (https://www.hydroffice.org/manuals/soundspeed/index.html).
10. Conclusions
Sound Speed Manager is the result of a project jointly developed by NOAA Coast Survey and UNH CCOM. It has already been adopted as the preferred tool to process sound speed profiles onboard a large number of NOAA and UNOLS vessels.
Sound Speed Manager has been designed to ease integration into existing data acquisition workflows. The liberal open source license used by the project (specifically, GNU LGPL) provides for understanding of the chosen processing solutions through ready inspection of the source code, as well as the ability to adapt the application to specific organization needs. This adaptation is eased by the modular design of the application, with the NOAA-specific functionalities organized so that they can be easily deactivated for non-NOAA users. The main functionalities include: wide support of commonly-used sound speed profile formats in in- put and output, full integration with common data acquisition/integration applications (e.g., Kongsberg SIS), profile enhancement based on real-time and climatologic models, and database management of the collected data with built-in functionalities for analysis and visualization.
With a long-term support and development plan, Sound Speed Manager is a turnkey application ready to be used (and extended) by professionals and institutions in the hydrographic community.
11. Acknowledgments
The authors would like to thank Lt. Cmdr. Sam Greenway (NOAA OCS HSTB) for his enthusiastic and proactive support of the joint development solution; Paul Johnson (UNH CCOM and MAC P.I.) for his help in the identification of application requirements; UNOLS and NOAA vessels for many comments and feedback on both SSP Manager and Sound Speed Manager; and NSF for founding and continuing to fund the MAC project. This work was partially supported by NOAA grants NA10NOS4000073 and NA15NOS4000200.
12. References
Beaudoin, J., Furlong, A., Smyth, S., Floc’h, H., and Lurton, X. “Streamlining Sound Speed Profile Pre-Processing: Case Studies and Field Trials”, Proceedings U.S. Hydro 2011, Tampa, FL, USA, 2011.
Douglas, D. H., and Peucker, T. K. (1973). “Algorithms for the reduction of the number of points required to represent a digitized line or its caricature”, Cartographica: The International Journal for Geographic Information and Geovisualization, 10, 2, p. 112-122.
Gallagher, B., Masetti, G., Zhang, C., Calder, B. R., and Wilson, M. “Sound Speed Manager: an open-source initiative to streamline the hydrographic data acquisition workflow”, Proceedings U.S. Hydro 2017, Galveston, TX, USA, 2017.
Johnson, P., Beaudoin, J., and Ferrini, V. “Improving Multibeam Data Quality Across the U.S. Academic Research Fleet”, Proceedings 2012 Fall Meeting, American Geophysical Union (AGU), 2012.
Levitus, S., Antonov, J., Baranova, O. K., Boy- er, T., Coleman, C., Garcia, H., Grodsky, A., Johnson, D., Locarnini, R., and Mishonov, A. V. (2013). “The world ocean database”, Data Science Journal, 12, 0, p. WDS229-WDS234.
Mehra, A., Rivin, I., Tolman, H., Spindler, T., and Balasubramaniyan, B. “A Real-time Operational Global Ocean Forecast System”, Poster GODAE OceanView – GSOP – CLIVAR Workshop on Observing System Evaluation and Intercomparisons, Univ. of California Santa Cruz, CA, USA, 13-17 2011.
Spindler, T., Rivin, I., and Burroughs, L. (2006). NOAA real-time ocean forecasting system RTOFS-(Atlantic). National Weather Ser-vice, EMC/MMAB, Technical Procedure Bulletin.
13. Authors’ biographies
Giuseppe Masetti received a MS degree in Ocean Engineering (UNH, USA) in 2012, and a Master in Marine Geomatics (2008) and a Ph.D. degree (2013) in System Monitoring and Environmental Risk Management (University of Genoa, Italy). As Research Assistant Professor at CCOM/JHC he is focusing on hydrographic data processing and acoustic backscatter.
Barry Gallagher is an IT Specialist with the NOAA Office of Coast Survey for more than twenty-five years. He works as a software developer at the Hydrographic Systems and Technology Branch in Silver Spring, Maryland.
Brian Calder is a Research Associate Professor and Associate Director at CCOM (UNH, USA). He has a Ph.D. in Electrical and Electronic Engineering, completing his thesis on Bayesian methods in sidescan processing (1997). He is currently focusing on statistically robust automated data processing approaches and tracing uncertainty in hydrographic data.
Chen Zhang is an IT Specialist with the NOAA Office of Coast Survey. He received a MS degree in Physical Oceanography (IOCAS, China) in 1996, and a MS degree in Computer Engineering (NCSU, USA) in 2001. He works as a software developer at the Hydrographic Systems and Technology Branch in Silver Spring, Maryland.
Matthew Wilson is a physical scientist with the NOAA Office of Coast Survey. He received a MS degree in Ocean Mapping (UNH, USA) in 2012, and a MBA degree (Penn State, USA) in 2016. He works at the Atlantic Hydrographic Branch in Norfolk, Virginia, involved in data verification, software development, and ship support.