Abstract
1. Background
The background to this sustainment initiative is described in Halls (2015). Several of the sustainment activities have been completed and two major system replacements are scheduled early in 2017. The program remains generally on track and is scheduled for completion in 2018. This paper describes the sustainment tasks that have either been completed or will be implemented within the next 4 months.
2. Bathymetric Information Management
The original DHDB implementation used CARIS GIS to manage the set of validated bathymetry point data. The bathymetry model was also tightly coupled to a tidal polygon model. Both data types became more difficult to maintain due to increased data volumes and data type from modern bathymetry collection sensors and the limitations of the current technology. The AHS evaluated CARIS Bathy Database (BDB) and, for the past 18 months, has been working through the numerous requirements and data migration activities to not only ensure a successful implementation, but also to retain the maximum content of current bathymetric data.
CARIS BDB will be implemented in February 2017. The implementation phase addressed the following:
- Migration of over 2 billion sounding point features into CARIS BDB from the CARIS GIS Most Detailed Data Base (MDDB) structure;
- Each sounding being migrated needed to be correlated with associated data from a survey data quality database (ZOC). This will in- corporate the data quality metadata fields in the sounding record;
- Investigate and rectify any rogue MDDB data;
- Undertake a number of MDDB vs BDB sounding comparison checks;
- Enable a new variable grid tidal surface within the business process. The gridded tidal model is generated and maintained by the Tides Information System (TIS);
- Determine new data deconfliction procedures to build and maintain the validated bathy- metric surface;
- Determine new business processes for the cartographers to maximise the benefits of CARIS BDB processing tools to improve charting tasks;
- Perform parallel operations to phase-in the BDB capability within a period that is as short as possible to ensure a smooth handover of data operations; and
- Once CARIS BDB is implemented and operational, commence a phased decommissioning of related DHDB system software and tools.
The proposed conceptual workflow for bathymetry using CARIS BDB is shown in Figure 1.
The implementation of CARIS BDB replaces a core component of the old DHDB technology and will be a major milestone achievement. It will provide enhanced capability to support traditional national charting roles as well as new capabilities to support national and military charting and support tasks that require high resolution bathymetry (e.g. high resolution AU6 ENCs, littoral operations, tsunami modelling, coastal zone management, etc).
3. Wrecks Management
The AHS manages about 10,000 wrecks within the Australian Area of Charting Responsibility. Of these, approximately 1,300 are shown on charting products. To date, the wrecks data has been managed in a non-geospatial MS ACCESS database. The initiative has undertaken a task to develop and implement a Web- based Wrecks management system on the new enterprise ArcGIS environment (see Figure 2). The new system has just been accepted into operation.
To assist with managing the wrecks shown on national charting products, ArcGIS is used to manage all of the “validated” wrecks (10,000) and will export the “charted wrecks” (1,300) to a CARIS HPD source usage where the representations with the product usages will be established and managed. This will ensure changes to national charting products are triggered from changes to source. The CARIS HPD source data and the ArcGIS validated information are synchronised through a software routine that checks for any differences between features in the two systems.
The ArcGIS database will also export an Additional Military Layers (AML) Large Bottom Object (LBO) product for use in military applications. The LBO product will include a much larger number of wrecks features than is currently shown on national chart products.
4. Maritime Boundaries Information Management
The AHS manages a range of maritime boundaries and regulated area features for use in national and military products. The agency responsible for Australia’s jurisdictional maritime boundaries is Geosciences Australia (GA). The AHS includes GA data in the national ENC and paper charts. For military purposes beyond the national charting area, the AHS uses published information and/or calculates the outer limits using the CARIS Limits and Boundaries software tools. Other regulated areas, military areas, limits, etc. are managed and are either included in national ENCs or exported as an Additional Military Layers (AML) product.
The section responsible for managing these themes has traditionally used two different database schemas and systems to manage the data. Under the improvement initiative, both schemas will be merged into a single schema using S-121 constructs where possible. At this time, the single schema has been created. Data from both current databases have been migrated and the system is being tested. Implementation is due to be completed in November 2016.
5. Tides Information System (TIS)
Development of a system that would streamline the production process and output a digital version of the Australian National Tide Tables (ANTT) commenced in the mid-2000s. Due to the ad-hoc funding arrangement, the system was developed in a fragmented manner with no guarantee of a next stage. Under the sustainment initiative, the TIS development continues in a more structured approach. The TIS stores the tidal levels for all tidal stations in the Australian region (see Figure 3). From this data, the ANTT can be published from the database in PDF as an authorised publication.
The tidal levels were also stored in the DHDB Most Detailed Data Base (MDDB) as a set of polygons to provide a maintained polygonal surface for tidal datum adjustment of sounding data for use in charts. With the introduction of CARIS BDB which requires a gridded tidal sur- face instead, the TIS incorporated a link to ESRI analysis tools to generate and maintain a gridded tidal surface. The grid resolution needs to be variable to support high resolution grid models in key areas such as ports (usually supplied by the port authorities) (see Figure 4) and choke points as well as lower resolution grids for deeper water offshore areas.
an important data issue.
A number of enhancements are currently being developed for the TIS and will be implemented on a prioritised basis:
- Improved output for the ANTT and other product formats (e.g. tidal level panels);
- Inclusion of tidal stream data and tools to analyse and validate;
- Manage surveyed vertical datum levels and control marks collected by AHS surveyors;
- Analysis of tide gauge data;
- Spatial interface;
- New software capability to replace the current AusTides application; and
Future capability to support various S-100 tides related product specifications.
6. Management of Chart Notes
It is easy to overlook the importance of managing cautionary and explanatory chart notes that are used on paper charts and ENCs. When compiling paper charts, the notes were often managed on a product basis. The introduction of ENCs and the need to identify a chart note using the TXTDSC attribute meant that chart notes needed to be managed more effectively.
In the past, the AHS had in excess of 730 chart notes for approximately 470 charts. The AHS adopted a chart note identification scheme and commenced a lengthy process of evaluating each chart note and where possible consolidating like-notes into a single note content. The outcome of the consolidation was a reduction in the number of chart notes to around 310. The following naming scheme was adopted to enable each note to have a unique filename and be versioned to populate the ENC TXTDSC attributes:
AUxxx_yy (e.g. AU245_02)
where:
AU – AHO identifier as per S-62 (IHO, 2016)
xxx – a unique number
yy – a sequential version number
Prior to the sustainment initiative, chart notes were managed in a MS ACCESS database. To improve the management of the notes for all paper chart, ENC and AML products, a sustainment task was undertaken and implemented in 2016.
The new database manages a note’s content, version (current and previous), references to linked paper chart, ENC and unclassified AML products that include the note, as well as references to the AHS corporate file management system, narratives, etc. (refer to Figure 5). The notes can be exported in a number of formats to meet ENC requirements (ascii text), CARIS Paper Chart Composer/Editor formats (.rtf using the note RGB colours), PDF and HTML.
The notes can be exported as a single note or can be embedded together in a parent/child relationship – often used in AML products.
The Chart Notes system also has a routine to analyse CARIS HPD source and product usages to identify any chart notes attached to features that may not be listing the current version in the TXTDSC attribute (see Figure 6). It identifies potential issues and allows the user to decide on the actions they want to take and then apply those actions in a single operation.
7. Chart Product Management System (CPMS)
The adoption of CARIS HPD in 2013 together with the development of new tools and systems to manage metadata has enabled:
- More consistent compilation practices, system interfaces, data encoding, QC and HPD process workflow;
- Improved ENC product quality – the implementation of CARIS HPD has improved data encoding processes and product management. This has resulted in reducing data encoding errors and improved success rates through IC-ENC validation reports;
- Simplified the current complex QMS documents;
- Reduced overheads in training and software support.
In February 2017, a new Chart Product Management System (CPMS) will be implemented and addresses the follow activities:
- Data migration of paper chart metadata (470 charts) from the existing DHDB Product Profile database to CARIS HPD Product Management Centre (Paper) database fields and Paper Chart Editor;
- Data migration of high resolution one-bit paper chart repromat TIFF image files;
- Spatial capture, georeferencing and alignment of paper chart (one-bit raster) repromat files;
- Software updates to associated applications that interface to the current chart management systems (notice to mariner tool, web tools, etc.) (see Figure 7);
- Redefining the AusGeotiff product format.
The implementation of CPMS replaces a core component of DHDB technology and will be a major milestone achievement for the overall project.
An example of the test CPMS interface is shown at Figure 8.
The interface comprises several components with the specific information accessible through the set of tabs (e.g. Details, Files, References, S-11 & INT Chart, Panels, etc.). The window in the top RH corner will enable the user to view images of the paper chart (single repromat, Geotiff), ENC, web service, etc.
8. Enterprise ArcGIS Implementation
Hydro METOC Information Environment (HMIE) is evolving into a distributed information management environment in terms of systems, technology and processes. Under the previous DHDB system and work practices, a number of bespoke databases were created and maintained where DHDB processes were found wanting. To bring together these data- bases and to ensure that the AHS had a scalable enterprise geospatial environment, ESRI ArcGIS was implemented in 2016 as an enterprise capability. Key components (see Figure 9) to the architecture were:
- Scalable at each of the web, GIS application and data tiers to support growth in data volumes, processing and presentation;
- Fault tolerant with redundant components where feasible;
- Compatible with service oriented design principles to enable support for evolving business needs and integration with other applications;
- Able to support the production and publication of web services internally to the AHS and externally to the public web; and
- Capable of participating in overarching high- level business processes being implemented via the current (and future) workflow management tools.
9. Workflow
The current DHDB workflow has not been reviewed or re-aligned to current business processes. To address workflow, the AHS engaged a consultant to review its “as is” state, develop a set of requirements and identify a “to be” state and project plan. A new workflow management system (WfMS) will be developed and implemented over the next 6 months.
The functionality of the WfMS is decomposed as follows (refer to Figure 10):
- Authoring Tools: The capability that allows the development and maintenance of Workflow Models;
- Workflow Library: A persistent store of Activities and Conditionals that enable the rapid development of Workflow Models through reuse and refinement;
- Workflow Engine: The entity responsible for the orchestration and execution of Workflow
Models. This entity manages the flow, events and actions of a Workflow Task (being the individual instantiation of a Workflow Model). The Workflow Engine also maintains the state of Tasks (such that it can report at any time on the status of any given Task) and keeps an audit trail to support forensic activities;
- Workflow Queues: Queues are used by individuals and functions to manage and track their current workflow responsibilities. The Queues are maintained as part of the Workflow Engine state mentioned above, and collectively represent the current work underway within the WfMS at any given time;
- Reporting Tools: The business intelligence toolset for querying state and statistics of Tasks. Note that this capability may not be best served within the WfMS and may benefit from a solution that has a wider remit than just Workflow (e.g. potentially a Business Intelligence suite may regard the state store within the WfMS as just one of many sources it can derive a report on).
10. DHDB – what’s in a name?
The original project that commenced in the mid 1990s was titled the SEA1430 Phase 1 – Digital Hydrographic Data Base (DHDB). Unfortunately to the uninitiated, this was often interpreted as a relatively simple database solution – what was so hard about developing that? In reality, the DHDB was a complex, tightly bound system. Further, the DHDB was initially delivered as a standalone (air-gapped) system which made it difficult to deliver hydrographic products and information services to customers.
In sustaining the capability within a contemporary context, the capability needed to be renamed to remove the simple and usually mis-understood perception of the DHDB as a standalone system or “database”. The name that was decided was the HMIE.
A rolling program of ICT infrastructure improvements has resulted in a connected (HMIE) system which makes it easier to share information and products with external customers. For the purpose of managing complexity, and in order to assist in maintaining security, HMIE was broken down into a number of separate zones. The HMIE zones are described in Table 1 and the interfaces between the
While a significant effort has been invested into securely connecting HMIE, there is a need to continually improve HMIE ICT infrastructure in order to enable future business agility. The current activities include:
- Incorporation of email services into Fed Link, which will enable email to be securely exchanged with Australian Government Agencies;
- Migration of AHS Web site (hydro.gov.au) and supporting servers to Zone 3. This activity will allow work to begin on the automated publication and dissemination of information; and
- Implementation of remote access (similar to CITRIX) to HMIE Zone 1 is being considered, with planning to commence towards the end of 2016.
11. Organisation Impact
Organisations are social entities that have a culture, a formal and informal structure, goals, activities and resources.
On the whole, the AHS organisation structure has not experienced much change since the original implementation of the DHDB in early 2000s. Through implementation activities the AHS is:
- Moving towards a “information-centric” rather than “product-centric” organisation;
- Implementing changes to the physical organisation structure supporting such activities; and
- Broadening aspects of training to extend staff core competencies.
The change to the organisation structure is a challenge, particularly in a government sector employment framework. Due to the Wollongong location, staff turnover is quite low and this imposes limited flexibility and opportunities for staff when it comes to implementing organisation structure and process changes.
12. The next 18-24 months
The sustainment of HMIE still has a number of key systems changes to be worked through. Over the next 18-24 months, the sustainment activities will focus on:
- Survey and Chart Planning capability;
- Hydrographic Product Sales and Distribution System (HPSDS);
- Integration of the Source Information Receipt Framework (SIRF) into the enterprise ArcGIS system;
- Source Data and records management system;
- Business analytics;
- Satellite Derived Bathymetry;
- Management of oceanographic and meteorological information;
- Online services and the exposure of information via Web Service technology;
- Implementation of a new test and development capability – potentially cloud based.
13. Conclusions
The evolutionary sustainment of the DHDB (now HMIE) is a lengthy programme that commenced in 2011 and is now planned for completion in 2018. Adopting an Enterprise Architecture (EA) approach to the sustainment activity is providing considerable benefits for the AHS in aligning its business requirements, strategy and goals and systems.
The evolutionary sustainment of the HMIE is already positively impacting AHS operations. In reality, the sustainment never ends, but at least this overall activity will provide the AHS with a contemporary solution that meets the legislative requirements of Defence and its customers.
14. Acknowledgements
The authors would like to acknowledge the support of the Hydrographer of Australia and the AHS Directors and all staff for their support of the sustainment initiative. The diligence of the Capability Manager and the expertise of the BAE Systems project team supported by the software vendors are much appreciated. This activity is a collaborative effort.
15. References
Halls, I. (2015). “An enterprise approach to the next generation systems environment for the Australian Hydrographic Service (AHS)”, International Hydrographic Review, Publication P-1, No. 13, May 2015.
IHO (2016). S-62 List of Data Producer Codes, International Hydrographic Organization, Monaco.
16. The Authors’ biographies
Ian HALLS commenced work at the Australian Hydrographic Office (AHO) in 1979 as a trainee nautical cartographer and has been involved in the development of hydrographic information management and chart production systems since the mid-1980s. This period included serving several years on IHO ECDIS/ S-57 technical committees. He is a past Director of HSA Systems Pty Ltd and resumed working at the AHO in 2009 after 15 years in private industry undertaking systems engineering, hydrographic surveying and charting activities. He is currently managing a team that is responsible for maritime boundaries, large and small seabed objects, navigation aids, place names, and littoral information depicted on national and military hydrographic data, products and services within of the AHO. Ian is also working with a small dedicated team to sustain the Digital Hydrographic Database (DHDB now HMIE) solution developed in early 2000. This involves software, hardware and ICT refresh of the various source data receipt, validated information, production, distribution, and workflow sub-systems using an enterprise architecture approach.
Matt JONES commenced work at the Australian Hydrographic Office (AHO) in 2000 as an ICT support officer and has been involved in the design, development and maintenance of hydrographic and defence information systems since. This period has included several years as a systems administrator, systems developer, security manager and as the manager of the ICT section. He is a Professional Member of the Australian Computer Society (ACS). He is currently working as a business analyst and is responsible for the cloud environment which is utilised to securely share hydrographic information via the internet. Matt is also working with a small dedicated team to sustain the Digital Hydrographic Database (DHDB now HMIE) solution developed in early 2000. This involves software, hardware and ICT refresh of the various source data receipt, validated information, production, distribution, and workflow sub-systems using an enterprise architecture approach.