Abstract
1. INTRODUCTION
The signatory countries to the SOLAS Convention (Safety of Life at Sea – International Maritime Organization) are committed to preparing and publishing documents and nautical charts to provide safe navigation. In order to fulfill this responsibility, member countries seek to follow the terms and guidelines of the International Hydrographic Organization (IHO), whose international standards are intended to reduce uncertainties that may result in accidents to navigation. The parameters used for hydrographic surveys, making nautical charts and other activities inherent to hydrography were then standardized to ensure that this uncertainty reached the acceptable value range, guaranteeing 95% reliability (SOLAS, 2014 and IHO, 2008).
Within this scope we have the Chart Datum (CD), also known in some literature as Water Level Datum or Sounding Datum, which is the reference plane used to reduce the depths represented on nautical charts (IHO, 2005). Although there are specifications for hydrographic surveys and for the various activities that involve this work, IHO has concluded that each Hydrographic Service in each country will have the responsibility to choose the methods and prepare their standards, as long as they are able to do so and fulfill the minimum requirements for Safety of Navigation (IHO, 2008). This means that there is not a single method used by Hydrographic Services in different countries for the hydrographic activities that are within the competence of each. This is true of the identification of CD, as each country uses a method that it considers appropriate.
In the case of Brazil, the Directorate of Hydrography and Navigation (DHN) chose to use the Balay method to determine CD, associated with the Courtier Criterion for tide classification (BALAY, 1952, COURTIER, 1939). However, this method is the result of the sum of the most significant components of the astronomical tide, which means that it does not consider the effect of noise and meteorological tide, the non-tidal components (FRANCO, 2009). This factor causes the method to deviate from the ideal in places with great influence of that component.
The Balay method is consolidated, and its use has allowed the DHN to make nautical charts that comply with the concept of safe navigation. Nevertheless, the method is a sum of harmonic constituents of the astronomical tide. The greater the influence from shallow water constituents or from the non-tidal component, the greater occurrence of sea-level heights observed below the CD.
For the first case, calculating the Lowest Astronomical Tide (LAT) from a harmonic forecast would be a solution that would allow the identification of a more accurate CD. For the second case, even a harmonic forecast can deviate from the desired value. For the case of a microtidal regime with great meteorological influence, as is the case in the south of Brazil, LAT can constantly present negative values for the height of the sea level. A statistical method allows the use of both the astronomical portion and the meteorological portion for the calculation of CD, being a solution for both cases.
During this study, the level of reduction was calculated using a statistical method, with the CD being calculated from Probability Density Functions (PDF) for the astronomical tide and non-tidal components of the sea-level, for different tide stations throughout the Brazilian territory. This calculation was carried out for the periods of 15 days, 30 days, 3 months, 6 months, 1 year and 5 years, being limited superiorly by the size of the existing sample for each of the stations. This result was then compared with those observed using the Balay Method to identify which would be more stable and which would be more secure.
Although the statistical method is relatively old, it has not been applied consistently along the Brazilian coast, nor considering the diversity of the types of tides and adverse meteoceanographic conditions. Also, this methodology might be implemented in areas where sea level records are scarce or poorly reliable. Thus, it may be valuable for planning and building port and coastal infrastructure, as well as for implementing risk reduction measures.
2. AREA OF STUDY
The present study looked for places with different tidal characteristics and meteorological influence throughout the Brazilian territory. The astronomical tide follows semi-diurnal patterns, typically two high waters (HW) and two low waters (LW) are observed during a lunar day. This pattern can vary according to the region and according to the local geography or latitude, among other aspects that alter the harmonic constituents (PUGH and WOODWORTH, 2014).
In the extreme north of the country, the prevailing tide is macrotidal, with hipertidal being found in some cities. From 4ºS to 18.5ºS the mesotidal regimen prevails, and between 18.5ºS and 33ºS the microtidal regimen is predominant. Also, the semidiurnal tide pattern (SD) is observed from the extreme north to 22ºS, and between 22ºS and 30ºS, predominant tidal pattern is the semidiurnal tide with diurnal inequalities (SDI). From 30ºS to the extreme south of the Brazilian Coast the prevailing tidal patterns are mixed tide and diurnal tide (D) (VELLOZO and ALVES, 2005).
In order to cover those different tide patterns, the tide stations of Pecém Port, Recife Port, Shipyard Naval Base (EBN), Itajaí Port and the Southern Nautical Signaling Service (SSN-5) were selected, where simulations were carried out within the existing continuous data limit for each of the stations. All tide gauge stations in this study used a float operated shaft encoder, presenting discrete data with sampling rates between 5 minutes and 1 hour. The tide stations on the Brazilian coast can be seen in Figure 1.
The tide datasets used at the Pecém Port station have a temporal resolution of 5 min, starting on November 12, 2016, at 00:00 and ending on February 02, 2017, at 23:55. This data set was the same processed by DHN for the calculation of the station’s Mean Sea Level (MSL) and CD, and it was possible to use this result as a reference for the calculations performed. For the Recife Port, the data set had a sampling rate of 1 hour, starting on November 9, 1978, at 00:00, and ending on October 31, 1986, at 23:00. The tide dataset of the EBN station has a temporal resolution of 10 minutes, starting on May 24, 2014, at 00:00, and ending on May 24, 2015, at 22:00. For the port of Itajaí, the dataset has a temporal resolution of 1 hour, starting on March 31, 1960, at 00:00 and ending on March 22, 1961, at 23:00. Finally, the tide station of the SSN-5 has a dataset with a 5-minute temporal resolution, from January 1, 2015, at 00:00 until January 1, 2016, at 23:55, also being processed by DHN for calculating the station’s MSL and CD.
3. METHODS
During this work, the Harmonic Method and the Statistical Method were applied in the tide datasets for the calculation of the CD, to compare the results obtained for the simulations carried out. At first, the consistency of the dataset was analyzed, looking for time series gaps, and checking whether the tidal curve and the MSL were consistent with what was expected at the site.
Both methods were initiated by carrying out the tide harmonic analysis, in order to identify the harmonic constituents. Those studies were made using the software PacMare, a package of tidal analysis and prediction based on frequency domain. The PacMare was developed by Franco (2009) according to the harmonic analysis method proposed by Franco and Rock (1971).
3.1 Harmonic Method
For the tide classification, the Courtier criterion, “C”, was used, which uses the harmonic constituents obtained by the harmonic analysis. For this, the method must be able to correctly identify the diurnal principal lunar (O1), diurnal principal lunar and solar (K1), semidiurnal principal lunar (M2) and semidiurnal principal solar (S2) constituents (PUGH and WOODWORTH, 2014), applying them in equation 3.1. The result obtained represents the tide class observed at the location, from the values present in Table 1.
(3.1)
Table 1. Tide classification as a function of C.
For the case of the semidiurnal tide with diurnal inequalities, which is divided into three types, it is also necessary to verify the type of diurnal inequality, using equation 3.2 (BALAY, 1952). Then, the Balay method is used to calculate the distance between the MSL and the CD. It depends directly on the Courtier Criterion C and the diurnal inequality 2K, and the equations for calculating the level of reduction can be seen in Table 2. During this work, all the CD calculated by the Balay Method will be called as BCD.
(3.2)
Table 2. Equations for calculating the distance between the Mean Sea Level and the Chart Datum, depending on the values of C and 2K.
3.2 Statistical Method
For the calculation of CD using the Statistical Method (SCD), the probability distributions of the tidal and the non-tidal components of sea-level are considered separately (PUGH and VASSIE, 1978). Separating those tidal and the residual non-tidal components allows us to evaluate the influence of the observed residual component over the sea-level variations that were found at the astronomical tide (PUGH and VASSIE, 1978).
To separate the astronomical tide from the noise, the sea level must be considered as a function of time, containing 3 main components, as given by equation 3.9 :
(3.9)
where 
is the height of the tide referred to the zero of the tide gauges as a function of time,
is the astronomical component of the tide,
is the non-tidal component as a function of time and
is the local MSL (FRANCO, 2009).
The astronomical tide is the sum of periodic constituents generated mainly by the Earth-Moon and Sun-Earth systems interaction, with their physical representation presented by equation 3.10.
(3.10)
Since 
is the amplitude of each of the harmonic constituents,
the angular velocity of each constituent,
the equilibrium phase and
the phase difference in relation to the equilibrium tide (PUGH and VASSIE, 1978).
The identification of the non-tidal component will depend directly on the size of the sample. This is because for a few days, such as 15 days or 1 month, few meteoceanographic phenomena generating significant anomalies at sea level may have occurred, making it difficult to differentiate between the first ones and the astronomical component. In addition, some important astronomical tidal constituents cannot be identified by the harmonic analysis, which means that the astronomical tide will not be completely reliable. In such cases, that will be shown at the non-tidal component.
In general, the non-tidal influence can be considered small, when compared with the astronomical tide (PUGH and VASSIE, 1978), which allows investigating the extent to which that component affects the reliability of the method for smaller samples.
An important observation to be made is that equation 3.9 considers that the components are independent of each other, due to the generating force of each one of them (PUGH and WOODWORTH, 2014). Thus, it is possible to consider that the astronomical and non-tidal components have their Probability Density Function (PDF), with zero covariance (PUGH and VASSIE, 1978).
From this concept, if 
is a given tidal height, the probability of occurrence of is equal to the sum of the probabilities of all products of the probabilities of the possible combinations of astronomical and non-tidal heights that result in .
Thus, being 
, the PDF 
for the astronomical component and the PDF for the non-tidal component, the PDF for sea level 
is given by the convolution of the PDF’s, resulting in equation 3.11 below.
(3.11)
Bearing in mind that the tide gauge data offers data in a discrete distribution, equation 3.11 can be rewritten in the form presented in equation 3.12. To calculate the probability associated with each height, 10 cm class intervals were used (FRANCO, 2009).
(3.12)
With the probabilities associated with each height 
, it is possible to find the Cumulative Distribution Function (CDF) for any level 
of interest, as can be seen in equations 3.13 and 3.14.
(3.13)
(3.14)
Considering η the CD, the value of 
found in equation 3.13 shows the probability of finding a safe level for navigation, that is, a level higher than that shown in the chart, while the value found by equation 3.14 presents a Complementary CDF, calculating the probability of being below the CD, that is, below 
(PUGH and VASSIE, 1978).
With the CDF curve in hand, it is possible to identify the accumulated density for each value of 
, or to identify an associated value for each accumulated density. During this study, the probabilities of 0.5%, 1% and 5% were chosen as SCD in order to compare with the BCD.
With this method it is possible to make different combinations of the astronomical and non-tidal components. In this way, it will be possible to analyze a component of the astronomical tide resulting from an equinoctial spring tide combined with the largest surge observed. Although this simultaneous occurrence is rare, it still is possible, and that methodology includes such cases in the range of probabilities. (PUGH and VASSIE, 1978).
For the application of the statistical method, it was considered that the portion of the astronomical tide is equivalent to the predicted tidal curve from the harmonic constituents calculated for each of the sampled periods. The non-tidal component was calculated from equation 3.9. Although this approximation deviates from reality, the further away from the actual value the predicted tide height is, this error is inserted in the non-tidal values, and appears to maintain safe results during the simulations.
In order to verify the results obtained by the tide harmonic analysis carried out, it was necessary to use a low-pass filter, which would allow the identification of the daily variation of the sea level in each of the analyzed samples. A simplified application of the Godin filter was used for this operation (FRANCO, 2009). This filter methodology causes the first and last 36 hours to be lost, but it allows the results to be compared to those obtained by the main methodology.
The implementation of the statistical method and process automation was made in a Python environment. In addition to the CD values, the code aided the analysis with the comparative graphs of the observed tide, predicted tide and the non-tidal residual, in addition to histograms and graphs of the PDF calculated for each of the locations and periods, probability graphs for each of the classes of astronomical tide, noise and non-astronomical tide as a function of time, and the PDF values and the accumulated density for all possible tide heights found.
4. RESULTS AND DISCUSSIONS
In order to discuss whether the method currently used to define CD is superior to the statistical method, the tide data observed in some ports on the Brazilian coast were analyzed in the light of the literature, each of which was analyzed for periods of 15 days, 30 days, 3 months, 6 months, 1 year and 5 years, with the upper limit of the sample size provided by the DHN. The distance between the MSL and the CD (Z0) values obtained from the methods is summarized in Table 3.
Following the concept that the CD should be “a plane so low that the tide rarely drops below it” (FRANCO, 2009), among the results obtained, the calculated Statistical CD (SCD) of 0.5% proved to be the safest, as it has the lowest occurrence of negative heights. The exceptions occurred in the 3-month sample for the Port of Recife and in the 15-day sample for the EBN. Observing the variation of the SCD of 0.5% between the samples of 6 and 3 months, it is noticeable the reduction in the value of Z0 calculated for the tide gauges of Port of Recife, Port of Itajaí and SSN-5. As the variation between the other samples presented a random profile, and due to the small number of repetitions, it is not possible to assess whether there is a trend. It is important to consider that, apart from the tide observed by the SSN-5 tide gauge, the results obtained in the tide forecasts were able to similarly represent the astronomical tides for the periods of 6 and 3 months, indicating that the variation present in the SCD of 0.5% resulted from extra-tidal effects, which, in turn, are predominantly random.
The forecast for the 15-day sample of EBN tide gauge data, in addition to the significant variation in the SCD, also showed a large variation in the calculated MSL. This joint variation may be due to the presence of a meteorological effect in the period, causing an increase in the local Sea Level during the period.
In all ports, the closest proximity between the Z0 values calculated for the different simulations was observed in the calculated CD using the Balay Method (BCD), making it possible to consider this as the most stable method. In the SSN-5, for example, the difference between the highest and lowest BCD calculated was 1.87 cm, while the SCD 0.5% varied by 18 cm. This is already expected, considering that the statistical method uses the data present in the sample, and extremes do not occur very often, which is why years of observation are needed to identify tidal extremes (PUGH and VASSIE, 1978).
Table 3. Distance between the Mean Sea Level and the Chart Datum calculated, in centimeters, separated by tide station and sample size.
Table with the results obtained from the carried simulations. For each station, the SCD of 0.5%, 1% and 5%, the BCD and the MSL are presented in centimeters, separated by the simulation sample size. “-” was used when there were no results obtained for one sample size.
On the other hand, as mentioned, the calculated value of Z0 was much higher for the statistical method. Referring to the CD calculated from zero of the rulers, in the SSN-5, we have that the safest BCD was found for the 6-month sample (224.9cm), while the less safe 0.5% SCD was found for the 15-day sample (218.9 cm), which was still 6 cm below the best BCD for the site. For the 6-month sample, the SCD of 0.5% was 188.4 cm, that is, 36.5 cm below the BCD calculated for the period. These results are shown in Figure 2.
If we analyze each station individually, it is possible to observe that the value of Z0 calculated by the Balay Method varied little depending on the size of the sample. The greatest variation was 8.4 cm, between samples of 6 months and 15 days for the Pecém Terminal. The MSL, however, changed more significantly, reaching 16.55 cm between samples of 6 months and 15 days for the SSN-5 station.
The large difference observed between the calculated BCD and SCD of 0.5% for the 6-month sample is indicative of the ability to more safely identify the CD using the statistical method, when the influence of the non-astronomical tide is higher than the astronomical tide of a location. From the data in Table 3, we see that the BCD calculated for the SSN-5 from the data set with a period of 1 year was 237.77 cm in relation to the zero of the tide rulers, 6.62 cm above the MSL calculated for the 6-month period.
5. CONCLUSIONS AND FINAL CONSIDERATIONS
In this work, two methods for calculating CD were compared. The first was proposed by Balay (1952), whose principle is harmonic, and the second was implemented according to that indicated by Pugh and Vassie (1978), whose principle is statistical. Both methods were applied and analyzed for the five stations along the Brazilian coast.
Regarding the calculated Reduction Levels, it was possible to observe that in most of the simulations performed, the calculated SCD of 0.5% was below the BCD found for the same temporal sample. The exceptions were the 3-month sample for the Port of Recife, in which the SCD of 0.5% was 5.68 cm above, and the 15-day sample for the EBN, in which the 0.5% NRE was 4.68cm above.
One case that could not fail to be noticed was the CD calculated from the data from the SSN-5 tide gauge. When comparing the SCD and the BCD, it is possible to notice that, if the SCD were used for the location, the reference plane used would be 42.39 cm below the one currently used. In other words, just by inspecting the nautical chart of the place, the navigator cannot be sure if his vessel can navigate the area, which represents a risk to the safety of navigation.
It was also observed that the Z0 calculated by the Balay Method proved to be very stable since they present slight variation in their value, when different sizes of time series are used, considering the MSL to be fixed. The most significant variation was 8.4 cm between the 6-month and 15-day samples from the Port of Recife. On the other hand, there was a remarkable variation in the values calculated by the statistical method, according to the size of the sample used. During the simulations, while the CD calculated by the statistical method varied with the sample size mainly due to the computed Z0 value, the CD calculated by the Balay Method changed with the sample size primarily due to the calculated MSL.
It is important to note that CD is calculated as a function of both Z0 and MSL calculated from the sample used. As shown in table 3, for the tide station of the SSN-5, when considering only the samples of six months and one year, it is possible to identify a difference of 10.99cm. This significant variation reinforces the need for tide stations with an observation period longer than one year to reduce the seasonal influence in the calculation of MSL. Also, since there was a high frequency of negative tide heights for the CD currently used in SSN-5, it is considered extremely important to use other methods, such as the statistical method presented here, for locations with high influence of meteorological tide.
Those results show that the statistical method is feasible to be used to determine a safe CD, being a safe alternative for the Balay Method even when there are small time series or for data from tidal regimen such as those from the Brazilian south coast. Also, the statistical method was able to perform well for the four tidal patterns where the simulations were carried.
For future work, it is recommended to reproduce these simulations in others tide gauge stations to see if there are influences that this study cannot see for different tidal patterns. Also, another suggestion would be to carry those simulations in rivers, in order to find if the method should be able to offer a safe CD in those places.
6. REFERENCES
– Balay, M. A. (1952). Determination of Plane of Reduction of Soundings in any Place. The International Hydrographic Review.
– Courtier, A. M. (1938). Marées. Service Hydrographique de la Marine.
– MARINHA, D. B. (2017). Normas da Autoridade Marítima para Levantamentos Hidrográficos: NORMAM 25. Diretoria de Hidrografia e Navegação, 2a Revisão, Niterói.
– FRANCO, A. D. S., & Alvarenga, J. B. (2009). Marés: fundamentos, análise e previsão. DHN, Niterói, RJ.
– Franco, A., & Rock, N. J. (1971). The Fast Fourier Transform and its application to tidal oscillations. Boletim do Instituto Oceanográfico, 20(1), 145-199.
– IHO. (2005). Manual on Hydrography (C-13). 1st ed. corrections to 2011. Monaco.
– IHO. (2008). IHO Standards for Hydrographic Surveys (S-44).5th ed. Monaco.
– MIGUENS, A. P. (1996). Marés e correntes de maré; correntes oceânicas. Navegação: a ciência e a arte-navegação costeira, estimada e em águas restritas. Niterói: DHN- Diretoria de Hidrografia e Navegação, órgão da Marinha do Brasil, 1, 227-274.
– International Maritime Organization (IMO) (2014). International Convention for the Safety of Life at Sea – SOLAS. Retrieved from https://www.ccaimo.mar.mil.br/sites/default/files/solas_indice-2014_2.pdf.
– Pugh, D. T. (1987). Tides, surges and mean sea level.
– Pugh, D. T., & Vassie, J. M. (1978). Extreme sea levels from tide and surge probability. In Coastal Engineering 1978 (pp. 911-930).
– Pugh, D., & Woodworth, P. (2014). Sea-level science: understanding tides, surges, tsunamis and mean sea-level changes. Cambridge University Press.
– Vellozo, T. G., & Alves, A. R. (2005). Características gerais do fenômeno da maré no Brasil. An. Hidrogr, 61, 121-129.
7. BIOGRAPHIES
Lieutenant Vinícius Fuchs:
Assistant of the Oceanography Division of the Research and Survey Vessel “Vital de Oliveira”. B.Sc. in Naval Science (Naval College, 2016), he completed the Hydrographic Course IHO/FIG/ ICA Category A in 2020 (Directorated of Hydrography and Navigation – DHN).
E-mail : vinicius.fuchs@marinha.mil.br
Gregorio Luiz Galvão Teixeira:
Assistant Professor of Oceanic Tides and of Geology and Geophysisc on Admiralty Radler de Aquino Instruction and Training Center. He received her master degree in Envromental Engeneering from Federal University of Espirito Santo, in Brazil. His current research interests include morphogravimetric variabilities on Brazil continental margin and estatistical methods to compute chart and tidal datums. He has published papers in national and international journals in oceanography and hydrography areas.
E-mail : gregorio.luiz@marinha.mil.br
Teodósio das Neves Milisse Nzualo,
Since 2015 is a Postdoctoral Research Associate in Coastal Engineering at the Federal University of Rio de Janeiro (UFRJ) in Brazil. He holds a Ph.D. (2015) and M.Sc. in Coastal Engineering with emphasis on hydrodynamic and sediment transport modeling on shallow-water regions, from Federal University of Rio de Janeiro (UFRJ); He also received B.Sc in Oceanography from Eduardo Mondlane University (UEM) in Mozambique. Dr. Teodósio Nzualo has 12 years of experience with computational numerical modeling in the areas of: hydrodynamics of rivers, estuaries, bays, lagoons etc; hydraulic modeling; fluvial mechanics; hydraulic works; sediment transport; dredging; Water quality models; control of water pollution and tidal current energy resource assessment.
E-mail: nzualo@oceanica.ufrj.br