JIMAR CONTRIBUTION NO. 98-319
This manual serves as a user's guide for performing sea level data processing on IBM PC compatible microcomputers or clones. The goal is to prepare a scientifically valid data set. The package was assembled and documented by the Joint Archive for Sea Level, a collaboration between the University of Hawaii Sea Level Center and the the National Oceanographic Data Center, a branch of the National Oceanic and Atmospheric Administration. The software includes M.G.G Foreman's tidal analysis and prediction routines. The tidal predictions should be applied to quality control the data. The tidal predictions should not be used for nautical or coastal engineering purposes.
The software is interactive and self-descriptive. However this manual will serve as a useful reference. Step by step instructions are given within for the operation of each program and procedure. Examples of input formats are given in the appendices. Also, examples of output files and plots are detailed. The user is advised to pay close attention to the descriptions in this manual.
The University of Hawaii Sea Level Center (UHSLC) in collaboration with the National Oceanographic Data Center (NODC) has prepared a software package for sea level data processing designed for IBM PC compatible microcomputers under the DOS operating system. The package includes three principle tasks: 1) tidal analysis and prediction, 2) quality control, and 3) filtering. The software is geared to those with prior knowledge of the DOS environment as well as a basic understanding of sea level processing. The goal is to share software on a commonly available platform (the personal computer) that will enhance the quality of sea level data sets. This in turn will promote the growth of the international archives of sea level, thus, providing scientists with greater opportunities to understand the ocean. The Joint Archive for Sea Level, a collaboration between the UHSLC and the US NODC, encourages contributions to its expanding global archive [3].
This package is derived from existing routines which were commonly used for data processing at the Tropical Ocean Global Atmosphere (TOGA) Sea Level Center (TSLC), under direction of Dr. Klaus Wyrtki [7]. Mr. Bernie Kilonsky and Mrs. Shikiko Nakahara wrote the earlier versions of some core routines, many of which date back to the 1970s under the North Pacific Experiment (NORPAX) [12]. Under the TOGA program, major efforts were made to centralize hourly sea level for a scientific archive. Mr. Patrick Caldwell was hired by the NODC to support this task through the JASL. With the goal of transferring the latest information technology and enhancing the quality of data for various agencies in South and Central America, Mr. Caldwell put together the first public domain package in 1988 (Version 1.0). In 1990, the experts of Intergovernmental Oceanographic Commission (IOC) Global Sea Level Observing System (GLOSS) suggested improvements for the first version, resulting in a more complete second version which was finalized in 1991 (Version 2.0), with copies available in English and Spanish. Over 150 copies have been distributed and the package has been demonstrated at several GLOSS technical workshops [10,11]. The TOGA program ended in 1995 but the JASL program continued at the UHSLC. Under UHSLC guidance, modifications to the 1991 package have been made to comply with the Year 2000. Improvements of this manual and software relative to the earlier versions are summarized in Appendix A.
After an explanation of system requirements and loading procedures, a detailed discussion of sea level data processing will be given. But first, a few important points.
The package has been designed to work with hourly data sets. Data stored at higher sampling intervals must be filtered to the hourly level and placed in the UHSLC data processing format. The choice of filtering method is left up to the individual. At the UHSLC, we have used a three-point Hanning filter centered on the hour with weights of 0.25, 0.50, and 0.25, respectively.
This package contains quality control procedures which require modification of data; therefore, make copies of all original data prior to application of this software. Store the original data in a safe location.
Data files provided with this package and examples in this manual are fabricated. These files were purposely altered to show examples of data errors. Do not use the sample data for scientific or engineering purposes.
An option is available for the creation of tide tables. These are to be used as a general reference only. Do not use these for navigational purposes. For official nautical tidal predictions, please refer to the National Ocean Service (NOS) of the National Oceanic and Atmospheric Administration (NOAA).
Graphical subroutines are provided by
¯ ¯ PLOTWORKS,INC,
16440 Eagle Crest Road
Ramona, CA 92065 USA
PLOTWORKS, INC. copyright@1984-89. All rights reserved. The above copyright restricts duplication and distribution of this software.
This package has been developed for use on IBM PC XT, AT, 386, 486, and pentium compatible microcomputers and clones. Because of the plotting subroutines and the complex mathematical algorithms of the tidal analysis program, the execution of the programs will be much faster if the host system has a math coprocessor. For an indication of the differences in execution time for various host systems, the Tidal Analysis Function takes the following amount of time:
machine execution time
pentium ~seconds
386, 20 Mhz 1.5 minutes
XT with 8087, 4.77 Mhz 14 minutes
XT, no 8087, 4.77 Mhz 2 hours
The plotting subroutines require a display card capable of supporting graphics in the EGA, CGA, VGA, or Hercules modes, and support hardcopy output on most IBM and Epson printers as well as on most HP PC laser and wet ink plotters. An option for PostScript output is also available. The package works best with at least 512 Kbyte of memory and assumes the system supports a hard disk drive and optionally a 1.4 megabyte or 740 kilobyte 3.5 inch disk drive.
The batch files (*.BAT) of the UHSLC/NODC package utilize PC-DOS or MS-DOS operating system versions 3.1 or greater, and the source codes of the programs were compiled with the Micro-Soft FORTRAN Optimizing Compiler version 4.10. One undesirable characteristic of this compiler is its inability to overwrite files. Thus, if a program must be run more than once, be sure to eliminate the working files from the prior attempt. This is the most common cause for programs to crash.
The package has been designed for work within the MS DOS prompt environment. However, for Windows users, many of the executables (*.EXE) can be run from within a file manager. However, batch (*.BAT) files which require command-line parameters must be run from the MS prompt environment.
Within this manual, we expect the user to be familiar with basic DOS concepts, such as subdirectories and file names. If you are not familiar with these concepts please read about them and the related commands in your DOS manual. A text editor is also necessary for many of the steps. We have found the Norton Editor to work well for this application. The software in this package contains no internal consistency checks. The user is expected to provide input data files in the correct format, and to answer all prompts correctly. The programs will respond to incorrect answers and formats in an unpredictable fashion, including occasionally stopping the system. If you are having trouble with the package, first check the correctness of your formats and responses. Then execute the package with the sample data files. If the package will not process the sample data properly, reinstall the package from your original distribution disks. As a last resort, contact Mr. Caldwell for help.
The package is supplied on diskettes or can be downloaded from an FTP site (Appendix B). The package must be loaded onto the hard disk of the host system.
The software is provided in a self-expanding ZIP file, SLPAQ2.EXE. First, create a directory with the name SLPR2. There must not be an existing subdirectory with the name \SLPR2. If there is, then rename the existing one. The software of this package is fixed to operate from Hard Disk C: from a subdirectory named \SLPR2 off the root directory; ie., C:\SLPR2.
Next, change directory to SLPR2. Now copy in the file SLPAQ2.EXE.
To unpack the file, enter:
Automatically, the directory tree is created and files are expanded and copied into the appropriate directories. The software is to be operated from the hard disk of the host system from drive C: and has been designed to enhance organization and facilitate file management by utilizing a fixed directory structure (Appendix C). The entire package requires about 4 megabytes of hard disk space.
The first step after loading the software is to set up the STAtion INFOrmation Data INput (STAINFO.DIN) file (Appendix E.1). One must assign a three-digit number to each unique tide station. Next, provide a station name, followed by the latitude and longitude. The next field is for the time meridian, although concatenation of -E or -W is not necessary since this information is obtained from the longitude field. The following field is the time-of-day schema, with 0 denoting hours 1-24 (British system) and 1 denoting hours 0-23. The date of the start of the station and the country are given in the following fields; however, this information is not used by any software, so it can be left blank. Finally, a mandatory parameter is the plot factor flag that determines the scale of the Y-axis on plots of observed data. This is variable because the range of tides varies from place to place. For an explanation of the code, see Appendix E.2. All DIN files are used by FORTRAN programs, which are sensitive to column positions. Therefore, do not alter the column positions of the parameters away from those provided in the original, unaltered files.
The file naming convention conforms to DOS 8.3 standards; that is, 8 characters for the root of the name and 3 characters for the file extension. All formats are ASCII and can be viewed with a text editor (Appendix D). Within this package, the format is referred to as the "processing" format. This is to distinguish it from the UHSLC/NODC ärchive" format.
All data files are kept in subdirectory \SLPR2\DAT. The predicted tides are placed in subdirectory \SLPR2\PRD.
The observed hourly data, the predicted tides, and the residuals (defined as observed data minus predicted tides) have a similar format and file naming convention. Each file consists of a year of values at hourly sampling intervals2.
The UHSLC processing format of the OBSERVED HOURLY DATA is described in Appendix D.1 and the file names have the following form:
C: century (U:1800-1899, V:1900-1999, W:2000-2099) V: file version (letters A-Z) SSS: station number YY: the last two digits of the year For example, VA00385.DAT is the observed hourly data file (version A) for station 003 and year 1985. WA00385.DAT would be the same for year 2085.
The file version is useful for keeping track of modifications to a data file. For instance, the original data could be named version A. Then, after editing and interpolation, the modified version could be called version B, and so on. A useful convention at the UHSLC is to rename the "final" data as file version "Z" to denote that the data have received as much processing as possible.
A routine is available (Section 2) to facilitate conversion of observed data into the UHSLC observed data format.
The PREDICTED TIDES files have the same format as the observed hourly data files, except columns 7-9 of the monthly headers are filled with the letters "PRD". As mentioned in the previous footnote, each predicted tides file has suffixed one day from the previous year and has appended one day from the following year. This is to accommodate some routines which require time shifts. The file naming convention for the predicted tides is as follows:
C: century (O:1800-1899, P:1900-1999, Q:2000-2099) V: file version (letters A-Z) SSS: station number YY: the last two digits of the year For example, PA00385.DAT is the predicted tides file (version A) for station 003 and year 1985. QA00385.DAT would be the same for year 2085.
The file version for the predicted files can be used to keep track of which version of harmonic constants was used, as discussed in Section 4.2. Versions of harmonic constants depend on the time span which was input into the tidal analysis routine.
The RESIDUALS FILES also have the same format as the observed hourly data files, except columns 7-9 of the monthly headers are filled with the letters "RES". The first letter of the file name is always "R" and the remaining letters are the same as the observed data file (file version, station number, and two-byte year). For example, RA00385.DAT is the residuals file (version A) for station 003 and year 1985. RA00385.DAT would also be the file name for year 2085. A convention to denote the century was not incorporated because the residuals files are temporary and one normally only works with a few years at a time during quality control, so the need for multiple centuries would not arise.
The file version for the residuals file normally mirrors the file version of the observed hourly data file from which it was computed. For example, the observed hourly data file, VA00385.DAT, would have a corresponding residuals file, RA00385.DAT.
As will be described in Section 5.2 on reference level quality control, monthly files can be maintained for preliminary incoming data. The file naming convention is as follows:
V: fixed (always "V") SSS: station number YY: the last two digits of the year MM: month For example, V0038807.DAT denotes the file for station 003 during July of 1988.
Because these files are temporarly, a convention to account for the next century was not required.
Related with the monthly file is the PAIR file (Section 5.2) as described in Appendix D.4. It has a file name the same as the monthly file above, except the first letter is an "X". For example, X0038807.DAT denotes the file for station 003 during July of 1988. The PAIR file is maintained in subdirectory \SLPR2\CAL.
Daily and monthly means are obtained through use of a sophisticated filter as described in Section 6. All years for a given site are kept in a single file per type (daily or monthly). The daily data file has an ASCII format (Appendix D.2) and a file naming convention as follows:
D: fixed (always "D") V: file version (letters A-Z) SSS: station number For example, DA003.DAT is the daily data file for station 003, version A.
The monthly data file has an ASCII format (Appendix D.3) and a file naming convention as follows:
M: fixed (always "M") V: file version (letters A-Z) SSS: station number For example, MA003.DAT is the monthly data file for station 003, version A.
Several utilities are available for a variety of applications. The programs can be found under subdirectory \SLPR2\UTIL.
The package has been designed to work with hourly data sets in the UHSLC data processing format. Data stored at higher sampling intervals must be filtered to the hourly level and placed in the UHSLC hourly format. The choice of filtering method is left up to the individual. At the UHSLC, we have used a three- point Hanning filter centered on the hour with weights of 0.25, 0.50, and 0.25, respectively.
One of the early tasks for use of this software is the conversion of your local format to the UHSLC format for hourly data. To facilitate this task, a conversion routine, DTDCNV, has been provided. The input to this conversion program must be of the form such that each record (line) contains Date-Time-Data for a given hour. Thus, there will be one line for each hour with data. An example is given in Appendix E.3. Other aspects of this input file:
Note, in May 2005, programs were added to the SLPR2 package to convert the SLPR2 format to CSV format. The programs are HSL2CSV, DSL2CSV, and MSL2CSV for the hourly, daily, and monthly values, respectively. These are found in the \SLPR2\UTIL directory. This allows easy importation of the files into spreadsheets.
The UHSLC format maintains scientific units in the following forms:
1. feet (in hundredths, eg. UHSLC format 132 = 1.32 feet)
2. millimeters (in whole digits, 1141 = 1141.0 mm)
3. centimeters (in whole digits, 556 = 556.0 cm)
One can convert among these three forms using the program, CHUNIT. To run the program, go to directory \SLPR2\UTIL and enter:
The program is interactive- you will be prompted to supply information about the input and output files and other options.
One can obtain a list of gaps in an observed hourly data file using program GAPCOU on directory \SLPR2\UTIL. To run the program, enter
The program is interactive and prompts the user for file name attributes. The output is placed in file MISsss.TXT in the same directory from which the program was executed. This utility is useful for identifying short gaps (see Section 5.4) and also for maintaining a log of missing data.
Some programs, such as the time shifting program (Section 2.5) and the filtering program (Section 6) require the hourly data files to have either data or missing data flags for each hour of the year. To facilitate the creating of missing data flags, one can use the program FILLVM. To run the interactive program, go to directory \SLPR2\UTIL and enter:
The output is placed in the directory \SLPR2\DAT within file, FILL.DAT. One must then use an editor to paste the missing data flags into the appropriate place within the hourly data file.
The hourly data files can be moved forward or backward in time (maximum of 12 hours in each direction) by exact increments of an hour using the utility, TSALL. This program is applied to change the time zone reference of the data or to correct timing errors, as will be discussed in Section 5.3. This package does NOT provide a utility to shift the data by fractional hours in time. For example one can NOT shift in time by 15 minutes. If a time zone is not exactly on the hour, for example, locations in India have half-hourly zones, one can not use this program to shift the data to GMT. The program can run on multiple consecutive years. However, one must ensure that each observed hourly data file has either a data flag or a data value for each hour of each year. To create blocks of missing data flags, see utility FILLVM (Section 2.4). To run this interactive program, enter:
The following prompts will be given:
Plotting programs supplement many of the core functions in this package. The routines are introduced here, but one will refer back to them many times in the sections that follow. The plotting programs are found in subdirectory \SLPR2\PLOT.
Graphical subroutines are provided by
¯ ¯ PLOTWORKS,INC,
16440 Eagle Crest Road
Ramona, CA 92065 USA
The hourly data plot program, HOURYR, is the most commonly used program of this package and supplements the tidal analysis and prediction routines, as well as the quality control routines. One of the first tasks after conversion of your local format to the UHSLC format is to plot the observed data. To run this program, type:
Several options are requested from the user:
Y-Axis Scale Settings. For the observed hourly data and predicted tides, the Y-axis scale will depend on the tidal range of the given location. For each station, a plot factor is coded in file STAINFO.DIN on \SLPR2\DIN, (Section 1.4 and Appendix E.1) to set an appropriate range. Thus, choose "0", ßcale determined by data". For plots of residuals, a fixed scale is normally chosen.
Output Destination of Plot. One has a variety of options for plotting on a monitor, a printer, a plotter, or to a PostScript file as described in Appendix E.2. If the size of the plot on your selected output media is not appropriate, one can edit file PLOTSIZ.DIN on \SLPR2\DIN to expand or contract the scale.
If the HOURYR program does not allow enough detail, or if one is concerned about a feature exactly at the border of the plot (start or end of the month), then one can expand the plot using program, EXPLOT. To run the program, type:
You will need to enter the file version and station number, the date, the number of days for the plot (maximum of 31), a code for the Y-axis scale, and an output destination.
A basic scatter diagram of the tide staff and tide gauge pairs (Appendix D.4) (found in \SLPR2\CAL) can be plotted by typing:
One simply enters the file name of the tide staff/gauge pair (eg. X0038807.DAT) and a code for the destination of the output. This program generates four output files which contain summaries of the input data: RL16.OUT, RL17.OUT (Appendix D.5), RL18.OUT, and RL19.OUT. Prior to re-running SCAT, one must rename or delete these files from the directory from which SCAT was executed.
The daily and monthly plot programs work similarly. Both are interactive and require information regarding file version and station number, scientific units of input data, start year and number of years (maximum 30 for daily and 150 for monthly), Y-axis scale, X-axis ticking and numbering scheme, and output destination. To run the daily program, enter:
and to run the monthly program, enter:
Both programs can also plot the difference between two stations, as described in Section 6.
The goal of this package is to quality control the hourly data files. The techniques have been well documented [2], [8]. The basis for quality control is the inspection of plots of residuals, defined as observed data minus predicted tides. In the following sections, the tidal analysis and prediction routines, the plotting programs, and the various quality control procedures are outlined. Lastly, a filter routine for the creation of daily and monthly means from the hourly data is described. Many of these routines must be performed in a cyclical fashion; thus, one will need to refer backwards and forwards within the manual as necessary.
In this manual, a majority of the routines are fixed to work with the UHSLC hourly formats grouped by year (one year per file). For one section of the manual (section 5.2), the software works with hourly data grouped by month (one month per file). This is for a special case for those that have tide staff observations available to calibrate the tide gauge values. If so, please jump to this section and review the material. When calibration is complete, these monthly files in turn are concatenated to form UHSLC hourly files grouped by year, on which the remaining routines can be applied.
This software package utilizes the tidal height analysis and prediction programs of Dr. M. Foreman of the Institute of Ocean Sciences, Victoria, British Colombia [4]. Various notes regarding his procedures are found in Appendix F. The analysis program applies a linear least squares fit for specified harmonic constituents (astronomical arguments), which are based on earth, moon, and sun astronomical configurations. The output of the analysis is a set of harmonic constants (Appendix G). These in turn become the input for the tidal prediction function, which can produce predicted tides for any year of any century. The UHSLC/NODC package has facilitated the use of the Foreman routines by providing self-descriptive, interactive batch jobs and programs which allow input (output) of the UHSLC formats to (from) the Foreman programs.
This program analyzes the hourly heights tide gauge data for a given period of time. Amplitudes and Greenwich phase lags are calculated via a least squares fit method coupled with nodal modulation for only those constituents that can be resolved over the length of the input record. The output is a file of harmonic constants which become input for the tidal height prediction program.
The first step in use of the tidal analysis is to determine a suitable time span as input. This involves the use of the hourly data plot program, HOURYR, found on subdirectory, \SLPR2\PLOT, which was described in Section 3.1. Plot and inspect the available years of observed data.
This package is fixed to allow a maximum of 68 constituents (Appendices F and G), which can be achieved through analysis of a record length of 366 days. The program will not work (it will abort) if an input record greater than 13 months is requested. If one would like to apply Foreman software for longer periods of analysis, then one has the liberty to use the Foreman routines, but you will need to provide input in the Foreman format [4]. The program will function for record lengths less than 366 days; however, the number of output harmonic constituents will be fewer and subsequently the quality of the constituents will be less. For instance, a record length of 30 days will produce 30 constituents and a record length of 14 days will produce 11 constituents.
From the hourly plots choose a time span of at least 366 consecutive days for a period with apparently good data (no obvious spikes or corruption) and with minimal data gaps. From the example shown in the Figures 1 and 2, a time span from the first hour of 01 April, 1985 to the last hour of 01 April, 1986 would satisfy this criteria. The Foreman Analysis program can handle data gaps, but it is preferable to choose a period without them. The time span can begin on any hour of any day and can end on any hour of any day in the following year, as long as one does not exceed 13 months. One restriction is that the time span CAN NOT cross the century boundary; that is, the input span can not include the consecutive months of December 1999 and January 2000.
Once a time period for analysis has been selected, one is ready to run the Foreman program. The procedures are found in subdirectory, \SLPR2\TIDE\ANA. Go to this directory and execute the TIDEANL.BAT batch job by entering:
P1 denotes a three-digit station number for the chosen station; eg., 003. The interactive job will scroll messages to the screen to describe the procedure. One must hit a carriage control (RETURN) key to scroll from page to page of the initial screens with notes.
The following options will be displayed:
If a grossly bad value exists in the input hourly data file within the time span provided to the TIDEANL job, then the Foreman program will abort. A useful way to find the input record with the bad data is to do the following.
2) Rerun the TIDEANL job. When the new PAUSE command is reached (just after execution of FITTIDE), hit CNTRL-C, then answer Y to exit the batch job.
3) Edit file TGOUTPUT. If a bad record was encountered, it will be identified.
4) Go to the input data file on subdirectory \SLPR2\DAT and replace the questionable value with a missing data flag, 9999. If the questionable record is not identifiable, replace all values on that line (12 values) with 9999s.
5) Rerun TIDEANL.
If the above methods do not help, choose a different time span as input. If you are still having difficulties, contact Mr. Caldwell at the University of Hawaii.
Tidal prediction utilizes the harmonic constituents calculated by the tidal analysis (Section 4.1) to compute predictions in the form of either equally-spaced (in time) hourly values in the UHSLC processing format or high-low heights in the form of a tide table3.
Prior to running the Foreman tidal prediction program, one may need to alter some default settings that are set in file PRDVP.DIN located in the \SLPR2\DIN subdirectory:
1) *units of equally-spaced predictions = 2
2) units of harmonic constants = 3
3) start month of predictions = 12
4) interval between predictions = 60
5) time scheme (0:1-24 1:0-23) = 1
*units code 1:feet (hundredths)
2:millimeters (whole digits)
3:centimeters (whole digits).
For example, PRDVP.DIN contains one record in the following
format: 2 3 12 60 1
As seen above for parameter 1, the scientific units for the predictions defaults to millimeters. For the quality control routines that follow, one should choose the units of the predicted tides to be the same as the units of your observed hourly data. Thus, if you do not have your units as millimeters, YOU MUST MODIFY PRDVP.DIN.
The parameters 2 - 4 above should not be altered. An explanation for obtaining predicted tides for equally-spaced intervals less than one hour is given in Appendix H. If the user's data files have a time scheme convention of 1 to 24 hours, then parameter 5 should be changed to a value of 0, so that the raw data files and the predicted tides files will be compatible.
To initiate the tidal prediction step, enter:
CD \SLPRC\TIDE\PRD
TIDEPRD p1 p2 p3 p4 p5 p6
where p1 : file version for output predictions file (A-Z)
p2 : station number (i.e. 005)
p3 : year to be predicted (i.e. 88)
p4 : time reference meridian of the predicted data
which should coincide with the reference meridian
of the raw hourly data file that is input into
the tidal analysis program (note the hemisphere
(E or W) is not needed)
(i.e. 075 or 000 for GMT)
p5 : numerical code for the form of predicted data
(0:high-low 1:equally-spaced)
p6 : code for century
(18: 1800-1899, 19:1900-1999, 20:2000-2099)
Because it is awkward to remember these parameters each time, one may initiate the job by typing:
The second message screen has the parameters defined. Thus, one can enter CONTROL-C to stop the program and Y to terminate the batch job. Then one has the parameters visible on the monitor screen which makes it easier to initiate the job with the correct parameter entries.
The output of the program will be placed in subdirectory \SLPR2\PRD either file CVSSSYY.DAT (Section 1.5) or HLsssyy.DAT for equally-spaced (hourly) or tide tables, respectively. For convenience, the file name specifics are repeated below:
C: century (O:1800-1899, P:1900-1999, Q:2000-2099) V: file version (letters A-Z) SSS: station number YY: the last two digits of the year
For example, PA00385.DAT is the predicted tides file (version A) for station 003 and year 1985. QA00385.DAT would be the same for year 2085. HL00385.LIS is the tide table for station 003 in year 1885, 1985, or 2085. The filename for the tide table option does not distinguish century although the actual predictions do.
Special consideration must be given to the change of century years. If one is running this program for the last year of a century (eg, 1999), then the TIDEPRD batch job will send a message to the screen showing the last hour of that year. This value should be written down and saved. Then when making predictions for the following year (eg. 2000), the value just mentioned should be hand-edited into the year 2000 predictions to replace the missing data flag (9999) of the first hour of January. This step is essential for use of the gap interpolation programs (Section 5.4).
Quality control ensures the scientific validity of the data. Three main aspects are emphasized: 1) the linking of the data to a reference level (tidal datum), 2) the inspection of the timing quality, and 3) the replacement of short gaps and spikes. Technical aspects of quality control procedures have been well documented [2], [8].
The tidal analysis and prediction programs (Section 4) and the plotting routines (Section 3.1) are an integral part of the quality control task. Many of the routines are run in a cyclical fashion. In the discussion below, one will be referred to various sections in this manual.
Prior to initiating quality control of the data, make copies of the original observed data files. It is not uncommon to make mistakes during editing; thus, it is very important to preserve the original version of the data. Also, keep a log of modifications as you proceed. It is good practice to keep editing notes in a text file, which can be saved with the final version of the data to clearly indicate what changes were made to the original measurements.
The basis of quality control is the inspection of residuals, defined as observed data minus predicted tides. Residuals are made from program, RESIDM, in subdirectory \SLPR2\QC. To initiate the program, go to this directory and enter:
The interactive program will ask the following questions:
Next, the residuals are visually inspected through use of the HOURYR plotting program (Section 3.1). This program graphs one year at a time. One should make a paper copy of each year (file) of residuals.
If the observed data is of good quality and the tidal analysis estimated well the tidal species at the site (ie., good predictions), then the residual plots will have a smooth appearance (Figure 3). In this case, there are no obvious problems with the data.
If the observed data has erroneous values but the predicted tides are accurate, then the residuals plot will clearly show the faulty values (Figure 4). In this case, some periods within the plot are smooth, while others show significant fluctuations, gaps, and spikes . Techniques for correcting these problems are discussed in Sections 5.3 and 5.4.
In locations with highly non-linear tides due to shallow waters, influence of rivers or estuaries, or complex basin configurations, the Foreman tidal analysis routines will not be able to completely resolve all the tidal species. At such a location, assuming the observed data are of good quality, the residuals will show the tidal species that have not been resolved by the tidal analysis. This situation is shown in Figure 8 for a location on the western coast of Malaysia. Thus, quality control is limited at this site. Only grossly extreme spikes in the residuals could potentially be identified in the observed data file.
For most locations, the harmonic constants, which were produced by the tidal analysis, are applicable for prediction of any year. However, at some locations, the tidal species are not completely resolved and thus the quality of the predictions decreases as the predicted year moves further away in time from the time span used in the tidal analysis. Note the residuals in Figures 9 and 10. The first figure shows residuals of 1995 observed data based on a tidal analysis of December 1991 - December 1992. Note the significant fluctuations. In the second figure, the residuals are based on the same year as plotted. In this case, the residuals are smoother. Since the goal of the residuals is to identify potential errors in the data, then for some sites with tidal species similar to this example, it would be best to refresh the harmonic constants file using time spans of tidal analysis close to the year in review.
It is also possible that the harmonic constants could have been created from a time span that is less ideal than another available time span of the same time series. For instance, assume one creates a series of residual plots for various years, and notes in the residual plots that the time period selected for the tidal analysis had timing errors and other data problems. In this case, it would be best to recreate a harmonic constants file based on a more suitable time span as input to the tidal analysis. It is good practice to recreate a harmonic constants file after quality control is complete so that one has the best possible set of constituents to base future inspections.
The calibration of sea level data is necessary to relate the measurements recorded by a sensor to a stable datum [9]. Often called the tide gauge datum, this relationship defines the horizontal plane from which heights are measured at the tide gauge. For most float and stilling well gauges using analog or punched paper tape recorders, the tide staff zero is the fundamental calibration point, which is geodetically linked to a primary Tide Gauge Bench Mark and several auxiliary bench marks. The procedures for the establishment of a tide staff and the associated network of vertical control points are well described [5]. Visual staff readings are used in the calculations of the tide gauge datum as shown below. Many modern acoustic gauges have internal calibration systems so the following steps are unnecessary [6]. Also, automated calibration devices, such as the UHSLC switch, provide supplementary calibration information. Use of switch data is not discussed here (see [9]).
Improper calibration leads to reference level shifts. Most shifts are readily identifiable in hourly residual plots (see Figure 4 at end of December). They can also be seen in plots of the daily and monthly data (Section 6) and in plots of differences of daily or monthly values with nearby tide stations or with redundant sensors at the site in question. If a shift is identified, the proper means of correction is through analysis of the tide staff readings and corresponding tide gauge values.
Normally the link to the tide staff is made with the observed data of the highest frequency of samples; for instance, most data are digitized in 4-, 6-, 10-, 15-, or 60-minute intervals. For punched paper tape recorders, the tide staff reading is made exactly at the time of the punch. For analog data, the tide staff reading is taken exactly on the hour.
Since the tide staff readings and most digital and analog punch paper tape or strip chart rolls come in on a monthly basis, this procedure is performed each month. For this reason the UHLSC processing format is designed for files consisting of either one month of data or up to one year of data in monthly increments. The monitoring of the reference level stability is most naturally performed on monthly files. These monthly files (Section 1.5.2) in turn are concatenated into yearly files at a later date, at which time the core functions of this package can be applied. These monthly files of hourly data must be kept on \SLPR2\DAT. An important field in these files is the REF= in the header. The value given will be the value that must be subtracted from each hourly value in order for the series to be linked to the station tide gauge datum. The reference level quality control programs are found on \SLPR2\QC.
The method for calibration using tide staff readings is as follows.
As a rule of thumb, for float/well analog and punched paper type gauges, the UHSLC considers changes greater than 0.05 feet (1.5 cm) as significant and worthy of an investigation into the level stability. Most acoustic gauges have greater accuracy. With the float/well systems, the UHSLC employs redundant gauges [9] at a given tide station to bring the accuracy of datum correction to finer levels.
If a reference level shift has occurred, follow these guidelines:
Timing errors are introduced into the data due to mistakes during data processing, to incorrect setting of the timer on the tide gauge, or to inaccuracies in the gauge time clock. The errors are evident in the plot of residuals as periodic fluctuations. An example can be seen in Figure 4. Note the large fluctuations in November and December, and the mild fluctuations during other months.
As discussed in Section 5.1, residual plots can reflect inaccuracies in either the observed data or the tide predictions. The ability for the tidal analysis to fully model the tidal species depends greatly on the location. For gauges in ports adjacent to deep waters, the analysis is usually very good. For gauges in regions with influence of rivers, shallow coastal shelves, narrow basins, or complex basin bathymetry, the analysis is of low to moderate quality. In this case, assuming the observed data are good, the residuals with show periodic fluctuations for tidal species that were not resolved by the tidal analysis (Figure 8).
With this software package, one can only adjust the timing of observed hourly data files by exact increments of an hour. Once a timing error is identified in the residuals, there are three ways to determine the magnitude of the timing error:
As an example, see the timing error in November 1986 of Figure 4 and the inspection of magnitude in Appendix I. After applying the steps above for a positive one hour shift, the residuals are shown in Figure 11. Note that the November period is now smooth. Thus the choice of a positive one hour shift was correct. Also note in Figure 11 that other time periods now have significant periodic fluctuations. These can be ignored.
To know exactly which hour the shift began, one can look at the residual file (RA00386.DAT) of the original data file (VA00386.DAT). For the example given above (shift in November), there are gaps on both sides of the shift, so it is easy to identify where the time period with the shift begins and ends. When there is not a gap adjacent to a shift, then on uses the plot (Figure 11) to guide you to the day of the shift. For instance, there is a timing error on December 4-7 of this same example. It is obvious where the timing error begins because there is a gap proceeding it. But it is unclear on which hour the timing error ends. Looking at the residual file for the 6-8 December (first 11 columns of each line of file removed below to fit in text):
198612 61-1784-1449 -774 54 822 1335 1468 1188 571 -243-1039-1609 198612 62-1807-1602-1028 -209 646 1336 1689 1629 1177 447 -387-1116 198612 71-1566-1645-1339 -733 0 675 1120 1224 975 441 -254 -947 198612 72-1438-1601-1404 -882 -161 577 1176 1478 1415 1006 361 -372 198612 81 371 -149 -160 -159 -137 -109 -88 -75 -73 -81 -101 -124 198612 82 -135 -151 -158 -154 -135 -106 -77 -55 -41 -30 -33 -47
Note the dramatic difference from value to value during the period with the timing error, and the small difference between adjacent values for the periods without a timing error. In the example above, it appears that the timing error ends on hour 00 of 08 December. Make a note of the hour and date of the start and end of the period with the timing error.
The next step is to make a copy of the original file. For instance, if the file is VA00386.DAT, then copy it to VB00386.DAT. This will preserve the original in case the following edits need to be removed. As pointed out, original file versions must be preserved.
Use a text editor to modify the copy of the original (VB00386.DAT). One deletes the hourly values for the time period with the timing error. In a separate window, load the output of TSALL (for example, VT00386.DAT). Cut the corresponding period with the corrected shift from VT00386.DAT and paste into VB00386.DAT for the deleted time period. At UHSLC, we normally replace the start and end hour of the shift with a missing data flag (9999) since the exact hour of when the timing error began or ended is usually not clear. The missing data can later be interpolated as discussed in Section 5.4.
To verify the correction, create residuals for the edited file (create RB00386.DAT) and plot. If timing errors are still present, repeat the process above.
It is sometimes trial and error in discovering the magnitude and direction of a shift. One may have to iterate through the steps above several times before a correction is found. And recall, this package can only correct shifts of exact increments of an hour. If timing errors exist due to shifts of partial hours (such as 15 minutes, etc.), then this package is not applicable. Note in Figure 4 the small periodic fluctuations throughout January to October. It is suspected that these fluctuations are due to minor timing inaccuracies of the tide gauge clock (note how the amplitudes of fluctuations increase with time). This package can not correct this timing error. A note regarding these timing uncertainties would be made in the final document file that accompanies the data in the final archive. If the originally sampled data are at intervals less than an hour (6-, 10-, or 15-minutes are common), then one should try the technique mentioned but for exact increments of the sampling interval of those data. Unfortunately, one will have to develop their own routines, since this package is geared to hourly values.
Short gaps and data spikes are a common problem in most sea level time series records (See example in Figure 4 in December). This section describes a technique for using interpolation to correct such problems.
Program GAPCOU (Section 2.3) is a utility that lists the length (hours) and dates of missing data. Run this program to obtain this information.
The best procedure for filling gaps is to replace the missing data flags (ie. 9999) with quality controlled data from an auxiliary sea level gauge that is linked to the same datum. If a redundant sensor in not available, then another good method is linear interpolation via the predicted tide method through application of batch file GAPFALL.BAT (program FALLPTM) on directory \SLPR2\QC. Technical aspects of running this program will be discussed shortly, but first, a note about the methodology.
The predicted tide method for filling gaps requires yearly files of observed and corresponding predicted data. The predicted tides are shifted in time to match the timing characteristics of the observed series. The residuals between the predicted tides and the observed data are calculated. Then, a linear interpolation between the end points of the gap in the residual series is performed and each interpolation constant is added to the shifted predicted tides over the span of the gap.
The UHSLC recommends using this procedure only for gaps less than or equal to 24 hours. This is essential for the integrity of the daily data which can be calculated from the hourly data (Section 6).
To run the program, go to the \SLPR2\QC directory (mandatory), and enter:
P1 is the station number which must be three digits long (ie. 003). The program will query the user for the following information:
1) the file version of the observed data file with the
gap (ie. A),
2) the file version for the predicted data (typically A),
(ensure the scientific units of the observed data
and predicted tides are the same)
3) the file version for the output data with the gap
filled which must be different from the original
version (ie. B, C, etc.),
4) the year with the gap in form yyyy (ie. 1986),
5) if a gap occurs on last few days of December, then
one can optionally acquire data from January of
the following year. However, the observed and predicted
files of the following year must exist, and have the
same file versions.
6) On can add a header to the output listing of
file GAPsss.TXT. This output file details the
hours and dates that have been interpolated.
If one runs this program on a consecutive number of
years, one may choose to only place the header in
the output listing once during the first execution of
this program for a given station. Subsequent runs
of the program on other years are placed in the
same output file, GAPsss.TXT.
The following output is produced:
To correct, first identify the date and time by looking in the residuals data file. The spike should stand out as an outlier in the data. Second, use a text editor and replace the corresponding value in the observed data file for the date and time of the spike with a missing data flag, 9999. Finally, run the gap filling program as described above.
One should verify the output of the gap interpolation by making and plotting residuals.
Software is provided for obtaining daily values from hourly sea level data by use of a sophisticated, two-step filter and for obtaining monthly values from the daily values with a simple average. Plotting software is included for the daily and monthly data. Also, programs are available for calculating the difference between two daily or monthly files. A subdirectory \SLPR2\FILT has been created for these purposes. An overview of the filtering method, the input and output files, the plotting routines, and other important items are discussed below.
Daily values are obtained using a two-step filtering operation. First, the dominant diurnal and semi-diurnal tidal components are removed from the quality controlled hourly values. Secondly, a 119-point convolution filter [1] centered on noon is applied to remove the remaining high frequency energy and to prevent aliasing when the data are computed to daily values. The 95, 50, and 5% amplitude points are 124.0, 60.2, and 40.2 hours, respectively. The Nyquist frequency of the daily data is at a period of 48 hours which has a response of about 5% amplitude, thus, aliasing is minimal. The primary tidal periods have a response of less than 0.1% amplitude.
The filtering operation incorporates an objective procedure to handle gaps. This objective technique simply replaces the filter weight at any missing observation with a zero and renormalizes the sum of the modified weight function to unity. This technique is equivalent to interpolating the missing observation with an estimate of the local mean of the time series. The local mean is defined as the mean of a given segment of length equal to the length of the filter. The error associated with this technique can be estimated objectively and is used as a criterion for accepting or rejecting a daily value computed in an area of the time series which contains a gap or gaps. This error depends on the ratio of the standard deviations of the input (hourly) and the output (daily) data. Thus in order to keep the ratio low, it is essential to apply this technique to the residual series as defined above.
The monthly values are calculated from the daily data with a simple average of all the daily values in a month. If seven or fewer values are missing, the monthly value is calculated. The monthly value is stored in the file with the daily data and in a separate file as noted below.
The input files for this program are in the UHSLC processing format (Appendix D.1). A requirement of the filtering program is that each month have 12 consecutive months of input hourly data or missing data flags in the UHSLC processing format. If some months are missing in a year, one may use program FILLVM in the \SLPR2\UTIL directory (Section 2.4) to create complete months of missing data flags, which can then be inserted into the data file so that the file has twelve consecutive months.
The filter program will ask the user for the start year and number of years to be filtered. This defines the time window of the data to be filtered. The program also asks if data prior to and after the time window are available. This is because the filter length is 5 days long; thus, the first and last day of the time window can not be calculated without information prior and post of the time window. For example, one runs the program with a time window of 1982 to 1984. Thus if data (not missing data flags) begin on 1 January, 1982 and data are available for 30-31 December, 1981, then the program can automatically access this data and thus produce a filtered daily value for 1 January, 1982. Note for the prior and post year files, the UHSLC processing format must also be used with the same file version yet the file DOES NOT have to have 12 consecutive months (an exception to the requirement above).
To run the program, go to the \SLPR2\FILT directory and type:
The output of the program is in the UHSLC processing format for daily data (Appendix D.2) and monthly data (Appendix D.3) and is placed in directory \SLPR2\DAT. File names are Dvsss.DAT and Mvsss.DAT (v:version, s:station) for the daily and monthly data, respectively. All years for a given station are kept in the same file.
Programs PDALL and PMALL (Section 3.4) on the \SLPR2\PLOT directory are provided to plot the daily and monthly data, respectively.
The daily and monthly values are useful for studying the low frequency variability of sea level. In addition, they provide another opportunity to judge the quality of the reference level stability (Section 5.2). Large level shifts are clearly apparent in plots of daily and monthly values as step-functions.
Another way to judge the quality of the reference level is to compare one tide gauge with another gauge at the same or nearby site. For this task, this package provides the programs DIFFDAY and DIFFMON on the \SLPR2\FILT directory for the daily and monthly data, respectively.
The programs simply prompt the user for the station number and file version for the input data (daily or monthly). Output is placed in directory \SLPR2\DAT in files DDIFF.DAT or MDIFF.DAT for daily and monthly data, respectively. Prior to re-executing either program, the difference files must be renamed or removed or the program will abort.
The daily and monthly differences can be plotted using programs PDALL and PMALL. When running these programs provide "DIFF" at the "file version and number" prompt.
If a step-function signature is seen in the difference, then there is a possibility that a reference level shift occurred at one of the stations. One must return the the procedures discussed previously (Section 5.2) to investigate and correct the problem.
The UHSLC/NODC Sea Level Processing Software package for IBM PC compatible microcomputers is available for enhancing the use and quality of sea level data. More data centers and oceanographic institutes now have access to personal computers and the UHSLC/NODC would like to share its software products and data processing techniques. This self-descriptive, interactive software package utilizes the DOS environment in the command line mode (not available for Windows). Any questions or comments regarding this manual or software should be directed to:
¯ ¯ Mr. Patrick Caldwell
Joint Archive for Sea Level
Dept. of Oceanography
University of Hawaii at Manoa
1000 Pope Rd. MSB 317
Honolulu, Hawaii 96822 USA
phone: 808-956-4105
fax: 808-956-2352
Internet: caldwell@soest.hawaii.edu
Web: http://ilikai.soest.hawaii.edu/UHSLC/jasl.html
The Joint Archive for Sea Level (JASL), a collaboration of the UHSLC and the NODC, maintains an expanding global archive of scientifically valid data sets to support science and sustainable development [3]. The data sets are annually contributed to the World Data Center-A for Oceanography and to the Permanent Service for Mean Sea Level. The JASL strongly encourages your support of hourly data sets to this international data bank. For details on how to submit data, please contact Mr. Caldwell at the address above.
Figure 1 or download PostScript file
Figure 2 or download PostScript file
Figure 3 or download PostScript file
Figure 4 or download PostScript file
Figure 5 or download PostScript file
Figure 6 or download PostScript file
Figure 7 or download PostScript file
Figure 8 or download PostScript file
Figure 9 or download PostScript file
Figure 10 or download PostScript file
Figure 11 or download PostScript file
1. Package Organization
The base sudirectory has been changed from \SLPRC to \SLPR2.
New subdirectories were added:
- \CAL {hold calibration files}
- \PRD {hold predicted tide files}
- \UTIL {utility programs}
2. Foreman Tidal Analysis and Prediction (M. Foreman, 11/1992).
A) Incorporate a new calendar for analysis and predictions outside of the
20th century, both backward and forward. B) A higher order approximation
for the astronomical arguments, their derivatives, and the constituent
frequencies to maintain sufficient accuracy well outside this century.
3. Format Conversion Utility
A program has been written to facilitate conversion of foreign formats
into the UHSLC data processing format as used by this package. The
program, DTDCNV.FOR, is under the \slpr2\util subdirectory. An
accompanying file, DTDCNV.DIN is found in the \slpr2\din subdirectory.
This file must be updated to define the column-wise positions of the input
to the program. The basic idea is for the foreign data sets to first be
converted into a simple format with one data point along with the Date-Hour
per line (record). Then the conversion program converts this simple format
into the UHSLC format.
4. Year 2000 Compliance: The File Naming Convention
Observed hourly and predicted data files: CVSSSYY.DAT
Observed : C: U for 1800-1899, V for 1900-1999, W for 2000-2099.
Predicted: C: O for 1800-1899, P for 1900-1999, Q for 2000-2099.
(VSSSYY unchanged, ie., version, station number, and last two digits of year)
5. Year 2000 Compliance: General Upgrades
All routines that would be affected by change in century have been modified.
6. Time Shift Routine
The program, TSHFT60.FOR, which operated on one year at a time, has been
replaced with TSALL.FOR, which allows time shifts within consecutive years.
7. Manual Update
Significant improvements were made to better explain and present the package.
The package is packed into a self-creating ZIP file
(PKZIP) of the name: slpaq2.exe (~1.4 Mbytes)
and the manual: slman2.ps (~2.5 Mbytes)
Anonymous FTP
=============
enter: ftp kapau.soest.hawaii.edu
at user prompt enter: anonymous
at password prompt: your email address
(ie. caldwell@soest.hawaii.edu)
after login, enter: cd pub/sealev/software
set transfer to binary: binary
grab files: get read.1st {defines files}
get ver3hist.txt {history of changes to package}
get slread2.me {this announcement}
get slpaq2.exe {self-expanding PKZIP file,complete package}
get slman2.ps {PostScript version of manual} OR
get slman2.zip {PKZIP version of slman2.ps--easier to FTP}
exit ftp: bye
To unpack, create directory from your C: root, SLPR2
and copy in file slpaq2.exe. Now type:
slpaq2.exe -d
Directories with appropriate files will be created.
The manual can be printed on a PostScript printer.
1 A report of the University of Hawaii pursuant to the
National Oceanographic and Atmospheric Administration
Award No. NA67RJ0154.
2 The predicted
tides file for a given year
has suffixed one day from the previous year and has appended one
day from the following year. This is to accommodate some routines
which require time shifts.
3 An option is available for the creation of tide tables.
These are to be used as a general reference only. Do not use
these for navigational purposes. For official nautical tidal
predictions, please refer to the National Ocean Service (NOS)
of the National Oceanic and Atmospheric Administration (NOAA).
4 I acknowledge Dr. M.G.G Foreman for making his
tidal analysis and prediction programs publically available.
Also, thanks go out to Mr. Bernie Kilonsky and
Mrs. Shikiko Nakahara of the University of
Hawaii Sea Level Center for their contributions to the
development of routines in this package. Thanks to
Mr. Brent Miyamoto for his comments regarding calibration.
Finally, thanks to
users of the 1991 version who have given me suggestions for
improvement. Production of this package is also supported
through funding of Cooperative
Agreement #NA67RJ0154 from National
Oceanographic and Atmospheric Administration (NOAA). The views expressed
herein are those of the author and do not necessarily reflect
the view of NOAA or any of its sub-agencies.
C Directory Tree
\SLPR2
|
|--\DIN {data information files}
|
|--\DMT {manual}
|
|--\DAT {data files}
|
|--\PRD {predicted tide files}
|
|--\CAL {calibration data files}
|
|--\TIDE
| |---\ANA {tidal analysis}
| |---\PRD {tidal prediction}
| |---\HARM {harmonic constituents}
| |---\SRC {source code}
|
|--\PLOT {plotting software}
| |--\SRC {source code}
|
|--\QC {quality control}
| |--\SRC {source code}
|
|--\FILT {filtering software}
| |--\SRC {source code}
|
|--\UTIL {utility programs}
| |--\SRC {source code}
|
|--\SAMP {sample data}
D Formats
D.1 Hourly Data Processing Format
Each observed hourly data file is given a name, CVSSSYY.DAT, where
C: century (U:1800-1899, V:1900-1999, W:2000-2099)
V: file version (letters A-Z)
SSS: station number
YY: the last two digits of the year
Example: VA00385.DAT is the observed hourly data file (version A) for
station 003 and year 1985.
The file contains header records, data records and
terminator records. Each month begins with a one-record
header and ends with two terminator records. Each data
record contains one half day of data or 12 hourly values.
The time scheme convention centers the data on hours
0-11 and 12-23 for the respective data records.
The header for each month is coded as:
field bytes comment
------------------ ------- ---------------------------------
station id 1-3 station number - must be 3 digits
station name 4-14 station name - note special cases:
7-9 file type; PRD for predicted and
RES for residual
latitude 19-26 latitude in degrees, minutes, tenths
of a minute, and hemisphere
longitude 33-41 longitude as above
time zone 50-55 time meridian reference for data,
ie. GMT or 090W
reference offset 61-65 constant value that is subtracted from
each data point in the month in order
for the data to be relative to the
station tide staff zero
time interval 67-68 time in minutes between each data point
month 70-72 first 3 characters of month, ie. JAN
year 74-75 year without century indication, ie. 78
field bytes comment
------------------ ------- ---------------------------------
units 77 scientific units of data as:
F : hundredths of feet (1121 = 11.21 ft)
M : integer millimeters(1121 = 1121.0 mm)
C : integer centimeters(1121 = 1121.0 cm)
# days in month 79-80 not a mandantory field
For example:
# of days in month
STATION time interval between values |
ID ref level offset | units|
| STATION time zone | | year | |
| | latitude and longitude | | | month| | |
__|_______| ____________________________| _____________:_________|__|___|__|_|_|
003BALTRA LAT=00 26.8S LONG=090 17.2W TMZONE=GMT REF=00000 60 APR 85 M 30
The data records are coded as:
field bytes comment
------------------ ------- ---------------------------------
station id 1-3 station number - must be 3 digits
station name 4-10 station name
year 12-15 year ie. 1988
month 16-17 numerical month, ie 12
day 18-19 numerical day, ie 31
record count 20 1 for 1st record; 2 for 2nd record of day
data values 21-80 Fortran format 12i5
For example:
STATION YEAR
ID | MONTH
| STATION | | DAY
| | | | | RECORD # HOURLY VALUES IN feet
__|_______|___|_|_|_|__________________________________________________________|
003BALTRA 1985 4 11 1492 1668 1879 2054 2152 2161 2067 1890 1663 1445 1285 1218
003BALTRA 1985 4 12 1266 1431 1667 1921 2123 2235 2230 2097 1893 1650 1432 1299
An extraction from an hourly file for Baltra is shown below.
003BALTRA LAT=00 26.8S LONG=090 17.2W TMZONE=GMT REF=00000 60 APR 85 M 30
003BALTRA 1985 4 11 1492 1668 1879 2054 2152 2161 2067 1890 1663 1445 1285 1218
003BALTRA 1985 4 12 1266 1431 1667 1921 2123 2235 2230 2097 1893 1650 1432 1299
003BALTRA 1985 4 21 1277 1385 1604 1868 2101 2268 2303 2201 1974 1681 1397 1185
003BALTRA 1985 4 22 1103 1180 1392 1695 2023 2295 2430 2401 2239 1971 1634 1337
003BALTRA 1985 4 31 1165 1141 1288 1581 1922 2236 2426 2451 2306 2027 1663 1303
003BALTRA 1985 4 32 1057 987 1112 1413 1813 2216 2533 2681 2604 2347 1961 1537
. . . .
. . . .
. . . .
003BALTRA 1985 4291 1823 2014 2145 2191 2124 1964 1740 1519 1345 1260 1294 1432
003BALTRA 1985 4292 1649 1896 2110 2245 2274 2195 2020 1787 1561 1392 1333 1387
003BALTRA 1985 4301 1549 1773 1997 2166 2231 2183 2032 1796 1545 1341 1234 1252
003BALTRA 1985 4302 1396 1638 1930 2186 2359 2402 2314 2108 1828 1550 1341 1251
99999999999999999999999999999999999999999999999999999999999999999999999999999999
99999999999999999999999999999999999999999999999999999999999999999999999999999999
003BALTRA LAT=00 26.8S LONG=090 17.2W TMZONE=GMT REF=00000 60 MAY 85 M 31
003BALTRA 1985 5 11 1301 1480 1742 2021 2242 2344 2302 2117 1840 1537 1278 1142
003BALTRA 1985 5 12 1161 1338 1638 1983 2302 2510 2555 2423 2147 1796 1456 1200
003BALTRA 1985 5 21 1093 1167 1405 1736 2076 2345 2460 2400 2173 1834 1470 1175
003BALTRA 1985 5 22 1024 1069 1307 1676 2094 2457 2679 2704 2524 2174 1736 1322
. . . .
. . . .
D.2 Daily Data Processing Format
The filename consists of the letter "d" followed by the file version
and the station number. For example, for station 027, file version a,
the file name is "da027.dat". All years for the station are in this file.
Header Record:
NNNxxxxxxxxxxxx Daily values 119-point filter
where NNN is the station number
xxxxxxxxxxxx is station name
The output data record format reserves the first eleven spaces for
station identification, the next four for the year of the data record,
the next two spaces for the month of the data record, the next space
for a sequential number that positions the record within the month.
The next space is a blank. The next 55 spaces contain the daily
filtered values in groups of five spaces per observation, 11 per
record. The data is units of mm. The averages are centered on
the 12th hour GMT of the day. There are three records per month,
and the last record contains the monthly average in the 11th
data record group. Missing data is flagged by four nines (9999). i.e.
station
id year
| | month
| | | indicator for 1ST, 2ND, or 3RD part of the month
| | | |
| | | | |_________________________data_mm______________________
_|________|___|_|_|____|____|____|____|____|____|____|____|____|____|____
01PONAPE 1983 21 538 562 574 567 573 547 555 558 548 532 538
01PONAPE 1983 22 582 600 586 587 590 606 620 592 589 592 569
01PONAPE 1983 23 574 581 590 614 638 638 580
^
|
monthly average
D.3 Monthly Data Processing Format
The filename consists of the letter "m" followed by the file version
and the station number. For example, for station 027, file version a,
the file name is "ma027.dat". All years for the station are in this file.
Header Record:
Consists of the station number, name, and starting year (bytes 55-58),
IE.
003Baltra MONTHLY MEANS 1985-
Data record
Contain a record header (station number and abbrevated name in columns
1-10), year (columns 11-14), a fixed "M" in column 15, and 12 sea level
values of four-bytes each in the remaining columns. Missing data given
by 9999.
Last Record: string of 9s.
Example
station
id year
| | fixed
| | |
| | |
| | ||_________________________data_mm______________________
_|________|__||___|___|___|___|___|___|___|___|___|___|___|___
003Baltr 1985M999999991785179317791801177017551771175517541746
003Baltr 1986M180417981802183318141842182817921813185218971831
99999999999999999999999999999999999999999999999999999999999999999999999999999999
D.4 Tide Staff-Gauge Pair: File Format
Example of a PAIR file of tide staff readings (highs and lows)
and corresponding gauge observations. The filename is as follows:
XSSSYYMM.DAT where
X: fixed (always "X"); SSS: station number;
YY: the last two digits of the year; MM: month
Record 1: Byte 1 : A special field coded as follows:
code meaning
-------- ----------------------------------------------
blank Input data in hundredths of feet (ie. 250 =
2.50 feet). Plot of high and low readings
in hundredths of feet.
A Input data same as above. Plot of averages of
the highs and lows.
M Input data in centimeters (ie. 250 = 250 cm)
Plot of high and low readings in hundredths of feet
N Input data in centimeters but plot of averages
in hundredths of feet
- Input gauge data values that should be negative
are converted to negative (ie if gauge value 1 = 0001
and the tide is falling and gauge value 2 = 9998, then
the program will convert 9998 to -0002)
Bytes 2 - 80 Header of plot; include station number, name, time period
Record 2 TO N: byte contents
--------- -----------------------------------
1-4 highest readings
5-8 lowest readings
Note if only one reading; both the same
9-13 gauge value
Last record: 99999999999999
Example:
STATION 30 SANTA CRUZ 01 JUL - 01 AUG 1988
250 250 1281
330 330 1373
.
.
570 570 1602
360 360 1400
9999999999999
D.5 Tide Staff-Gauge Pair:SCAT Output
Output of program SCAT.EXE in file RL17.OUT
STATION 30 SANTA CRUZ 01 JUL - 01 AUG 1988
HI LOW AVE GAUGE DIFF
250 250 250 1281 1031
330 330 330 1373 1043
410 410 410 1431 1021
440 440 440 1478 1038
390 390 390 1441 1051
460 460 460 1500 1040
610 610 610 1665 1055
650 650 650 1688 1038
640 640 640 1659 1019
590 590 590 1626 1036
390 390 390 1431 1041
480 480 480 1522 1042
640 640 640 1669 1029
380 380 380 1420 1040
500 500 500 1538 1038
330 330 330 1367 1037
430 430 430 1467 1037
380 380 380 1418 1038
340 340 340 1369 1029
340 340 340 1384 1044
610 610 610 1639 1029
D.6 Tide Staff-Gauge Pair:REFLEV Output
Output of program REFLEV.EXE in file REFLEV.OUT
STATION 30 SANTA CRUZ 01 JUL - 01 AUG 1988
MEAN OF HIGH VALUES : 489
MEAN OF LOW VALUES : 489
MEAN OF HIGH AND LOW MEANS : 489
MEAN OF GAUGE VALUES : 1525
CALIBRATION LEVEL : 1036
NUMBER OF OBSERVATIONS : 32
GAUGE - STAFF(AVE)
STANDARD DEVIATION : 8
MEAN
ZERO REFERENCE LEVEL VALUE: 1036
D.7 Calibration Log File
STATION = Santa Cruz, 003
GAUGE = Fischer and Porter Analog-to-Digital Recorder (ADR)
TIDE DATUM = TIDE STAFF ZERO
TIDE GAUGE BENCH MARK = BM UH 1
AUXILLARY BENCH MARKS = BM UH 2, BM UH3, BM II, BM III, BM 4, BM 5,
BM INOCAR 1, BM WS2
MEAN ZERO
DATES REFERENCE LEVEL NUMBER OF
VALUE (units=feet*) OBSERVATIONS
-------------------- ------------------- --------------------
01 JAN - 01 FEB 1988 1036 32
01 FEB - 01 MAR 1988 1037 28
01 MAR - 01 APR 1988 1035 26
01 APR - 01 MAY 1988 1036 30
01 MAY - 01 JUN 1988 1037 32
01 JUN - 01 JUL 1988 1035 31
01 JUL - 01 AUG 1988 1036 32
*units in feet to implied hundredths, ie 1036 = 10.36'.
(the numbers above are fictitous--this is only an example)
E Data INformation (DIN) Files
E.1 Station Information File
*Example of a STAINFO.DIN file:
The station information file contains specifications for a
given station that can be utilized by application programs. Each
recipient of the UHSLC/NODC package has the liberty of using
ones own station numbers with the limit of three digits per number.
The first record is a header to define the fields. The fields are :
field bytes comment
--------------- ---------- -----------------------------------
### 4-6 station number - must be exactly
3 digits
station name 8-27 station name - limit to 20 bytes
latitu 29-34 latitude in degrees and minutes
longitu 36-42 longitude " " "
mer 44-46 hour meridian of station
S 48 time scheme (0:1-24 1:0-23)
strtdate 50-57 start date of station data history
country 59-73
P 75-75 plot factor where:
PLOT FACTOR Y-AXIS SCALE
----------------------------------------------------
1 +- 1600 MM
2 +- 800 MM
3 +- 400 MM
4 +- 250 MM
5 +- 160 CM
6 +- 80 CM
7 +- 40 CM
8 +- 25 CM
Example:
### station name latitu longitu mer S strtdate country P
001 Ponape 06 59N 158 14E 165 1 Apr 1974 Micronesia 2
002 Tarawa,Betio 01 22N 172 56E 180 1 May 1974 Kiribati 2
003 Baltra 00 27S 090 17W 090 1 Aug 1974 Ecuador 2
004 Nauru 00 32S 166 54E 165 1 May 1974 Nauru 2
E.2 Plot Settings File
The contents of FILE C:\SLPR2\DIN\PLOTSIZ.DIN are given below.
The following are plot size factors used by the PLOTWORKS routines.
The size factor is a function of the output device. The output device
options are as follows:
* 0 => IBM PC CGA
* 1 => IBM PC EGA
* 2 => IBM PC Hercules
* 3 => Epson FX-80 series printer, LPT1
* 4 => Epson FX-100 & 200 series printer
* 5 => HP 7000 series pen plotter, COM1
* 6 => HP LaserJet, LPT1
* 7 => HP QuietJet, LPT1
* 8 => HP Thinkjet, LPT1
* 9 => IBM PC VGA
* 10 => POST SCRIPT PRINTER
If your plot does not fit on the output device, then the size
factor is too large. Decrease the size factor below for the desired
device by a small percentage and replot. Repeat this process until the
plot neatly fills the entire screen (or page). For plots that are too
small on the output device, increase the size factor below and replot.
DEVICE--> 0 1 2 3 4 5 6 7 8 9 10
HOURYR 0.75 0.90 0.70 0.95 0.95 0.95 0.95 0.95 0.95 0.92 0.93
OTHERS 0.65 0.80 0.60 0.85 0.85 0.85 0.85 0.85 0.85 0.78 0.83
For program HOURYR, use the first line above for each of the 11 output
device options.
For the other plot programs (EXPLOT, PDALL, PMALL, SCAT) use the line
labelled OTHERS.
IMPORTANT
This data is read into the program by skipping 26 lines (25 for
HOURYR) and using FORTRAN format (6x, 11(2x,f4.2)). Thus do not add
lines to this file or change the length (in bytes) of each plot factor
above.
E.3 DIN File for Conversion Program
Format of input data record
Specify start and end column positions
year start column = 13
year end column = 16
month start column = 18
month end column = 19
day start column = 21
day end column = 22
hour start column = 26
hour end column = 27
data start column = 33
data end column = 39
Example of data file corresponding to
the default settings above:
"8771510","1998/01/01","00:00", 1.399,
"8771510","1998/01/01","01:00", 1.335,
"8771510","1998/01/01","02:00", 1.286,
"8771510","1998/01/01","03:00", 1.177,
"8771510","1998/01/01","04:00", 1.122,
"8771510","1998/01/01","05:00", 1.065,
"8771510","1998/01/01","06:00", 1.129,
"8771510","1998/01/01","07:00", 1.187,
"8771510","1998/01/01","08:00", 1.210,
"8771510","1998/01/01","09:00", 1.291,
Note, this format is used by the National
Ocean Service for their online data holdings
accessible through the World Wide Web. The
important aspect of the format is that there
is one line per hourly value. If data are
missing, then no line exists for that hour.
F Notes on Foreman Tidal Analysis Program
1) Interpretation of the output file of the tidal analysis program.
For each tidal component there are four columns (after the time span for
analysis which is repeated on every line) corresponding to A, G, AL, and GL.
The last two are the fitted amplitude (AL) and phase (GL) direct from the least
squares analysis. The phase is relative to a reference time origin at 0000 01
Jan 1976. The first two are the amplitude (A) and phase (G) after applying
the so-called nodal corrections (see next paragraph).
Applying nodal corrections allows the fitted components to be used further
from the actual time period used to fit the components. Each fitted component
is in reality a sum of a major component and several satellite components that
are too nearby in frequency to resolve in the least squares analysis. Therefore
the AL and GL values do not accurately reflect the amplitude and phase of the major
component, but show a modulation due to the satellites. By assuming that the ratio
is equal to the ratio of the equilibrium components, it is possible to correct the
major component to obtain the amplitude and phase that would be fit if the
satellites were resolved.
2) Foreman updated his programs in 1992 to comply with change of century and to
allow analysis and prediction for previous centuries. Two main changes have been
made: 1) the calendar was extended for calculations outside of the 20th century,
and 2) the position of the moon and sun (i.e., the astronomical arguments) are
now approximated by higher order polynomials (in time) so that sufficient accuracy
is maintained for dates well outside the 20th century.
3) Doodson number conventions (from Geoff Lennon)
The Doodson numbers define the tidal component frequency and also determine the
astronomical argument. The astronomical argument is the phase of the equilibrium
tide at some reference time origin. Unfortunately, the Doodson numbers for a number
of low frequency components and a few high frequency ones are not the same as those
used in other programs. This causes the fit to be done at a slightly different
frequency and with a different reference phase (the astronomical argument). This
will not affect prediction, but will cause disparities between our analysis and
those done at the IOS (UK) and the IHO. There is no simple fix for this situation
since we do not know exactly which components are affected and what the preferred
Doodson numbers are for each of them.
G Harmonic Constants: Analysis Output
003 Baltra 0 00 26S 090 17W
1 Z0 .00000000 0 785/ 786178.6955 .00 178.6955 .00
2 SA .00011407 0 785/ 786 3.7802 109.26 3.7802 112.59
3 SSA .00022816 0 785/ 786 .3527 117.53 .3527 278.78
4 MSM .00130978 0 785/ 786 .7544 62.29 .7544 239.82
5 MM .00151215 0 785/ 786 1.0036 13.21 1.0036 99.16
6 MSF .00282193 0 785/ 786 .6430 25.45 .6430 288.92
7 MF .00305009 0 785/ 786 2.0005 32.02 2.0005 96.75
8 ALP1 .03439657 0 785/ 786 .0856 107.06 .1100 214.71
9 2Q1 .03570635 0 785/ 786 .1114 122.13 .1575 42.03
10 SIG1 .03590872 0 785/ 786 .0962 148.30 .1145 344.99
11 Q1 .03721850 0 785/ 786 .3442 170.27 .4370 179.94
12 RHO1 .03742087 0 785/ 786 .1030 150.67 .1169 74.11
13 O1 .03873065 0 785/ 786 1.0145 47.51 1.1655 148.29
14 TAU1 .03895881 0 785/ 786 .3383 24.71 .2187 113.63
15 BET1 .04004043 0 785/ 786 .0398 310.27 .0474 47.30
16 NO1 .04026859 0 785/ 786 .2466 344.70 .2599 311.91
17 CHI1 .04047097 0 785/ 786 .0600 339.19 .0698 159.43
18 PI1 .04143851 0 785/ 786 .1035 64.26 .1028 70.57
19 P1 .04155259 0 785/ 786 2.6462 51.36 2.6214 61.10
20 S1 .04166667 0 785/ 786 1.3144 16.16 .8941 349.24
21 K1 .04178075 0 785/ 786 7.8056 49.21 8.5533 44.58
22 PSI1 .04189482 0 785/ 786 .1121 202.07 .1138 196.65
23 PHI1 .04200891 0 785/ 786 .1356 40.90 .1264 192.38
24 THE1 .04309053 0 785/ 786 .1683 52.94 .1881 231.27
25 J1 .04329290 0 785/ 786 .5376 69.84 .8075 165.32
26 SO1 .04460268 0 785/ 786 .2183 167.59 .2512 66.73
27 OO1 .04483084 0 785/ 786 .6323 87.87 .9812 171.10
28 UPS1 .04634299 0 785/ 786 .1186 113.25 .2156 278.08
29 OQ2 .07597494 0 785/ 786 .1910 173.92 .1815 14.83
30 EPS2 .07617731 0 785/ 786 .5028 164.52 .4839 273.35
31 2N2 .07748710 0 785/ 786 1.7823 188.99 1.6968 115.44
32 MU2 .07768947 0 785/ 786 2.5555 193.17 2.4735 27.63
33 N2 .07899925 0 785/ 786 14.9349 217.48 14.4488 229.16
34 NU2 .07920162 0 785/ 786 3.0899 222.69 2.9874 142.91
35 H1 .08039733 0 785/ 786 .7249 86.90 .6907 358.86
36 M2 .08051140 0 785/ 786 70.0426 246.43 67.9384 344.16
37 H2 .08062547 0 785/ 786 .6465 79.72 .6352 180.32
38 MKS2 .08073957 0 785/ 786 .2083 84.68 .2550 353.46
39 LDA2 .08182118 0 785/ 786 .5082 269.73 .4899 5.34
40 L2 .08202355 0 785/ 786 1.5764 280.41 1.6704 303.40
41 T2 .08321926 0 785/ 786 1.3464 301.44 1.3464 298.11
42 S2 .08333334 0 785/ 786 18.7974 292.37 18.8300 292.29
43 R2 .08344740 0 785/ 786 .3483 284.44 .4250 102.53
44 K2 .08356149 0 785/ 786 4.9431 286.66 6.2274 97.63
45 MSN2 .08484548 0 785/ 786 .0401 202.46 .0377 288.43
46 ETA2 .08507364 0 785/ 786 .2544 289.95 .3487 189.60
47 MO3 .11924210 0 785/ 786 .0436 218.71 .0486 57.22
48 M3 .12076710 0 785/ 786 .1659 356.64 .1585 143.42
49 SO3 .12206400 0 785/ 786 .0158 274.13 .0182 14.83
50 MK3 .12229210 0 785/ 786 .0193 109.26 .0205 202.36
51 SK3 .12511410 0 785/ 786 .0507 14.73 .0557 10.01
52 MN4 .15951060 0 785/ 786 .0261 359.37 .0245 108.78
53 M4 .16102280 0 785/ 786 .0837 64.54 .0787 260.00
54 SN4 .16233260 0 785/ 786 .0304 322.51 .0295 334.11
55 MS4 .16384470 0 785/ 786 .0394 116.72 .0383 214.37
56 MK4 .16407290 0 785/ 786 .0116 100.18 .0142 8.89
57 S4 .16666670 0 785/ 786 .0614 209.11 .0616 208.95
58 SK4 .16689480 0 785/ 786 .0182 114.89 .0230 285.78
59 2MK5 .20280360 0 785/ 786 .0124 143.05 .0128 333.88
60 2SK5 .20844740 0 785/ 786 .0068 238.65 .0075 233.86
61 2MN6 .24002200 0 785/ 786 .0311 357.93 .0283 205.08
62 M6 .24153420 0 785/ 786 .0407 337.75 .0371 270.95
63 2MS6 .24435610 0 785/ 786 .0819 37.54 .0771 232.93
64 2MK6 .24458430 0 785/ 786 .0306 17.82 .0362 24.25
65 2SM6 .24717810 0 785/ 786 .0326 64.01 .0317 161.58
66 MSK6 .24740620 0 785/ 786 .0212 108.36 .0259 16.98
67 3MK7 .28331490 0 785/ 786 .0023 348.03 .0023 276.60
68 M8 .32204560 0 785/ 786 .0205 302.46 .0182 333.39
H Make Predictions other than Hourly
1) edit tideprd.bat
just before tidep.exe (before run tide prediction)
place a "pause" command.
2) run tideprd.bat
hit cntrl-C when pause is hit (the one just before tidep)
3) edit TIDEDATA
on last non-blank record, in field just after EQUI,
enter fraction of hour (default is hourly = 1.000)
eg, for 15 minutes change it to 0.25000)..... its
fortran so be byte specific (dont delete any zeros).
4) now just type TIDEP, which runs prediction.
Your output will be in eqsprd.dat in the Foreman format.
I Test for Magnitude of Timing Error
One way to determine the magnitude of a timing error is to compare
observed data to predicted tides. In separate windows for the same
time span with the suspect timing error, place the observed data
and predicted files. For example:
Window 1: Observed File, VA00386.DAT, 14-16 November, 1996
003BALTRA 198611141 1243 1389 1684 2055 2416 2683 2767 2645 2360 1983 1591 1274
003BALTRA 198611142 1120 1166 1399 1751 2131 2454 2637 2627 2437 2124 1761 1439
003BALTRA 198611151 1258 1269 1463 1793 2176 2523 2730 2748 2559 2211 1799 1405
003BALTRA 198611152 1129 1050 1177 1477 1867 2262 2564 2683 2604 2356 2002 1638
003BALTRA 198611161 1357 1227 1301 1547 1905 2291 2596 2733 2660 2402 2016 1596
003BALTRA 198611162 1239 1029 1026 1230 1573 1977 2351 2592 2651 2511 2209 1841
^
|
Note high tide at hour 20 of 11-16-1986
Window 2. Predicted Tides File, PA00386.DAT, 14-16 November, 1996
003Baltra 198611141 1277 1238 1384 1682 2059 2420 2675 2752 2627 2333 1939 1536
003Baltra 198611142 1219 1068 1120 1358 1718 2110 2439 2621 2608 2412 2088 1720
003Baltra 198611151 1402 1223 1235 1437 1778 2174 2525 2737 2747 2552 2199 1773
003Baltra 198611152 1372 1093 1010 1140 1446 1850 2254 2559 2685 2605 2348 1985
003Baltra 198611161 1607 1314 1190 1268 1528 1903 2300 2615 2759 2689 2423 2027
003Baltra 198611162 1592 1222 1011 1014 1226 1588 2014 2401 2651 2701 2547 2236
^
|
Note high tide at hour 21 of 11-16-1986
As can be seen above, it appears that the observed data file lags the predicted
tides by one hour. Thus, one can applied the timing error correction technique
to shift the observed file for this time span by +1 hour.
Footnotes:
File translated from TEX by TTH, version 2.32.
On 26 Dec 2000, 16:43.