Comments, queries or suggestions can be emailed
To Programmer's Report
Data Quality Report
Ruth Lawless, October 2001
Data Currency Voyage Description Logging System Data Sampling Interval
Data Gaps Software Data Quality
Data Start Time: 03 January 1991 12:28 UTC
Underway Data End Time: 19 March 1991 23:07 UTC
This was primarily a marine science voyage. Technical support personnel were provided on board, but underway data were quality checked well post-voyage (October 2001).
|16 - 17 January||Mawson|
|19 January - 01 March||Marine Science, Prydz Bay area|
|04 - 05 March||Mawson|
|07 - 08 March||Davis|
DLS data types were logged.
4. Data Sampling Interval: 60 seconds
5. Data Gaps
All logged parameters have the following data gaps:
Underway data were quality checked using Java based software created by Gordon Keith and Peter Wiley. Depth data were already processed; it is not documented when, how and by whom this was done.
Back to Contents
7. Data Quality
of Checked Parameters
The following is a description of what criteria were used to flag data as "good" or "bad". Underway data users should also refer to the section on Data Gaps.
Bathymetry (Water Depth)
There is no documentation relating to the processing protocol or quality of these data.
Ship’s Heading (SH)
SH is a measurement of the direction that the bow of the ship is facing.
These data were accepted as good.
Ship’s Speed (SS)
Ship's Speed data are derived from the electromagnetic current meter output (Ship's Log). SS is a measurement of ship's speed through the water and is directly affected by waves.
Ship's Speed data are believable with respect to the ship's schedule and capability and have therefore been left as good.
Fluorometer, Thermosalinograph and Low Resolution
According to the Computer Logbook, the uncontaminated seawater (USW) line was open, although very frequently blocked by ice. However, it is evident that USW flow was not logged accurately as Fluorometer Flow (FLU_FLOW) values are constant at zero for the entire voyage.
Accurate Fluorometer Value (FLU_VALUE), Thermosalinograph Salinity (TSG_SALIN), Conductivity (TSG_CNDUCT) and Temperature (TSG_TEMP), and Low Resolution Water Temperature (W_TEMP) data all rely on USW flow. As this could not be assessed, these datasets were flagged as bad for this voyage.
Air Temperature Port and Starboard (A_TEMP_P
Warm air from the exhaust funnel aft of the air temperature sensors corrupts air temperature data when upwind of them. As apparent wind direction data for this voyage are not loaded onto the marine database, it cannot be determined when this occurred.
Therefore, data from both air temperature sensors were marked bad where they differ by more than about 1.0º C or where the shapes of the traces don't correlate well. Outliers were also rejected.
Barometric Pressure (BAR)
Apart from occasional outliers, BAR data were accepted as good.
Humidity Port and Starboard (HUMID_P and
The humidity sensors are located close to the air temperature sensors and humidity data are therefore also affected by exhaust air from the funnel, although to a lesser extent. When a sensor is downwind of the funnel, humidity data are sometimes decreased relative to the unaffected sensor. However, as with air temperature data, humidity data could not be quality checked with respect to apparent wind direction.
Therefore, data from both humidity sensors were marked bad where values differ by more than about 10% or where the shapes of the traces don't correlate well. Outliers were also rejected.
Photosynthetically Active Radiation (LICOR_R)
Note that this sensor is affected at times by shadows from the ship's superstructure.
Values greater than 2000 µmol/m2/s were rejected, as were occasional night-time outliers.
Wind Direction True Port (WDPT) and Wind
Speed True (WSPT)
According to the Computer Logbook, the anemometer was replaced on 31 January 14:00 UTC. Data prior to this are suspect and have been allocated bad quality flags.
Otherwise, very scattered WDPT data corresponding to low wind speed were rejected. Occasional WDPT outliers were also marked bad. WSPT data were left as good, apart from a few outliers.
Apparent wind direction and wind speed data should exist, but have not as yet been loaded onto the marine database.
Back to Contents
Author and date not recorded
All shipboard personnel had access to this software, the intention being that researchers could conduct their own post-processing on board. In most cases people did use this system; only a few relied upon ADP staff to extract data and generate plots for them. CHART was never used for post-processing of logged data.
Users entered the post-processing menu by typing "LOOKSEA" at the dollar prompt. The opening menu contained options for CTD, DLS, XBT, and data management processing. Appendix 1 shows each looksea menu screen to indicate the options provided.
Two Fortran programs were used for plotting DLS data. A "property vs property" routine allowed users to plot any DLS logged parameter against any other. A customised version of this program was provided for plotting the ship's track and coast lines.
A gateway to TINY allowed users to invoke a series of PowerHouse screens and reports. Enquiry screens enabled meteorological, satellite navigation and ship status information to be displayed from individual logged DLS records. Reports generated tables of meteorological and oceanographic hourly averages and meteorological values at user specified intervals.
In anticipation of the use of the telex facility by most passengers command procedures were written that generated and removed user accounts in bulk. The facility worked well and was well received by the ship's radio officer, voyage leader, and passengers.
The HP Vectra PC in the MET lab failed and would not reboot. A faulty VGA card was the cause. Two HP Vectra PC keyboards failed. The VGA repeater on the bridge from the MET PC has lost one colour gun. Four BARCO monitors have failed. In all cases it was the RGB video driver transistors that failed. The DLS HP Draftmaster track plotter occasionally hangs, saying "Put in carousel" in the LCD display. Jiggling the carousel usually fixes this problem.
The Fluorometer burst it's plumbing and filled with sea water on 06-Mar-1991. It was replaced by a spare instrument.
Certain XBT probes cause the XBT HP Vectra PC to hang, requiring a reboot.
DLS HP-7979 tape drive number two occasionally displayed an incorrect number of dots in its "tape used" bargraph indicator. At these times the display should have contained seven or eight dots. However, only three dots actually were displayed. The bargraph display was used to estimate when the tape would be filled - this fault may lead to data loss.
On two occasions the known TK-70 firmware fault appeared, locking up the tape drive. It remained un-useable until the next reboot. One VT320 keyboard has a faulty return key -multiple characters can be sent for a single keystroke.
The bridge console failed to operate at other times for no apparent reason. Powering the bridge console off and on occasionally crashed TRAC some time later with error 314 "Receive buffer overflow". Again, a BASIC reset followed by "RUN" was required to clear this error. Error 314 and a TRAC crash also occured if the printer was left offline for more than an hour or so.
The VAX LAN connection was lost several times for no apparent reason.
Once again the ETA calculation proved unsatisfactory. This calculation is performed by integrating the distance along the ship's intended route between the current position and the ETA point. No account is taken of the off-track distance - i.e. only that part of the route which lies along the intended track is included in the calculation. A meaningful ETA estimate must allow for the time required to rejoin the intended track should the vessel deviate, e.g. due to ice or bad weather. In the event of the intended route crossing itself or of the ship deviating such that the nearest point on the intended route becomes unclear, an ambiguity arises as to which parts of the track are to be included in the ETA calculation. To resolve this, the calculation is based on the concept of a "current segment"; all parts of the route following the current segment contribute to the ETA time. However, the algorithm for assigning the current segment often chose the first segment, independent of the ship's location. Incorrect ETA times resulted. Recognising this problem, QUBIT have provided a means to manually set the current segment. Unfortunately this would often either fail to work, or fail to indicate which segment was current. Manual segment selection was based on a "trial and error" technique which usually succeeded only after several attempts.
DLS logs some, but not all, EA200 settings. In particular, the "minimum depth" setting is not recorded. On occasion we have seen flat-topped features in the digital bottom trace. There is no way of telling from the logged data if these are real features or if they have been clipped at the top due to the minimum depth being greater than the actual depth.
Pressing the "Enable CTD" soft key causes the "Start CTD dip" soft key to disappear. This was not the case on the HIMS cruise. It means that we could not check that the CTD data summary display was functioning prior to recording a CTD time-stamp in the DLS LOG file. Whilst DLS could correctly read the CTD fish ident number it could not allow for different pressure transducers on different fish. Thus, when we were forced to change CTD fish, the DLS display no longer gave correct pressure values. Occasional CTD data files failed to transfer from the CTD PC across the LAN to the VAX due to a "record too long" error. We solved this problem by using PCSA's NFT utility and specifying "/image" on the copy command. The QUBIT menu option should do likewise for all transfers.
"Transit fix age" was logged every ten seconds with no gaps. However, examining the data revealed that the logged values increased by eleven seconds per ten second logging interval. Examining the satellite navigation screen showed the same fault with the value displayed as "time since last fix".
DLS occasionally corrupted tape records, writing fragments of one record at the start of the next. Without correction, the remainder of the tape was unreadable, because all character fields in all remaining records became displaced by the number of extra characters in the rogue "fragment". This fault was present before the Heard island cruise; it remains to be corrected. On occasion an extra CR-LF sequence appeared in the middle of an EK-500 record, making the record unreadable to software expecting stream-cr or stream-lf record formatting.
DLS generates position "fixes" at regular intervals. At these times the ship's current postion is output to the DLS line printer. There is a facility to draw crosses on the graphics devices to indicate the fix, and to annotate these with either a time or a sequential fix number. It is useful to have hourly positions printed. However graphics devices become cluttered if fixes are indicated at this frequency. A menu option is provided to specify, for each graphics device, that only every i-th fix be indicated and that only every j-th of these be annotated. However, for at least part of the cruise, every fix was indicated on the graphics devices - independent of the values of i and j. A partial solution was attempted, by setting the automatic fixing interval to 86400 seconds - i.e. once per day. TRAC truncated this value, without notification, to 31980 seconds, or 8.8833 hours. As a compromise we chose a 21600 second fix interval (6 hours). In any case the approach was unsatisfactory, as hourly positions were no longer sent to the printer. Further, when using the option to select frequency of fix marking, old values were not cleared from the screen before new values were written. A confusing display resulted if a new value contained less digits than the old.
Only the absolute value of meteorolgical sea water temperature was logged. I.E. negative temperatures (in degrees C) were logged as positive. The fault was traced to the STEEDMANS single board computer (SBC) in the met lab, which dropped the sign bit from sea water temperature, and from air temperature. The latter was not a problem on this voyage because an approximately 22 degree C offset was applied to air temperature. This was to shift the range of values for which data could be logged toward temperatures colder than those for which the STEEDMANS unit was originally designed. For sea water temperature, the problem was solved by placing the met HP-Vectra PC between DLS and the met system. A short BASIC program on the PC read both the original STEEDMANS met station data (which did contain valid sea temperatures) and the SBC data (which didn't). It used the former data stream to correct the latter, and transmit the result to DLS. This meant that the PC could not perform it's usual task of printing ten-minute averages for use by the met bureau. These data will need to be extracted from DLS records for them at the end of the voyage.
The "big number" display for the Met PC does not clear old values before writing new ones, making a mess very quickly.
Chart's "INIT TAPE MEDIA" also offers the option of initialising AURORA's system disk! This option should be removed as it is a nonsense option - it should not work. If, by chance it does work, it would be extremely dangerous! We weren't game to try.
Attempts to copy a single database to tape using Chart's media options menu produced tapes that TRAC reported as containing no files. However, if all the files in the VAX directory are copied to tape by Chart then TRAC has no trouble at all in reading any file from that tape.
Chart does not display system error messages for long enough to be read or even noticed. At one stage we resorted to setting the terminal to communicate at 300 baud in order to read these messages! On more than one occasion inability to read system messages led us to believe that tape operations had been successful when in fact they had failed. This could easily lead to loss of data.
As we have no manual for Chart, we were unable to use it to plot any
quantity against any other quantity for any time interval. We did manage
to plot database files and use some of the LAN server options and media
utilities. Provision of such a software package as Chart without a manual
is totally unacceptable and must be remedied before next season. On this
voyage, post processing of DLS logged data was performed exclusively by
Many passengers bring their own PC (IBM or Macintosh) aboard, not only for word processing but also to run applications relevant to their own programs. We perceive a need for two general access machines (one IBM PC compatible and one Macintosh) whose main role would be to provide access to network resources such as printers, plotters, and VAX file services. Limited time on these machines may be given for general use to individuals without PCs. Should demand become heavy a booking system may be required. Presumably, these machines would be Antarctic Division property and be removed from the ship during the off-season.
The two proposed machines would be located in the lower conference room. They would be connected to the network via thin-wire ethernet. To complete the word processing facilities the HP Laserjet printer would need to be upgraded with extra memory and support for Postscript. A few standard packages would be provided. These would reside as read-only disk services on the VAX, free from interference by enthusiastic users. The local hard disks would hold user files. Backups would be each user's own responsibility. Software to consider providing would include Microsoft Word, Microsoft Excel, kermit, disk utilities and anti-virus packages.
Possible hardware configurations for these machines are as follows:
IBM compatible PC
We recommend that the Division consider including a full time data quality control officer on future marine science voyages. With the provision of suitable software, this individual would be responsible for inspecting logged data and setting quality flags, to the best of their ability, to indicate the data reliability. The intention would be that by the conclusion of a marine science voyage a reliable database would be available for immediate use.
TINY currently has 4 Mbytes of memory, of which VMS and DECNET consume 2.5 Mbytes. It's performance, even with a single user, is much slower than expected for a CPU of it's capacity. This is particularly noticeable when running, EVE, PowerHouse, WPS, etc. We recommend that TINY be upgraded to it's maximum capacity of 6 Mbytes.
On this cruise there has been heavy demand for the HP Draftmaster plotter. Most plots have been A4 or A3 size. For certain applications, Chart in particular, this plotter cannot be queued because two way communications are required. Further, it does not make sense to queue a plotter unless it has some sheet feed mechanism or continuous roll media mounted. Consequently, we consider it desirable to install a second, queue-able, A4 plotter in the conference room. Perhaps this requires no more than upgrading the Laserjet printer to support HPGL (as well as Postscript). Alternatively, a small multipen HPGL plotter with sheet feed would suffice.
During Voyage 6 PowerHouse software was developed that ran on TINY and accessed the logged data on AURORA. This software proved useful but awkward to use as the postprocessing menu was invoked from AURORA. Therefore, we recommend that a run-time only license for QUICK be obtained for AURORA. Due to the nature of QUIZ and QTP a full license for QUIZ, QTP for AURORA would be required. Were the Division to obtain the "Inquisitive" product then this would also prove useful for interrogating the cruise and wildlife databases.
Obtaining such PowerHouse licenses for AURORA would mean that the wildlife databases (Fish, Birds, Seals and Whales) could be accessed from AURORA rather than TINY. A full PowerHouse license could be retained for TINY so that development work could be done during a voyage.
Back to Data Quality Report