Marine Science Support Reports

RSV Aurora Australis Season 1990-1991
Voyage 6 (AAMBER2)

Comments, queries or suggestions can be emailed to All_STS@antdiv.gov.au
 

To Programmer's Report

Data Quality Report
Ruth Lawless, October 2001
 

Contents
Data Currency  Voyage Description  Logging System  Data Sampling Interval
Data Gaps  Software  Data Quality



1. Underway Data Start Time: 03 January 1991 12:28 UTC
    Underway Data End Time: 19 March 1991 23:07 UTC

2. Description
This was primarily a marine science voyage. Technical support personnel were provided on board, but underway data were quality checked well post-voyage (October 2001).

    Schedule:
03 January Hobart
16 - 17 January Mawson
19 January - 01 March Marine Science, Prydz Bay area
04 - 05 March Mawson
07 - 08 March Davis
19 March Hobart

3. Logging System
DLS data types were logged.

4. Data Sampling Interval: 60 seconds

5. Data Gaps
All logged parameters have the following data gaps:

Back to Data Quality of Checked Parameters

6. Software
Underway data were quality checked using Java based software created by Gordon Keith and Peter Wiley. Depth data were already processed; it is not documented when, how and by whom this was done.

Back to Contents


7. Data Quality of Checked Parameters
The following is a description of what criteria were used to flag data as "good" or "bad". Underway data users should also refer to the section on Data Gaps.

Bathymetry (Water Depth)
There is no documentation relating to the processing protocol or quality of these data.

Ship’s Heading (SH)
SH is a measurement of the direction that the bow of the ship is facing.

These data were accepted as good.

Ship’s Speed (SS)
Ship's Speed data are derived from the electromagnetic current meter output (Ship's Log). SS is a measurement of ship's speed through the water and is directly affected by waves.

Ship's Speed data are believable with respect to the ship's schedule and capability and have therefore been left as good.

Fluorometer, Thermosalinograph and Low Resolution Water Temperature
According to the Computer Logbook, the uncontaminated seawater (USW) line was open, although very frequently blocked by ice. However, it is evident that USW flow was not logged accurately as Fluorometer Flow (FLU_FLOW) values are constant at zero for the entire voyage.

Accurate Fluorometer Value (FLU_VALUE), Thermosalinograph Salinity (TSG_SALIN), Conductivity (TSG_CNDUCT) and Temperature (TSG_TEMP), and Low Resolution Water Temperature (W_TEMP) data all rely on USW flow. As this could not be assessed, these datasets were flagged as bad for this voyage.

Air Temperature Port and Starboard (A_TEMP_P and A_TEMP_S)
Warm air from the exhaust funnel aft of the air temperature sensors corrupts air temperature data when upwind of them. As apparent wind direction data for this voyage are not loaded onto the marine database, it cannot be determined when this occurred.

Therefore, data from both air temperature sensors were marked bad where they differ by more than about 1.0º C or where the shapes of the traces don't correlate well. Outliers were also rejected.

Barometric Pressure (BAR)
Apart from occasional outliers, BAR data were accepted as good.

Humidity Port and Starboard (HUMID_P and HUMID_S)
The humidity sensors are located close to the air temperature sensors and humidity data are therefore also affected by exhaust air from the funnel, although to a lesser extent. When a sensor is downwind of the funnel, humidity data are sometimes decreased relative to the unaffected sensor. However, as with air temperature data, humidity data could not be quality checked with respect to apparent wind direction.

Therefore, data from both humidity sensors were marked bad where values differ by more than about 10% or where the shapes of the traces don't correlate well. Outliers were also rejected.

Photosynthetically Active Radiation (LICOR_R)
Note that this sensor is affected at times by shadows from the ship's superstructure.

Values greater than 2000 µmol/m2/s were rejected, as were occasional night-time outliers.

Wind Direction True Port (WDPT) and Wind Speed True (WSPT)
According to the Computer Logbook, the anemometer was replaced on 31 January 14:00 UTC. Data prior to this are suspect and have been allocated bad quality flags.

Otherwise, very scattered WDPT data corresponding to low wind speed were rejected. Occasional WDPT outliers were also marked bad. WSPT data were left as good, apart from a few outliers.

Apparent wind direction and wind speed data should exist, but have not as yet been loaded onto the marine database.

Back to Contents



Programmer's Report
Author and date not recorded

VOYAGE 6 1991 - AAMBER2 - MARINE SCIENCE

 

1.0 TASKS ACHIEVED

1.1 General description of the voyage

Voyage 6 1990/91, code named AAMBER2, departed Hobart at 13:02 U.T. on 03-Jan-1991. Arrived Horseshoe Harbour, Mawson, 13:41 U.T. on 16-Jan-1991, departed 13:06 U.T. on 17-Jan-1991. Then followed 44 continuous days of marine science. During this phase of the voyage bottom and mid-water trawling, CTD casts, accoustic biological surveys and several krill swarm studies were conducted. For the duration of the marine science program we worked alternate 12 hour shifts. The ship returned to Mawson, arriving at Horseshoe Harbour at 05:07 U.T. on 03-Mar-1991. After taking on returning expeditioners and cargo departed for Davis on 05-Mar-1991 at 08:35 U.T.. Arrived Davis at 04:18 on 7-Mar-1991. After taking on returning expeditioners and cargo departed Davis for Hobart at 14:45 U.T. on 08-Mar-1991. Figure 1 indicates the entire cruise track, whilst figure 2 shows, on an expanded scale, our cruise track within the marine science survey area.

1.2 Hardware installations prior to departure

Several hardware installations were carried out in conjunction with QUBIT and DEC. A second hand RA-82 disk drive was installed on AURORA, in the bottom of the rack which houses open-reel tape drive MUB0. A 650 megabyte magneto-optical disk drive (325 MB/side) was installed, also on AURORA. This required the removal of the LPV-11 line printer interface card as the M.O.D. needed a Qbus to SCSI adaptor card and AURORA's backplane was full. The LG-02 printer is now driven through a terminal server port, which has an added advantage in that TINY has ready access to it. An RD-54 disk, in it's own standalone cabinet, was added to TINY. Files on both TINY and AURORA were shuffled around such that their new disks hold user accounts and non-system applications software. AURORA's DUA0: also contains CHART and PCSA software.

1.3 Post-processing software

Post-processing software written by Antarctic Division personnel consisted of a suite of fortran programs, some powerhouse screens and reports, and DCL command procedures to run these via a simple menu interface. Data analysis menus included procedures to display plotfiles on VT screens or produce hardcopy on the HP-Draftmaster plotter.

All shipboard personnel had access to this software, the intention being that researchers could conduct their own post-processing on board. In most cases people did use this system; only a few relied upon ADP staff to extract data and generate plots for them. CHART was never used for post-processing of logged data.

Users entered the post-processing menu by typing "LOOKSEA" at the dollar prompt. The opening menu contained options for CTD, DLS, XBT, and data management processing. Appendix 1 shows each looksea menu screen to indicate the options provided.

1.3.1 CTD software

Some CTD programs from Dave Watts' "MARPAK" facility were modified slightly to handle Aurora Australis data and to operate from within the "LOOKSEA" system. These programs did castplots, transect contours and temperature-salinity plots. LOOKSEA also included a new program called CTD_BLOBS, which drew 3-dimensional pictures of water masses defined by a user-specified limiting value of some CTD parameter, as well as coastlines and bottom topography. Sample output from CTD_BLOBS forms the cover illustration for this report. Another new program, CTD_SUMMARY, was provided on request from the voyage leader to generate tables of maxima and minima of CTD parameter values. One menu option, the "property vs property" program, was not completed and remained unavailable throughout the voyage.

1.3.2 DLS software

Qubit logging files were unable to support analysis schemes requiring access to the entire cruise. A new indexed file structure was defined and updated on a daily basis from the most recent QLFs. In house software used the indexed file which contained the whole cruise.

Two Fortran programs were used for plotting DLS data. A "property vs property" routine allowed users to plot any DLS logged parameter against any other. A customised version of this program was provided for plotting the ship's track and coast lines.

A gateway to TINY allowed users to invoke a series of PowerHouse screens and reports. Enquiry screens enabled meteorological, satellite navigation and ship status information to be displayed from individual logged DLS records. Reports generated tables of meteorological and oceanographic hourly averages and meteorological values at user specified intervals.

1.3.3 XBT software

A Fortran routine to enable XBT profiles to be plotted.

1.3.4 Data management menu

Several Fortran and DCL programs under this menu assisted ADP staff in carrying out housekeeping tasks. These included pre-processing CTD data, basic quality checking of DLS tapes, copying DLS tapes to high density tapes, backing up the M.O.D. and managing a number of batch procedures.

1.4 TELEX facility

A command procedure was made available to all users which automated the production of telexes. Messeges edited on the VAX were checked for validity and length (if personal, for which a limit of 100 words was imposed in the last week of the voyage). A series of questions established the telex address; this information was attached to the message in a format acceptable to BIGEAR. The message was mailed to a special TELEX account for inspection by the voyage leader and forwarding to the radio officer.

In anticipation of the use of the telex facility by most passengers command procedures were written that generated and removed user accounts in bulk. The facility worked well and was well received by the ship's radio officer, voyage leader, and passengers.

1.5 One-off data processing jobs

On many occasions, particularly towards the end, we received requests for "one-off" data processing work, especially the transfer of data files between the VAX and PCs.
 

2.0 PROBLEMS ENCOUNTERED

2.1 Hardware

2.1.1 HP and Scientific instruments

The DLS HP9000 has, since the HIMS cruise, occasionally failed to boot, returning and error message of "Unexpected use of FFFFFFAC". This problem was worked on by an HP engineer before the AAMBER2 cruise and supposedly solved. However, it reappeared several times during this voyage. Fortunately, on these occasions, the HP9000 rebooted successfully after having been powered off and on. Clearly the original fault remains; we are concerned that in future it may not be recoverable.

The HP Vectra PC in the MET lab failed and would not reboot. A faulty VGA card was the cause. Two HP Vectra PC keyboards failed. The VGA repeater on the bridge from the MET PC has lost one colour gun. Four BARCO monitors have failed. In all cases it was the RGB video driver transistors that failed. The DLS HP Draftmaster track plotter occasionally hangs, saying "Put in carousel" in the LCD display. Jiggling the carousel usually fixes this problem.

The Fluorometer burst it's plumbing and filled with sea water on 06-Mar-1991. It was replaced by a spare instrument.

Certain XBT probes cause the XBT HP Vectra PC to hang, requiring a reboot.

DLS HP-7979 tape drive number two occasionally displayed an incorrect number of dots in its "tape used" bargraph indicator. At these times the display should have contained seven or eight dots. However, only three dots actually were displayed. The bargraph display was used to estimate when the tape would be filled - this fault may lead to data loss.

2.1.2 DEC

Our most serious problem with respect to DEC hardware was the failure of the second hand RA-82 disk drive (DUA3). Between 04-Feb-1991 and 23-Feb-1991 DUA3 logged many hundreds of soft disk errors. Most errors were bad disk blocks or positioner mis-seeks. On 16-Feb-1991 DUA3 went into "Mount verification in progress" state and appeared to not recover, forcing a reboot. No errors were logged from DUA3 after 25-Feb-1991. Both AURORA and TINY's DUA0 (i.e. system) drives made loud buzzing noises from time to time. The Magneto-Optical disk started giving read errors over the last few days of the cruise, for several different disk platters. The reason for this is unknown. Possibly similar quantities of soot had deposited on the read/write head as had deposited everywhere else in the instrument room. If so, it is hard to imagine the M.O.D. working at all.

On two occasions the known TK-70 firmware fault appeared, locking up the tape drive. It remained un-useable until the next reboot. One VT320 keyboard has a faulty return key -multiple characters can be sent for a single keystroke.

2.2 Software

2.2.1 T.R.A.C. V

On many (but not all) occasions changing the TRAC database resulted in TRAC error 59 in 7448 (sometimes in 7333). The accompanying error text was "End of file or buffer found". Following this TRAC V would restart. However tape drives were left in "POWER" state which meant they were unusable until they had been powered off and on. Also the bridge console ceased to operate. Having occured, the error 59 repeated every fifteen minutes. To recover required a BASIC reset followed by the "RUN" command.

The bridge console failed to operate at other times for no apparent reason. Powering the bridge console off and on occasionally crashed TRAC some time later with error 314 "Receive buffer overflow". Again, a BASIC reset followed by "RUN" was required to clear this error. Error 314 and a TRAC crash also occured if the printer was left offline for more than an hour or so.

The VAX LAN connection was lost several times for no apparent reason.

Once again the ETA calculation proved unsatisfactory. This calculation is performed by integrating the distance along the ship's intended route between the current position and the ETA point. No account is taken of the off-track distance - i.e. only that part of the route which lies along the intended track is included in the calculation. A meaningful ETA estimate must allow for the time required to rejoin the intended track should the vessel deviate, e.g. due to ice or bad weather. In the event of the intended route crossing itself or of the ship deviating such that the nearest point on the intended route becomes unclear, an ambiguity arises as to which parts of the track are to be included in the ETA calculation. To resolve this, the calculation is based on the concept of a "current segment"; all parts of the route following the current segment contribute to the ETA time. However, the algorithm for assigning the current segment often chose the first segment, independent of the ship's location. Incorrect ETA times resulted. Recognising this problem, QUBIT have provided a means to manually set the current segment. Unfortunately this would often either fail to work, or fail to indicate which segment was current. Manual segment selection was based on a "trial and error" technique which usually succeeded only after several attempts.

DLS logs some, but not all, EA200 settings. In particular, the "minimum depth" setting is not recorded. On occasion we have seen flat-topped features in the digital bottom trace. There is no way of telling from the logged data if these are real features or if they have been clipped at the top due to the minimum depth being greater than the actual depth.

Pressing the "Enable CTD" soft key causes the "Start CTD dip" soft key to disappear. This was not the case on the HIMS cruise. It means that we could not check that the CTD data summary display was functioning prior to recording a CTD time-stamp in the DLS LOG file. Whilst DLS could correctly read the CTD fish ident number it could not allow for different pressure transducers on different fish. Thus, when we were forced to change CTD fish, the DLS display no longer gave correct pressure values. Occasional CTD data files failed to transfer from the CTD PC across the LAN to the VAX due to a "record too long" error. We solved this problem by using PCSA's NFT utility and specifying "/image" on the copy command. The QUBIT menu option should do likewise for all transfers.

"Transit fix age" was logged every ten seconds with no gaps. However, examining the data revealed that the logged values increased by eleven seconds per ten second logging interval. Examining the satellite navigation screen showed the same fault with the value displayed as "time since last fix".

DLS occasionally corrupted tape records, writing fragments of one record at the start of the next. Without correction, the remainder of the tape was unreadable, because all character fields in all remaining records became displaced by the number of extra characters in the rogue "fragment". This fault was present before the Heard island cruise; it remains to be corrected. On occasion an extra CR-LF sequence appeared in the middle of an EK-500 record, making the record unreadable to software expecting stream-cr or stream-lf record formatting.

DLS generates position "fixes" at regular intervals. At these times the ship's current postion is output to the DLS line printer. There is a facility to draw crosses on the graphics devices to indicate the fix, and to annotate these with either a time or a sequential fix number. It is useful to have hourly positions printed. However graphics devices become cluttered if fixes are indicated at this frequency. A menu option is provided to specify, for each graphics device, that only every i-th fix be indicated and that only every j-th of these be annotated. However, for at least part of the cruise, every fix was indicated on the graphics devices - independent of the values of i and j. A partial solution was attempted, by setting the automatic fixing interval to 86400 seconds - i.e. once per day. TRAC truncated this value, without notification, to 31980 seconds, or 8.8833 hours. As a compromise we chose a 21600 second fix interval (6 hours). In any case the approach was unsatisfactory, as hourly positions were no longer sent to the printer. Further, when using the option to select frequency of fix marking, old values were not cleared from the screen before new values were written. A confusing display resulted if a new value contained less digits than the old.

Only the absolute value of meteorolgical sea water temperature was logged. I.E. negative temperatures (in degrees C) were logged as positive. The fault was traced to the STEEDMANS single board computer (SBC) in the met lab, which dropped the sign bit from sea water temperature, and from air temperature. The latter was not a problem on this voyage because an approximately 22 degree C offset was applied to air temperature. This was to shift the range of values for which data could be logged toward temperatures colder than those for which the STEEDMANS unit was originally designed. For sea water temperature, the problem was solved by placing the met HP-Vectra PC between DLS and the met system. A short BASIC program on the PC read both the original STEEDMANS met station data (which did contain valid sea temperatures) and the SBC data (which didn't). It used the former data stream to correct the latter, and transmit the result to DLS. This meant that the PC could not perform it's usual task of printing ten-minute averages for use by the met bureau. These data will need to be extracted from DLS records for them at the end of the voyage.

The "big number" display for the Met PC does not clear old values before writing new ones, making a mess very quickly.

2.2.2 CHART V

It is possible to lose data after a LAN failure. Under some conditions the LAN file is overwritten instead of appended to. The "Close, rename, and reopen" option of Chart's lan server manager was observed to fail. Detailed descriptions of these events are found in the "Computer Log" book for the AAMBER2 voyage.

Chart's "INIT TAPE MEDIA" also offers the option of initialising AURORA's system disk! This option should be removed as it is a nonsense option - it should not work. If, by chance it does work, it would be extremely dangerous! We weren't game to try.

Attempts to copy a single database to tape using Chart's media options menu produced tapes that TRAC reported as containing no files. However, if all the files in the VAX directory are copied to tape by Chart then TRAC has no trouble at all in reading any file from that tape.

Chart does not display system error messages for long enough to be read or even noticed. At one stage we resorted to setting the terminal to communicate at 300 baud in order to read these messages! On more than one occasion inability to read system messages led us to believe that tape operations had been successful when in fact they had failed. This could easily lead to loss of data.

2.2.3 VMS

During our second visit to Mawson we had intended to upgrade GORT from uVMS V4.7 to VMS V5.3-1. We anticipated only 24 hours ashore. Further, there was to be no one familiar with VAX system management at Mawson after we left. In view of these restrictions and in consultation with the UAP computing officer it was decided that the risks of attempting the major upgrade outweighed the benefits. Whilst at Mawson we generated a stand-alone backup kit on a TK-50 and used this to do a stand-alone backup of GORT's system disk.

2.2.4 Manuals

Generally, we found the TRAC Manual superficial and of little help in answering specific questions. The manual describes features that do not exist in the Division's version of the system, e.g. 'Text Overlay' option for graphic displays does not exist.

As we have no manual for Chart, we were unable to use it to plot any quantity against any other quantity for any time interval. We did manage to plot database files and use some of the LAN server options and media utilities. Provision of such a software package as Chart without a manual is totally unacceptable and must be remedied before next season. On this voyage, post processing of DLS logged data was performed exclusively by in-house software.
 

3.0 RECOMMENDATIONS FOR NEXT SEASON

3.1 PC/Macintosh facilities and word processing

Most scientific passengers have a need for word processing. We have found that few passengers have any experience with VAX based text processing. Even those that do may not have access to the same software back at their own institution. Because of this it may be worthwile removing WPS from TINY. However, some crew members do use this package. Its removal would leave the ship without a word processing system, which may reduce its attractiveness to potential charterers during the off-season. P&O should be consulted.

Many passengers bring their own PC (IBM or Macintosh) aboard, not only for word processing but also to run applications relevant to their own programs. We perceive a need for two general access machines (one IBM PC compatible and one Macintosh) whose main role would be to provide access to network resources such as printers, plotters, and VAX file services. Limited time on these machines may be given for general use to individuals without PCs. Should demand become heavy a booking system may be required. Presumably, these machines would be Antarctic Division property and be removed from the ship during the off-season.

The two proposed machines would be located in the lower conference room. They would be connected to the network via thin-wire ethernet. To complete the word processing facilities the HP Laserjet printer would need to be upgraded with extra memory and support for Postscript. A few standard packages would be provided. These would reside as read-only disk services on the VAX, free from interference by enthusiastic users. The local hard disks would hold user files. Backups would be each user's own responsibility. Software to consider providing would include Microsoft Word, Microsoft Excel, kermit, disk utilities and anti-virus packages.

Possible hardware configurations for these machines are as follows:

IBM compatible PC

Macintosh

3.2 Data quality assurance

Many of the parameters logged are subject to periods of unreliability, for example, due to noise, sensor failure, or limitations in the measurement technique. Currently, cruise data is collected into one large file which is available to general users for analysis. Quality flags associated with each parameter are contained in this file. It is intended that software will be provided to automatically detect certain classes of unreliable measurements and set the corresponding quality flags appropriately. However, automated routines are limited in their ability to detect more subtle occurances of data corruption. Ultimately, human intervention is required.

We recommend that the Division consider including a full time data quality control officer on future marine science voyages. With the provision of suitable software, this individual would be responsible for inspecting logged data and setting quality flags, to the best of their ability, to indicate the data reliability. The intention would be that by the conclusion of a marine science voyage a reliable database would be available for immediate use.

3.3 Hardware enhancements

We recommend the installation of a third DECServer 200 terminal server in the Instrument Room. Prior to Voyage 6 two free terminal server ports were available here. During the voyage these have been taken up by the LG02 line printer and as a console to the MOD (Magneto-Optical Disk drive). Currently, we have no spare ports. One terminal is doubling as a console to the QD. An extra terminal server would enable any terminal to connect to the QD as a console, once a dedicated service was set up. On many occasions we had to disconnect a little-used terminal to gain one-off access to an extra terminal server port.

TINY currently has 4 Mbytes of memory, of which VMS and DECNET consume 2.5 Mbytes. It's performance, even with a single user, is much slower than expected for a CPU of it's capacity. This is particularly noticeable when running, EVE, PowerHouse, WPS, etc. We recommend that TINY be upgraded to it's maximum capacity of 6 Mbytes.

On this cruise there has been heavy demand for the HP Draftmaster plotter. Most plots have been A4 or A3 size. For certain applications, Chart in particular, this plotter cannot be queued because two way communications are required. Further, it does not make sense to queue a plotter unless it has some sheet feed mechanism or continuous roll media mounted. Consequently, we consider it desirable to install a second, queue-able, A4 plotter in the conference room. Perhaps this requires no more than upgrading the Laserjet printer to support HPGL (as well as Postscript). Alternatively, a small multipen HPGL plotter with sheet feed would suffice.

3.4 Software enhancements

The following software upgrades should be carried out before next season: upgrade AURORA and TINY to VMS V5.4; upgrade AURORA to the latest version of FORTRAN; upgrade AURORA to the latest version of PCSA.

During Voyage 6 PowerHouse software was developed that ran on TINY and accessed the logged data on AURORA. This software proved useful but awkward to use as the postprocessing menu was invoked from AURORA. Therefore, we recommend that a run-time only license for QUICK be obtained for AURORA. Due to the nature of QUIZ and QTP a full license for QUIZ, QTP for AURORA would be required. Were the Division to obtain the "Inquisitive" product then this would also prove useful for interrogating the cruise and wildlife databases.

Obtaining such PowerHouse licenses for AURORA would mean that the wildlife databases (Fish, Birds, Seals and Whales) could be accessed from AURORA rather than TINY. A full PowerHouse license could be retained for TINY so that development work could be done during a voyage.

3.5 The question of "clustering" AURORA and TINY

Neither of us feel we have enough experience managing VAX clusters to make a definite recommendation. Rather, we list here advantages and disadvantages which we see in clustering the two machines.

Advantages:

Disadvantages:


Back to Data Quality Report