Ferret supports collections of netCDF files that are regarded as a single netCDF data set. Such data sets are referred to as "MC" (multi CDF) data sets. They are defined via a descriptor file, in the style of TMAP-formatted data sets. These are FORTRAN NAMELIST-formatted files. Slight variations in syntax exist between systems. The requirements for an MC data set are described in the chapter "Data Set Basics", section "Multi-file NetCDF data sets".
Beginning with Ferret/PyFerret v7, the command TSERIES (an alias for DEFINE DATA/T) may be used as an alternative to multi-CDF datasets. This command defines an aggregation in the time direction, creating a virtual dataset in the Ferret session. Each method has advantages. Multi-file datasets will initialize more quickly than making an aggregation definition, as the step of defining the aggregation needs to initialize all of the member datasets to determine the time axis, but in a multi-CDF dataset, that information is read from the descriptor file. With a TSERIES aggregation, however, the correct axis bounds are applied, and missing time steps are handled by filling the data for those time steps with missing-value flags.
Previous to version 5.2 Ferret performs sanity checking on the data set by comparing these time coordinates with those found in the data files as the data are read. In version 5.3 and higher no sanity checks are performed. This means that the MC descriptor mechanism can be used to associate into time series groups of files that are not internally self-documenting with respect to time, however, it also shifts an additional burden onto the user of carefully checking the validity of the STEPFILE records in the descriptor files.
Ferret v6.84 and higher the default file type is netCDF. Ferret will recognize a descriptor dataset only if its file extension is .des or .DES. When opening the dataset, include its extension with the file name. In V6.85 and higher, files with extension .mc are also recognized as a descriptor files.
The stepfiles may contain different scale and offset values for the variables they contain, see reference. Ferret reads and applies the scale and offset values as data from each stepfile is read. Note that the commands
yes? SAY `var, RETURN=nc_offset` yes? SAY `var, RETURN=nc_scale`
return the latest scale and offset value that were applied.
The fields which are essential to consider are
$FORMAT_RECORD
D_TYPE = ' MC' ,
D_FORMAT = ' 1A',
which must be exactly as shown.
$BACKGROUND_RECORD
D_TITLE = 'Put your data set title here',
where you can insert a data sets title to appear on plots and listings;
D_T0TIME = '14-JAN-1980 14:00:00',
which corresponds exactly to the /T0 qualifier on the DEFINE AXIS command
D_TIME_UNIT = 3600.0,
which contains the same information as /UNITS= on the DEFINE AXIS command encoded as numbers of seconds . (/UNITS="minutes" corresponds to D_TIME_UNIT = 60., /UNITS="hours" corresponds to D_TIME_UNIT = 3600., etc.)
D_CALTYPE = '360_DAY',
to specify the name of the calendar if your time axis is not on the standard Gregorian calendar. See the discussion of time axes and calendars for more on the calendars available.
$STEPFILE_RECORD
S_FILENAME = 'mtaa063-nc.001',
which points to the individual file names. Typically you will need one STEPFILE_RECORD for each file in the series, however if the files are named with extension .001, .002, ... you can use S_NUM_OF_FILES below.
S_START = 17592.0,
which contains the time step value of the first time step in the file. (For help determining the time step values use DEFINE AXIS to create the desired time axis (say, my_t_ax) and then use LIST T[gt=my_t_ax] to see the dates and time steps.)
S_END = 34309.0,
which contains the time step value of the last time step in the file (or group of files if S_NUM_OF_FILES is greater than 1). If there is only a single time step in the file set S_END identical to S_START.
S_DELTA = 73.0,
which contains the delta value separating time steps within in the file. If there is only a single time step in the file set S_DELTA = 1.
S_NUM_OF_FILES = 23,
Normally S_NUM_OF_FILES should be omitted or should have a value of 1. Use it for the special case that your files are named with ending .001, .002, .... in which case you can describe them all with a single STEPFILE_RECORD as in the example. ( S_DELTA must then also describe the delta between the last time step in each file and the first time step in the next file of the series.)
10.4.1 Tools for making descriptor files
Two Ferret Users have written tools to create multi-file netCDF data sets. They are
1) A unix shell script, nc2mc which is available and documented at
http://www.pa.op.dlr.de/~PatrickJoeckel/nc2mc
2) A perl script, make_des which is available and documented at
http://www.gfdl.noaa.gov/~atw/ferret/make_des
Note that Ferret v6.84 and higher will recognize a descriptor file only if the file extension is .des or .DES. When opening the file, use its entire name. Either syntax will work:
yes? use file.des or yes? set data file.des
In Ferret v6.85 and higher, files with ".mc" as the extension are also recognized as descriptor files, and in addition "SET DATA/FORMAT=descriptor" will cause any file to be opened as a descriptor dataset.
yes? use file.mc yes? SET DATA/FORMAT=descriptor filename.anything yes? USE/FORMAT=des file.nnn
10.4.2 Example descriptor file
A typical MC descriptor file is given below. This file ties into a single data set the 23 files named mtaa063-nc.001 through mtaa063-nc.024. The time steps are encoded in the descriptor file through the S_START and S_END values.
Descriptor files have a space before the $ in the record headers, and comment lines begin with a *. In addition there are differences in the formatting of descriptor files depending on the operationg system. (For Ferret v5.0 through 5.51, here is an FAQ which addresses these differences: Using Multi-file netCDF descriptors under Linux)
*************************************************************************** * NOAA/PMEL Tropical Modeling and Analysis Program, Seattle, WA. * * created by MAKE_DESCRIPT rev. 4.01 * *************************************************************************** $FORMAT_RECORD D_TYPE = ' MC', D_FORMAT = ' 1A', D_SOURCE_CLASS = 'MODEL OUTPUT', $END $BACKGROUND_RECORD D_EXPNUM = '0063', D_MODNUM = ' AA', D_TITLE = 'MOM model output forced by Sadler winds', D_T0TIME = '14-JAN-1980 14:00:00', D_TIME_UNIT = 3600.0, D_TIME_MODULO = .FALSE., D_ADD_PARM = 15*' ', $END $MESSAGE_RECORD D_MESSAGE = ' ', D_ALERT_ON_OPEN = F, D_ALERT_ON_OUTPUT = F, $END ************************************************* $EXTRA_RECORD $END $STEPFILE_RECORD s_filename = 'mtaa063-nc.001', S_AUX_SET_NUM = 0, S_START = 17592.0, S_END = 34309.0, S_DELTA = 73.0, S_NUM_OF_FILES = 23, S_REGVARFLAG = ' ', $END ************************************************** $STEPFILE_RECORD s_filename = '**END OF STEPFILES**' $END **************************************************