National Oceanic and
Atmospheric Administration
United States Department of Commerce

Using data that spans multiple files.

Using data that spans multiple files.

Question:

How can I plot the time series of a variable which spans multiple data files?

Example:

> I have some netcdf-files with a lot of data. Each file represents the
> status of a certain model time. Now I want to plot one variable (say the
> instantanous mean) as a function of time. How can I do it?
>
> And can I save the calculated time series in a seperate (binary or netcdf)
> file from Ferret?

Explanation:

The answer to this depends on how your existing NetCDF files are arranged. Yousay that each file has a single time step. Let us suppose there are 3 spatialdimensions for the variables in your NetCDF files. There are two ways that thiscould be encoded: either as 3D (time-independent) variables or as 4D variableson a single point time axis. "ncdump -h mynetcdf.cdf" will show if thevariables have a 1-point time axis.

Solution:

==> If the files have a 1 point time axis there are three solutions:

  1. If you have a THREDDS server installed, use THREDDS aggregation to make the data into a single virtual dataset. THREDDS aggregation may be used to combine files that contain different timesteps of the same variable(s). It also includes other powerful tools for modifying and combining datasets.
  2. Define a virtual dataset in the Ferret session (Ferret v7 and higher) using DEFINE DATA/AGGREGATE/T

    yes? let file_list = SPAWN("ls -1 tmp/step_*.nc")
    yes? TSERIES all_times = file_list
  3. Use ferret to concatenate the files:
     use file1
    save/file=concat.cdf var1, var2, var3
    use file2
    save/append/file=concat.cdf var1,var2,var3
    .
    .
    .

    note that the names "var1", "var2",... refer to the "current" data set since no explicit data set was specified as in "var1[d=file1]".

  4. Create an "MC" (multi-cdf) data set by writing a descriptor file thatties the files into a single data set. An example of such adescriptor is below.

==> If your files do not have a time word,

    then you must use Ferret to impose atime word. Suppose you have 10 files, each separated by one model day, running1-jan-1980 to 10-jan-1980. Then you might use this technique:
 ! *** set up
define axis/t=1-jan-1980:10-jan-1980:1/unit=days t10days
define grid/t=t10days gg

let var1_with_T = t[g=gg]*0 + var1
set variable/title="xxx"/units="uuu" var1_with_T

! *** loop over input files
use file1.cdf
save/file=concat.cdf var1_with_T[l=1]

use file2.cdf
save/file=concat.cdf/append var1_with_T[l=2]

...

--------------------------------------------------
Here is an example "MC" descriptorfile: my_mc_descriptor.des:

 $FORMAT_RECORD
D_TYPE = ' MC',
D_FORMAT = ' 1A',
D_SOURCE_CLASS = 'MODEL OUTPUT',
D_SOURCE = 'PHIL/SIEG',
D_SUBSOURCE = 'TROPICAL PACIFIC',
$END
$BACKGROUND_RECORD
D_EXPNUM = '0068',
D_MODNUM = ' AA',
D_TITLE = 'MOM model output; Hindcast, Double Ramp; NCEP 79-95 twice daily winds'
D_FORCING_SET = 'NCEP 79-95 daily',
D_T0TIME = '14-JAN-1978 14:00:00',
D_TIME_UNIT = 3600.0,
D_TIME_MODULO = .TRUE.,
D_ADD_PARM = 15*' ',
$END
$MESSAGE_RECORD
D_MESSAGE = ' ',
D_ALERT_ON_OPEN = F,
D_ALERT_ON_OUTPUT = F,
$END
*************************************************
*************************************************
$EXTRA_RECORD
$END

$STEPFILE_RECORD
s_filename = 'mtaa068-nc.001',
S_AUX_SET_NUM = 0,
S_START = 87623.0,
S_END = 183599.0,
S_DELTA = 24.0,
S_NUM_OF_FILES = 400,
S_REGVARFLAG = ' ',
$END
**************************************************
$STEPFILE_RECORD
s_filename = '**END OF STEPFILES**'
$END
**************************************************