Chapter 4: GRIDS AND REGIONS
Information describing a region in space/time, a data set, and a grid is collectively referred to as the "context." The current context may be examined with the commands SHOW DATA_SET, SHOW REGION, and SHOW GRID. The context may be set explicitly with the commands SET DATA_SET, SET REGION, and SET GRID.
The context may be modified for the duration of a single command with qualifiers to the command name (separated by slashes). The same qualifiers in square brackets may also modify single variables, changing the context only of that variable:
yes? PLOT/D=levitus_climatology temp, salt
yes? CONTOUR rose[D=etopo20]
yes?
FILL/Z=0 temp[L=2] - temp[L=1]
Every variable has an underlying grid which defines a coordinate space. All grids are in a sense 4 dimensional (X, Y, Z, and T) but axes normal to the data are represented as "normal" (such as the Z axis of the surface wind stress).
Grids can be viewed, specified and created using SHOW GRID, SET GRID, DEFINE AXIS, and DEFINE GRID. These commands are all in the Commands Reference section of this manual. Data can be regridded by the G= modifier. (See this chapter, section "Regridding," p. 153)
Axes and grids can be explicitly created by DEFINE AXIS and DEFINE GRID. NetCDF and TMAP-formatted data set variables have all of the necessary grid and axis definitions embedded in the data set files, but if you are reading data from an ASCII or binary file, you must tell Ferret about the underlying grid of your data.
If you are creating abstract expressions entirely from pseudo-variables, you may want to define a grid in order to define the coordinate space of your calculation. This will also help produce a nicely labeled plot. (See the chapter "Variables and Expressions", "Grids and axes of pseudo-variables" (p. 61) and the example in the section on "Abstract Variables," p. 63.)
Example
This example is taken from the demonstration script "file_reading_demo.jnl". An ASCII file contains a grid of numbers, 50 rows by 6 columns. Suppose the data are on a 2D grid of 6 longitudes by 50 latitudes (Figure 4_1).
yes? DEFINE AXIS/X=10E:60E:10/UNIT=DEGREE xlong
yes? DEFINE AXIS/Y=0:49N:1/UNIT=DEGREE
ylat
yes? DEFINE GRID/X=xlong/Y=ylat gsnoopy2d
! By default only 1 column
is read, /COLUMNS= specifies 6 columns
yes? FILE/VAR=my_2D_var/COL=6/GRID=gsnoopy2d
snoopy.dat
yes? CONTOUR my_2D_var
Ch4 Sec2.2. Time axes and calendars
Data, particularly outputs from models, may be defined with time axes that are not on the standard Gregorian calendar. The netCDF conventions document discusses and defines usage for different calendars. These conventions for calendars are implemented in Ferret version 5.3 See:
http://www.cgd.ucar.edu/cms/eaton/cf-metadata/CF-current.html
NetCDF conforms to the conventions in the UDUNITS software package
http://www.unidata.ucar.edu/packages/udunits/
The concept of time units and formatted time needs some thought and explanation. The possibility of using different calendar definitions also compilcates the question. Time coordinates (seconds, days, years, etc) are used by the software for computation and comparison. Formatted time (30-DEC-2003 00:00:00) is for the convenience of the human user.
Time coordinates, given as so many units (seconds, days, years, etc) since a reference time is generally impossible to comprehend at a glance. There has to be internal code to convert to formatted time. Conversion between the time-since-reference form and formatted time requires that we know the calendar. The calendar says how many days there are in each month, and hence also implies the length of the year, which therefore depends on the calendar.
The units second, minute, hour and day (24 hours) are always the same in all calendars we use for Earth and so the utilities can assume this. Models would expect to use these units when they write out times in timesteps.
Conversion to units of time (year month day hour minute second) is also needed when processing data to calculate means over months or other calendar-related intervals and climatological statistics. For computation, comparisons, plotting and regridding, Ferret makes the choice to adopt a common length of year for all calendars.
TThe default calendar in Ferret is the Gregorian calendar. This is implemented as a "proleptic" calendar, where the definition of a year is consistent throughout time and does not have an offset in the 1500's as the historical calendars did. However, files written using the NOAA/CDC standard for the "blended" Julian/Gregorian calendar are read correctly by Ferret: If a time axis has a time origin of 1-1-1 00:00:00, and uses the default calendar, and if the coordinates of axis lie entirely after the year 1582, then the historical 2-day shift is applied.
Other calendars may be defined using DEFINE AXIS/CALENDAR= or by reading a variable with a calendar attribute from a netCDF file (see p. 40). You can set the calendar type in a descriptor file, with the D_CALTYPE attribute.
Example:
$BACKGROUND_RECORD
D_TITLE = 'Model Output, Daily Averages',
D_T0TIME = '30-DEC-0000 00:00:00',
D_TIME_UNIT =
3600.,
D_CALTYPE = 'NOLEAP',
$END
The calendars that are defined for use in Ferret are
|
calendar name |
number of days/year |
notes |
|
GREGORIAN or STANDARD |
365.2425 |
default calendar (proleptic gregorian) |
|
JULIAN |
365.25 |
with leap years |
|
NOLEAPor COMMON_YEAR |
365 |
no leap years |
|
360_DAY |
360 |
each month is 30 days |
Calendar names are matched using the first three characters.
Example:
Define a calendar axis and regrid an existing variable to this axis:
yes? DEFINE AXIS/CALENDAR=JULIAN/T="15-JAN-1982":"15-DEC-1985":30/UNITS=days tmodel yes? LET twind = uwnd[GT=tmodel@NRST]
Regridding between different calendars is allowed using the transformations @LIN (the default), @ASN, or @NRST. When regridding with @LIN from one calendar axis to another the length of a year is assumed to be constant, therefore the regridding calculates a scale factor based on the length of a second in each calendar, computed from the number of seconds per year for the calendars.
The analysis of multi-year daily data is often awkward, because the length of the year changes for leap years. The analysis can often be made simpler by regridding the data to a NOLEAP calendar.
Ch4 Sec2.3. Dynamic grids and axes
The commands DEFINE AXIS and DEFINE GRID, described in the preceding section, should be used when the grid or axis will be referenced more than once and/or shared among several variables. In many cases it is more convenient to use dynamic (a.k.a. "implicit") grids and axes. Two quick examples:
PLOT SIN(X[X=0:3.14:.1])
dynamically creates an axis from 0 to 3.14 by 0.1
SHADE SST[X=140E:160W:5, D=coads_climatology]
dynamically creates a longitude axis extending from 140E to 160W by 5 degrees, dynamically creates a grid which is like the grid upon which the variable SST is defined but with the X axis replaced by the new dynamic axis, and automatically regrids to this new grid.
It is often possible to avoid explicitly defining grids. This is useful in two common situations:
Regridding to specified axes without the need for defining the destination
grid.
Syntax: G*=name@transform, where
Example:
sst[GX=x10deg]
Suppose the variable SST is defined on a 2×2 degree grid in latitude/longitude
(e.g., SET DATA coads_climatology). If we wish to regrid to 10-degree spacing
in longitude over a range from 175W to 75W we could use the commands
DEFINE AXIS/X=175w:75w:10/UNITS=degrees x10deg
LET sst10 = sst[GX=x10deg]
Several axes can be specified together when they are to be regridded similarly.
For example, instead of sst[GX=x10deg, GY=y10deg] one can use the more
concise sst[GXY=x10deg]
Similarly, av_sst[GZ=@AVE, GT=@AVE] can be condensed to av_sst[GZT=@AVE]
Ferret will dynamically create a grid equivalent to new_grid in
DEFINE GRID/LIKE=sst/X=x10deg new_grid.
Figure 4_2 shows the effects of regridding the 2×2 degree COADS data to a 10-degree
spacing in longitude using (default) linear interpolation.
The command SHOW GRID SST10 will show the dynamically created grid. The
names of dynamic grids and axes will always be displayed in parentheses.
Note that the transformation method to be used for regridding may also
be specified, so LET SST10 = SST[GX=x10deg@ave] would create a 10-degree
spaced result in which each grid point was computed as the weighted sum
of the source points that fell within its grid box. The default
method for regridding is linear interpolation.
Situation 2
Automatic reconciliation of incompatible grid shapes
Syntax: G=name@transform
where
|
name |
The name of a grid or of another variable defined on the desired grid |
|
@transform |
The (optional) name of a regridding transform |
Example:
VAR1[g=VAR2]
If two variables are defined on grids that are mutually non-conformable
because axes exist in one grid but do not exist (are NORMAL) in another,
Ferret will now create a dynamic grid to resolve the non-conformabilities.
This means that an expression of the form VAR1[G=VAR2] will be meaningful
as long as the grid domains overlap.
For example, TEMP[d=levitus_climatology] is defined on an XYZ (time-independent)
grid whereas SST[d=coads_climatology] is defined on an XYT grid. So to
evaluate the expression SST[d=coads_climatology,G=TEMP[d=levitus_climatology]]
Ferret will create a dynamic intermediate grid equivalent to
DEFINE GRID/LIKE=sst[D=coads_climatology]/X=temp/Y=temp
so that regridding occurs on the X and Y axes but the original grid structure
is maintained with respect to depth and time.
The command SHOW GRID will reveal the resulting dynamically created grid
structure.
The syntax "GX=lo:hi:delta" can be used in square brackets modifying a variable name to indicate the dynamic creation of an axis with the indicated range and spacing and the immediate regridding of the variable to a grid containing that axis. For example, SST[GX=175W:75W:10] will create a dynamic axis of 10-degree regular point spacing, will create a dynamic grid incorporating this axis (see previous section), and will regrid the variable SST to this grid.
Similarly, by referring to the grid indices rather than their world coordinates, the expression SST[GX=1:100:5] will create a dynamic axis that subsamples every 5th longitude point from SST. In this case the points of the resulting axis may be irregularly spaced if the points of the original axis were also irregular.
As with the dynamic regridding described above, transformations can be specified to indicate the regridding technique. Thus SST[GX=1:100:5@AVE] will use averaging instead of the default linear interpolation to perform the regridding.
As a notational convenience the "G" may be dropped when referring to dynamic axes. Thus SST[X=175W:75W:10] is equivalent to SST[GX=175W:75W:10] and SST[I=1:100:5@AVE] is equivalent to SST[GX=1:100:5@AVE]. When using this notational convenience keep in mind that a regridding is taking place, so the transformation applied (if any) must be a regridding transformation (see SHOW TRANSFORMS in the command reference section, p. 442).
The lower plot of Figure 4_2 illustrates the effect of dynamic axes in the command
SHADE SST[GX=175W:75W:10]
Ch4 Sec2.3.3. Dynamic pseudo-variables
The same notation used for dynamic axes may also be applied to pseudo-variables providing a simple means for creating arrays of values. For example, X[GX=0.2:1:0.2] is a vector of 5 points from 0.2 to 1 at a regular spacing of 0.2 units. The vector is oriented in the X direction.
An example of using such a vector is (Figure 4_3)
PLOT SIN(X[GX=0:3.14:.1])
Note that when the lo:high:delta notation is applied to T or L expressed as calendar dates the units of the delta value will be hours. For example, L[GT=1-jan-1980:1-feb-1980:24] is the integers 1 to 32 defined on an axis of 32 days, 24 hours apart.
As a notational convenience the "G" may be dropped when referring to dynamic pseudo-variables. Thus X[X=0.2:1:0.2] is equivalent to X[GX=0.2:1:0.2].
See also the discussion of grids for pseudo-variables in section 3.1.3, p. 61.
Syntax:
var[G=name] for (default) linear interpolation to new grid
or
var[G=name@trn] to regrid all axes using transform "trn" (see below)
or
var[G=name,GX=@TRN,GY=@TRN,...] to control regridding transformations along each axis separately
where
|
var |
is the name of the variable to be regridded (e.g., temp, u, tau, ...) |
|
name |
is the name of a variable (e.g., temp[G=u]) or the name of a grid (e.g., temp[G=gu01]) |
|
trn |
is the name of a special transformation (e.g., @AVE, @ASN, @LIN) |
The syntax var[G=name,GX=@TRN,GY=@TRN,...] can be compressed when specifying
that several axes are to be regridded similarly. For example, instead of
var[GX=sst,
GY=SST]
one can now use the more concise
var[GXY=sst]
Similarly, if using a regridding transformation,
var[GZ=@AVE, GT=@AVE]
can
be condensed to
var[GZT=@AVE]
Note that in Ferret Version 5 and after when the limits of a variable are unspecified v2[g=v1] will default to the full extent of the v1 grid. Previously, it would become the size of whatever region of the v2 native grid overlapped with the v1 grid.
The Ferret distribution provides a demonstration of many regridding techniques:
yes? GO regridding_demo
Regridding is essential for algebraic operations that combine variables on incompatible grids. Ferret provides the commands DEFINE AXIS and DEFINE GRID to assist with the creation of arbitrary grids.
The result grid of a regridding operation does not necessarily match exactly the destination grid requested. For example, suppose the native grid of variable TEMP3D (Ocean Temperature) is 1 degree resolution in X and Y and 50 meter spacing in Z. If the syntax "[G=sst]" is used to request regridding to the grid of variable SST (Sea Surface Temperature), which is 2 degree resolution in X and Y, but normal to Z, then the resulting grid will be generated dynamically inheriting X and Y axes from SST as requested, but retaining the Z axis of TEMP3D.
Examples
1) Suppose the variables u and temp are on staggered X, Y, and Z axes but share the same T axis. Then the two variables can be multiplied together on the axes (grid) of the u variable as follows:
yes? CONTOUR u * temp[G=u]
This will regrid temp onto the u grid by multi-axis linear interpolation
before performing the multiplication.
2) Two variables, v1 and v2, are defined on distinct 4-dimensional grids (X, Y, Z, and T axes). The T axes of the two grids are identical but the X, Y, and Z axes all differ between the two variables. (This is often the case in numerical model outputs.)
To obtain the variable v1 on its original Z (depth) locations but regridded
in the XY plane to the grid locations of the variable v2, define a new
grid (say, named "new_grid") that has the X and Y axes of v2 but the Z
axis of v1.
yes? DEFINE GRID/LIKE=v2/Z=v1 new_grid !define new grid
yes? LIST/X=160E:140W/Y=5S:5N
v1[G=new_grid] !request regridding
3) In this example we look at temperature data from two data sets. levitus_climatology, an annual climatology, has the variable "temp" on an XYZ grid which is 1×1 degree in XY, and coads_climatology, a monthly climatology, has the variable "sst" on an XYT grid which is 2×2 degrees in XY. Suppose we wish to look at the sea surface temperatures in January at the higher XY resolution of the Levitus data.
yes? SET DATA levitus_climatology
yes? SET DATA coads_climatology
yes? SET
REGION/L=1/Z=0
yes? !get the name of the grid on which temp is defined
yes?
SHOW GRID temp[D=levitus_climatology] ! > "Glevitr1"
yes? DEFINE GRID/X=glevitr1/Y=glevitr1/Z=sst/L=sst
glevitus_xy
yes? LIST/X=150E:155E/Y=0:5N sst[G=glevitus_xy]
Ch4 Sec2.4.1. Regridding transformations
Ferret supports several regridding transformations. Use the SHOW TRANSFORMATIONS command to obtain a list of the supported transformations from Ferret. The choice of regridding transformation determines the computation by which data from the source grid determine the values on the destination grid.
Regridding transformations provide a means to perform a given calculation over a limited span of coordinates and repeat that calculation for a series of contiguous spans. For example, if we wish to compute the variance of the variable SST over 10-degree longitude range from 180 to 170W we could use the syntax sst[X=180:170w@VAR]. Now, if we wish to perform the same operation 10 times in 10-degree wide bands from 180 to 80W we could instead use G=@VAR regridding as in (see Dynamic Grids, p. 148, for an explanation of the "GX=" syntax):
DEFINE AXIS/X=175w:85w:10/UNITS=degrees ×10deg
LET sst10 = sst[GX=x10deg@VAR]
The regridding transformations are:
@LINlinear interpolation (the default if no transform is specified)
Performs regridding by multi-axis linear interpolation.
Computes the length-weighted average of all points on the source grid that lie partly or completely within each grid cell of the destination grid. If any portion of a source grid cell containing data overlaps a given destination grid cell, then data from that source cell contributes to the destination cell, weighted by the fraction of the destination cell overlapped by the source cell. The source data are treated as continuous, extending to the edges of the grid cells.
Note: When @AVE is applied simultaneously to the X and Y axes, where X
and Y are longitude and latitude, respectively, an area-weighted average
(weighted by cos(latitude)) is used. The @AVE transformation is unique
in this respect. In multiple axis applications other than X and Y @AVE
will be applied sequentially to the axes, computing the "average of the
average." This may not be the desired weighting scheme in some cases. See
@VAR below for an example.
Associates by subscript (blindly) the points from the source grid onto destination coordinates.
Computes the variance of the points from the source grid that fall within each destination grid cell. This is a length-weighted computation like the @AVE transformation.
Note: This transformation is suitable for regridding only in a single axis.
When applied simultaneously to two axes, for example, it will compute the
variance of the variance. For example, V[gx=130E:80W:10@VAR, gy=205:20W:10@VAR]
is equivalent to tmp[X=130E:80W:10@VAR] where tmp=V[y=20S:20N:10@VAR].
Compute the number of points from the source grid that fall within each destination grid cell. Note that the results of this calculation need not be integersthis is a length-weighted computation like the @AVE transformation. It is common for a grid cell on the source grid to span the boundary between grid cells on the destination grid, thereby contributing a fraction of its weight to multiple destination grid cells.
Note: This transformation is suitable only for regridding on a single axis.
When applied simultaneously to two axes, for example, it will compute a
constant. See @VAR for an example.
Nearest coordinate regridding VAR[GX=newaxis@NRST] chooses the value from the source axis coordinate closest to the destination axis. If source coordinates above and below are equally close to a destination coordinate the value at the lower coordinate will be chosen. (This is most useful for regridding between axes whose coordinate values are very close, though not exactly matched -- e.g. between equally and unequally spaced monthly time axes.)
Computes the length-weighted sum of the points from the source grid that fall within each destination grid cell. This is a length-weighted computation like the @AVE transformation.
Finds the minimum value of those points from the source grid that lie within each destination grid cell. Note that this is NOT a weighted calculation; the destination grid cell that "owns" a source point is determined entirely from the coordinate location of the source point, not from the limits of the source grid cell.
(As of Ferret V5.1) If a point on the source axis lies exactly on the
boundary between grid cells on the destination axis it will be included
in the calculations for the higher indexed cell on the destination axis.
If a point on the source axis lies exactly on the upper cell boundary of
the highest point on the destination axis, then it will be included in
the calculations for that highest destination grid cell.
If you have data on a single point axis and you wish to embed it in a larger
axis range you can achieve this by using either the G=@MIN or G=@MAX.
For example,
yes? define axis/x=163e/npoints=1 x1pt
yes? let var_1pt = randu(x[gx=x1pt])
! a random value at a single coordinate
yes? list var_1pt
RANDU(X[GX=X1PT])
LONGITUDE: 163E
0.4914
yes? define axis/x=161e:165e:1
x5pt
yes? list var_1pt[gx=x5pt@max] ! same value embedded within 5 point
axis
RANDU(X[GX=X1PT])
regrid: 1 deg on X@MAX
161E / 1: ....
162E / 2: ....
163E / 3: 0.4914
164E / 4:
....
165E / 5: ....
Finds the maximum value of those points from the source grid that lie within
each destination grid cell. Note that this is NOT a weighted calculation;
the destination grid cell that "owns" a source point is determined entirely
from the coordinate location of the source point, not from the limits of
the source grid cell..
(As of Ferret V5.1) If a point on the source axis lies exactly on the
boundary between grid cells on the destination axis it will be included
in the calculations for the higher indexed cell on the destination axis.
If a point on the source axis lies exactly on the upper cell boundary of
the highest point on the destination axis, then it will be included in
the calculations for that highest destination grid cell.
The @MAX transformation is useful as a mechanism to perform regridding
between two axes that do not quite match. A common example would be to
regrid between two monthly axes one of which has points located at the
15th of each month and the other having points exactly at mid-month. These
Ferret commands illustrate the example using a 5-month axis in 1993:
! define axes for 15th of month and mid-month
yes? DEFINE AXIS/UNIT=DAYS/T0=1-JAN-1900 month_15 =
DAYS1900(1993,I[I1:5],
15)
yes? DEFINE AXIS/UNIT=DAYS/T0=1-JAN-1900/EDGES month_mid =
DAYS1900(1993,I[I=1:6],
1)
yes? let my_var = L[gt=month_15
yes? list my_var
L[GT=MONTH_15]
15-JAN-1993 00 / 1: 1.000
15-FEB-1993 00 / 2: 2.000
15-MAR-1993 00 / 3: 3.000
15-APR-1993 00 / 4: 4.000
15-MAY-1993
00 / 5: 5.000
yes? list my_var[gt=month_mid]
L[GT=MONTH_15]
regrid: on T
16-JAN-1993 12 / 1: 1.048
15-FEB-1993 00
/ 2: 2.000
16-MAR-1993 12 / 3: 3.048
16-APR-1993 00 / 4: 4.033
16-MAY-1993 12 / 5: .... ! unable to interpolate
yes? list my_var[gt=month_mid@max]
L[GT=MONTH_15]
regrid: on T@MAX
16-JAN-1993 12 / 1: 1.000
15-FEB-1993 00 / 2: 2.000
16-MAR-1993
12 / 3: 3.000
16-APR-1993 00 / 4: 4.000
16-MAY-1993 12 / 5:
5.000
Regridding with G=@XACT (or GX=@XACT, etc.) is a request to transfer values from a source grid to a destination grid only at those positions where there is an exact coordinate match between the source and destination axis points on the axis in question. Other destination points will be set to "missing". This transformation is especially useful for taking multiple in-situ data profiles, such as oceanographic cast data, and regridding them onto a regular (sparse) grid. For example: grep
yes? LET xcoarse = sin(x[x=0:20:10])
yes? LIST xcoarse
SIN(X[X=0:20:10])
0 / 1: 0.0000
10 / 2: -0.5440
20 / 3: 0.9129
yes? DEFINE AXIS/X=0:20:5 xfine
yes? LIST
xcoarse[gx=xfine@XACT]
SIN(X[X=0:20:10])
regrid: 5 delta on X@XACT
0 / 1: 0.0000
5
/ 2: ....
10 / 3: -0.5440
15 / 4: ....
20 / 5: 0.9129
Creates climatologies from time series by regridding to a time series axis with a modulo regridding transformation. See the section on Modulo Regridding (p. 159) for details.
1) Let variable temp be defined on a grid with points spaced regularly at 1-degree intervals in both longitude and latitude (X and Y). Let grid "g10" possess points spaced regularly at 10-degree intervals in both X and Y.
yes? PLOT temp[G=g10] ! uses linear interpolation (@LIN)
yes?
PLOT temp[G=g10@AVE] ! area-averages 10x10 degrees of source\
points into each destination point.
yes? PLOT
temp[G=g10,GX=@AVE] ! averages 10 degrees of longitude but\
interpolates (@LIN) in Y.
2) @ASN has the effect of bypassing Ferret's protections against misrepresenting data (Figure 4_4).
yes? SET DATA levitus_climatology
yes? SET REGION/X=180/Y=0 ! true profile
yes?
PLOT/Z=0:5000 temp
yes? DEFINE AXIS/DEPTH /Z=100:2000:100 zfalse
yes? DEFINE
GRID/LIKE=temp /Z=zfalse gfalse ! false profile
yes? PLOT/Z=0:5000/OVER
temp[G=gfalse@ASN]
Ferret can create climatologies from time series simply by regridding to a climatological axis with a modulo regridding transformation. For example, if the axis named month_reg is a 12-point monthly climatological (modulo) axis then the expression
LET sst_climatology = sst[D=coads,GT=month_reg@MOD]
is a 12-month climatology computed by averaging the full time domain of the input variable (576 points for coads) modulo fashion into the 12 points of the climatological axis.
Ferret has three pre-defined climatological axes: seasonal_reg (Feb, May, Aug, Nov), month_reg (middle of every month), and month_irreg (15th of every month). In addition, there is an FAQ that describes how to create a daily climatological series, How do I compute a daily climatology for a time series? The analysis of multi-year daily data is often awkward, because the length of the year changes for leap years. The analysis can often be made simpler by regridding the data to a NOLEAP calendar.
yes? USE climatological_axes
*** NOTE: regarding ... climatological_axes.cdf
***
NOTE: Climatological axes SEASONAL_REG, MONTH_REG, and MONTH_IRREG
defined
yes?
CANCEL DATA climatological_axes ! the axes are still defined
To generate a climatology based on a restricted range of input data simply define an intermediate variable with the desired points. For example, a monthly climatological time series based on data from the 1960s could be computed using
LET sst_1960s = sst[D=coads,T=1-jan-1960:31-dec-1969]
PLOT sst_1960s[GT=month_reg@MOD]
In a similar fashion intermediate variables can be defined that mask out certain input points.
This example shows the entire sequence necessary to generate a plot of climatological SST at 40N, 40W based on the January 1982 to December 1992 Fleet Numerical wind data set. (Figure 4_5).
! use the predefined climatological axes
USE climatological_axes
CANCEL DATA
climatological_axes
! use the Fleet Numerical winds
SET DATA monthly_navy_winds
!
plot the raw data (top figure)
SET REGION/X=40w/Y=40n
plot uwnd
! plot the
12 month climatology (middle figure)
LET uwnd_clim = uwnd[GT=month_reg@MOD]
PLOT
uwnd_clim
! since uwnd_clim is on a climatological axis
! Ferret can also
plot it on the calendar axis with the raw data
PLOT/T=16-jan-1982:17-dec-1992
uwnd,uwnd_clim
In many cases the volume of input data needed to perform climatological calculations is very large. In the example above the command
CONTOUR/X=0:360/Y=90s:90n sst_climatology[L=1]
to plot January from the climatology would require Nx*Ny*Nt=72*72*576=3 Megawords of data. Such calculations may be too large to fit into memory. However, if the region is fully specified (as shown for the X and Y limits in the example) Ferret's internal memory manager will break up the calculation as needed to produce the result. (See Memory Use in the chapter "Computing Environment", p. 259, for further details.)
Unlike other transformations and regridding, modulo regridding is performed as an unweighted average: each non-missing source point contributes 100% of its weight to the destination grid box within which it falls. If the source and destination axes are not properly aligned this can lead to apparent shifts in the data. For example, if a monthly time series has data points at the first of each month and a climatological axis is defined at midmonths, then unweighted modulo averaging will lead to an apparent 1/2-month shift. To avoid situations of this type simply regrid to the climatological axis using linear interpolation prior to the modulo regridding.
Here is an example that illustrates the situation and the use of linear interpolation to repair it. (Figure 4_6).
! define test_var, an illustrative variable with 1 year periodicity
! Note:
test_var points are at the **beginnings** of months
DEFINE AXIS/T=1-jan-1970:1-jan-1974:`365.25/12`/UNITS=days
t10years
DEFINE GRID/T=t10years gg
LET test_var = SIN(L[G=gg]*2*3.14/12)
!
plot 4 years of the cycle
PLOT test_var
! define climatological axes at the
midpoints of months
USE climatological_axes
CANC DATA climatological_axes
!
notice that modulo regridding appears to shift the data
! (due to mis-aligned
source and destination axes) (top figure)
PLOT/OVER/T=1-jan-1970:1-jan-1974
test_var[GT=month_reg@MOD]
! to avoid the shift we can first regrid test_var
to mid-month
! points using linear interpolation (the default regridding
method)
! notice that the function test_var is largely unchanged by this
LET
test_var_centered = test_var[GT=month_reg]
PLOT/OVER/T=1-jan-1970:1-jan-1974
test_var_centered
! finally perform a modulo regridding on well-aligned
data
! notice that the shift is gone (bottom figure)
PLOT/OVER/T=1-jan-1970:1-jan-1974
test_var_centered[GT=month_reg]
Ch4 Sec2.5.1. Modulo regridding statistics
In addition to the modulo averaging calculation performed by @MOD Ferret provides other statistics of the regridding. All modulo regridding calculations are unweighted as discussed under @MOD.
the variance of source points within each destination grid box (SUM(var-varbar)^2)/(n-1))
the sum of the source points within each destination grid box
the number of source points contributing to each destination grid box
the number of source points that are missing in the contribution to each destination grid box
the minimum value of the source points contributing to each destination grid box
the maximum value of the source points contributing to each destination grid box
The region in space and time where expressions are evaluated may be specified in 3 different ways:
1) with the command SET REGION
2) with qualifiers to the command name (slash-delimited)
3) with qualifiers to variable names (in square brackets, comma-delimited)
If SET REGION is used, Ferret remembers the region as the default context for future commands, whereas a qualifier to a command name specifies the region for that command only, and a qualifier to a variable name specifies the region for that variable and command only.
Regions may be manipulated using DEFINE REGION, SET REGION, @ notation, and CANCEL REGION. The Commands Reference section of this manual covers all of these topics.
Region information is normally specified in the following form:
QUAL=val or
QUAL=lo_val:hi_val or
QUAL=val@transform (as a variable qualifier
only) or
QUAL=lo_val:hi_val@transform (as a variable qualifier only)
When the region for an axis is specified as a single value (instead of a range) Ferret, by default, selects the grid point of the grid box containing this value. The Ferret mode "interpolate" can control this behavior. See command SET MODE INTERPOLATE in Commands Reference, p. 414.
Examples: Regions
Examples of valid region specifications.
1) Fully specify the region in an XY plane with the first vertical (Z) level and time 27739.
yes? SET REGION/X=140E:160W/Y=10S:20N/K=1/T=27739
2) Contour vertical heat advection within whatever region is the current default (previously set with SET REGION).
yes? CONTOUR qadz
3) Define, modify and set a named region and then modify with delta notation.
yes? DEFINE/REGION/Y=5S:5N YT !define region YT to be 5S:5N
yes?
DEFINE REGION/DY=-1:+1 YT !modify region YT to be 6S:6N
yes? SET
REGION/@YT !set current region to YT
yes? SET REGION/DY=-1:+1
!modify current region to 7S:7N
4) List meridional currents calculated by averaging values between the surface and a depth of 50 m.
yes? LIST v[Z=0:50@AVE]
5) Equivalent to v[Z=10] - v[Z=0:100@AVE], the anomaly at z=10 between v and the 0 to 100 meter depth average of v.
yes? LIST/Z=10 v - v[Z=0:100@AVE]
Specify latitude or a latitude range with the qualifier Y or J. Specifications using J are integers between 1 and the number of points on the Y axis. Specifications using Y are in the units of the Y axis.
The units may be examined with SHOW GRID/Y. If the Y axis units are degrees of latitude then the Y positions may be specified as numbers followed by the letters "N" or "S".
Examples
yes? CONTOUR temp[Y=15S:10N]
yes? LIST/J=50 u
Specify longitude or a longitude range with the qualifier X or I. Specifications using I are integers between 1 and the number of points on the X axis. Specifications using X are in the units of the X axis.
The units may be examined with SHOW GRID/X. If the units are degrees, then X values may be given as numbers followed by "W" or "E" (e.g., 160E, 110.5W) or as values between 0 and 360 with Greenwich at 0 increasing eastward. Note: If the X axis is "modulo" then it is sometimes desirable to use X greater than 360.
Examples
yes? CONTOUR temp[Y=160E:140W]
yes? LIST/I=100 u
yes? SHADE/X=100:460
temp !360 degrees centered at 100W
See the chapter "Grids and Regins", section "Modulo Axes" (p. 167), for help with globe-encircling axes.
Specify depth or a depth range with the qualifier Z or K. Specifications using K are integers between 1 and the number of points on the Z axis. Specifications using Z are in the units of the Z axis.
The units may be examined with SHOW GRID/Z.
Examples
yes? CONTOUR temp[Z=0:100]
yes? LIST/K=3 u
Specify time or a time range with the qualifier T or L. Specifications using L are integers between 1 and the number of points on the T axis. Specifications using T may refer to calendar dates or to the time step units in which the time axis of the data set is defined.
Calendar date/time values are entered in the format dd-mmm-yyyy:hh:mm:ss, for example 14-FEB-1988:12:30:00. At a minimum the string must contain day, month, and year. If the string contains any colons it must be enclosed in quotation marks to differentiate from colons used to designate a range. If a time increment is specified with the repeat command given in calendar format (e.g., REPEAT/T="1-JAN-1982":"15-JAN-1982":6) it is interpreted as hours always. Calendar dates in the years 0000 and 0001 are regarded as year-independent dates (suitable for climatological data). Ferret cannot work with years larger than year 9999.
SHOW GRID/T can be used to display time step values. (Units may vary between data sets.) The commands SET MODE CALENDAR and CANCEL MODE CALENDAR can be used to view date strings or time steps, respectively.
Examples
yes? LIST/T="1-JAN-1982:13:50":"15-FEB-1982" density
yes? CONTOUR temp[T=27740:30000]
yes?
LIST/L=90 u
See the section in this chapter on "Modulo Axes" (p. 167) for help with climatological axes.
The notation q=lo:hi:delta (e.g., Y=20S:20N:5) specifies that the data in the requested range is regularly subsampled at interval "delta."
This notation is valid only for the REPEAT, SHOW GRID, and DEFINE AXIS commands, and the qualifiers /HLIMITS and /VLIMITS used in action commands with graphical output.
It can also be used in square brackets when specifying variable context:
yes? LIST temp[l=40:90:5]
(but this is NOT allowed: LIST/L=40:90:5 temp )
Regions may be named and referred to using the syntax "@name". Some commonly used regions are predefined. See commands SET REGION (p. 421) and DEFINE REGION (p. 347) in the Commands Reference section for further information.
If a region is specified using a combination of "@" notation and explicit axis limits the explicit axis limits will be evaluated after the "@" specification, possibly superseding the "@" limits.
Note: It is not advised to use the @notation inside of variable definitions, as redefinitions of the named region can cause code errors that lead to wrong results.
Using the @ notation only sets/alters the axis limits specified in the named region. For example, suppose that region "XY" is defined for the X and Y axes, but not for the Z and T axes. Then
yes? SET REGION/@XY
modifies only X and Y limits. BUT,
yes? SET REGION XY
modifies all axesX and Y to the limits specified by XY, and Z and T to unspecified (even if they were previously specified).
Examples
1) Contour the 25th time step of temperature data at depth 10 within region T, the "Tropical Pacific."
yes? CONTOUR/@T/Z=10/L=25 temp
2) Produce a contour plot over region W, the "Whole Pacific Ocean," in the XY plane (the variable to be contoured as well as the depth and time will be inferred from the current context).
yes? CONTOUR/@W var1
3) Set the default region to "T", the Tropical Pacific Ocean (latitude 23.5S to 23.5N).
yes? SET REGION/@T
4) Define a region and then supersede with an axis limit specification.
yes? DEFINE REGION/X=180:140W/Y=2S:2N/Z=5 BOX1
yes? SET REGION/@BOX1/Z=15
!replace Z
As a convenience in the analysis of the Tropical Pacific Ocean the following regions are pre-defined:
These may be redefined by the user for the duration of a Ferret session or until the definitions are canceled.
Some axes are inherently "modulo," indicating that the axis wraps aroundthe first point immediately following the last.
To determine if an axis is modulo use SHOW AXIS or SHOW GRID. A letter "m" following the number of points in the axis indicates a modulo axis. The command SHOW GRID qualified by the appropriate axis limits can be used to examine any part of the axisincluding points beyond the nominal length of the axis. The commands SET AXIS/MODULO and CANCEL AXIS/MODULO can be used to control this feature on an axis-by-axis basis. Starting with Ferret version5.5, longitude axes which span 360 degrees or less, and climatological time axes are always detected as modulo, unless Ferret is specifically directed that the axis is NOT modulo, e.g. by a CANCEL AXIS/MODULO command.
Example
yes? SET DATA coads_climatology
yes? SHOW GRID/I=180:183 sst !range request
beyond last point
GRID COADS1
name axis # pts
start end
COADSX LONGITUDE 180mr 21E
19E(379)
[text omitted]
I X BOX_SIZ
180> 19E(379) 2
181> 21E(381) 2
182> 23E(383) 2
183> 25E(385) 2
The most common uses of modulo axes are:
1) As longitude axes for globe-encircling data sets. This allows any starting and any ending longitudes to be used, for example, X=140E:140E indicates the entire earth with data beginning and ending at 140E.
2) As time axes for climatological data. By this device the time axis appears to extend from 0 to infinity and the climatological data can be referred to at any point in time. This facilitates comparisons with data sets at fixed times.
Ch4 Sec3.7.1. Subspan Modulo Axes
Ferret V5.5 introduces the concept of a "sub-span modulo axis" -- an axis where the range is a sub-range of a fullmodulo cycle. As of V5.5, longitude axes and climatological time axes are always detected as modulo, or as sub-span modulo when appropriate, unless Ferret is specifically directed that the axis is NOT modulo, e.g. by a CANCEL AXIS/MODULO command. If the user does not specify the modulo length, it is set to 360 degrees for a longitude axis, or a year for a time axis. Time axes of lenght less than or equal to one year, and starting in year 0000 or 0001 are taken to be climatological axes.
The modulo length of an axis defined on the Ferret command line is set with an argument to the MODULO qualifier, or with an argument to the netCDF modulo attribute. Here is an example showing an axis defined explicitly as a modulo axis, and another which is modulo by default.
yes? DEFINE AXIS/MODULO=100/x=41:55:1 xax_subspan
yes? DEFINE AXIS/X=100:300:10/UNITS=degrees_longitude
xax_lonspan
The output of SHOW AXIS includes the modulo length and span of the axis:
yes? show axis xax*
name axis # pts start end
XAX_SUBSPAN
6mr 41 46
Axis span (to cell edges)
= 6 (modulo length = 100)
XAX_LONSPAN LONGITUDE 21mr 100E
60W
Axis span (to cell edges) = 210 (modulo length = 360)
In netCDF output files you will now see the modulo attribute taking a value. Continuing the example above, we write some variables using the axes to a file and use ncdump to show the modulo attribute in these files.
yes? LET v1 = X[GX=xax_subspan] +10
yes? LET v2 = SIN(X[GX=xax_lonspan])
yes?
SAVE/FILE=test_subspan_modulo.nc v1, v2
yes? SPAWN ncdump -c test_subspan_modulo.nc
netcdf
test_subspan_modulo {
dimensions:
XAX_SUBSPAN = 15 ;
XAX_LONSPAN = 21 ;
variables:
double
XAX_SUBSPAN(XAX_SUBSPAN) ;
XAX_SUBSPAN:modulo = 100. ;
XAX_SUBSPAN:point_spacing
= "even" ;
XAX_SUBSPAN:AXIS = "X" ;
float V1(XAX_SUBSPAN) ;
V1:missing_value
= -1.e+34f ;
V1:_FillValue = -1.e+34f ;
V1:long_name = "X[GX=XAX_SUBSPAN]
+ 10" ;
double XAX_LONSPAN(XAX_LONSPAN) ;
XAX_LONSPAN:units = "degrees_east"
;
XAX_LONSPAN:modulo = 360. ;
XAX_LONSPAN:point_spacing = "even" ;
XAX_LONSPAN:AXIS
= "X" ;
float V2(XAX_LONSPAN) ;
V2:missing_value = -1.e+34f ;
V2:_FillValue
= -1.e+34f ;
V2:long_name = "SIN(X[GX=XAX_LONSPAN])" ;
// global attributes:
:history
= "FERRET V5.50 15-Jan-03" ;
data:
XAX_SUBSPAN = 41, 42, 43, 44, 45, 46,
47, 48, 49, 50, 51, 52, 53, 54, 55 ;
XAX_LONSPAN = 100, 110, 120, 130,
140, 150, 160, 170, 180, 190, 200, 210,
220, 230, 240, 250, 260, 270,
280, 290, 300 ;
}
The importance of the sub-span modulo axes is to take the first of two steps that will make it possible for users largely to ignore differences in encodings of longitude and climatological time -- e.g. the blending of data in plots and analyses where the data come from data sets that are encoded variously as -180:180, 0:360, etc. Thus, for example, in V5.5 you can refer to my_subspan_var[g=another_var] and get a meaningful answer as long as the grids occupy the same region on the globe, regardless of longitude encoding. (The second step, for a future release, will address the longitude encoding of scattered data.)
Example:
Suppose we have data on an axis that was defined as follows
yes? DEFINE AXIS/X=520:550:1/UNITS=degrees xax
and supposet we want to overlay it on a map showing the regional topography.
yes? USE etopo05
yes? SHOW GRID rose ! We will want the names of the axes
GRID GOZ1
name axis # pts start
end
ETOPO05_X LONGITUDE 4320mr 0E 0.079987W
ETOPO05_Y LATITUDE 2161 r 90S 90N
normal
Z
normal T
yes? SET REGION W
yes? SHADE/PAL=land_sea rose[d=1] ! draw
the shade plot
yes? USE my_data.nc ! The dataset containing the x=520:550
data
yes? SHOW AXIS xsub
name axis # pts start
end
XSUB LONGITUDE 31mr 160E(520)
170W(550)
Axis span (to cell edges) = 31 (modulo length = 360)
yes?
SHADE/OVER/PAL=greyscale a[GX=etopo05_x,GY=etopo05_y]
There is an implied void point in a sub-span modulo axis that fills the gap between the end of the axis and the start of the next modulo cycle. The data value at this point will always be the missing value flag (except for pseudo-variables such as "X[g=my_subspan_axis]"). Transformations such as smoothers do not operate across the void point.
In netCDF files, the modulo attribute is specified as follows:
1) Specify the modulo length of the axis with the attribute modulo = <value>, e.g. var:modulo=100;
2) The modulo attribute from previous netCDF files remains unchanged: modulo = " ". To set a modulo axis of standard length (360 degrees or one year). The modulo length is 360 degrees for a longitude axis, or one year for a climatological time axis.
3) The attribute value modulo = "FALSE", modulo = "NO", modulo="OFF" tells Ferret that the axis is not to be treated as modulo
Conflicting region information can be given to Ferret in obvious ways such as
LIST/I=1:3 I[I=1:10]
in which it is not clear if the request is for 10 points or for 3, or in subtler, disguised ways such as
LET A = I[I=1:10] LIST/I=1:3 A
In both examples Ferret would resolve the conflict by listing just the three values I=1:3.
Internally, Ferret uses the region closest to the variable to perform the calculation. Thus, in both of the examples above Ferret will perform the calculation on I=1:10, since the "[I=1:10]" directly modifies the variable name. If Ferret sees conflicting regions it attempts to use the regions further from the variable to clip the calculation. Thus 10 points are clipped to 3 in the above examples.
Unresolvable conflicts such as
LIST/I=11:13 I[I=1:10]
will result in a warning message that invalid limits have been ignored.
Ch4 Sec4. FERRET PROGRAM LIMITS
There are a number of hard limits in the Ferret code: the number of variables that may be defined, the number of datasets open at a time, the length of certain strings, etc. Some of these limits have been relaxed with successive Ferret versions as computing resources have expanded. Here are the limits as of Ferret version 5.41:
|
Parameter Name |
Value |
Description |
|
Parameter Name |
Value |
Description |
|
memsize |
6.4 |
Initial size of memory at startup, in Megawords. You can always change the memory at startup with the -memsize option (p. 6), or during a Ferret session with the SET MEMORY command. SHOW MEMOR gives the current size of the memory cache. |
|
cmnd_buff_len
|
2048 |
Length of the command buffer. You can make long commands more readable using the continuation character backslash \ (p. 14) |
|
Number of arguments to go scripts |
99 |
Maximum number of arguments to a go script. Use the syntax ($nn) or $nn in the script. (p. 25) |
|
Length of arguments to go scripts |
511 |
Maximum length in characters of each argument to a go script. |
|
maxvars |
2000 |
Maximum number of all variables defined by SET DATA (including aliases USE and FILE) |
|
max_uvars |
2000 |
Maximum number of all user-defined variables (LET var = ) |
|
maxezvars |
100 |
Maximum number of variables that can be read from a single delimited ASCII file, using SET DATA/FORMAT=DELIMITED (p. 402) |
|
maxezfreefmtvars |
20 |
Maximum number of variables that can be read in free format from a single ASCII file, e.g. in SET DATA/EZ/VARIABLES="var1,var2" (p.405) |
|
maxdsets |
100 |
Maximum number of data sets simultaneously open (as seen through SHOW DATA) |
|
maxstepfiles |
5000 |
Maximum number of files with time step data. These are read via descriptor files (p. 38). This is a limit on the cumulative sum of all files in all open multi-file data sets. |
|
s_filename |
128 |
Maximum length of the filenames listed within descriptor files (p. 289). |
|
length of variable names |
128 |
Maximum length of all variable names. |
|
length of label text |
2048 |
Maximum length of labels. |
|
max_grids |
500 |
Maximum number of static grids (grids defined by DEFINE GRID). |
|
max_dyn_grids |
1000 |
Total number of grids that can be defined at any time, static and dynamic. Dynamic grids are created by opening data sets and by implicit regridding operations such as strides (e.g. var[i=1:100:10]), regridding operations between grids of different dimensionality (e.g. temp4d[g=sst]), and external functions that create new grids (e.g. EOF_SPACE(A, F)) . |
|
max_lines |
1000 |
Maximum number of static axes. Static axes are axes defined by DEFINE AXIS |
|
max_dyn_lines |
1500 |
Total number of axes, static and dynamic, thatcan be defined. Dynamic axes are defined by opening data sets and by implicit regridding operations such as strides (e.g. var[i=1:100:10]), regridding operations between grids or axes of different dimensionality (e.g. temp4d[gx=sst]), and external functions that create new grids (e.g. SAMPLEXY(sst, xpts, ypts)) |
|
maxlinestore |
250000 |
Maximum number of coordinates in all irregular axes. This is the sum of all the coordinates of irregular axes currently defined via opening files and DEFINE AXIS, and includes storage for the edges of the grid cells defined by these axes. Coordinate storage may be recovered with the CANCEL AXIS command. |
|
abstract_line_dim |
20480 |
Dimension of the default abstract axis for reading ASCII data (p. 47). To read larger amounts of data, explicitly define an axis or grid. |
|
ef_max_args |
9 |
Maximum number of arguments that may be passed to an external function. |
|
ef_max_work_arrays |
9 |
Maximum number of work arrays defined by an external function for use by that function. |
|
spec_size |
250 |
Maximum number of levels in a spectrum, or palette file (.SPK) (p. 203) |
|
pattern_num |
50 |
Maximum number of patterns defined in a pattern file (.PAT) (p. 372) |
|
year |
Years in dates may take values from 0000 to 9999 |
For help with Ferret see our Support Policy