Chapter 3: VARIABLES AND EXPRESSIONS 

Ch3 Sec1.  VARIABLES 

Variables are of 2 kinds: 

Both types may be accessed identically in all commands and expressions. 

Variables, regardless of kind, possess the following associated information:

    1)  grid—the underlying coordinate structure
    2)  units
    3)  title
    4)  title modifier (additional explanation of variable)
    5)  flag value for missing data points 

Use the commands SHOW DATA and SHOW VARIABLES to examine file variables and user-defined variables, respectively. 

The pseudo-variables I, J, K, L, X, Y, Z, T and others may be used to refer to the underlying grid locations and characteristics and to create abstract variables. 

For a description of string variables and arrays, see the chapter on "Handling String Data", p. 229. 

Ch3 Sec1.1.  Variable syntax 

Variables in Ferret are referred to by names with optional qualifying information appended in square brackets. See DEFINE VARIABLE (p. 349) for a discussion of legal variable names. 

The information that may be included in the square brackets includes 

D=data_set_name_or_number       ! indicate the data set
G=grid_or_variable_name         ! request a regridding
X=,Y=,Z=,T=,I=,J=,K=,L=         ! specify region and transformation
                                 e.g. LIST V[x=1:50:5,l=1:30@ave] 

See the chapter "Grids and Regions",  section  "Regions" (p. 162) for more discussion of the syntax of region qualifiers and transformations. 

Some examples of valid variable syntax are 

Myvar                ! data set and region as per current context
myvar[D=2]           ! myvar from data set number 2 (see SHOW DATA)
myvar[D=a_dset]      ! myvar from data set a_dset.cdf or a_dset.des
myvar[D=myfile.txt]  ! myvar from file myfile.txt
myvar[G=gridname]    ! myvar regridded to grid gridname
myvar[G=var2]        ! myvar regridded to the grid of var2
                    ! which is in the same data set as myvar
myvar[G=var2[D=2]]   ! myvar regridded to the grid of var2
                    ! which is in data set number 2
myvar[GX=axisname]   ! myvar regridded to a dynamic grid which
                    ! has X axis axisname
myvar[GX=var2]       ! myvar regridded to a dynamic grid which
                    ! has the X axis of variable var2
myvar[I=1:31:5]      ! myvar subsampled at every 5th point
                    ! (regridded to a subsampled axis)
myvar[X=20E:50E:5]   ! myvar subsampled at every 5 degrees
                    ! (regridded to a 5-deg axis by linear
                    !  interpolation) 

Ch3 Sec1.2.  File variables 

File variables are stored in disk files. Input data files can be ASCII, binary, netCDF, or TMAP-formatted (see the chapter "Data Set Basics", p. 33). File variables are made available with the SET DATA (alias USE) command. 

In some netCDF files the variable names are not consistent with Ferret's rules for variable naming. They may be case-sensitive (for example, variables "v" and "V" defined in the same file), may  be restricted names such as the Ferret pseudo-variable names I, J, K, L, X, Y, Z, T, XBOX, YBOX, ZBOX, or TBOX, or they may include "illegal" characters such as "+", "-", "%", blanks, etc. To access such variable names in Ferret, simply enclose the name in single quotes. For example, 

yes? PLOT 'x' 

yes? CONTOUR 'SST from MP/RF measurements' 

By the same token when using Ferret to output into netCDF files that Ferret did not itself create, the results may not be entirely as expected. Case-sensitivity of names is one aspect of this. Since Ferret is (by default) case insensitive and netCDF files are case-sensitive writing into a "foreign" file may result in duplicated entities in the file which differ only in case. Starting with Ferret v6.0 you may specify 

yes? CANCEL MODE upcase_output 

and output to netCDF files will retain the original case of the variables and attribute names. Also see the documentation on SET ATTRIBUTE/OUTPUT for more about controlling which attributes are written to netCDF files. 

Ch3 Sec1.3.  Pseudo-variables 

Pseudo-variables are variables whose values are coordinates or coordinate information from a grid. Valid pseudo-variables are 

A grid box is a concept needed for some transformations along an axis; it is the length along an axis that belongs to a single grid point and functions as a weighting factor during integrations and averaging transformations. 

The pseudo-variables I, J, K, and L are subscripts; that is, they are a coordinate system for referring to grid locations in which the points along an axis are regarded as integers from 1 to the number of points on the axis. This is clear if you look at one of the sample data sets: 

yes? USE levitus_climatology
yes? SHOW DATA
   1> /home/e1/tmap/fer_dsets/descr/levitus_climatology.des  (default)
      Levitus annual climatology (1x1 degree)
            diagnostic variables: NOT available
     name   title                       I         J         K       L
     TEMP   TEMPERATURE               1:360     1:180     1:20     ...
       ... on grid GLEVITR1   X=20E:20E(380) Y=90S:90N  Z=0m:5000m
     SALT   SALINITY                  1:360     1:180     1:20     ...
       ... on grid GLEVITR1   X=20E:20E(380) Y=90S:90N  Z=0m:5000m
 

We see that there are 20 points along the z-axis (1:20 under K), for example, and that the z-axis coordinate values range from 0 meters to 5000 meters. Pseudo-variables depend only on the underlying grid, and not on the variables (in this case, temperature and salt). 

Examples:  Pseudo-variables 

    1) yes? LIST/I=1:10 I
    2) yes? LET xflux = u * xbox[G=u] 

Ch3 Sec1.3.1.  Grids and axes of pseudo-variables 

The name of a pseudo-variable (p. 61), alone, ("I", "T", "ZBOX", etc.) is not sufficient to determine the underlying axis of the pseudo-variable. The underlying axis may be specified explicitly, may be inherited from other variables used in the same expression, may be generated dynamically, or may be inherited from the current default grid. The following examples illustrate the possibilities: 

TEMP + Y        ! pseudo-variable Y inherits the y axis of variable TEMP 

Y[G=TEMP]       ! explicit: Y refers to the y axis of variable TEMP 

Y[GY=axis_name] ! explicit: Y refers to axis axis_name 

Y[Y=0:90:2]     ! y axis is dynamically generated (See "dynamic axes">,
               ! p. 148) 

In the expression 

LET A = X + Y 

in which the definition provides no explicit coaching, nor are there other variables from which Y can inherit an axis, the axis of Y will be inherited from the current default grid. The current default grid is specified by the SET GRID command and may be queried at any time with the SHOW GRID command. SHOW GRID will respond with "Default grid for DEFINE VARIABLE is grid". 

Note that when pseudo-variables are buried within a user variable definition they do not inherit from variables used in conjunction with the user variable. For example, contrast these expressions involving pseudo-variable Y 

USE coads_climatology ! has variable SST 

LET A = Y             ! Y buried inside variable A (axis indeterminate) 

LIST SST + A          ! y axis inherited from current default grid 

LIST SST + Y          ! y axis inherited from grid of SST 

LIST SST + A[G=SST]   ! y axis inherited from grid of SST 

Ch3 Sec1.4.  User-defined variables 

New variables can be defined from existing variables and from abstract mathematical quantities (such as COS(latitude)) with command DEFINE VARIABLE (alias LET). The section later in this chapter, Defining New Variable (p. 143) expands on this capability. 

See command DEFINE VARIABLE (p. 349) and command LET (p. 363) in the Commands Reference. Example 3 shows the use of masking, a useful concept in constructing variables. 

Examples: User-defined variables 

1) yes? LET/TITLE="Surface Relief x1000 (meters)" r1000=rose/1000
2) yes? LET/TITLE="Temperature Deviation" tdev=temp - temp[Z=@ave] 

3) yes? LET a = IF (sst GT 20. AND sst LT 30.) THEN sst ELSE 20. 

Ch3 Sec1.5.  Abstract variables 

Ferret can be used to manipulate abstract mathematical quantities such as SIN(x) or EXP(k*t)—quantities that are independent of file variable values. Such quantities are referred to as abstract expressions. 

Example: Abstract variables 

Contour the function 

COS(a*Y)/EXP(b*T)  where a=0.25 and b=-0.02 

over the range 

Y=0:45 (degrees) and  T=1:100 (hours) 

with a resolution of 

    0.5 degree on the Y axis and 2 hours on the T axis. 

Quick and dirty solution: 

yes? CONTOUR COS(0.25*Y[Y=0:45:0.5])/EXP(-0.2*T[T=1:100:2]) 

Nicer (Figure 3_1); plot is documented with correct units and titles): 

yes? DEFINE AXIS/Y=0:45:0.5/UNIT=DEGREES yax
yes? DEFINE AXIS/T=1:100:2/UNIT=HOURS tax
yes? DEFINE GRID/T=tax/Y=yax my_grid
yes? SET GRID my_grid
yes? LET a=0.25
yes? LET b=-0.02
yes? CONTOUR COS(a*Y)/EXP(b*T) 

See the chapter "Grids and Regions", section "Grids" (p. 145), for more information on grids. 

Ch3 Sec1.6.  Missing value flags 

Data values that are absent or undefined for mathematical reasons (e.g., 1/0) will be represented in Ferret with a missing value flag. In SHADE outputs a missing value flag embedded at some point in a variable will result in a transparent rectangular hole equal to the size of the grid cell of the missing value. In a CONTOUR or FILL plot it will result in a larger hole—extending past the grid box edge to the coordinate location of the next adjacent non-missing point—since contour lines cannot be interpolated between a missing value and its neighboring points. In the output of the LIST command for cases where the /FORMAT qualifier is not used the missing value will be represented by 4 dots ("...."). For cases where LIST/FORMAT=FORTRAN-format is used the numerical value of the missing value flag will be printed using the format provided. 

Ch3 Sec1.6.1.  Missing values in input files 

Ferret does not impose a standard for missing value flags in input data sets; each variable in each data set may have its own distinct missing value flag(s). The flag(s) actually in use by a data set may be viewed with the SHOW DATA/VARIABLES command. If no missing value flag is specified for a data set Ferret will assume a default value of –1.E+34. 

For EZ input data sets, either binary or ASCII, the missing data flag may be specified with the SET VARIABLE/BAD= command. A different value may be specified for each variable in the data set. 

For netCDF input data sets the missing value flag(s) is indicated by the values of the attributes "missing_value" and "_FillValue." If both attributes are defined to have different values both will be recognized and used by Ferret as missing value indicators, however the occurrences of _FillValue will be replaced with the value of missing_value as the data are read into Ferret's memory cache so that only a single missing value flag is apparent inside of Ferret. The command SET VARIABLE/BAD= can also be applied to netCDF variables, thereby temporarily setting a user-imposed value for _FillValue. If there are values of NaN in the file, then NaN must be listed in either the as either the "missing_value" OR  "_FillValue" attribute and then NaN is the missing value. Or, the user may specify SET VARIABLE/BAD=NAN (case insensitive) to  designate the Fortran value NaN (not a number) as the bad value flag for a given variable in a netCDF dataset. 

Ch3 Sec1.6.2.  Missing values in user-defined variables 

User-defined variables may in general be defined as expressions involving multiple variables. The component variables need not in general agree in their choice of missing value flags. The result variable will inherit the bad value flag of the first variable in the expression. If the first component in the expression is a constant or a pseudo-variable, then Ferret imposes its default missing value flag of –1.E+34. 

The function MISSING(variable,replacement) provides a limited control over the choice of missing values in user-defined variables. Note, however, that while the MISSING function will replace the missing values with other values it will not change the missing value flag. In other words, the replacement values will no longer be regarded as missing. 

Ch3 Sec1.6.3.  Missingvalues in output NetCDF files 

Values flagged as missing inside Ferret will be faithfully transferred to output files—no substitution will occur as the data are written. In the case of netCDF output files both of the attributes missing_value, and _FillValue will be set equal to the missing value flag. 

Under some circumstances it is desirable to save a user-defined variable in a netCDF file and then to redefine that variable and to append further output. (An example of this is the process of consolidating several files of input, say, moored measurements, into a gridded output.) The process of appending will not change any of the netCDF attributes—neither long_name (title), units, nor missing_value or _FillValue. If the subsequent variable definitions do not agree in their choice of missing value flags the resulting output may contain multiple missing value flags that will not be properly documented. 

An easy "trick" that avoids this situation is to begin all of the variable definitions with an addition of zero, "LET var = 0 + ...." The addition of zero will not affect the value of the output but it will guarantee that a missing value flag of –1.E+34 will be consistently used. Of course, you will want to use the SET VARIABLE/TITLE= command in conjunction with this approach. 

Ch3 Sec1.6.4.  Displaying the missing value flag 

If the LIST command is used, missing values are, by default, displayed as "...."  To examine the flag as a numerical value, use LIST/FORMAT=(E) (or some other suitable format). 

Ch3 Sec1.7.  Returning properties of variables 

The keyword RETURN= can reveal the size and shape, title, bad flag,  units, and other properties of a variable or expression. See p. 138for a description of this useful construct. 

Ch3 Sec1.8.  Variable and dataset attributes 

Beginning with Ferret V6.0, Ferret has access to attributes of all variables, including netCDF variables, netCDF coordinate variables, user-defined variables and variables from non-netCDF datasets.. When a netCDF dataset is opened, its variables and attributes are stored. For other file variables and user variables, a basic set of attributes is created, including at least the missing_value flag, _FillValue, and a long-name attribute with the variable's definition. If a variable is defined with /UNITS or /TITLE then those are also included among the attributes. 

The power of this feature is in its access to attribute information as strings or numeric data. The text or values can be examined, edited, and used in computations. The attributes of a variable can be changed, removed, and new attributes defined, and when writing netCDF files, we can control which attributes that are written to the file. 

The general syntax for this access is 

varname.attname 

For example, 

yes? USE coads_climatology 

yes? LIST sst.units 

             VARIABLE : SST.UNITS 

             FILENAME : coads_climatology.cdf 

             FILEPATH : /home/porter/tmap/ferret/linux/fer_dsets/data/ 

        "Deg C" 

To refer to a coordinate variable, put the name of the coordinate variable in parentheses. The RETURN= keyword is helpful for getting the names of coordinate axes. 

yes? LIST (coadsx).point_spacing 

             VARIABLE : (COADSX).POINT_SPACING 

             FILENAME : coads_climatology.cdf 

             FILEPATH : /home/porter/tmap/ferret/linux/fer_dsets/data/ 

        "even" 

yes? LIST (`sst,RETURN=taxis`).time_origin 

 !-> list (TIME).time_origin 

             VARIABLE : (TIME).TIME_ORIGIN 

             FILENAME : coads_climatology.cdf 

             FILEPATH : /home/porter/tmap/ferret/linux/fer_dsets/data/ 

        "1-JAN-0000 00:00:00" 

The dataset itself can have attributes; the global attributes. To refer to the dataset, use a dot. The history attribute of the current dataset can be referred to by 

..history 

For example, 

yes? LET a = 1; save/clobber/file=new.nc a 

 LISTing to file new.nc 

yes? USE new.nc 

yes? LIST ..history 

             VARIABLE : ..HISTORY 

             FILENAME : new.nc 

        "FERRET V6    5-Jul-06" 

When there is more than one dataset open, the dataset specifier [d=1] or [d=datasetname] comes at the end of the varname.attname string: 

yes? use dataset_1 

yes? use dataset_2 

yes? list var.units[d=1] 

Ch3 Sec1.8.1.  SHOW ATTRIBUTE commands 

Example: 

yes? USE etopo60 

yes? SHOW DATA/ATT 

     currently SET data sets: 

    1> /home/porter/tmap/ferret/linux/fer_dsets/data/etopo60.cdf  (default) 

 VARIABLE               ATTRIBUTE NAME   TYPE   SIZE OUTFLAG VALUE 

------------------------------------------------------------------------ 

 .                      history          CHAR    28   T      FERRET V4.45 (GUI) 22-May-97 

(ETOPO60X)              units            CHAR    12   T      degrees_east 

                        modulo           CHAR    1    T 

                        point_spacing    CHAR    4    T      even 

(ETOPO60Y)              units            CHAR    13   T      degrees_north 

                        point_spacing    CHAR    4    T      even 

 ROSE                   missing_value    FLOAT   1    T      -1.000000E+34 

                        _FillValue       FLOAT   1    T      -1.000000E+34 

                        long_name        CHAR    34   T      RELIEF OF THE SURFACE OF THE EARTH 

                        history          CHAR    12   T      From etopo60 

                        units            CHAR    6    T      METERS 

yes? SHOW ATT rose.missing_value 

     attributes for dataset: /home/porter/tmap/ferret/linux/fer_dsets/data/etopo60.cdf 

 ROSE.missing_value = -1.000000E+34 

yes? SHOW ATT/ALL rose 

     attributes for dataset: /home/porter/tmap/ferret/linux/fer_dsets/data/etopo60.cdf 

 ROSE.missing_value = -1.000000E+34 

 ROSE._FillValue = -1.000000E+34 

 ROSE.long_name = RELIEF OF THE SURFACE OF THE EARTH 

 ROSE.history = From etopo60 

 ROSE.units = METERS 

yes? SAY `rose.missing_value,RETURN=SIZE` 

 !-> MESSAGE/CONTINUE 1 

1 

yes? say `rose.long_name,RETURN=size` 

 !-> MESSAGE/CONTINUE 34 

34 

Ch3 Sec1.8.2.  Attribute keywords 

In addition to the attributes of  variables we can also learn about the number and names of variables in a dataset, the dimensions and number of attributes for each variable,  and the same information about the dimensions (coordinate variables). New keywords are introduced: 

Examples: 

yes? use mydata.nc 

yes? let names = ..varnames 

yes? list names 

yes? let anames = `names[i=2]`.attname 

yes? list ..ndims 

yes? list myvar.dimhnames 

Ch3 Sec1.8.3.  Programmatic access to attributes 

To form a variable from attribute information simply use a LET command:  

yes? let a = varname.missing_value 

yes? let b = (axisname).units 

yes? let h = ..history 

Or, use attribute keywords 

   ! The number of variables (not including coordinate variables) 

yes? let nv = ..nvars 

   ! The number of coordinate variables 

yes? let nc = ..ndims 

   ! A variable with a list of all coordinate variables 

   ! from data set 1 

yes? let lnames = ..dimnames[d=1] 

   ! If two datasets have a variable TEMP, list the units 

   ! of the variable in each dataset 

yes? let uu = temp.units 

yes? list/d=1 uu 

yes? list/d=2 uu 

Ch3 Sec1.8.4.  Editing attributes 

We can change an existing attribute for a variable, add new attributes, and control which are written to output netCDF files. We can define attributes to be of type STRING or FLOAT. (on ouput we will be able to request type conversions to other numeric types). Numeric attributes may be lists of values. If the type is note specified by the  /TYPE qualifier, the type is inferred from the value of the attribute. The outupt flag is set with the /OUTPUT qualifier: SET ATTRIBUTE/OUTPUT varname.attname marks the attribute for output to netCDF files, and CANCEL ATTRIBUTE/OUTPUT causes that attribute to be hidden on output. 

Example: add new attributes with DEFINE ATTRIBUTE 

yes? USE etopo60 

yes? DEFINE ATT/TYPE=float rose.floatval = 22 

yes? DEFINE ATT rose.pp = {1.5, 1.9} 

yes? DEFINE ATT/TYPE=string rose.strval = 2 

yes? DEFINE ATT rose.some_text = "some text about ROSE " 

yes? SHOW ATT/ALL rose 

     attributes for dataset: /home/porter/tmap/ferret/linux/fer_dsets/data/etopo60.cdf 

 ROSE.missing_value = -1.000000E+34 

 ROSE._FillValue = -1.000000E+34 

 ROSE.long_name = RELIEF OF THE SURFACE OF THE EARTH 

 ROSE.history = From etopo60 

 ROSE.units = METERS 

 ROSE.floatval = 22 

 ROSE.pp = 1.5, 1.9 

 ROSE.strval = 2 

 ROSE.some_text = some text about the ROSE variable 

   ! /D= specifies the dataset if needed 

yes? DEFINE ATT/D=1 rose.another_attribute = 6 

   ! /OUTPUT sets the output flag: this attribute will be 

   ! written to output files with the variable. 

yes? DEFINE ATT/OUTPUT rose.more_text = "Another notation" 

We can also change an existing attribute. As in the commands DEFINE VARIABLE; SET VARIABLE; CANCEL VARIABLE we can use DEFINE ATTRIBUTE to redefine an existing attribute or SET ATTRIBUTE and CANCEL ATTRIBUTE to change settings or values of an attribute. 

Examples: continuing the above example 

   ! Change the text in long_name 

yes? DEFINE ATT rose.long_name = "Relief of the Surface of the Earth" 

   ! The rose.history attribute will NOT be written to output files 

yes? CANCEL ATT/OUTPUT rose.history 

   ! The new attribute rose.pp we defined above WILL be written 

yes? SET ATT/OUTPUT rose.pp 

   For a coordinate variable, SET AXIS and SET ATTRIBUTE commands do the same thing: 

yes? USE levitus_climatology 

   ! These commands are equivalent 

yes? SET AXIS/POSITIVE="up" `temp,RETURN=zaxis` 

yes? SET ATT (`temp,RETURN=zaxis`).positive = "up" 

Another way to edit attributes is to inherit them from another variable. Use the qualifer SET ATTRIBUTE/LIKE= 

Example (note the long_name and units) 

yes? use levitus_climatology 

yes? SET ATT/LIKE=salt temp 

yes? SHOW ATTRIBUTE/ALL temp 

     attributes for dataset: /home/porter/tmap/ferret/linux/fer_dsets/data/levitus_climatology.cdf 

 TEMP.missing_value = -1.E+10 

 TEMP._FillValue = -1.E+10 

 TEMP.long_name = SALINITY 

 TEMP.history = From levitus_climatology 

 TEMP.units = PPT 

As noted at the start of this section, the attribute  commands can be applied to any variable, not just those in netCDF datsets. A user variable has a few default attributes when it is defined; a missing_value, _FillValue, and a long_name, and units if they are included in the definition. Here a user variable inherits all of the attributes from a file variable. Here salt2 initially has only a long-name, missing_value, and _FillValue. We can inherit the units and a more descriptive long_name from the dataset variable. 

yes? USE levitus_climatology 

yes? LET salt2 = 2* salt 

yes? SHOW ATT/ALL salt2 

     attributes for user-defined variables 

 SALT2.long_name = 2* SALT 

 SALT2.missing_value = -1.000000E+34 

yes? SET ATT/LIKE=salt salt2 

yes? SHOW ATT/ALL salt2 

     attributes for user-defined variables 

 SALT2.missing_value = -1.E+10 

 SALT2._FillValue = -1.E+10 

 SALT2.long_name = SALINITY 

 SALT2.history = From levitus_climatology 

 SALT2.units = PPT 

Finally, fix the long_name of our new variable 

yes? set att salt2.long_name = "2 * `salt.long_name`" 

 !-> set att salt2.long_name = "2 * SALINITY" 

 *** NOTE: Changing the value of attribute 

Ch3 Sec1.8.5.  Output attributes to NetCDF fies 

As noted in the previous section, the value of the attribute output flag, set by the commands  DEFINE ATTRIBUTE/OUTPUT, SET ATTRIBUTE/OUTPUT and CANCEL ATTRIBUTE/OUTPUT controls whether an attribute is written to a netCDF file when the variable is written. The SET ATTRIBUTE/OUTPUT= allows more precise control over the writing of attributes. 

Example: 

yes? LET aa = 12. 

yes? LET bb = {3,4.5,6,7,4} 

yes? DEFINE ATT/OUTPUT att bb.my_title = "This is my new variable bb" 

yes? DEFINE ATT bb.another_attr = 6 

! Output just bb.mytitle, along with the default 

! ones, missing_value, _FillValue, and long_name. 

yes? SAVE/CLOBBER/FILE=ab.nc aa,bb 

! Output all attributes 

yes? SET ATT/OUTPUT=all bb 

yes? SAVE/CLOBBER/FILE=ab.nc aa,bb 

! Output default attributes 

yes? SET ATT/OUTPUT=default bb 

yes? SAVE/CLOBBER/FILE=ab.nc aa,bb 

We can suppress output of an attribute that Ferret would otherwise add, the "axis" attribute for coordinate axes. 

yes? USE levitus_climatology 

yes? CANCEL ATT/OUT (`temp,RETURN=xaxis`).axis 

Ch3 Sec1.8.6.  Output Variables to NetCDF files 

The attribute-handling structure gives us flexibility in writing variables to netCDF files. We can specify that the  upper- or lower-case spelling of variables and attributes from an input netCDF file be preserved on output, or that these should be upper-cased as has been done previously in Ferret. By default, Ferret still upcases variable names when it writes variables to netCDF files. If we want to keep the case of the names that they had on input, use 

CANCEL MODE upcase_output 

We can also control the data type of variables written to output netCDF files, with SET VAR/OUTTYPE=]. The netCDF library is used to convert the data type of a Ferret FLOAT value to the requested output type. The types allowed are FLOAT, INT, SHORT, and BYTE, or INPUT to preserve the type the data had on input. To write integers, for instance, 

yes? SET DATA etopo60 

yes? SET VAR/OUTTYPE=int4 rose 

yes? SAVE/X=180/FILE=r_int.nc rose 

 LISTing to file a.nc 

 *** NOTE: Converting data type of missing_value NC_FLOAT to match output type of variable NC_INT 

 ** netCDF error: 

 data in attribute missing_value not representable in output type NC_INT 

Here, the missing_value of the variable cannot be represented as an integer, and the file was not written. Correct for this by assigning a new missing_value to the variable before writing: 

yes? SET DATA etopo60 

yes? SET VAR/OUTTYPE=int4/BAD=200000 rose 

yes? SAVE/X=180/FILE=r_int.nc rose 

 LISTing to file a.nc 

 *** NOTE: Converting data type of missing_value NC_FLOAT to match output type of variable NC_INT 

Note that not all data can be represented in all types. When data is written to DOUBLE, it is always converted from the Ferret internal representation of single precision FLOAT data, even if the original data was double precision. 

This mechanism lets us pack data when writing it to a netCDF file by using the add_offset and scale_factor attributes. Say a dataset has a variable called elev which is packed using these attributes. By default Ferret scales this data when it is read, and writes it in scaled (unpacked) form. To pack it, we turn on output of the scale_factor and add_offset attributes. Then Ferret will apply this scaling to data and its missing_value and _FillValue are rescaled on output. 

yes? USE my_scaled_dset.nc 

yes? SET ATT/OUTPUT elev.scale_factor 

yes? SET ATT/OUTPUT elev.add_offset 

yes? SET ATT/OUTTYPE=input elev 

yes? SAVE/CLOBBER/FILE=scaled.nc/J=1 elev 

We can use this technique to apply new scale factors to a variable or  define scale and offset attributes when writing any variable. 

Ch3 Sec2.  EXPRESSIONS 

Throughout this manual, Ferret commands that require and manipulate data are informally called "action" commands. These commands are: 

    PLOT
    CONTOUR
    FILL  (alias for CONTOUR/FILL)
    SHADE
    VECTOR
    POLYGON
    WIRE
    LIST
    STAT
    LOAD 

Action commands may use any valid algebraic expression involving constants, operators (+,–,*,...), functions (SIN, MIN, INT,...), pseudo-variables (X, TBOX, ...) and other variables. 

A variable name may optionally be followed by square brackets containing region, transformation, data set, and regridding qualifiers. For example, "temp", "salt[D=2]", "u[G=temp"], "u[Z=0:200@AVE]", "v[k=1:50:5] 

The expressions may also contain a syntax of: 

    IF  condition  THEN  expression_1  ELSE  expression_2 

Examples:  Expressions 

i) temp ^ 2
    temperature squared 

ii) temp - temp[Z=@AVE]
for the range of Z in the current context, the temperature deviations from the vertical average 

iii) COS(Y)
the cosine of the Y coordinate of the underlying grid (by default, the y-axis is implied by the other variables in the expression) 

iv) IF (vwnd GT vwnd[D=monthly_navy_winds]) THEN vwnd ELSE 0
use the meridional velocity from the current data set wherever it exceeds the value in data set monthly_navy_winds, zero elsewhere. 

Ch3 Sec2.1.  Operators 

Valid operators are 

    +
    –
    *
    /
    ^    (exponentiate)
    AND
    OR
    GT
    GE
    LT
    LE
    EQ
    NE 

For instance the exponentiate operator can compute the square root of a variable as var^0.5
 

Ch3 Sec2.2.  Multi-dimensional expressions 

Operators and functions (discussed in the next section, Functions)  may combine variables of like dimensions or differing dimensions. 

If the variables are of like dimension then the result of the combination is of the same dimensionality as inputs. For example, suppose there are two time series that have data on the same time axis; the result of a combination will be a time series on the same time axis. 

If the variables are of unlike dimensionality, then the following rules apply: 

1)    To combine variables together in an expression they must be "conformable" along each axis. 

2)    Two variables are conformable along an axis if the number of points along the axis is the same, or if one of the variables has only a single point along the axis (or, equivalently, is normal to the axis). 

3)    When a variable of size 1 (a single point) is combined with a variable of larger size, the variable of size 1 is "promoted" by replicating its value to the size of the other variable. 

4)    If variables are the same size but have different coordinates, they are conformable, but Ferret will issue a message that the coordinates on the axis are ambiguous. The result of the combination inherits the coordinates of the FIRST variable encountered that has more than a single point on the axis. 

Examples: 

Assume a region J=50/K=1/L=1 for examples 1 and 2. Further assume that variables v1 and v2 share the same x-axis. 

1)    yes? LET newv = v1[I=1:10] + v2[I=1:10]      !same dimension (10) 

2)    yes? LET newv = v1[I=1:10] + v2[I=5]          !newv has length of v1 (10) 

3)    We want to compare the salt values during the first half of the year with the values for the second half. Salt_diff will be placed on the time coordinates of the first variable—L=1:6. Ferret will issue a warning about ambiguous coordinates.

yes? LET salt_diff = salt[L=1:6] - salt[L=7:12] 

4)    In this example the variable zero will be promoted along each axis. 

yes? LET zero = 0 * (i+j)
yes? LIST/I=1:5/J=1:5  zero         !5X5 matrix of 0's 

5)    Here we calculate density; salt and temp are on the same grid. This expression is an XYZ volume of points (100×100×10) of density at 10 depths based on temperature and salinity values at the top layer (K=1). 

yes? SET REGION/I=1:100/J=1:100
yes? LET dens = rho_un (salt[K=1], temp[K=1], Z[G=temp,K=1:10] 

Ch3 Sec2.3.  Functions 

Functions are utilized with standard mathematical notation in Ferret. The arguments to functions are constants, constant arrays, pseudo-variables, and variables, possibly with associated qualifiers in square brackets, and expressions. Thus, all of these are valid function references: 

• EXP(-1)  
• MAX(a,b) 
• TAN(a/b) 
• SIN(Y[g=my_sst]) 
• DAYS1900(1989,{3,6,9},1) 

A few functions also take strings as arguments. String arguments must be enclosed in double quotes. For example, a function to write variable "u" into a file named "my_output.v5d", formatted for the Vis5D program might be implemented as 

• LOAD WRITE_VIS5D("my_output.v5d", a) 

You can list function names and argument lists with: 

yes? SHOW FUNCTIONS        ! List all functions 

Yes? SHOW FUNCTIONS *TAN    ! List all functions containing string 

Valid functions are described in the sections below. They are: 

See also the section on string functions (p. 230). 

Grid-changing functions 

It is generally advisable to include explicit limits when working with functions that replace axes. For example, consider the function SORTL(v). The expression 

LIST/L=6:10 SORTL(v) 

is not equivalent to 

LIST SORTL(v[L=6:10]) 

The former will list the 6th through 10th sorted indices from the entire l range of variable v. The latter will list all of the indices that result from sorting v[l=6:10]. 

These functions in Ferret, including XSEQUENCE, SAMPLXY, and so on, are "grid-changing" functions. This means that the axes of the result may differ from the axes of the arguments. In the case of XSEQUENCE(sst), for example, the input grid for SST is 

    lon
    lat
    normal
    time 

whereas the output grid is 

    abstract
    normal
    normal
    normal 

so all axes of the input are replaced. 

Grid-changing functions create a potential ambiguity about region specifications. Suppose that the result of XSEQUENCE(sst[L=1]) is a list of 50 points along the ABSTRACT X axis. Then it is natural that 

LIST/I=10:20 XSEQUENCE(sst[L=1]) 

should give elements 10 through 20 taken from that list of 50 points (and it does.) However, one might think that "I=10:20"  referred to a subset of the longitude axis of SST. Therein lies the ambiguity: one region was specified, but there are 2 axes to which the region might apply. 

It gets a degree more complicated if the grid-changing function takes more than one argument. Since the input arguments need not be on identical grids, a result axis (X,Y,Z, or T) may be replaced with respect to one argument, but actually taken from another (consider ZAXREPLACE, for example.)  Ferret resolves the ambiguities thusly: 

If in the result of a grid-changing function, an axis (X, Y, Z, or T) has been replaced relative to some argument, then region information which applies to the result of the function on that axis will NOT be passed to that argument. 

So, when you issue commands like 

SET REGION/X=20E:30E/Y=0N:20N/L=1
LIST XSEQUENCE(sst) 

the X axis region ("20E:30E") applies to the result ABSTRACT axis -- it is not passed along to the argument, SST. The Y axis region is, in fact, ignored altogether, since it is not relevant to the result of XSEQUENCE, and is not passed along to the argument. 

Ch3 Sec2.3.1.  MAX 

MAX(A, B) Compares two fields and selects the point by point maximum.
MAX( temp[K=1], temp[K=2] )  returns the maximum  temperature comparing the first 2 z-axis levels. 

Ch3 Sec2.3.2.  MIN 

MIN(A, B) Compares two fields and selects the point by point minimum.
MIN( airt[L=10], airt[L=9] )  gives the minimum air temperature comparing two timesteps. 

Ch3 Sec2.3.3.  INT 

INT (X) Truncates values to integers.
INT( salt ) returns the integer portion of variable "salt" for all values in the current region. 

Ch3 Sec2.3.4.  ABS 

ABS(X) absolute value.
ABS( U ) takes the absolute value of U for all points within the current region 

Ch3 Sec2.3.5.  EXP 

EXP(X) exponential  e^x; argument is real.
EXP( X ) raises e to the power X for all points within the current region 

Ch3 Sec2.3.6.  LN 

LN(X) Natural logarithm  log_eX; argument is real.
LN( X ) takes the natural logarithm of X for all points within the current region 

Ch3 Sec2.3.7.  LOG 

LOG(X)     Common logarithm  log₁₀X;  argument is real.
LOG( X ) takes the common logarithm of X for all points within the current region 

Ch3 Sec2.3.8.  SIN 

SIN(THETA) Trigonometric sine; argument is in radians and is treated modulo 2*pi.
SIN( X ) computes the sine of X for all points within the current region. 

Ch3 Sec2.3.9.  COS 

COS(THETA ) Trigonometric cosine; argument is in radians and is treated modulo 2*pi.
COS( Y ) computes the cosine of Y for all points within the current region 

Ch3 Sec2.3.10.  TAN 

TAN(THETA)   Trigonometric tangent; argument is in radians and is treated modulo 2*pi.
TAN( theta ) computes the tangent of theta for all points within the current region 

Ch3 Sec2.3.11.  ASIN 

ASIN(X)  Trigonometric arcsine (-pi/2,pi/2) of X in radians.The result will be flagged as missing if the absolute value of the argument is greater than 1; result is in radians.
ASIN( value ) computes the arcsine of "value" for all points within the current region 

Ch3 Sec2.3.12.  ACOS 

COS(X)  Trigonometric arccosine (0,pi), in radians. The result will be flagged as missing of the absolute value of the argument greater than 1; result is in radians.
ACOS ( value ) computes the arccosine of "value" for all points within the current region 

Ch3 Sec2.3.13.  ATAN 

ATAN(X) Trigonometric arctangent (-pi/2,pi/2); result is in radians.
ATAN( value ) computes the arctangent of "value" for all points within the current region 

Ch3 Sec2.3.14.  ATAN2 

ATAN2(X,Y)  2-argument trigonometric arctangent of Y/X (-pi,pi); discontinuous at Y=0.
ATAN2( X,Y ) computes the 2-argument arctangent of Y/X for all points within the current region 

Ch3 Sec2.3.15.  MOD 

MOD(A,B) Modulo operation ( arg1 – arg2*[arg1/arg2] ). Returns the remainder when the first argument is divided by the second.
MOD( X,2 ) computes the remainder of X/2 for all points within the current region 

Ch3 Sec2.3.16.  DAYS1900 

DAYS1900(year,month,day) computes the number of days since 1 Jan 1900. This function is useful in converting dates to Julian days on the standard Gregorian calendar. If the year is prior to 1900 a negative number is returned. This means that it is possible to compute Julian days relative to, say, 1800 with the expression
LET jday1800 = DAYS1900 ( year, month, day) - DAYS1900( 1800,1,1) 

Ch3 Sec2.3.17.  MISSING 

MISSING(A,B) Replaces missing values in the first argument (multi-dimensional variable) with the second argument; the second argument may be any conformable variable.
MISSING( temp, -999 )  replaces missing values in temp with   –999
MISSING( sst, temp[D=coads_climatology] )  replaces missing sst values with temperature from the COADS climatology 

Ch3 Sec2.3.18.  IGNORE0 

IGNORE0(VAR) Replaces zeros in a variable with the missing value flag for that variable.
IGNORE0( salt ) replaces zeros in salt with the missing value flag 

Ch3 Sec2.3.19.  RANDU 

RANDU(A)  Generates a grid of uniformly distributed [0,1] pseudo-random values. The first valid value in the field is used as the random number seed. Values that are flagged as bad remain flagged as bad in the random number field.
RANDU( temp[I=105:135,K=1:5] )  generates a field of uniformly distributed random values of the same size and shape as the field "temp[I=105:135,K=1:5]" using temp[I=105,k=1] as the pseudo-random number seed. 

Ch3 Sec2.3.20.  RANDN 

RANDN(A) Generates a grid of normally distributed pseudo-random values. As above, but normally distributed  rather than uniformly distributed. 

Ch3 Sec2.3.21.  RHO_UN 

RHO_UN(SALT, TEMP, P)    Calculates the mass density rho (kg/m^3) of seawater from salinity SALT(salt, psu), temperature TEMP(deg C) and pressure P(dbar) using the 1980 UNESCO International Equation of State (IES80). Either in-situ or potential density may be computed depending upon whether the user supplies in-situ or potential temperature. 

Note that to maintain accuracy, temperature must be converted to the IPTS-68 standard before applying these algorithms. For typical seawater values, the IPTS-68 and ITS-90 temperature scales are related by T_68 = 1.00024 T_90 (P. M. Saunders, 1990, WOCE Newsletter 10). The routine uses the high pressure equation of state from Millero et al. (1980) and the one-atmosphere equation of state from Millero and Poisson (1981) as reported in Gill (1982). The notation follows Millero et al. (1980) and Millero and Poisson (1981).  

RHO_UN( salt, temp, P ) 

Ch3 Sec2.3.22.  THETA_FO 

THETA_FO(SALT, TEMP, P, REF)  Calculates the potential temperature of a seawater parcel at a given salinity SALT( psu), temperature TEMP(deg. C) and pressure P(dbar), moved adiabatically to a reference pressure REF(dbar). 

This calculation uses Bryden (1973) polynomial for adiabatic lapse rate and Runge-Kutta 4th order integration algorithm. References: Bryden, H., 1973, Deep-Sea Res., 20, 401–408; Fofonoff, N.M, 1977, Deep-Sea Res., 24, 489–491. 

THETA_FO( salt, temp, P, P_reference ) 

Ch3 Sec2.3.23.  RESHAPE 

RESHAPE(A, B)  The result of the RESHAPE function will be argument A "wrapped" on the grid of argument B. The limits given on argument 2 are used to specify subregions within the grid into which values should be reshaped.
RESHAPE(Tseries,MonthYear) 

 Two common uses of this function are to view multi-year time series data as a 2-dimensional field of 12-months vs. year and to map ABSTRACT axes onto real world coordinates. An example of the former is 

DEFINE AXIS/t=15-JAN-1982:15-DEC-1985/NPOINTS=48/UNITS=DAYS tcal
LET my_time_series = SIN(T[gt=tcal]/100) 

! reshape 48 months into a 12 months by 4 year matrix 

DEFINE AXIS/t=1982:1986:1 tyear
DEFINE AXIS/Z=1:12:1 zmonth
LET out_grid = Z[GZ=zmonth]+T[GT=tyear]
LET my_reshaped = RESHAPE(my_time_series, out_grid)
SHOW GRID my_reshaped
  GRID (G001)
name       axis              # pts   start                end
normal    X
normal    Y
ZMONTH    Z                   12 r   1                    12
TYEAR     T                    5 r   1982                 1986 

For any axis X,Y,Z, or T if the axis differs between the input output grids, then limits placed upon the region of the axis in argument two (the output grid) can be used to restrict the geometry into which the RESHAPE is performed. Continuing with the preceding example: 

!  Now restrict the output region to obtain a 6 month by 8 year matrix 

LIST RESHAPE(my_time_series,out_grid[k=1:6])
        RESHAPE(MY_TIME_SERIES,OUT_GRID[K=1:6])
             1       2       3       4       5       6
             1       2       3       4       5       6
1982   / 1:  0.5144  0.7477  0.9123  0.9931  0.9827  0.8820
1983   / 2:  0.7003  0.4542  0.1665 -0.1366 -0.4271 -0.6783
1984   / 3: -0.8673 -0.9766 -0.9962 -0.9243 -0.7674 -0.5401
1985   / 4: -0.2632  0.0380  0.3356  0.6024  0.8138  0.9505
1986   / 5:  0.9999  0.9575  0.8270  0.6207  0.3573  0.0610
 

For any axis X,Y,Z, or T if the axis is the same in the input and output grids then the region from argument 1 will be preserved in the output. This implies that when the above technique is used on multi-dimensional input, only the axes which differ between the input and output grids are affected by the RESHAPE operation. However RESHAPE can only be applied if the reshape operation preserves the ordering of data on the axes in four dimensions. The RESHAPE function only "wraps" the variable to the new grid, keeping the data ordered as it exists in memory, that is, ordered by X (varying fastest) then -Y-Z-T (slowest index). It is an operation like @ASN regridding. Subsetting is done if requested by region specifiers, but the function does not reorder the data as it is put on the new axes. For instance, if your data is in Z and T: 

SHOW GRID G001
  GRID (G001)
name       axis              # pts   start                end
normal    X
normal    Y
ZMONTH    Z                   12 r   1                    12
T_ABSTR   T                    5 r   1                     5 

and you wish to put it on a new grid, GRIDYZ 

SHOW GRID gridyz
  GRID (GRIDYZ)
name       axis              # pts   start                end
normal    X
YAX       LATITUDE             5 r   15N                 19N 

ZMONTH    Z                   12 r   1                    12
normal    T  

then the RESHAPE function would NOT correctly wrap the data from G001 to GRIDYZ, because the data is ordered with its Z coordinates changing faster than its T coordinates, and on output the data would need to be reordered with the Y coordinates changing faster then the Z coordinates. 

The following filled contour plot of longitude by year number illustrates the use of RESHAPE in multiple dimensions by expanding on  the previous example: (Figure 3_2) 

!  The year-by-year progression January winds for a longitudinal patch
!  averaged from 5s to 5n across the eastern Pacific Ocean. Note that
!  k=1 specifies January, since the Z axis is month