Ferret V5.2 release notes

New features and enhancements:

Plotting:

  • XLAB and YLAB now suppressed by /NOLABELS
  • New qualifiers simplify line plots: PLOT/COLOR=/THICK=
  • New qualifiers simplify plots using symbols: PLOT/COLOR=/SIZE=/THICKNESS=
  • New qualifiers simplify outline characteristics: POLYGON/LINE/COLOR=/THICK=
  • New qualifiers give easier control over labels: CONTOUR/SIZE=/SIGDIG=/SPACING=
  • The /PEN qualifier on the CONTOUR and VECTOR commands has been replaced by /COLOR
  • Contour plots of all-missing data are no longer treated as errors
  • New command qualifiers /HLIMITS and /VLIMITS allow you to specify the horizontal and vertical axis limits and tic spacing of all graphics commands
  • Able to PLOT/VS/OVER or POLY/OVER on top of a time axis
  • Point and click positioning of labels and other plot annotations
  • New alias WHERE to get mouse click position
  • New script "GO digitize" to get positions and values on a plot

    • Functions:

    External Functions:

    Data management:

    Grids, variables:

    Other:

    Bug fixes/ minor changes:

    New features/ enhancements:

    Plotting:

     

  • XLAB and YLAB now suppressed by /NOLABELS

    • This means that the axis labels (e.g. the units of the variable on the vertical axis of the PLOT command) are now suppressed by /NOLABELS
     
  • New qualifiers simplify line plots: PLOT/COLOR=/THICK=

    • For line plots it is now possible to control line thickness and color with self-evident commands such as

    PLOT/COLOR=blue/THICK=2 I[i=1:3]

    This is equivalent to the (still supported) use of the /LINE qualifier as in

    PLOT/LINE=10 I[i=1:3] ! 4(blue) + 6*(2-1)

    The available color names are Black, Red, Green, Blue, LightBlue, Purple, and White (not case sensitive), corresponding the the /LINE values 1-7, respectively.

    (/COLOR also accepts numerical values.) The line thickness may be 1, 2, or 3 corresponding to pixel thickness on the screen or corresponding to multiples of the default line thickness on hard copy.
    that /COLOR=WHITE is available only in the default /THICKNESS of 1
     
     

  •  New qualifiers simplify plots using symbols: PLOT/COLOR=/SIZE=/THICKNESS=

    • For symbol (scatter) plots (PLOT/VS or PLOT/SYMBOL) it is now possible to control the color, size, and line thickness of the symbols with self-evident commands such as

    PLOT/COLOR=red/SIZE=0.2/THICKNESS=2/SYMBOL=4 I[i=1:5]

    The available color names are Black, Red, Green, Blue, LightBlue, Purple, and White (not case sensitive), corresponding the the /LINE values 1-7, respectively. (/COLOR also accepts numerical values.) The line thickness may be 1, 2, or 3 corresponding to pixel thickness on the screen or corresponding to multiples of the default line thickness on hard copy.  /COLOR=WHITE is available only in the default /THICKNESS of 1.
    The /SIZE is given in units of "inches", consistent with the PLOT+ usage of "inches". (These are the same units as in, say, "ppl axlen 8,6", to specify plot axes of lengths 8 and 6 inches for horizontal and vertical axes, respectively.)

    Functions:

    Data management:

    Note that when writing netCDF files Ferret, by default, does NOT include the point_spacing attribute. This is because Ferret's default file characteristic is to be append-able, with no guarantees that the appended time steps will be regularly spaced. For output files of fixed length with regular time steps it is advisable to use the SAVE/RIGID qualifier. This allows Ferret to include the point_spacing="even" attribute. If the files will be very large (too large for the full time range to be in memory), then use the /RIGID//TLIMITS= qualifiers to specify the full, ultimate fixed size and use SAVE/APPEND to insert data into the file piecemeal.

  • Added strides, reversals, and permutations to netCDF reads

    • Internally, the functioning of netCDF reads has been changed when "strides" are involved. Suppose that CDFVAR represent a variable from a netCDF file. In version5.0 and earlier the command PLOT CDFVAR[L=1:1000:10] would have read the entire array of 1000 points from the file; Ferret's internal logic would have subsampledevery 10th point from the resulting array in a manner that was consistent for NetCDF variables, ASCII variables, user defined variables, etc. In V5.1 strides applied to netCDF variables are given special treatment -- subsampling is done by the netCDF library. The primary benefit of this is to make network access to remote data sets via DODS more efficient. A remote satellite image of size, say, 1000x1000 points x 8 bit depth (8 megabytes) can efficiently be previewed using
        SHADE DODS_VAR[i=1:1000:10,j=1:1000:10]
     
  • Compatibility with the netCDF encodings used by IRI

    • Ability to access the netCDF encodings at the Ingrid system at the Lamont Doherty Laboratory makes the large collection of data sets served from the IRI available to the Ferret user. For example
        yes? use
        "http://ingrid.ldeo.columbia.edu/SOURCES/.NOAA/.NODC/.WOA98/.ANNUAL/.analyzed/dods"

  • DODS caching

    • This feature allows caching of frequently accessed DODS-served datasets to produce a quicker response when requesting the remote data. A few words about caching:

    The first time you access a dods data set, a file in the users home directory will be created called
    .dodsrc
    and will contain:

          USE_CACHE=1     ! Turn cache on or off - 0=off, 1=on
      MAX_CACHE_SIZE=20   ! max cache size in Mbytes
      MAX_CACHED_OBJ=5    ! max cache entry size in Mbytes
      IGNORE_EXPIRES=0    ! 0: Honor expiration informatiom from servers,1: ignore them
      CACHE_ROOT=/home/kobrien/.dods_cache/  !pathname to cache directory
      DEFAULT_EXPIRES=86400    ! expiration time, in seconds, of cached data

    Also created will be a .dods_cache directory, which by default (as mentioned above) is created in the users home directory. This is where all the cached information is stored. To clear the DODS cache, simply delete the .dods_cache directory and all of it's contents (for example, rm -r ~/.dods_cache). This directory will be recreated and repopulated with caching information the next time data is accessed via DODS and caching is turned on. Of course, all of the above values can be modified to better suit individual needs, and will be incorporated the next time Ferret is run. For example, to turn caching off, simply set USE_CACHE to 0, and restart Ferret.

    NOTE: DODS caching is currently only available for the Solaris, Compaq Unix/OSF and RedHat Linux operating systems.

    Grids, variables:

  • Added units "minutes" and "months" to the list of units understood by Ferret

    • Since "months" is not truly a unit (its length varies with context) the unit "months" will be interpreted as the length of an average month in the Gregorian calendar, including leap years -- 1/12 or 365.2485 days. Thus the command

    DEFINE AXIS/T0=1-JAN-0000/T=0:12:1/EDGES/units=months/MODULO month_reg

    defines a climatological monthly axis which does not "drift" over time due to leap years. This non-drift behavior can be observed using commands like

    SHOW AXIS /l=1:12001:1200 month_reg

    which will show every 100th January over 1000 years.
     

  •  The grids and axes of netCDF datasets are now "dynamic"

    • "Dynamic" means that these grids and axes are automatically deleted when a data set is closed. This should eliminate the problem that was occasionally reported of running out of grids or axes in long REPEAT loops involving many files.

    If a grid or axis from a netCDF file is used in the definition of a LET-defined variable (e.g. LET my_X = X[g=sst[D=coads_climatology]]) that variable definition will be invalidated when the data set is canceled (CANCEL DATA coads_climtology, in the preceding example). There is a single exception to this behavior: netCDF files such as climtological_axes.cdf, which define grids or axes that are not actually used by any variables. These grids and axes will remain defined even after the data set, itself, has been canceled. They may be deleted with
    explicit use of CANCEL GRID or CANCEL AXIS.

  • New commands CANCEL GRID and CANCEL AXIS

    • These commands form the complement to DEFINE GRID and DEFINE AXIS. They are also applicable to "persistent" grids or axes which are defined by netCDF files such as climatological_axes.cdf -- grids or axes which are not associated with any variables in the netCDF file, itself, and are not automatically deleted when the data set is canceled.

    Attempts to CANCEL GRID or CANCEL AXIS on a grid or axis which is used by a variable in a currently open data set will be rejected with a message indicating the reason.

  •  Command qualifiers LET/BAD= and SET VARIABLE/BAD=

    • allow user to control the missing value of user-defined variables. When the command SET VARIABLE/BAD= is be used to set one of the two missing value flags of a file variable, the bad value which is specified will be used in subsequent outputs and calculations involving this variable.

    The specified value will be used whenever the variable is LISTed (or SAVEd) to a file. Note that the missing value will revert to its default (-1E34) when this variable is combined in further calculations. The example below illustrates:

    yes? LET/BAD=3 gap_3 = I[I=1:5]
    yes? LIST gap_3
                I[I=1:5]
     1  / 1: 1.000
     2  / 2: 2.000
     3  / 3:  ....
     4  / 4: 4.000
     5  / 5: 5.000
    yes? LET new_var = gap_3 + 5
    yes? LIST new_var
                GAP_3 + 5
     1  / 1:  6.00
     2  / 2:  7.00
     3  / 3:  ....
     4  / 4:  9.00
     5  / 5: 10.00
    yes? LIST/FORM=(1PG15.3) new_var
                GAP_3 + 5
                X: 0.5 to 5.5
          6.00
          7.00
        -1.000E+34
          9.00
          10.0

  • Bad value from SET VARIABLE/BAD= will be used in subsequent outputs and calculations involving this variable

    • When the command SET VARIABLE/BAD= is used to set one of the two missing value flags of a file variable, the bad value which is specified will be used in subsequent outputs and calculations involving this variable.

    Ferret is aware of up to two missing value flags for each variable in a netCDF file. Under most circumstances, netCDF file variables use only a single flag. With a command like SET VARIABLE/BAD=-999 my_file_var you can specify -999 as an additional missing value flag. It is this value which will be present in all subsequent SAVEs to files and calculations.

    Note that if the netCDF file contains two distinct flag values specified by the netCDF attributes "missing_value" and "_FillValue", then this command will migrate the value specified by missing_value to the position previously occupied by _FillValue and replace replace the one specified by missing_value. Thus a double usage of this command allows you to control both flags. You can use the command "SHOW DATA/VARIABLES" to see both bad value flags.

  • User-defined grids may be used in all situations

    • Previous versions of Ferret did not permit the (implied) grids of user-defined variables to be used in all situations. Those restrictions have been lifted. Grids implied by the LET command may now be used :

  • Units of "decibar" (or "dbar"), and "millibar" (or "mbar") now recognized as valid units for axes.

    • Integration and differentiation operations (e.g. @ddf) do not attempt to convert these units to a standard unit.

    External Functions:

  • The External Functions utility functions are now linked into the Ferret executable

    • The user does not need platform-specific utility functions to compile user-written external functions. Get new makefiles and example code at EF_Fer5_2.tar.Z (External Functions web page not yet updated with this change)
     
  • Example external function interface to Perl programs

    • An example external function PPPP.F is provided that illustrates the ability of Ferret's external functions to call routines written in the Perl programming language. This is a powerful capability, for example, in accessing relational data bases from Ferret. The example code downloads the recent past history of the Dow Jones industrial average from yahoo.com over the Web.
     
     
  • Example external function interface to Matlab programs

    • An example external function MMMM.F is provided that illustrates the ability of Ferret's external functions to call routines written in the Matlab environment. This capability gives the Ferret user access to the mathematical horsepower of Matlab and to the many suites of routines already written for that environment. The example code invokes a graphics demonstration that is distributed with Matlab.
  • The order of arguments to the SAMPLE functions is reversed relative to earlier versions

    • This is for consistency among the various SAMPLE functions. They all follow the pattern SAMPLE*(DATA, sample-points)
  • The scatter2grid* functions have been renamed to scat2grid*

    • scatter2gridgauss_xy is now scat2gridgauss_xy
    scatter2gridgauss_xz is now scat2gridgauss_xz
    scatter2gridgauss_yz is now scat2gridgauss_yz
    scatter2gridlaplace_xy is now scat2gridlaplace_xy
    scatter2gridlaplace_xz is now scat2gridlaplace_xz
    scatter2gridlaplace_yz is now scat2gridlaplace_yz
     
     
  • Many of the included external functions are statically linked and so are available for systems where dynamic linking is not available.

    •  

    See SHOW FUNCTIONS/EXTERNAL  or SHOW FUNCTIONS/EXTERNAL/DETAILS for more information. The statically linked functions are:
     
     
    ffta  FFT amplitude spectrum
    fftp FFT phase
    sampleij Sample data at a subset of grid points
    samplet_date Sample data at one or more times
    samplexy Sample data at a set of (x,y) points such as a track line
    samplei sample data at a set of I indices
    samplej sample data at a set of J indices
    samplek sample data at a set of K indices
    samplel sample data at a set of L indices
    scat2gridgauss_xy  Grid scattered data using Gaussian algorithm
    scat2gridgauss_xz
    scat2gridgauss_yz
    scat2gridlaplace_xy Grid scattered data using Laplacian/Spline algorithm
    scat2gridlaplace_xz
    scat2gridlaplace_yz
    sorti returns indices of sorted data
    sortj
    sortk
    sortl
    tauto_cor autocorrelation
    xauto_cor

    Other:

  • Backslash escapes protect quotations in GO script arguments

    • Example:
    file test.jnl:
        let a = 5
        set variable/title=$1 a

    yes? go test "\"multi word title\""     Results in:
    let a = 5
    set variable/title=$1 a
     !-> set variable/title="multi word title" a
     

  •  Change to the RETURN= option in immediate mode expressions

    • The RETURN= option in immediate mode expressions (grave accent enclosed) no longer actually computes the result unless it must. For example, the expression `sst, RETURN=TEND` will return the formatted coordinate for the last point on the T axis of variable sst without actually reading or computing the values of sst. This allows Ferret scripts to be constructed so that they can anticipate the size of variables and act accordingly.

    Note that this does not apply to variable definitions which involve grid-changing variables that return results on ABSTRACT axes. For those variables the size and shape of the result may depend on data values, so the entire result must be computed in order to determine many of the RETURN= attributes.
     

  • New RETURN= options for immediate mode expressions (grave accent enclosed)

    • Several new options have been added to the immediate mode RETURN=
        syntax::
           o return=size - returns size (number of points) in the
             expression
           o return=xstart (also y,z,t) - returns start of specified world
             coord region
           o return=xend (also y,z,t)   - returns end of specified world
             coordinate region
           o return=istart (also j,k,l) - returns start of specified index
             region
           o return=iend (also j,k,l)   - returns end of specified index
             region
           o return=T0 (returns the T0 string from the time axis)
           o return=unit (returns the units string from the variable)
           o return=title (returns the title of a variable)
           o return=grid (returns the grid name of a variable)
        Still available are the previous RETURN= options: SHAPE,
        [I,J,K,L]SIZE, [X,Y,Z,T]SIZE.

        For example, `sst,RETURN=SIZE` will return the total number of point in the variable sst -- Nx*Ny*Nz*Nt
            SET REGION/X=130e:80w/T=1-JAN-1990:15-MAR-1993
            `sst,RETURN=SIZE`
     


    2-June-2000