Personal tools
You are here: Home Documentation Users Guide 8. Working with Special Data Sets 8.2 POINT DATA
Document Actions

8.2 POINT DATA

by rohan — last modified 2014-10-17 14:03

In a gridded context point data is best viewed as a collection of 1-dimensional variables, where the axis of each variable is the index value, 1, 2, 3, ... of the individual point in the scatter. Thus, continuing our example of an oceanic NO3 data set, we would want to view this as four variables, longitude, latitude, date, and burden, where each variable was defined on a one-dimensional axis of earthquake number. Typically, this sort of data is organized in a table of the form








 Index  longitude  latitude  year  month  day  N03

1

160

30

1968

11

-999

6.2

2

33.1

60.2

1992

5

13

5.5

...









8.2.1 Getting point data into Ferret

Since point data sets are most commonly available in table form, where the columns of the table are the variables and each row of the table is a separate point. In the chapter "Data Set Basics", section "Reading ASCII Files" , example 2 and subsequent examples show how such a file might be read into Ferret.

For example, let us suppose that the file above is introduced to Ferret with the command

yes? FILE/VAR="index,lon,lat,yr,mn,day,NO3"/SKIP=1 my_data_file.dat
yes? SHOW DATA my_data_file.dat

currently SET data sets:
1> ./my_data_file.dat (default)
name title I J K L
LON LON 1:20480 ... ... …
LAT LAT 1:20480 ... ... …
YR YR 1:20480 ... ... …
MN MN 1:20480 ... ... …
DAY DAY 1:20480 ... ... …
NO3 NO3 1:20480 ... ... …

Note that the SET VARIABLE command would normally be used as well to assign titles, units, and missing value flags to the variables.


Beginning with Ferret v6.93, if the FILE command is given without a grid, Ferret determines the size of the file and defines an axis and grid on which to read the variables. It takes any /skip and /columns qualifiers into account. For v6.93 and higher, the number of I indices listed above will be the size of the data in the file.

[For versions prior to Ferret v6.93,  until the first data is actually requested from the file, Ferret does not know the size of the file. The /GRID= option may be used to tell Ferret what size to expect. Lacking a /GRID specification the "1:20480" is the size of the default grid "EZ." After the first data access SHOW GRID will reveal the true size of the file, instead. If the size still appears to be 20480 it may be that the default grid EZ was not large enough, and the /GRID qualifier must be used to pre-allocate sufficient space.]


8.2.2 How point data is structured in Ferret

In table form (above) each column represents a dependent variable; the column for "burden" and the column for "latitude" have equal status. In many cases this is an adequate representation. For example, a plot of NO3 burden versus latitude could be produced with the command

yes? PLOT/VS lat, NO3

To combine point data organized in tables with gridded data sources, say a gridded field of oceanic temperature two approaches are available. Either the gridded data may be viewed in the structure of the table, or the scattered data may be viewed in a geo-referenced 1-dimensional grid structure. The problem to be solved determines which approach is suitable. The next two sections describe these two approaches.


8.2.2.1 Working with dates

Ferret V5.0 does not understand formatted dates inside of generic data ASCII files. To use the dates intelligibly inside of Ferret you

1. Need to get the year, month, and day fields broken out separately or provide a Julian day. SET DATA/FORMAT=DELIMITED is helpful for inputting date information.

2. Can create a Julian date from year, month, day using function DAYS1900. If a time origin other that 1-jan-1900 is needed subtract DAYS1900(year0, mon0, day0). For help in creating the dates, see the FAQ, "How can I create a time axis from variables containing year, month, day, etc?" at http://ferret.pmel.noaa.gov/static/FAQ/axes_and_data/time_axis_from_variables.html

3. Can create an axis of dates as done in the preceding latitude axis example.

See the chapter "Grids and Regions", section "Time" and the section in the chapter "Converting to NetCDF" on "Converting time word data to numerical data" for details of creating time axes.


8.2.3 Subsampling gridded fields onto point locations and times

Ferret can be used as a tool to extract variables from gridded data sets at time/space locations to match the scatter of the point data. In this form they may, effectively, be combined into the table of data read from the ASCII (or binary) file. For example, suppose we want to obtain values of sea surface temperature at the locations of our NO3 samples, from a climatological annual average SST field. This may be accomplished simply with

yes? use coads climatology
yes? let ssttav = sst[l=1:12@ave]
yes? let my_lon = lon[d=my_data_file.dat]
yes? let my_lat = lat[d=my_data_file.dat]
yes? LET sst_xy = SAMPLEXY(ssttav, my_lon, my_lat)

Suppose that instead we defined our XY sampling based upon the 12 month time series of SST grids as in

     yes? LET sst_xy = SAMPLEXY(sst, my_lon, my_lat)

The variable sst_xy as defined above would then have a two-dimensional structure: sample index by 12 months. To sample this in time we use

yes? LET zero = 0 + 0*mn
yes? LET sst_t = SAMPLET_DATE(sst_xy,zero,mn,day,zero,zero,zero)

Note that the year is entered simply as 0, since SST is a climatological variable, and the variable "zero" is the same length as the variables "mn" and "day". To sample a field at hour 12 of each day, we could use "zero+12" for the hours argument.

In this example we sampled a field in X, Y, and T. The sst data was sampled at each time. If we were sampling a field which had a Z axis, that axis would be inherited from the first argument to SAMPLEXY in the same way; it would be sampled at the (x,y) points at each Z level.


8.2.4 Defining gridded variables from point data

There are functions to interpolate scattered data onto a grid. See the scat2gridgauss and scat2gridlaplace functions. These functions map irregular locations to a regular grid.

For some calculations one may want to let Ferret know which of the variables are dependent (measurements) and which are independent (coordinates). For example, suppose we wish to compute the average column burden of NO3 as a function of latitude. Burden here is an integral of the concentration NO3 over depth. We will want to see our variable burden on an axis of latitude.

The steps to do this are

1. In general, the latitude variable will not be sorted into strictly increasing order — needed to create an axis. Determine the sorting order for latitude using

    yes? LET lat_index = SORTI(lat)

2. Create a latitude grid

   yes? DEFINE AXIS/FROM/NAME=lat_ax/Y/UNITS=degrees SAMPLEI(lat, lat_index)
yes? DEFINE GRID/Y=lat_ax glat
yes? LET NEW = Y[g=glat] ! a dummy variable to use in RESHAPE below

3. Define your function for the burden based on the variable NO3, on the command line or using your script my_brdn.jnl.

    yes? GO my_brdn NO3 burden

4. Define a new variable burden_on_lat using this axis

   yes? LET sorted_burden = SAMPLEI(burden, lat_index) 
yes? LET burden_on_lat = RESHAPE( sorted_burden, new )

5. Now, to plot the NO3 burden averaged into 5 degree latitude bands we could use

    yes? PLOT burden_on_lat[Y=60s:30n:5@AVE]


8.2.5 Visualization techniques for point data

Scattered point data can be displayed in a number of ways.

A simple scatter plot showing the locations of points

yes? PLOT/VS lon,lat
yes? GO land

Use GO/help land for an explanation of resolving incompatible longitude encodings, should they arise.

A scatter plot in which the symbols are colored by value with control over the color palette and resolution can be made using the polymark.jnl script. For example, to plot using stars symbols in color levels by 10s use

yes? GO polymark POLYGON/LEV=(0,100,10) lon lat NO3 star.

Type GO/HELP polymark for more options.

See also the chapter "Customizing Plots", section "Map Projections" for guidance on plotting scattered data. The map projection scripts can be used in conjunction with the above.



Powered by Plone CMS, the Open Source Content Management System

This site conforms to the following standards: