How can LAS access data stored in a relational database?
Many types of non-gridded data are stored in relational database management systems (RDBMS). Examples would include oceanographic and atmospheric bottle data and profiles. Often, individual data records are stored as individual records (rows) in a table in the RDBMS. To be recoverable a record must include at least longitude, latitude, time, depth/height (where necessary) and columns for each variable measured. LAS code exists that makes it easier to communicate with the RDBMS, extract and reformat data so that it can be delivered with LAS. This FAQ describes the process.
Note 1: Other strategies might involve storing entire profiles in a single record rather than individual data points. The code provided with LAS is written to handle records consisting of individual data points but should prove a good starting point for writing your own code to deal with other data management strategies.
Note 2: The code provided is written to use the perl DBI.pm module that should allow you to use a wide variety of DBMS. The TMAPDBI.pm module assumes the MySQL DBMS. Another module is available that show how this module was extended to access the Joint Global Ocean Flux Study DBMS.
Note 3: This code is still experimental as of June 30, 2003l. Significant changes should be expected.
To get LAS working with your RDBMS you'll have to install some custom code distributed with LAS but not installed by default. Here's the installation process:
1) Perform the initial configuration of your LAS and say 'yes' to questions about a 'custom' directory:
Use custom directory? [no] yes <CR> Name of custom directory: [custom] <CR> Created custom directory server/custom
2) Test the resulting default LAS installation.
3) Install the 'in-situ' code:
cd las/contrib/insitu make install
The following files are included in the in-situ support code:
insitu:
CVS/ Makefile* server/
insitu/CVs: Entries Repository Root Tag
insitu/server: custom/ CVs/ insitu_operations.xml insitu_options.xml insitu_ui.xml
insitu/server/custom: abort_no_data.jnl insitu_poly_xt.jnl abort_single_profile.jnl insitu_poly_xy.jnl abort_too_big.jnl insitu_poly_yt.jnl custom.pl insitu_poly_zt.jnl custom_standalone.pl* insitu_property_depth_.jnl CVs/ insitu_property_.jnl insitu_gaussian_xy.jnl insitu_property_setup.jnl insitu_gaussian_xz.jnl insitu_refmap.jnl insitu_gaussian_yz.jnl insitu_setup.jnl insitu_list_1_text.jnl insitu_setup_stations.jnl insitu_list_cdf.jnl insitu_std_gif.tmpl insitu_list_text.jnl insitu_std_list.jnl insitu_meta_xz.jnl insitu_waterfall_key.jnl insitu_meta_yz.jnl insitu_waterfall_xz.jnl insitu_pie_key.jnl insitu_waterfall_yz.jnl insitu_pie_station_depth.jnl insitu_waterfall_zt.jnl insitu_pie_xt.jnl TMAPDate.pm insitu_pie_xy.jnl TMAPDBConfig.pl insitu_pie_xz.jnl TMAPDBI.pm insitu_pie_yt.jnl TMAPJGOFS.pm insitu_pie_yz.jnl TMAPWODB.pm insitu_pie_zt.jnl
4) Modify the las/server/custom/TMAPDBConfig.pl to reflect your database configuration. During the (ongoing) initial development of LAS-RDBMS functionality, this configuration file is merely a perl hash. This may change in the future. You will need to specify at least one named 'DBDataset' and the configuration information associated with it. The examples in the TMAPDBConfig.pl file should be enough to get you started.
5) Copy las/server/custom/custom_standalone.pl up one level into the las/server/custom directory. This is the directory from which LAS will try to access your RDBMS. Inside of custom_standalone.pl you will need to modify the following section:
# The following information should be changed to match # your database:
my $dsetname = "Arabian_Sea/bottle_arabian";
my @regArgs = (54, 71, 8, 26, 0, 300, "01-jan-1995", "31-dec-1995");
my $output_file = "database_test.nc";
my @variable = ("sal_bot","salinity","mol/L");
6) Try to access your RDBMS by running custom_standalone.pl. If successful, you are ready to configure LAS to present your in-situ data.
7) You will have to construct the dataset configuration file by hand, describing a default grid that matches the domain of the data. A simple dataset configuration file for a surface only, in-situ dataset might look like this:
<datasets> <obs_surface name="Surface Observations" url="obs/surface" doc=""> <institution name="USGODAE" url="http://usjgofs.whoi.edu/"/> <properties> <ui> <default>file:ui.xml#insituSurface</default> </ui> </properties>
<variables>
<sst units="deg C" name="SST">
<link match="/lasdata/grids/obs_surface_grid"/>
</sst>
</variables>
</obs_surface>
</datasets>
<grids> <obs_surface_grid> <link match="/lasdata/axes/obs_surface_X"/> <link match="/lasdata/axes/obs_surface_Y"/> <link match="/lasdata/axes/obs_surface_T"/> </obs_surface_grid> </grids>
<axes> <obs_surface_X units="degrees_east" type="x"> <arange start="-180" step="1" size="360"/> </obs_surface_X> <obs_surface_Y units="degrees_north" type="y"> <arange start="-90" step="1" size="180"/> </obs_surface_Y> <obs_surface_T units="day" type="t"> <arange start="2002-05-15" step="1" size="90"/> </obs_surface_T> </axes>
8) The las.xml file will need to be modified to include this dataset configuration file as will as the other in-situ specific configuration information copied from las/contrib. Here is an example las.xml file with changes hilighted in red.
<?xml version='1.0' ?> <!DOCTYPE spec SYSTEM "spec.dtd" [
<!ENTITY StdOperations SYSTEM "operations.xml"> <!ENTITY insitu_operations SYSTEM "insitu_operations.xml">
<!ENTITY obs_surface SYSTEM "obs_surface.xml">
]>
<lasdata>
<institution name="Pacific Marine Environmental Lab"
url="http://www.pmel.noaa.gov"/>
<!-- Define properties -->
<properties>
<ferret>
<land_type>shade
</land_type>
<fill_type>fill
</fill_type>
<view_centered>no
</view_centered>
<fill_levels/>
<format>netcdf
</format>
</ferret>
</properties>
<operations url="http://stout.pmel.noaa.gov/las-bin/LASserver.pl"> &StdOperations; &insitu_operations; </operations>
&obs_surface;
</lasdata>
9) Rebuild the las interface with CD las/server; make' and you should be able to access your data.
Jonathan Callahan: Jonathan.S.Callahan@noaa.gov
Last modified:June 30, 2003