[Thread Prev][Thread Next][Index]
[ferret_users] Tips and Tricks to avoid memory problems in Ferret
Hi Ferreters,
We all must have seen the following error message from
Ferret, quite a number of times....
**ERROR: insufficient memory: ???????? words were requested.
Here is few tips/tricks to avoid this error. With this tricks, you will
also be able to :
- say goodbye to "set memory/size=???mb" command
- save some of your precious time
- monitor the progress of your script .... interesting??
Tips
====
1. Memory error happens when the entire XYZT grid is written out in a single
stretch, or when some regridding/transforming operations are
performed on multiple axes.
2. Therefore, a straight solution is to split the writing out process into
multiple slices, along a PROPER axis.
3. There is no neat method (i guess so) to do such a slicing if you intend
to display the result of a complex calculation/regridding. A work
around is to write out the result to an intermediate file, using 2.
above, and read it back to create a display!
4. Remember that Ferret is not an "immediate evaluator" by default, and
it does so only when you force with grave accents (``). Hence you
can write all the calculation steps before the writing out part.
Tricks
======
From Ferret/NetCDF point of view, all the axes will not behave in the same
manner. Hence different cases are discussed separately.
case 1 : calculation on X/Y/Z axes, and NOT on T axis
-----------------------------------------------------
Say the calculation involves only the X/Y/Z axes, so that it is safe to work
on time "slices". The TIME axis have a special nature that, always it will be
the RECORD dimension. Hence you can append data without any special features
requested for the output file. The best solution will be doing the writing
out part within a REPEAT loop on "L".
sp rm -f outfile.nc ! keep it here
let l_start = `myvar,r=lstart`
let l_end = `myvar,r=lend`
let slices = l_end - l_start + 1
REPEAT/L=`l_start`:`l_end`:1 (;\
save/file=outfile.nc/APPEND/quiet myvar ;\
say " Time slice `(L-l_start+1)`/`slices`" ;\
)
case 2 : calculation on T axis
------------------------------
If the calculation is done on T-axis (time), then we have to slice the
writing process along any other axis, which does not affect the calculation.
The priority goes like Z, Y and X. The procedure remains same for all three,
since none of them are RECORD axis. For the same reason, if we have to open
a new file for writing out, we have to specify the corresponding abstract
LIMITS in the first step and respective index of data in subsequent steps.
If we are slicing along Z-axis, the Ferret commands will be
sp rm -f outfile.nc ! keep it here
let k_start = `myvar,r=kstart`
let k_end = `myvar,r=kend`
let slices = k_end - k_start + 1
save/file=outfile.nc/KLIMITS=1:`slices`/K=1/CLOBBER/quiet myvar
say " Depth slice 1/`slices`"
REPEAT/K=`k_start+1`:`k_end`:1 (;\
save/file=outfile.nc/APPEND/K=`K`/quiet myvar ;\
say " Depth slice `(K-k_start+1)`/`slices`" ;\
)
Please note that the "K=1" step is done separately, since it needs the
"/KLIMITS" qualifier.
Cheers !!
Jaison
--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.
[Thread Prev][Thread Next][Index]
Contact Us
Dept of Commerce /
NOAA /
OAR /
PMEL /
TMAP
Privacy Policy | Disclaimer | Accessibility Statement