General GAMAP usage

GAMAP is now obsolete. We recommend using GCPy for analyzing output from recent GEOS-Chem versions. But we will preserve the GAMAP wiki documentation for reference.

On this page we list some general GAMAP tips & tricks. Please also see the following pages for more tips & tricks:

--Bob Y. 16:05, 26 November 2008 (EST)

GAMAP and IDL 7

Version 7 of IDL comes with a new development environment (called Workbench) based on Eclipse. If you are not using the IDL Workbench, you will not notice any change. But if you do, you will see a major upgrade, ... and may soon have a problem with the !PATH definition. For GAMAP to work properly, we provide an idl_startup.pro template file for you to modify for your system. In this routine, you used to see the !PATH defined like that:

IdlPath = Expand_Path( '+' + !PATH,  /All_Dirs ) 

!PATH       = EXPAND_PATH('+~/IDL/gamap2/', /ALL_DIRS) + ':' + $
              EXPAND_PATH('+~/IDL/idlpro/')            + ':' + $
              EXPAND_PATH('+~/IDL/jhuapl/')            + ':' + $
              IdlPath

Now, for GAMAP to work with the new IDL Workbench, you must use the following syntax instead:

; Call the IDL routine PATH_SEP to get the path separator token 
; (i.e. the character that you need to separate directories in !PATH). 
; This token is different depending on which OS you are using.
Sep     = Path_Sep( /Search_Path )

; Use the PREF_SET command to append your directories into the default
; IDL search path.  This approach is necessary if you want to use the 
; IDL Workbench in IDL 7.0 and later versions. (phs, bmy, 4/15/08)
PREF_SET, 'IDL_PATH',                                        $
           EXPAND_PATH('+~/IDL/gamap2/', /ALL_DIRS ) + Sep + $
           EXPAND_PATH('+~/IDL/idlpro/'            ) + Sep + $
           EXPAND_PATH('+~/IDL/jhuapl/'            ) + Sep + $
           '<IDL_DEFAULT>', /COMMIT

Note for Windows users

one user reported that replacing

Sep     = Path_Sep( /Search_Path )
PREF_SET, 'IDL_PATH',                                        $
          EXPAND_PATH('+~/IDL/gamap2/', /ALL_DIRS ) + Sep + $
          '<IDL_DEFAULT>', /COMMIT

with

IdlPath = Expand_Path( '+' + !PATH,  /All_Dirs )
Sep     = Path_Sep( /Search_Path )
PREF_SET, 'IDL_PATH',                                        $
           EXPAND_PATH('+~/IDL/gamap2/', /ALL_DIRS ) + Sep + $
           idlpath, /COMMIT

in idl_startup.pro solved the following problem under Windows: !path would not contain directories without a *.pro file.

NOTES:

The PREF_SET command was introduced in IDL 6.2.
This syntax can be added to your idl_startup.pro file even if you do not use the IDL Workbench. More about that issue can be found here.
You can add/delete directories to the IDL_PATH corresponding to the particular directory structure in your space. In this particular example we are adding the gamap2, idlpro, and jhuapl directories to IDL_PATH. If, for example, you don't have an idlpro directory, you can of course omit that.
The EXPAND_PATH function should expand a relative file path (i.e. with a ~) to a full file path. However, there may be some flavors of Linux for which ~ might be problematic.
Adding a + before the directory name in the call to EXPAND_PATH will make IDL search the directory and all of its subdirectories for files of the appropriate type (*.pro, *.sav) for the given path.
Using the /ALL_DIRS keyword in the call to EXPAND_PATH, such as EXPAND_PATH( '+~/IDL/gamap2/', /ALL_DIRS ), will cause IDL to return the full path names of all of the subdirectories under ~/IDL/gamap2, regardless of whether or not they contain *.pro or *.sav files. (IDL's default behavior is to only return path names for those directories containing *.pro or *.sav files.)
If the EXPAND_PATH function for some reason does not expand the filepaths, then you can specify them manually. Windows users might need to start the string with the drive letter (e.g. C:\).
It is best to create the <IDL_DEFAULT> string with PREF_SET. Do not attempt to add the file paths directly to the IDLDE or IDL Workbench GUI.

--phs 10:52, 30 March 2009 (EDT)

--Bob Y. 13:49, 7 July 2008 (EDT)

Note for Mac users

If you experience a problem starting up GAMAP on a Mac, make sure to place the gamap2 directory into your home directory. If you place it into your Applications/itt directory you may encounter path problems.

If you are the Mac Xterm utility to run IDL on a Unix machine, then please make sure that you have your color settings properly defined, as described in this wiki post.

--Bob Y. 10:03, 13 April 2012 (EDT)

Note for Unix users

If IDL cannot find the idl_startup.pro file, you can specify the location with an environment variable:

If you are using tcsh or csh, then add this line to your .cshrc file:

# IDL settings
setenv IDL_STARTUP "$home/IDL/idl_startup.pro"

and type

source ~/.cshrc

to apply the changes.

If you are using bash, then add this to your .bashrc file:

IDL_STARTUP="$home/IDL/idl_startup.pro"
export IDL_STARTUP

and type

source ~/.bashrc

to accept the changes.

--Bob Y. 15:22, 22 April 2011 (EDT)

Problems with Windows 7

Rebecca Fahringer wrote:

The IDL website stated that IDL 8 is fully compatible with any code written for 7, however, some users had noted that there were problems with ordering one's PATH correctly because the workbench in IDL 8 has all of the internal programs and functions written as *.pro files such that they may be confused with a user-written program or function of the same name.

To avoid possible conflict, I uninstalled IDL 8 and installed 7. I then sub-contracted the local IDL guru who was able to get gamap up and running on his Linux device. He then came to assist me. After much finagling on this clunky Windows 7 setup (where apparently Windows thinks it knows what I want better than I do), my guru was able to get IDL to find gamap. victory! But, alas, there were compilation errors:

   % Compiled module: GAMAP.
   % Compiled module: ROUTINE_NAME.
   % Compiled module: STRBREAK.
   % Compiled module: STRSPLIT.
   % Compiled module: GAMAP_INIT.
   % Compiled module: FILE_WHICH.
   % Compiled module: PATH_SEP.
   % GAMAP_INIT: Reading 
   % Compiled module: OPEN_FILE.
   % OPEN_FILE: Filename must not contain wildcards when NO_PICKFILE option is set!
   % GAMAP_INIT: *** Cannot open gamap.defaults file ! ***
   % Error occurred at: GAMAP_INIT        198 C:\Program Files\gamap2\internals\gamap_init.pro
   %                    GAMAP              94 C:\Program Files\gamap2\internals\gamap_cmn.pro
   %                    $MAIN$          
   % Execution halted at: GAMAP              94 C:\Program Files\gamap2\internals\gamap_cmn.pro

My guru decided that the Windows version of IDL was choking on this error in OPEN_FILE. So, in the end, we have decided I should install IDL and GAMAP on a Unix system.

Bob Yantosca replied:

The error listed below in routine OPEN_FILE is just an error trap that says that if you see a * in a filename, GAMAP should pop up a dialog box and ask the user to pick the file that he/she wants. It may be that the * wildcard character is different in Windows than in Unix (I thought they were the same though).

The offending block of code in routine gamap2/file_io/open_file.pro is:

       ; if filemask contains wildcards, always use pickfile dialog
       ; Abort if NO_PICKFILE is set
       if (strpos(filemask,'*') ge 0 OR strpos(filemask,'?') ge 0) then begin
          if (keyword_set(NO_PICKFILE)) then begin
             message,'Filename must not contain wildcards when '+ $
                   'NO_PICKFILE option is set!',/CONT
             lun = -1   ; yet another error indicator
             return
          endif
          pickfile = 1

where the test

   strpos( filemask, ‘*’)

is looking for the “*” wildcard character (for Unix/Linux) and

   strpos( filemask, ‘?’ )

is looking for the “?” wildcard character (MS/DOS and older versions of Windows). I think the problem is that the newer versions of Windows have changed the behavior of the wildcard character. I just did a Google search and found this post online.

So it may be the GAMAP code as written cannot handle the Windows 7 OS behavior, and in particular, how it searches for files. It may be that the best option is to just use GAMAP on Unix/Linux machines. Even better if you can run on the same machine where your GEOS-Chem output is stored, that way you won’t have to FTP data over from the Unix/Linux machine to your PC.

--Bob Y. 10:20, 25 April 2012 (EDT)

GAMAP and GDL

GAMAP is not fully compatible with GDL (GNU Data Language, the open-source version of IDL). Some IDL features that GAMAP relies heavily upon have not been fully implemented in GDL. From the GDL web page:

Graphical output is partially implemented. The PLOT, OPLOT, PLOTS, XYOUTS, CONTOUR, SURFACE, TVRD and TV commands (along with WINDOW, WDELETE, SET_PLOT, WSET, TVLCT) work (important keywords, some !P system variable tags and multi-plots are supported) for X windows, z-buffer and postscript output.

--Bob Y. 09:51, 26 January 2012 (EST)

Usage of TRACERINFO.DAT and DIAGINFO.DAT

User A said he keeps getting this message:

% UPDATE_TRACER_DATAINFO: WARNING: Use_DataInfo appears independent from global array!

User B said:

My ctm.bpch files have duplicate data blocks for some parameters. Something's wrong!

Answer:

These are symptoms of diaginfo.dat or/and tracerinfo.dat that do not match the binary punch file (user B), or that the bpch file was created with inadequate diag/tracerinfo.dat files and end up with duplicate 999 tracers without tracername for example (user B). The solution is to use correct diag/tracerinfo.dat when reading and creating bpch files. GEOS-Chem now writes those files for each runs in your run directory. If you use them when reading run outputs, you will not have a problem.

The first time a GAMAP routine is used, it will look for and read these files. First, in the current directory, then in the !path variable (which means it will used the default ones in ../gamap2/input_files/). So, to read those corresponding to your run you can:

start your IDL session in your run directory
start your IDL anywhere, and then: cd, 'your_run_directory', before using GAMAP
copy your diag/tracerinfo.dat from your run dir to your ../gamap2/input_files/ (i.e., overwrite default ones), before using GAMAP
Advanced users can also, at anytime in their session, force GAMAP to read specific diag/tracerinfo.dat:

ctm_diaginfo, /all, /force_reading, filename='your_diaginfo.dat'
ctm_tracerinfo, /all, /force_reading, filename='your_tracerinfo.dat'

If you are using a default location, double check that it is really used with the following query:

print, file_which('tracerinfo.dat')

--phs 15:52, 4 April 2008 (EDT)

Memory management

CTM_MAKE_DATAINFO

When using CTM_MAKE_DATAINFO, you create few pointers and allocate memory to pointed data. Three scenarios to free that memory are possible.

(1) By default, GAMAP keeps track of the pointers in a global structure, and you can clean up the memory by calling CTM_CLEANUP (with or without the keyword /NO_GC):

    ctm_make_datainfo(data, ...)
    CTM_WriteBpch, DataInfo, FileInfo, FileName=file
    ctm_cleanup

(2) If you use the keyword /NO_GLOBAL when calling CTM_MAKE_DATAINFO, the story is a little bit more subtle. Now, GAMAP has no idea of the created pointers. If you use CTM_CLEANUP without the keyword /NO_GC, everything is clean up because there is a call to heap_gc. But you are also loosing **all other** pointers and objects.

    ctm_make_datainfo(data, ..., /No_global)
    CTM_WriteBpch, DataInfo, FileInfo, FileName=file
    ctm_cleanup

Note if you do not call ctm_cleanup or call it with /No_GC, then the memory allocated by CTM_MAKE_DATAINFO is still allocated and the pointers that refer to it are alive... until you exit the routine (unless they are passed back). Once you are out of the routine, this memory remains allocated but is useless since unaccessible. In other words, you have memory leak.

(3) So, if you want to keep others objects and/or pointers alive in your code (i.e., you do not want to call heap_gc or you want to call ctm_cleanup with /no_gc), you need to free only the "no global" created pointers as follows:

    ctm_make_datainfo(data, datainfo, fileinfo, ..., /No_global)
    CTM_WriteBpch, DataInfo, FileInfo, FileName=file
    ptr_free, DataInfo.data
    ptr_free, fileinfo.gridinfo

To free only the heap memory created by multiple calls to CTM_MAKE_DATAINFO(..., /No_Global), the procedure is:

  for D=0L, NTracers-1L do begin
     
     Success = CTM_Make_DataInfo( Data[*,*,D], DataInfo, FileInfo, ..., /No_Global )
     
     ArrDataInfo = D eq 0l ? [ DataInfo ] : [ ArrDataInfo, DataInfo ]
     
     if D ne Ntracers-1l then ptr_free, Fileinfo.gridinfo
     
  endfor
     
  CTM_WriteBpch, ArrDataInfo, FileInfo, FileName=OutFileName
     
  for d=0, n_elements(ArrDataInfo)-1l do ptr_free, ArrDataInfo[d].data
  ptr_free, fileinfo.gridinfo

--phs 10:28, 13 February 2008

UNDEFINE

The UNDEFINE (as of GAMAP v2.13) routine is used to free memory. When called on a DataInfo structure, it just undefines the datainfo variable, but not the data pointer inside it, at the risk of memory leak if the datainfo.data pointer points to some data. This is opposite to the expected effect! Note that CTM_CLEANUP is doing a fine job at unallocating gamap pointers when needed, as long as the datainfo structure have not been undefined. In GAMAP, most calls to UNDEFINE, if not all, are probably not needed.

Note that an updated UNDEFINE routine is available at D. Fanning site. This update does free pointers within a structure, thus avoiding memory leak. Its usage in GAMAP is being tested, but not part of GAMAP v2.13.

Please also see this wiki post about using UNDEFINE with CTM_GET_DATABLOCK.

--phs 17:03, 8 October 2009 (EDT)
--Bob Y. 09:56, 26 January 2012 (EST)

GAMAP routines for statistical analysis

GAMAP ships with several routins for computing various statistical quantities:

CUM_TOTAL: Computes the cumulative total of an array.
MEAN: Computes the mean value of an array of any # of dimensions.
ORG_CORR: Calculates the reduced major-axis.
PERCENTILES: Computes percentiles of an array
QQNORM: Sorts a data array, assigns assign actual "probability" and calculates the expected deviation from the mean.
RUN_AV: Computes the running average or running total of a data vector.

Also, the IDL routine MOMENT computes the 4 statistical moments of a data set: mean, variance, skewness, and kurtosis.

--Bob Y. 09:42, 8 July 2008 (EDT)

Defining high-resolution grids

0.5 x 0.666 grids

Win Trivitayanurak wrote:

I'm looking for IDL code (i.e. codes in GAMAP package) that works for 0.5° x 0.667° resolution but apparently didn't find one in my gamap2 package.

Specifically i have to regrid the restart file from 4° x 5° to 0.5° x 0.667° resolution China nested-grid domain (for GEOS-5 47L vertical res.). But it seems to me that other utility codes like ctm_type.pro does not have any info about the 0.5 x 0.66 resolution too.

Bob Yantosca wrote:

You can specify the 0.5 x 0.667 grid with

   InType = CTM_TYPE( MODELNAME='GEOS5_47L', RES=[2D/3D, 1D/2D] ... )
   InGrid = CTM_GRID( InType )

etc. The trick is that you need to pass the resolution string as fractions of double precision #'s to get the lats & lons specified correctly.

You will probably have to define a global 0.5° x 0.667° grid for the regridding (i.e. pass that to REGRIDH_RESTART) and then cut the global file down to size with the CREATE_NESTED etc. routines (located in the gamap2/regridding subdirectories).

In GAMAP v2.16 and higher versions you now have the option of using the shortcut:

   InType = CTM_TYPE( MODELNAME='GEOS5_47L', RES=0.667 )

which will set the resolution for 0.5° x 0.666° automatically.

--Bob Y. 10:55, 25 April 2012 (EDT)

0.25 x 0.3125 grids

The GEOS-FP data product has native resolution of 1/4° x 5/16° (i.e. 0.25° x 0.3125°). As described above, you can specify this grid using double precision numbers:

   InType = CTM_TYPE( MODELNAME='GEOS5_47L', RES=[5D/16D, 1D/4D] ... )
   InGrid = CTM_GRID( InType )

or:

   InType = CTM_TYPE( MODELNAME='GEOS5_47L', RES=[0.3125D, 0.25D] ... )
   InGrid = CTM_GRID( InType )

or you may use the following shortcut in CTM_TYPE:

   InType = CTM_TYPE( MODELNAME='GEOS5_47L', RES=0.25 ... )

--Bob Y. 10:08, 20 August 2013 (EDT)