PGI Fortran compiler

---

---

Support for the PGI Fortran compiler (pgfortran) has been removed in GEOS-Chem 12.6.0 and later versions. This is because of two reasons:

1. We are migrating towards open-source software (using the GNU gfortran compiler)
2. GCHP is only compatible with ifort and gfortran compilers.

--Bob Yantosca (talk) 14:54, 18 July 2019 (UTC)

---

---

Overview

Please see the Portland Group Int'l vendor website for PGI Fortran compiler documentation. The PGI Fortran compiler is also known as pgfortran.

PGI Fortran versions tested with GEOS-Chem

The GEOS-Chem Support Team has experimented using different versions of GNU Fortran to build GEOS-Chem. Here are our results.

Version	Machine	Status
pgfortran 14.10	odyssey.rc.fas.harvard.edu	GEOS-Chem builds into an executable. GEOS-Chem simulations run to completion. We have not yet been able to validate the results of simulations done with PGI Fortran w/r/t simulations done with Intel Fortran. We hope to be able to do this in the near future.

We invite you to test with other versions of PGI Fortran. Please send your results to the GEOS-Chem Support Team.

--Bob Yantosca (talk) 15:03, 5 October 2016 (UTC)

Environment settings for PGI Fortran

Here is some information about how you can customize your Unix environment to use the PGI Fortran compiler.

Using the module command to load PGI Fortran and related libraries

On most Unix clusters, the module command is used to load the PGI Fortran compiler library into your Unix enviroment, along with any related libraries (e.g. netCDF, OpenMPI). For example, we use these commands on the Harvard cluster (odyssey.rc.fas.harvard.edu):

 # These commands load PGI Fortran 14 
 module load pgi/14.10-fasrc01 openmpi/1.10.0-fasrc01 netcdf/4.1.3-fasrc05

You can ask your IT staff what the corresponding commands would be on your particular cluster.

The module command should automatically define several environment variables for you:

Variable	Expected setting	Description
FC	pgfortran	Name of the PGI Fortran compiler
CC	pgcc	Name of the PGI C compiler
CXX	pgcxx	Name of the PGI C++ compiler
NETCDF_HOME	System-dependent	Path to the root netCDF folder
NETCDF_INCLUDE	System-dependent	Path to the netCDF include folder (e.g. $NETCDF_HOME/include)
NETCDF_LIB	System-dependent	Path to the netCDF library folder (e.g. $NETCDF_HOME/lib or $NETCDF_HOME/lib64)
NETCDF_FORTRAN_HOME	System-dependent	Path to the root netCDF Fortran folder
NETCDF_FORTRAN_INCLUDE	System-dependent	Path to the netCDF Fortran include folder (e.g. $NETCDF_FORTRAN_HOME/include)
NETCDF_FORTRAN_LIB	System-dependent	Path to the netCDF Fortran library folder (e.g. $NETCDF_FORTRAN_HOME/lib or $NETCDF_FORTRAN_HOME/lib64)

If these variables are not automatically set by the module command on your system (or if your system does not use the module command), then:

• Set the FC, CC, and CXX variables manually in your startup script (e.g. .bashrc or .cshrc).
• Ask your IT staff where the netCDF library paths are located, and set the NETCDF_HOME, NETCDF_INCLUDE, and NETCDF_LIB environment variables accordingly.

Depending on your system, you may find that the netCDF Fortran libraries are installed into a different folder than the rest of the netCDF library files. If this is the case, then the module command should automatically define the NETCDF_FORTRAN_HOME, NETCDF_FORTRAN_INCLUDE, and NETCDF_FORTRAN_LIB environment variables. If not, then ask your IT staff what the proper paths are so that you can set these variables manually.

--Bob Yantosca (talk) 14:58, 5 October 2016 (UTC)

Requesting sufficient stack memory for GEOS-Chem

In order to run GEOS-Chem with PGI Fortran, you must request the maximum amount of stack memory in your Unix environment. (The stack memory is where local automatic variables and temporary !$OMP PRIVATE variables will be created.) Add the following lines to your system startup file:

If you use bash. add this to your .bashrc file	If you use csh or tcsh, add this to your .cshrc file
ulimit -s unlimited export OMP_STACKSIZE=500m	limit stacksize unlimited setenv OMP_STACKSIZE 500m

The ulimit -s unlimited (for bash) or limit stacksize unlimited commands tell the Unix shell to use the maximum amount of stack memory available.

The environment variable OMP_STACKSIZE must also be set to a very large number. In this example, we are nominally requesting 500 MB of memory. But in practice, this will tell the GNU Fortran compiler to use the maximum amount of stack memory available on your system. The value 500m is a good round number that is larger than the amount of stack memory on most computer clusters, but you can change this if you wish.

NOTE: Setting the OMP_STACKSIZE environment variable will make it easier to switch between different compilers on your system. The KMP_STACKSIZE environment variable only works with the Intel Fortran Compiler but not with GNU Fortran.

--Bob Yantosca (talk) 14:58, 5 October 2016 (UTC)

Optimization

In this section we provide information about the optimization options that you can use with the PGI Fortran compiler.

Optimization options

Here is a quick reference table of pgfortran's optimization options (taken from the online PGI Compiler User's Guide.

Option	Description	How invoked in GEOS-Chem?
	Optimization flags
-O0	Level zero specifies no optimization. A basic block is generated for each language statement. At this level, the compiler generates a basic block for each statement. Performance will almost always be slowest using this optimization level. This level is useful for the initial execution of a program. It is also useful for debugging, since there is a direct correlation between the program text and the code generated. To enable debugging, include -g on your compile line.	DEBUG=yes or OPT=-O0
-O1	Level one specifies local optimization. Scheduling of basic blocks is performed. Register allocation is performed. Local optimization is a good choice when the code is very irregular, such as code that contains many short statements containing IF statements and does not contain loops (DO or DO WHILE statements). Although this case rarely occurs, for certain types of code, this optimization level may perform better than level-two (–O2). The PGI compilers perform many different types of local optimizations, including but not limited to: Algebraic identity removal Constant folding Common subexpression elimination Local register optimization Peephole optimizations Redundant load and store elimination Strength reductions	OPT=-O1
-O	When no level is specified, level two global optimizations are performed, including traditional scalar optimizations, induction recognition, and loop invariant motion. No SIMD vectorization is enabled.	OPT=-O
-O2	Level two specifies global optimization. This level performs all level-one local optimization as well as level two global optimization described in –O. In addition, more advanced optimizations such as SIMD code generation, cache alignment, and partial redundancy elimination are enabled. The -O or –O2 level performs all level-one local optimizations as well as global optimizations. Control flow analysis is applied and global registers are allocated for all functions and subroutines. Loop regions are given special consideration. This optimization level is a good choice when the program contains loops, the loops are short, and the structure of the code is regular. The PGI compilers perform many different types of global optimizations, including but not limited to: Branch to branch elimination Constant propagation Copy propagation Dead store elimination Global register allocation Induction variable elimination Invariant code motion	Default setting
-fast	The -fast option generally specifies global optimization; however, the -fast switch varies from release to release, depending on a reasonable selection of switches for any one particular release. The –Mvect option (see below) is included as part of -fast on all x86 and x64 targets. If your program contains computationally-intensive loops, the –Mvect option may be helpful. If in addition you specify -Minfo, and your code contains loops that can be vectorized, the compiler reports relevant information on the optimizations applied.	OPT=-fast
-O3	Level three specifies aggressive global optimization. This level performs all level-one and level-two optimizations and enables more aggressive hoisting and scalar replacement optimizations that may or may not be profitable.	OPT=-O3
-O4	Level four performs all level-one, level-two, and level-three optimizations and enables hoisting of guarded invariant floating point expressions.	OPT=-O4
	Vectorization flags
-Mvect	When a PGI compiler command is invoked with the –Mvect option, the vectorizer scans code searching for loops that are candidates for high–level transformations such as loop distribution, loop interchange, cache tiling, and idiom recognition (replacement of a recognizable code sequence, such as a reduction loop, with optimized code sequences or function calls). When the vectorizer finds vectorization opportunities, it internally rearranges or replaces sections of loops (the vectorizer changes the code generated; your source code’s loops are not altered). In addition to performing these loop transformations, the vectorizer produces extensive data dependence information for use by other phases of compilation and detects opportunities to use vector or packed Streaming SIMD Extensions (SSE) instructions on processors where these are supported. The –Mvect option can speed up code which contains well-behaved countable loops which operate on large REAL, REAL(4), REAL(8), INTEGER, INTEGER(4), COMPLEX(4) or COMPLEX(8) arrays in Fortran and their C/C++ counterparts. However, it is possible that some codes will show a decrease in performance when compiled with the –Mvect option due to the generation of conditionally executed code segments, inability to determine data alignment, and other code generation factors. For this reason, it is recommended that you check carefully whether particular program units or loops show improved performance when compiled with this option enabled.	OPT=-fast

--Bob Yantosca (talk) 21:14, 25 January 2016 (UTC)

Recommended compilation and optimization options for GEOS-Chem

Here are the pgfortran compilation options currently used by GEOS-Chem:

Option	Description	How invoked in GEOS-Chem?
	Normal compiler settings
-byteswapio	Specifies that the format will be big endian for integer data and big endian IEEE floating-point for real and complex data. This only affects file I/O to/from binary files (such as binary punch files) but not ASCII, netCDF, or other file formats.	Default setting
-Kieee	Adheres to the IEEE standard for floating-point operations (especially division). This should result in more precise floating-point computations.	Default setting
-Mfree	Tells pgfortran to expect a file in Fortran-90 free-format layout.	Done automatically in Makefile_header.mk
-Minfo	You can use the -Minfo option to display compile–time optimization listings. When this option is used, the PGI compilers issue informational messages to standard error (stderr) as compilation proceeds. From these messages, you can determine which loops are optimized using unrolling, SSE instructions, vectorization, parallelization, interprocedural optimizations and various miscellaneous optimizations. You can also see where and whether functions are inlined.	INFO=yes
-Mpreprocess	Turns on the C-preprocessor, to evaluate #if and #define statements in the source code.	Default setting
-O2	Optimizes the source code for speed, without taking too many liberties with numerical precision. For more information, please see the optimization options section above.	Default setting
	Special compiler settings
-r8	This option tells the compiler to treat variables that are declared as REAL as REAL*8 (as opposed to REAL*4. NOTE: This option is not used globally, but is only applied to certain indidvidual files (mostly from third-party codes like ISORROPIA. Current GEOS-Chem programming practice is to use either REAL*4 or REAL*8 instead of REAL, which avoids confusion.	Used as needed
-mcmodel=medium	This option is used to tell pgfortran to use more than 2GB of static memory. This avoids a specific type of memory error that can occur if you compile GEOS-Chem for use with an extremely high-resolution grid (e.g. 0.25° x 0.3125° nested grid).	Default setting
	Settings only used for debugging
-g	Tells the compiler to generate full debugging information in the object file. This will cause a debugger (like Totalview) to display the actual lines of source code, instead of hexadecimal addresses (which is gibberish to anyone except hardware engineers).	DEBUG=yes
-O0	Turns off all optmization. Source code instructions (e.g. DO loops, IF blocks) and numerical expressions are evaluated in precisely the order in which they are listed, without being internally rewritten by the optimizer. This is necessary for using a debugger (like Totalview).	DEBUG=yes
-Mbounds (aka -C)	Check for array-out-of-bounds errors. This is invoked when you compile GEOS-Chem with the BOUNDS=yes Makefile option. NOTE: Only use -Mbounds for debugging, as this option will cause GEOS-Chem to execute more slowly!	BOUNDS=yes
-traceback	This option tells the compiler to generate extra information in the object file to provide source file traceback information when a severe error occurs at run time. When the severe error occurs, source file, routine name, and line number correlation information is displayed along with call stack hexadecimal addresses (program counter trace). This option increases the size of the executable program, but has no impact on run-time execution speeds. It functions independently of the debug option.	TRACEBACK=yes
-Ktrap=fp	This option will cause GEOS-Chem to halt if any type of floating-point error is encountered. This can happen if an equation results in a denormal value, e.g. NaN, or +/-Infinity. Common causes of floating-point errors are divisions where the denominator becomes zero.	FPE=yes

--Bob Yantosca (talk) 20:46, 25 January 2016 (UTC)

Updates for GEOS-Chem v11-01

The following updates for the pgfortran compiler were made to GEOS-Chem v11-01f:

Compilation options settings

Module

Solution

Makefile_header.mk

The default value of Makefile variable $(COMPILER) now is taken from the $FC environment variable. NOTE: $FC is set in the .bashrc or when a module is loaded. Made the test for the PGI fortran compilers more robust (i.e. it should now work for pgf77, pgf90, or pgfortran.) Added -Kieee to specify IEEE floating-point math. Added -Ktrap=fp for the FPE=yes option. This should be analogous to the ifort -fpe0 -ftrapuv setting.

--Bob Yantosca (talk) 20:02, 25 January 2016 (UTC)

Fixes for syntax errors

Module	Solution
GeosCore/geos_timers_mod.F90	Added missing & continuation characters.
HEMCO/Core/hco_config_mod.F90	Removed an extraneous & character in the middle of PRINT statement. The pgfortran compiler assumes that & signals the end of a line of code.

--Bob Yantosca (talk) 19:30, 25 January 2016 (UTC)

Fixes for file I/O errors

Module	Solution
GeosUtil/file_mod.F	Added subroutine LINE_BUFFER, which calls intrinsic routine SETVBUF3F. Routine LINE_BUFFER will cause a Fortran logical file to be written to disk once each line is completed, instead of flushing all output to disk at the end of the simulation. (This is pgfortran's default behavior.
GeosCore/main.F	Added a call to LINE_BUFFER (from GeosUtil/file_mod.F) at the very top of main.F. This will cause the Unix stdout stream (i.e. output from all PRINT and WRITE statements) to be written to disk on a line-by-line basis.
GeosCore/readchem.F	Added a call to LINE_BUFFER (from GeosUtil/file_mod.F) after unit IO93 is opened. This will cause the smv2.log file to be written to disk on a line-by-line basis.
HEMCO/Core/hco_error_mod.F90	Added calls to intrinsic function SETVBUF3F routine after opening the HEMCO.log file. This will cause the HEMCO.log file to be written to disk on a line-by-line basis. NOTE: SETVBUF3F is called directly to avoid a dependence on GeosCore/file_mod.F in the standalone HEMCO code.

--Bob Yantosca (talk) 19:41, 25 January 2016 (UTC)

Fixes for other errors

Module	Solution
Headers/gigc_state_chm_mod.F90	In routine INIT_GIGC_STATE_CHM, declare the Input_Opt object with INTENT(INOUT). This is necessary because we overwrite the molecular weight fields of Input_Opt with values from the GEOS-Chem species database object here.
HEMCO/Core/hco_diagn_mod.F90	In routine DiagnList_Cleanup, check that the TmpCont object is associated before trying to deallocate it, i.e.: IF ( ( ASSOCIATED( TmpCont ) ) DEALLOCATE( TmpCont )

--Bob Yantosca (talk) 19:48, 25 January 2016 (UTC)

Updates for GEOS-Chem v10-01

Gan Luo (SUNY Albany) wrote:

Last night, I quickly tried to compile the [provisional release] version of v10-01 with PGI. However, there are numbers of errors during the compilation associated with HEMCO codes.

   PGF90-S-0034-Syntax error at or near identifier result (hco_clock_mod.F90: 1248)
   PGF90-S-0310-Illegal statement in the specification part of a MODULE  (hco_clock_mod.F90: 1266)
   PGF90-S-0310-Illegal statement in the specification part of a MODULE  (hco_clock_mod.F90: 1274)
   PGF90-S-0310-Illegal statement in the specification part of a MODULE  (hco_clock_mod.F90: 1276)

Bob Yantosca replied:

I was able to fix the problems with the PGI compiler. These were mostly syntax errors:

Module	Issue	Solution
Makefile_header.mk	Syntax errors	Changed #IF DEFINED( USE_REAL8 )<tt> to <tt>#if defined( USE_REAL8 ) The PGI compiler expects #ifdef statements to be in lower case.
GeosCore/fast_jx_mod>F	Syntax errors	Added the following code: #if defined( UCX ) CHARACTER*30  :: SPECFIL(8) #else CHARACTER*30  :: SPECFIL(6) #endif We needed to increase the length of SPECFIL from 6 to 8 for the UCX simulation, so that we will have as many entries in a DATA statement as there are elements in the SPECFIL array. The PGI compiler expects this.
GeosCore/physproc.F	Syntax errors	FUNCTION HCOX_SoilNOx_GetFertScale had to be changed to FUNCTION HCOX_SoilNOx_GetFertScale() The PGI compiler expects routine names taking no arguments to always end in ().
HEMCO/Core/hco_clock_mod.F90	Syntax errors	Changed FUNCTION HcoClock_SetMinResetFlag to FUNCTION HcoClock_SetMinResetFlag() Changed FUNCTION HcoClock_GetMinResetFlag to FUNCTION HcoClock_GetMinResetFlag() The PGI compiler expects routine names taking no arguments to always end in ().
HEMCO/Extensions/hcox_soilnox_mod.F90	Syntax errors	Changed FUNCTION HCOX_SoilNOx_GetFertScale to FUNCTION HCOX_SoilNOx_GetFertScale() The PGI compiler expects routine names taking no arguments to always end in ().

Also note that I made some extra modifications to the GEOS-Chem Makefile_header.mk file as part of this update.

--Bob Y. 13:06, 13 May 2015 (EDT)

Gan Luo (SUNY Albany) responded:

I finished the compilation of GEOS-Chem v10-1 with PGI compiler under our system. Some warning massages have been fixed during the compilation. The list of the patches has been attached.

messy_ncregrid_base.F90
Line 24-26:
!GanLuo+  PRIVATE   :: ASSOCIATED, PRESENT, PRODUCT, REAL, SIZE &
!GanLuo+             , INT, ABS, DBLE, MAX, MIN, SIGN, TRIM     &
!GanLuo+             , MAXVAL, MINVAL, IAND, NULL

hcoio_dataread_mod.F90
Line 926-927:
      !GanLuo+ncYr  = FLOOR( MOD(oYMDh1,10000000000) / 1.0d6 )
      !GanLuo+ncMt  = FLOOR( MOD(oYMDh1,1000000)     / 1.0d4 )
      ncYr  = FLOOR( MOD(oYMDh1*1.d0,10000000000.d0) / 1.0d6 )
      ncMt  = FLOOR( MOD(oYMDh1*1.d0,1000000.d0)     / 1.0d4 )

Line 1823-1826
   !GanLuo+origYr = FLOOR( MOD(prefYMDh, 10000000000) / 1.0d6 )
   !GanLuo+origMt = FLOOR( MOD(prefYMDh, 1000000    ) / 1.0d4 )
   !GanLuo+origDy = FLOOR( MOD(prefYMDh, 10000      ) / 1.0d2 )
   !GanLuo+origHr = FLOOR( MOD(prefYMDh, 100        ) / 1.0d0 )
   origYr = FLOOR( MOD(prefYMDh*1.d0, 10000000000.d0) / 1.0d6 )
   origMt = FLOOR( MOD(prefYMDh*1.d0, 1000000.d0    ) / 1.0d4 )
   origDy = FLOOR( MOD(prefYMDh*1.d0, 10000.d0      ) / 1.0d2 )
   origHr = FLOOR( MOD(prefYMDh*1.d0, 100.d0        ) / 1.0d0 )

Line 2313-2316
   !GanLuo+hrs = FLOOR( MOD(YMDh, 10000000000) / 1.0d6 ) * 8760 + &
   !GanLuo+      FLOOR( MOD(YMDh, 1000000    ) / 1.0d4 ) * 720  + &
   !GanLuo+      FLOOR( MOD(YMDh, 10000      ) / 1.0d2 ) * 24   + &
   !GanLuo+      FLOOR( MOD(YMDh, 100        ) / 1.0d0 )
   hrs = FLOOR( MOD(YMDh*1.d0, 10000000000.d0) / 1.0d6 ) * 8760 + &
         FLOOR( MOD(YMDh*1.d0, 1000000.d0    ) / 1.0d4 ) * 720  + &
         FLOOR( MOD(YMDh*1.d0, 10000.d0      ) / 1.0d2 ) * 24   + &
         FLOOR( MOD(YMDh*1.d0, 100.d0        ) / 1.0d0 )

hcox_paranox_mod.F90
Line 2202:
  !GanLuo+VALUE = MAX( VALUE,   MINVAL( NODES ) )
  VALUE = MAX( VALUE,   MINVAL( NODES )*1.d0 )

isoropiaIIcode.F
Warning of the use of FLOAT: change FLOAT to REAL

modis_lai_mod.F90
Line 207:
   !GanLuo+TYPE(MetState),  INTENT(IN)  :: State_Met     ! Meteorology State object
   TYPE(MetState),  INTENT(INOUT)  :: State_Met     ! Meteorology State object

ocean_mercury_mod.F
Line 1095-1097:
           !GanLuo+NPP(I,J) = MAX ( NPP(I,J) , NPPMINNUM )

           !GanLuo+CHL(I,J)    = MAX ( CHL(I,J) , CHLMINNUM )

           NPP(I,J) = MAX ( NPP(I,J)*1.d0 , NPPMINNUM )

           CHL(I,J)    = MAX ( CHL(I,J)*1.d0 , CHLMINNUM )

mercury_mod.F
Line 566:
     !GanLuo+CHARACTER(LEN=*), PARAMETER  :: METHOD='GoodsiteUpdate' \
     CHARACTER(LEN=*), PARAMETER  :: METHOD='GoodsiteUpdate'

tpcore_geos5_window_mod.F90 and tpcore_geosfp_window_mod.F90
Warning of the use of FLOAT: change FLOAT to REAL

--Melissa Sulprizio (talk) 19:56, 1 June 2015 (UTC)

Previous issues that have now been resolved

File Linking Error

These fixes were validated with 1-month benchmark simulation v11-01b and 1-year benchmark simulation v11-01b-Run0. This version was approved on 19 Aug 2015.

Zahra Hosseini (RWDI AIR Inc.) wrote:

I am trying to compile the GEOS-Chem v10-01 using: make MET=geosfp GRID=4x5 COMPILER=pgi but I get the following error:

       PGF90-F-0004-Unable to open MODULE file inquiremod.mod (hco_error_mod.F90: 728)
       PGF90/x86-64 Linux 14.7-0: compilation aborted
       make[6]: *** [hco_error_mod.o] Error 2

However, the following modifications in the Makefile_header.mk seems to magically resolve my problem! I am not sure why!

1. removing -Bstatic option in lines 984 to 991:

       # Pick compiler options for debug run or regular run
       REGEXP             := (^[Yy]|^[Yy][Ee][Ss])
       ifeq ($(shell [ [ "$(DEBUG)" =~ $(REGEXP) ] ] && echo true),true)
         FFLAGS           :=-byteswapio -Mpreprocess -g -O0
         USER_DEFS        += -DDEBUG
       else
         FFLAGS           :=-byteswapio -Mpreprocess $(OPT)
       endif

2. commenting the last lines:

     #headerinfo:
     #       @@echo '####### in Makefile_header.mk ########'
     #       @@echo "COMPILER    : $(COMPILER)"
     #       @@echo "DEBUG       : $(DEBUG)"
     #        @@echo "BOUNDS      : $(BOUNDS)"
     #       @@echo "F90         : $(F90)"
     #       @@echo "CC          : $(CC)"
     #       @@echo "INCLUDE     : $(INCLUDE)"
     #       @@echo "LINK        : $(LINK)"
     #       @@echo "USERDEFS    : $(USER_DEFS)"
     #       @@echo "NC_INC_CMD  : $(NC_INC_CMD)"
     #       @@echo "NC_LINK_CMD : $(NC_LINK_CMD)"

So now I run using:

    make -j8  MET=geos-fp GRID=4x5 COMPILER=pgi

and this time I do not get any errors!

--Lizzie Lundgren (talk) 19:01, 17 July 2015 (UTC)

Compatibility issues added in v9-01-02

Please see this list of minor bugs that were corrected in order to get the GEOS-Chem code to compile with the PGI compiler.

--Bob Y. 10:04, 23 February 2012 (EST)

Error with ADJUSTL and ADJUSTR

Win Trivitayanurak wrote:

In short, TRIM and ADJUSTL or ADJUSTR do not work together properly when compiled with Portland Group Fortran. I propose removing TRIM inside the subroutine StrSqueeze. This is not urgent and relevant to only the few PGI users.

So if you are using the PGI compiler, then you will have to modify the code in routine STRSQUEEZE "charpak_mod.f" such that the statements

STR = ADJUSTR( TRIM( STR ) )
STR = ADJUSTL( TRIM( STR ) )

are now replaced with

STR = ADJUSTR( STR )
STR = ADJUSTL( STR )

and this will solve the problem. We will incorporate this into a future release of GEOS-Chem.

Setting the stacksize

You may encounter an "out-of-memory" error (which may manifest itself as a segmentation fault) if your simulation uses a fine-resolution grid and/or a large number of advected tracers. This error can occur if your simulation is not using all of the available stack memory. Recall that the OpenMP parallelization utilizes the stack memory for storage of PRIVATE and THREADPRIVATE variables.

You can usually solve this problem by telling your simulation to use all of the available stack memory on the system. Here is an example script, intended for use with the PGI compiler:

Fangqun Yu wrote:

Attached is a csh script (also copied below) we use to run the APM aerosol microphysics within GEOS-Chem. It works on our 8-core Linux machine with pgi compiler for all GEOS5 4x5, 2x2.5, and nested grid simulations.

   #!/bin/csh
   setenv NCPUS 8
   setenv MPSTKZ 1024M
   limit stacksize unlimited
   geos

NOTES:

1. The limit stacksize unlimited command will set the Unix stack size memory to its maximum value.
2. The PGI compiler environment variable MPSTKZ increases the stack size for threads executing in parallel regions. Please see the Online PGI compiler manual for more information.

--Bob Y. 15:49, 14 December 2010 (EST)