Difference between revisions of "Setting Up the GCHP Environment"

Latest revision as of 15:40, 8 December 2020

The GCHP documentation has moved to https://gchp.readthedocs.io/. The GCHP documentation on http://wiki.seas.harvard.edu/ will stay online for several months, but it is outdated and no longer active!

Previous | Next | Getting Started with GCHP | GCHP Main Page

1. Hardware and Software Requirements
2. Setting Up the GCHP Environment
3. Downloading Source Code and Data Directories
4. Compiling
5. Obtaining a Run Directory
6. Running GCHP: Basics
7. Running GCHP: Configuration
8. Output Data
9. Developing GCHP
10. Run Configuration Files

Create an Environment File

You must load all necessary libraries and export certain environment variables before compiling and running GCHP. We recommend simplifying the environment setup process by customizing one or more GCHP-specific environment files that work on your system and storing them in a central location for repeated use.

An example of an environment file is below. This particular file was also used to build ESMF and therefore includes ESMF-only environment variables as well as those required by GCHP. It also sets certain system limits to avoid potential problems with memory.

if  $- = *i*  ; then
  echo "Loading modules, please wait ..."
fi

#==============================================================================
# Load modules
#==============================================================================

module purge
module load git/2.17.0-fasrc01
module load intel/18.0.5-fasrc01
module load openmpi/4.0.1-fasrc01
module load netcdf-fortran/4.5.2-fasrc01
module load cmake/3.16.1-fasrc01

#==============================================================================
# Environment variables
#==============================================================================

# Make all files world-readable by default
umask 022

# Specify compilers
export CC=gcc
export CXX=g++
export FC=ifort

# MPI
export MPI_ROOT=${MPI_HOME}

# For ESMF
export ESMF_COMPILER=intel
export ESMF_COMM=openmpi
export ESMF_DIR=/n/jacob_lab/Lab/ESMF/ESMF_8_0_1
export ESMF_INSTALL_PREFIX=${ESMF_DIR}/INSTALL_ifort18_openmpi4

# For GCHP
export ESMF_ROOT=${ESMF_INSTALL_PREFIX}

#==============================================================================
# Set limits
#==============================================================================

ulimit -c 0                      # coredumpsize
ulimit -l unlimited              # memorylocked
ulimit -u 50000                  # maxproc
ulimit -v unlimited              # vmemoryuse
ulimit -s unlimited              # stacksize

#==============================================================================
# Print information
#==============================================================================

module list
echo ""
echo "Environment:"
echo ""
echo "CC: ${CC}"
echo "CXX: ${CXX}"
echo "FC: ${FC}"
echo "ESMF_COMM: ${ESMF_COMM}"
echo "ESMF_COMPILER: ${ESMF_COMPILER}"
echo "ESMF_DIR: ${ESMF_DIR}"
echo "ESMF_INSTALL_PREFIX: ${ESMF_INSTALL_PREFIX}"
echo "ESMF_ROOT: ${ESMF_ROOT}"
echo "MPI_HOME: ${MPI_HOME}"
echo "NetCDF C: $(nc-config --prefix)"
echo "NetCDF Fortran: $(nf-config --prefix)"
echo ""
echo "Done sourcing ${BASH_SOURCE[0]}"

System memory limits and stack size should be set to unlimited to avoid memory problems. Such problems would manifest as sudden termination upon file read or a segmentation fault during advection. You can find out what you system limits are by typing the following at the command prompt:

ulimit -a

You will see something like this:

core file size          (blocks, -c) unlimited
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 1030083
max locked memory       (kbytes, -l) unlimited
max memory size         (kbytes, -m) unlimited
open files                      (-n) 100000
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) unlimited
cpu time               (seconds, -t) unlimited
max user processes              (-u) 4096
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited

Previous | Next | Getting Started with GCHP | GCHP Main Page