Developing GCHP

Previous | Getting Started with GCHP

1. Hardware and Software Requirements
2. Downloading Source Code
3. Obtaining a Run Directory
4. Setting Up the GCHP Environment
5. Compiling
6. Basic Example Run
7. Run Configuration Files
8. Advanced Run Examples
9. Output Data
10. Developing GCHP

Overview

GCHP works as a layer around GEOS-Chem, simulating the more complex environment of a full atmospheric global circulation model (AGCM). Most model updates will involve editing GEOS-Chem source code as you would with GEOS-Chem "classic" (GCC). However, certain updates such as specifying output variables or adding new input fields will require development within the GCHP-specific source code. In addition, sometimes debugging will lead you into the MAPL source code. This page provides an overview of the code structure to help navigate and debug GCHP.

High-level Execution of GEOS-Chem Classic

GCC primarily consists of a single, monolithic code. When running the GEOS-Chem executable geos, the main routine in GeosCore/main.F performs the following functions:

1. Read in simulation settings from input.geos
2. Set up arrays to hold data such as species concentrations and meteorological data
3. Loop though the following steps until the simulation is complete:
   1. Read meteorological data into the State_Met object from pre-determined locations
   2. Calculate emissions in each grid box via HEMCO
   3. Calculate chemistry in each grid box via FlexChem
   4. Calculate transport between grid boxes

Although the code for each of these functions is found in different files (e.g. chemistry_mod.F, transport_mod.F), all of the routines are called from main.F.

High-level Structure of GCHP

The primary difference with GCHP is that main.F is replaced by the GMAO MAPL framework. MAPL provides an interface with ESMF which allows the different components to be entirely unaware of each other's existence and all communication is standardized. The functional flow now looks more like this:

1. Initialize the MAPL_Cap process. MAPL will:
   1. Establish a generic input component, called ExtData
   2. Establish a generic output component, called History
   3. Establish a generic CTM component, called GCHP
2. Determine which modules will be performing which function. In GCHP, GEOS-Chem will calculate chemistry, FV3Dycore will calculate transport, and emissions are completed through HEMCO
3. For each component, send an “Initialize” command
4. Send the “Run” command to the CTM component. The CTM component will loop through the following steps:
   1. Request input data from ExtData
   2. Send a “Run” command to each component in the CTM
   3. Send output data to History
5. Once the CTM component is done, send a “Finalize” command to all components and exit

All ancillary operations, such as data regridding and parallelization, are handled by MAPL. Each core is unaware of the existence of each other core. This means that, in a 6-CPU run, there are six distinct instances of the GEOS-Chem component running; each one will see ⅙ of the available domain, and be fed data as if that domain were all that existed. Components can request data from other domains (e.g. the transport core will request data from adjacent domains) but this communication is all handled through MAPL.

Source Code Structure

GCHP source code can be sub-divided into five parts:

1. GEOS-Chem
2. GCHP wrapper interface routines (GCHP)
3. Cubed-Sphere Finite Volume Dynamical Core (FVDycore)
4. Earth System Modeling Framework (ESMF)
5. NASA-GMAO Mapping, Analysis and Prediction Layer (MAPL)

The GEOS-Chem source code is the same as you would download for running GCC. It contains C preprocessor directives specifying which parts of the code should be compiled in a high performance computing (HPC) environment. You can enable HPC in GEOS-Chem by additionally downloading the GCHP wrapper and storing it in the top-level GEOS-Chem source code directory as a sub-directory called GCHP. The GCHP directory is designed to use GEOS-Chem source code within a set of wrapper functions that interface GEOS-Chem's routines to the ESMF using, in part, MAPL and the HPC-capable cubed-sphere dynamics core FVDycore.

The GCHP directory contains four subdirectories and several Fortran-90 and header files. Description of each are as follows:

• FVdycoreCubed_GridComp: The HPC-capable cubed-sphere dynamics core. GEOS-Chem's serial dynamical core is not capable of operating in a distributed environment, requiring that an HPC-capable version be included within the GCHP system. In 2014, NASA-GMAO made available a stand-alone version of the Finite-Volume Cubed-Sphere Dynamical Core used in GEOS-5, which was adapted to the GCHP and resides within the FVdycoreCubed_GridComp subdirectory. The FV dycore is able to read in meteorological fields in either lat/lon or cubed-sphere formats.
• Shared: Contains both NASA-GMAO's MAPL and Shared library packages used to facilitate coupling between components and provide the primary interface with the ESMF and I/O software. Problems with MAPL will lead you into this directory. However, be aware that error traceback for many configuration file problems will lead you here. Carefully check that your configuration files are properly set before attempting to change MAPL code to fix the issue.
• ESMF: Contains ESMF infrastructure source code for version v5.2.0rp2. See ESMF/README in the source code for more information.
• Registry: Contains information used by MAPL at compile time to generate the Fortran interface between the various quantities needed by GEOS-Chem and the ESMF, MAPL, and FVdycore code.
• *.F90 and *.H files: These Fortran routines and header files replace the GEOS-Chem classic main.F functionality. They also consist of ESMF and MAPL interface code that couples GEOS-Chem routines in an ESMF environment. Scientific developers will in general not be expected to modify these files, as doing so will require some skill with using MAPL and ESMF. Files that begin with gigc_* call the various methods within GEOS-Chem necessary to input, initialize, calculate, and output GEOS-Chem scientific quantities. These represent the direct interface between GEOS-Chem and the GCHP wrapper. gigc_* stands for “Grid Independent GEOS-Chem” and refers to the chemistry component used by both GCHP and the GEOS-5 GCM whereas GCHP refers to its specific implementation as a stand-alone CTM. Files that contain GridComp in the name are ESMF gridded components which define Initialization, Run and Finalize (IRF) routines used to execute the various operations of GEOS-Chem within the ESMF. Within these is code used establish an interface between ESMF and MAPL structures and GEOS-Chem.

Previous | GCHP Home