GCHP Main Page
About High Performance GEOS-Chem
GEOS-Chem with the high performance option (GCHP) is the next generation of GEOS-Chem. It uses a distributed memory capability designed to enable efficient scaling across many cores, and thus enable finer resolution simulations. GCHP has the advantage of using a cubed sphere geometry that enables more accurate transport and eliminates the polar singularity inherent to lat-lon grids. Despite these advances you can still run the model on a single machine at coarse resolution if you choose to do so.
If you are working on a project using GCHP, please add your name and project description to the GCHP projects list. Also help us pool performance information across systems by contributing your GCHP run information on our GCHP Timing Tests page.
If you would like to stay informed of GCHP developments, please join the GEOS-Chem High Performance Working Group mailing list. We also encourage you to join our GCHP Slack workspace. Please contact the GEOS-Chem Support Team with GCHP questions and feedback.
GCHP Wiki Guide
Getting Started
Version History and Validation
- GEOS-Chem HP v11-02/12.0.0
- v11-02 Benchmarks
- User-reported Timing Information
- GEOS-Chem Model Development Priorities
Community
Help Resources and Quick Links
- Frequently Asked Questions
- Verifying a Successful Run
- Tools for Regridding and Post-processing Output
- Introduction to git
GCHP Cubed Sphere Grid Geometry
GCHP uses a cubed sphere grid rather than the traditional lat-lon grid used in GEOS-Chem Classic. While regular lat-lon grids are typically designated as ΔLat ⨉ ΔLon (e.g. 4⨉5), cubed sphere grids are designated by the side-length of the cube. In GCHP we specify this as CX (e.g. C24 or C180). The simple rule of thumb for determining the roughly equivalent lat/lon for a given cubed sphere resolution is to divide the side length by 90. Using this rule you can quickly match C24 with 4x5, C90 with 1 degree, C360 with quarter degree, and so on.
Harvard graduate student Jiawei Zhuang created the following interactive illustrations demonstrating key features of the GCHP cubed sphere geometry:
- Step-by-step illustration on how to create a cubed sphere grid
- Comparison of different cubed sphere map projections
- Creating a "stretched" cubed sphere
- This is a way of getting fine resolution in a selected region of the world (e.g. akin to GEOS-Chem "Classic" nested-grid simulations)
Using a cubed sphere grid allows for natural parallelization. At least one unique core is assigned to each face on the cubed sphere, resulting in a constraint of at least six cores to run GCHP. The same number of cores must be assigned to each face, resulting in another constraint of total number of cores being a multiple of six. Communication between the cores occurs only during transport processes.
While any number of cores is valid as long as it is a multiple of six, you will typically start to see negative effects due to excessive communication if a core is handling less than around one hundred grid cells or a cluster of grid cells that are not approximately square. You can determine how many grid cells are handled per core by analyzing your grid resolution and resource allocation. For example, if running at C24 with six cores each face is handled by one core (6 faces / 6 cores) and contains 576 cells (24x24). Each core therefore processes 576 cells. Since each core handles one face, each core communicates with four other cores (four surrounding faces).
You can configure approximately how the cores are assigned to grid cell geometry by using the NX and NY configuration variables in GCHP. Imagine lining up the six face grids adjacent to each other to get a single rectangular array. The rectangle will have N grid cells width (e.g. 24 if a C24 grid), and 6N grid cells height (since 6 faces). NX is the number of segments the width N is broken into for core distribution. NY is the number of segments the height 6N is broken into and must always be a multiple of six. NX * NY is always the total number of cores.
For the case of a six core run, NX is equal to 1 and NY is equal to 6. This is because the entire N grid cells width is handled by 1 core (NX) and the 6N grid cells height is handled by 6 cores (NY), or one per face. If you instead wanted each face to be handled by four cores, and further constrain each core to handle one face quadrant, you would set NX equal to two and NY equal to twelve. A simple way of thinking about this is that core distribution across each face is with geometry NX x NY/6. In this last example that would be equivalent to 2x2.
--Lizzie Lundgren (talk) 14:40, 28 June 2018 (UTC)
