GEOS-Chem HP

About High Performance GEOS-Chem

GEOS-Chem with the high performance option (GCHP) is the next generation of GEOS-Chem. It uses a distributed memory capability designed to enable efficient scaling across many cores, and thus enable finer resolution simulations. GCHP has the advantage of using a cubed sphere geometry that enables more accurate transport and eliminates the polar singularity inherent to lat-lon grids. Despite these advances you can still run the model on a single machine at coarse resolution if you choose to do so.

If you are working on a project using GCHP, please add your name and project description to the project table on the GEOS-Chem High Performance Working Group wiki page (see link below). If you would like to stay informed of GCHP developments, please join the GEOS-Chem High Performance Working Group mailing list. We also encourage you to join our GCHP Slack workspace.

Please contact the GEOS-Chem Support Team with GCHP questions and feedback.

GCHP Wiki Guide

Getting Started with GCHP

• Hardware and Software Requirements
• GCHP Online Manual
• v11-02 Tutorial

Versions and Validation

• GEOS-Chem HP v11-02
• v11-02 Benchmarks
• GEOS-Chem Model Development Priorities

Community

• GCHP Working Group
• GCHP on Slack

Help Resources and Quick Links

• Frequently Asked Questions
• Verifying a Successful Run
• Fortran tool for regridding between cubed-sphere and lat-lon created by Sebastian Eastham
• Introduction to git

GCHP Cubed Sphere Grid Geometry

GCHP uses a cubed sphere grid rather than the traditional lat-lon grid used in GEOS-Chem Classic. While regular lat-lon grids are typically designated as ΔLat ⨉ ΔLon (e.g. 4⨉5), cubed sphere grids are designated by the side-length of the cube. In GCHP we specify this as CX (e.g. C24 or C180). The simple rule of thumb for determining the roughly equivalent lat/lon for a given cubed sphere resolution is to divide the side length by 90. Using this rule you can quickly match C24 with 4x5, C90 with 1 degree, C360 with quarter degree, and so on.

Harvard graduate student Jiawei Zhuang created the following interactive illustrations demonstrating key features of the GCHP cubed sphere geometry:

• Step-by-step illustration on how to create a cubed sphere grid
• Comparison of different cubed sphere map projections
• Creating a "stretched" cubed sphere
  • This is a way of getting fine resolution in a selected region of the world (e.g. akin to GEOS-Chem "Classic" nested-grid simulations)

Using a cubed sphere grid allows for natural parallelization. At least one unique core is assigned to each face on the cubed sphere, resulting in a constraint of at least 6 cores to run GCHP. The same number of cores must be assigned to each face, resulting in another constraint of total number of cores being a multiple of 6. Communication between the cores occurs only during transport processes.

While any number of cores is valid as long as it is a multiple of six, you will typically start to see negative effects due to excessive communication if a core is handling less than around one hundred grid cells or a cluster of grid cells that are not approximately square. In our standard default 6-core c24 configuration, each face is handled by one core (6 faces / 6 cores) and contains 576 cells (24x24). Each core therefore processes 576 cells. Since each core handles on face, each core communicates with four other cores (four surrounding faces).

You can configure approximately how the cores are assigned to grid cells by using the NX and NY configuration variables in GCHP. Imagine stacking the six faces on top of each other to get a single rectangle. The rectangle will have N grid cells width (e.g. 24 if a C24 grid), and 6N grid cells height (since 6 faces). NX is the number of segments the width is broken into for core distribution, while NY is the number of segments the height is broken into. NX * NY is always the total number of cores. Our simple example of 6 cores handling the entire globe is equivalent to NX=1 and NY=6 since the entire N grid cells width is handled by 1 core (NX) and the 6N grid cells height is handled by 6 cores (NY). If you instead wanted each face to be handled by four cores, and further constrain each core to handle one quadrant, you would set NX=2 and NY=12. A simply way of thinking about this is that core distribution across each face is with geometry NX x NY/6. In this last example that would be 2x2 which is how to split a square into quandrants.

---

--Lizzie Lundgren (talk) 14:40, 28 June 2018 (UTC)