GEOS-Chem HP Status

From Geos-chem
Jump to: navigation, search

Go to GCHP Main Page

Overview

This page tracks the status of the GEOS-Chem High Performance (GCHP) model, under development since 2011. The GCHP development team is currently wrapping up the science validation phase which will culminate in release of the first GCHP benchmark comparing GEOS-Chem 1-month standard simulation output with and without MPI parallelization. We performed an initial benchmark comparing GCHP with GEOS-Chem classic v11-01 and will perform another benchmark after merging necessary GEOS-Chem classic source updates into v11-02b.

The next major phase of GCHP development is improving usability and performance. You can follow our progress on our software development Trello board at https://trello.com/b/cnNUDhlL/gchp. To get set up with GCHP, follow instructions at the Getting Started With GCHP wiki page.

Validation process

A critical step for validating GCHP is to ensure that it yields the same results as GEOS-Chem Classic (GCC) under controlled simulation conditions. For validation, we chose to isolate a subset of GEOS-Chem components in a series of isolation tests . Once all components were validated in isolation, we ran a series of exclusion tests where all components except one were turned on. Finally, we ran a standard one-month simulation with all components on using default GCHP run directory settings. For all simulations, the following setup is used: The GCHP simulation is performed with the following computational environment and model setup:

  • 6 cores on 1 Jacob Group computational node (Odyssey cluster)
  • Intel Fortran v15.0.0 compiler with MVAPICH2 v2.2 MPI
  • C24 cubed-sphere grid for GCHP (total of 3,456 grid cells on each level)
  • 4x5 latitude-longitude grid for GCC (total of 3,312 grid cells on each level)
  • Driven by GEOS-FP data
  • Simulation period 2013-07-01 00:00:00 UTC to 2013-07-02 00:00:00 UTC for isolation and exclusion tests
  • Simulation period 2013-07-01 00:00:00 UTC to 2013-08-01 00:00:00 UTC for standard 1-month tests

For output comparison, we regridded all GEOS-Chem 4x5 output to 1x1.25 degree resolution. We did the same for GCHP c24 cubed-sphere output. For all isolation and exclusion tests we compared instantaneous concentration results at the end of the 1-day run. For standard 1-month runs we compared both the instantaneous output as well as the 1-month mean.

Isolation test

The component is isolated in both GEOS-Chem Classic and GEOS-Chem High Performance, and a single simulation day is run with an identical restart file. This includes disabling all emissions, unless the emissions component is being specifically tested.

Exclusion test

The component is disabled in both GEOS-Chem Classic and GEOS-Chem High Performance, but all other components remain active.

Model status

GCHP v1.0.0 Benchmark (Comparison with GCC v11-01)

Results of the GCHP v1.0.0 benchmark runs comparing GCHP v1.0.0 to GEOS-Chem v11-01 are stored at the links below.

All comparisons show 1x1.25 degree resolution regridded from the GEOS-Chem 4x5 lat/lon and GCHP c24 cubed-sphere output files. Diagnostics are not yet implemented in GCHP and therefore the v1.0.0 benchmark plots only include species concentration comparisons. As a diagnostic, OH is not included as a species in benchmark plots at this time.

--Lizzie Lundgren (talk) 18:38, 24 April 2017 (UTC)

Pre-benchmark Component Tests

Individual members of the GCHP development team previously tested separate components and the results are stored in this section for reference. These tests helped identify and isolate remaining issues prior to running the preliminary benchmark. All results in this section use preliminary versions leading up to GCHP v1.0.0 and are superceded by the v1.0.0 benchmark.

For each entry below, a more detailed breakdown of the current status can be found by following the links. The column "Next completion date" refers to whichever test is currently underway (either isolation or exclusion). A specific test can be in one of three states: Not Started, WIP, and Complete.

Component Isolation test Developer(s) Version tested
Wet deposition Complete: 1-day Jiawei Zhuang 2017-02-14 (GC_BLEEDING_EDGE GCHP_Dev branch commit 45a3bdb, GCHP Dev branch commit 1aad4dc)
Convection Complete: 1-day Jiawei Zhuang 2017-02-14 (GC_BLEEDING_EDGE GCHP_Dev branch commit 45a3bdb, GCHP Dev branch commit 1aad4dc)
Dry deposition Complete: 7-day Lizzie Lundgren 2017-02-24 (GC_BLEEDING_EDGE GCHP_Dev branch commit 339e48, GCHP Dev branch commit ec8419)
Aerosol thermodynamics (ISORROPIA II) Complete: 1-day, Special Sebastian Eastham
KPP Complete: Mike Long
Heterogeneous Chemistry Complete Mike Long
Fast-JX Complete:
  • Full Met - Standard Chem: 3 day
Lizzie Lundgren; Mike Long; Sebastian Eastham 2017-03-16 (GC_BLEEDING_EDGE branch off of GCHP_Dev branch commit 7b409d, GCHP Dev branch commit 2abe5b)

NOTE: modifications were made in GCBE to set RCONST=0 for non-photolysis rates, turn off het chem, and turn on/off photolysis

Advection Complete: 7-day Sebastian Eastham 2017-02-19 (GCHP Dev branch commit 645f5d5)
Emissions Complete: 7-day Sebastian Eastham 2017-03-05 (GC_BLEEDING_EDGE GCHP_Dev branch commit d6e31a0, GCHP Dev branch commit 7ac929c, UT GCHP_Dev branch commit 2d877e9)
Boundary layer mixing Complete: 1-day Sebastian Eastham 2017-03-22 (GC_BLEEDING_EDGE GCHP_Dev branch commit a2218c3, GCHP Dev branch commit 2abe5b8)
Other Chemistry Testing - Sulfate_Mod Complete: 1-day Jiawei Zhuang 2017-03-07 (GC_BLEEDING_EDGE GCHP_Dev branch commit d6e31a0, GCHP Dev branch commit 2abe5b8)
Other Chemistry Testing - Seasalt_Mod Complete: 1-day Jiawei Zhuang 2017-03-07 (GC_BLEEDING_EDGE GCHP_Dev branch commit d6e31a0, GCHP Dev branch commit 2abe5b8)
Other Chemistry Testing - Carbon_Mod Complete: 1-day Jiawei Zhuang 2017-03-07 (GC_BLEEDING_EDGE GCHP_Dev branch commit d6e31a0, GCHP Dev branch commit 2abe5b8)
Other Chemistry Testing - Aerosol_Mod Complete Sebastian Eastham; Lizzie Lundgren
Stratospheric Chemistry Complete Sebastian Eastham; Lizzie Lundgren

Change Log (internal use)

The following table lists notable bugs and fixes discovered and corrected during the process of individual component testing and review of the preliminary benchmark. All updates are included in GCHP v1.0.0.

Update Comment Contact Date Urgency Scope Commit


Update OTD-LIS scaling and dust mass tuning factor to best match GCC Seb Eastham submitted the updated values. He writes: "This corresponds to an annual average total flash rate of 145.5 flashes/second in GCHP, and a dust mass flux rate in GCHP at C24 which is 8.34% greater than that observed in GCC at 4x5. This is assuming that the annual average ratio is roughly accurate for the target time period." Eastham 4/10/17 important Chemistry
  • UT_BLEEDING_EDGE, branch GCHP_Dev, commit c14e9c
Modified UCX surface VMR BC for non-lat-lon grids The UCX surface VMR boundary conditions were being set on the assumption that the grid was lat-lon. This assumption is no longer made, and the correct VMR will be set on all grids. Eastham 4/1/17 important UCX
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 9241c5
Assorted run directory updates for replicating GCC v11-01 public release settings Lundgren/Eastham/Long 3/30/17 important general
  • UT_BLEEDING_EDGE, branch GCHP_Dev, commit 9241c5 and earlier
Correct bug with CH4 in EXTERNAL_GRID simulations CH4 was being incorrectly set to an undefined value when EXTERNAL_GRID was defined, such as in GEOS-5 or GCHP. Eastham 3/29/17 important general
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit fbcf6d
Modified sun.H in MAPL to permit negative SZA > 90 GCC expects the solar zenith angle (SZA) to be defined beyond 90, given that photolysis can occur at angles up to about 98. MAPL, however, limits SZA to be <= 90. This resulted in excessive calls to Fast-JX, and excessive photolysis. The relevant routine in MAPL (sun.H) has been adjusted to ensure that it can now return SZA of greater than 90. Eastham 3/29/17 important photolysis
  • GCHP, branch Dev, commit 13408d


Use background concentrations from the species database Long 3/28/17 important general
  • GCHP, branch Dev, commits bd8c28 and 57ccff
Fix bug where NAT supercooling and ice supersaturation values from input.geos not broadcasted to all threads in GCHP Use Input_Opt%P_ICE_SUPERSAT and Input_Opt%T_NAT_SUPERCOOL in ucx_mod.F instead of the ucx module variables. The Input_Opt variables are broadcasted to all threads while the ucx module variables are only set for the root thread that calls READ_INPUT_FILE during initialization. Lundgren 3/20/17 important stratospheric aerosols
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 11fffa


Fix bug where NAT supercooling and ice supersaturation values from input.geos not broadcasted to all threads in GCHP Use Input_Opt%P_ICE_SUPERSAT and Input_Opt%T_NAT_SUPERCOOL in ucx_mod.F instead of the ucx module variables. The Input_Opt variables are broadcasted to all threads while the ucx module variables are only set for the root thread that calls READ_INPUT_FILE during initialization. Lundgren 3/20/17 important stratospheric aerosols
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 11fffa
Use current time cos(SZA) for State_Met variables suncos and suncosmid in GEOS-Chem classic for comparison with GCHP GCHP and GCC State_Met%suncos and State_Met%suncosmid differ in two ways:
  1. GCC State_Met%suncos is calculated with current hour not current time (this is a bug); GCHP uses current time.
  2. GCC State_Met%suncosmid is calculated with mid-point time of chemistry timestep (this is not a bug); GCHP uses current time.

For GCHP validation I am fixing the GCC bug and setting GCC State_Met%suncosmid to GCC State_Met%suncos to match GCHP. NOTE: We must remove this GCHP short-term fix when merging GCHP_Dev into Dev.

Lundgren 3/10/17 important general
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 7b409d
Use locally calculated PEDGE rather than the edge pressure from the ECTM Prior to this update, PEDGE was set equal to EXTERNAL_PEDGE rather than calculated from the wet floating pressure and GMAO A and B parameters. Now the edge pressure is calculated the same as as in GEOS-Chem classic: PEDGE = AP(L) + ( BP(L) * PFLT_WET(I,J) ). Long 3/8/17 important general
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit e73fbb
Bug fix in setting H2O species concentration and State_Met humidity when UCX is on Pass 'A' not 'T' flag to extract H2O species index in the conditional for calling SET_H2O_TRAC in gigc_chunk_mod. This bug caused differences in State_Chm%Spc(:,:,:,id_H2O), State_Met%SPHU and State_MET%RH between GCHP and GEOS-Chem classic when UCX=y. Lundgren 3/7/17 important general
  • GCHP, branch Dev, commits 23d7c0 and 2abe5b
Update all three branches to support validated emissions Various changes made to all three branches to ensure that emissions in GCHP are as close as possible to this in GCC Eastham 3/5/17 optional emissions
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit d6e31a0 (and prior)
  • GCHP, branch Dev, commit 7ac929c (and prior)
  • UT, branch GCHP_Dev, commit 2d877e9 (and prior)
Bug fix in calculation of LAI for GCHP When calculating LAI for GCHP only, an incorrect subrange was used. As a result, LAI used anywhere in GCHP would only include a small subset of vegetation classes. Eastham/Lundgren 3/2/17 recommended emissions
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 1579887
Fix to MAPL_Generic default ringTime setting TingTime was originally incremented by TIMEINT leading to undesireable ringTimes for components with timesteps != Heartbeat. This fix ensures primarly that all components are able to execute at T=0. NOTE: we will move the implementation of this fix out of MAPL and into GCHP-specific code in the future to avoid potential impact on GEOS-5. Long 2/23/17 important general
  • GCHP, branch Dev, commit ec8419
Use MODIS LAI in emissions instead of met-field LAI and apply LAI bug fix in GCC emissions Besides being an LAI source change, this update corrects a bug introduced in GCHP_Dev hcoi_gc_main_mod.F90 where GC_LAI was used in GCC but is no longer available. GC_LAI was replaced with State_Met%MODISLAI in the previous dry deposition updates for GCHP. Met-field LAI is still read into ExtData to avoid GCHP dev disruption. Lundgren 2/23/17 important HEMCO: POPs, MEGAN, soil nox
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 452711
Define State_Met%OPTD as the sum of met-fields TAUCLI and TAUCLW Prior to this commit, the sum was scaled by State_Met%CLDF, inconsistent with GCC Lundgren 2/22/17 important photolysis
  • GCHP, branch Dev, commit c96d33
Read ozone met-field TO3 using ExtData and store in State_Met%TO3 State_Met%TO3 is needed for the ozone column used in photolysis; prior to this commit all values were zero. Lundgren 2/22/17 important photolysis
  • GCHP, branch Dev, commit 87ecc1
  • UT_BLEEDING_EDGE, branch GCHP_Dev, commit 2321f8
Set State_Met%UVALBEDO to offline climatology Prior to this update, State_Met%UVALBEDO was set to met-field ALBD, inconsistent with GCC Lundgren 2/22/17 important photolysis
  • GCHP, branch Dev, commit 2effc2
Updated transport and pressure handling for consistency with GCC and to ensure mass conservation Also affects calculation of specific humidity and temperature, which were not previously consistent with GCC. This requires an update to ExtData.rc, hence the changes to UT_BLEEDING_EDGE. Eastham 2/19/17 important general
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 2f2f95a
  • GCHP, branch Dev, commit 645f5d5
  • UT_BLEEDING_EDGE, branch GCHP_Dev, commit ac76ce1
Set State_Met%SWGDN if using GEOS-FP or MERRA2 in GCHP GCC uses SWGDN not RADSWG for GEOS-FP/MERRA2 Lundgren 2/14/17 important light correction in dry deposition; soil nox in emissions
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 45a3bd
  • GCHP, branch Dev, commit 1aad4d
Set State_Met%LAI to met LAI in GCHP for use in emissions Omission bug fix Lundgren 2/13/17 important HEMCO: POPs, MEGAN, soil nox
  • GCHP, branch Dev, commit 6c3162
Extract XLAI import once per day rather than every timestep GCC interpolates monthly LAI at start of each new day while GCHP was interpolating every dt Lundgren 2/10/17 recommended dry deposition
  • GCHP, branch Dev, commit 97f4a2
Change GCC NetCDF diagnostic output type to instantaneous for comparison with GCHP GCC default is mean but GCHP output is instantaneous Lundgren 2/7/17 recommended general
  • GC_BLEEDING_EDGE, branch GCHP_Dev, commit 865e7b

Go to GCHP Main Page