Difference between revisions of "GEOS-Chem v11-02 benchmark history"

From Geos-chem
Jump to: navigation, search
(→‎v11-02b)
(→‎v11-02b 1-month comparison)
Line 602: Line 602:
 
== High Performance Option benchmarks ==
 
== High Performance Option benchmarks ==
  
−
=== v11-02b 1-month comparison ===
+
=== v11-02b HP 1-month ===
  
 
Assessment form for 1-month benchmark simulation using GEOS-Chem '''[[GEOS-Chem_v11-02#v11-02b|v11-02b]]''' with the high performance option (GCHP):
 
Assessment form for 1-month benchmark simulation using GEOS-Chem '''[[GEOS-Chem_v11-02#v11-02b|v11-02b]]''' with the high performance option (GCHP):

Revision as of 15:16, 12 June 2017

On this page we have posted complete information about all benchmark simulations (both 1-month and 1-year) for GEOS-Chem v11-02.

1-year full-chemistry benchmarks

v11-02a-Run1

This 1-year benchmark simulation was an unofficial benchmark to validate removal partitioning of NOx biomass burning emissions directly to PAN and HNO3 from v11-02a.

Three GEOS-Chem model versions were compared to each other:

Color Model Version Met Type Year Updates affecting the benchmark simulation Annual Mean OH
[105 molec/cm3]
Red v11-01k-Run0 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-01h:

Updates introduced in v11-01i:

Updates introduced in v11-01j:

Updates introduced in v11-01k:

12.004
Green v11-01-public-release-Run0 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-01 public release:

12.001
Blue v11-02a-Run1 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-02a:

11.829
Black Observations        

The output plots for Run0 may be downloaded from: 

ftp ftp.as.harvard.edu
cd gcgrid/geos-chem/1yr_benchmarks/v11-02/v11-02a/Run1/output
mget *

You may also view the PDF files online by pointing your browser to 

http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/v11-02/v11-02a/Run1/output/

Please also view the following pages comparing this version to past 1-year benchmarks:

--Melissa Sulprizio (talk) 16:37, 1 June 2017 (UTC)

v11-02a-Run0

This 1-year benchmark simulation was approved by the GEOS-Chem Steering Committee on 12 May 2017, with the recommendation to remove partitioning of NOx biomass burning emissions directly to PAN and HNO3.

Three GEOS-Chem model versions were compared to each other:

Color Model Version Met Type Year Updates affecting the benchmark simulation Annual Mean OH
[105 molec/cm3]
Red v11-01k-Run0 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-01h:

Updates introduced in v11-01i:

Updates introduced in v11-01j:

Updates introduced in v11-01k:

12.004
Green v11-01-public-release-Run0 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-01 public release:

12.001
Blue v11-02a-Run0 GEOS-FP,
72L, 4x5 		2013

Updates introduced in v11-02a:

11.750
Black Observations        

The output plots for Run0 may be downloaded from: 

ftp ftp.as.harvard.edu
cd gcgrid/geos-chem/1yr_benchmarks/v11-02/v11-02a/Run0/output
mget *

You may also view the PDF files online by pointing your browser to 

http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/v11-02/v11-02a/Run0/output/

Please also view the following pages comparing this version to past 1-year benchmarks:

--Melissa Sulprizio (talk) 18:14, 10 May 2017 (UTC)

1-month benchmarks

v11-02b

Assessment form for 1-month benchmark simulation of v11-02b:

Description
New features added into GEOS-Chem
Features not affecting the full-chemistry simulation:
Feature Submitted by
Source code updates for compatibility with GCHP v1.1.0 Seb Eastham (Harvard)
Lizzie Lundgren (GCST)
Mike Long (GCST)
Jiawei Zhuang (Harvard)
Bob Yantosca (GCST)
Bug fixes for diagnostics:
Ilya Stanevic (Toronto)
Lee Murray (Rochester)
Write initial and final Ox mass to file when using the tagged O3 simulation Bob Yantosca (GCST)
Convert CO2 emissions units using dry pressure used in advection Meemong Lee (JPL)
Add QFED emissions for 2014-2016 Christoph Keller (NASA GMAO)
Version, resolution, met fields used: v11-02, GEOS-FP (72L), 4x5, July 2013
1-month benchmark finished on: Thurs June 07 23:08:02 EDT 2017
Performance statistics:
  • Ran on 24 CPUs of holyjacob01.rc.fas.harvard.edu (Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50 GHz)
  • Wall time: 04:25:40
  • CPU time / wall time: 22.5723
  • % of ideal performance: 94.05%
  • Memory: 5.0852 GB
Compared to previous benchmark: v11-02a
This update will impact:
(select all that apply with boldface) 		Advection, BL Mixing, Convection, Met Fields, Dry Dep, Wet Dep, Stratosphere, Anthro Emiss, Biogenic Emiss, Biomass Emiss, Photolysis, Chemistry, Other (please specify):
Unit test results may be viewed at: http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02b/v11-02b.results.html
Plots may be viewed at: http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02b/

New in this version: We now include a subdirectory called gchp_comparison containing 1-month standard simulation output comparisons using the standard "classic" capabilities (GCC) and the high performance option (GCHP). Details and discussion of the GCHP benchmark are in the High Performance Option v11-02b 1-month Comparison benchmark form.

Metrics
Global mean OH (from log file): 12.3818817518568 x 105 molec/cm3
Methyl chloroform lifetime: 5.0764 years
Did either of these change by more than 5%? No, both values are identical to values of the previous version.
At the SURFACE, list all species that changed by 10% or more: none
Comments on SURFACE differences: none
At 500 hPa, list all species that changed by 10% or more: none
Comments on 500 hPa differences: No differences.
In the ZONAL MEAN differences, list all species that changed by 10% or more: none
Comments on ZONAL MEAN differences: No differences.
In the EMISSION RATIO maps, list all species that changed by 10% or more: none
Comments on EMISSION RATIO differences: none
Additional or summary comments: As expected, merging in the modifications for the high performance option (GCHP) has zero impact on the GEOS-Chem "classic" standard simulation.
Approval
Requires further investigation: TBD
Approved by: TBD
Date of approval: TBD

--Lizzie Lundgren (talk) 19:04, 30 May 2017 (UTC)

v11-02a

Here is the assessment form for 1-month benchmark simulation v11-02a.

Description
New features added into GEOS-Chem
Features affecting the full-chemistry simulation:
Feature Submitted by
Update chemistry rate constants based on JPL Publication 15-10 Barron Henderson (US EPA),
Mat Evans (U. York), &
Oxidants and Chemistry WG
Fixes to correct ALK4 lumping issue Barron Henderson (US EPA)
PAN updates (including emissions, deposition, species, chemistry) Emily Fischer (CSU)
Monthly mean NEI2011 emissions GCST &
Katie Travis (Harvard)
Bug fixes in the GEOS-Chem sulfate module:
Prasad Kasibhatla (Duke)
Prasad Kasibhatla (Duke)
Qianjie Chen (UW)
Viral Shah (UW)
Fix bug in dry deposition aerodynamic resistance Brian Boys (Dalhousie)
Fix acetone parameterization in hcox_seaflux_mod.F90 GCST
Bug fix in cos(SZA) for start of timestep Lizzie Lundgren (GCST)
Include TOA pressure when calculating dry pressure edges Seb Eastham (Harvard)
Features not affecting the full-chemistry simulation:
Feature Submitted by
Implement ISORROPIA v2.0 as a Fortran module Seb Eastham (Harvard) &
GCST
Updates to the HEMCO emissions component:
Christoph Keller (NASA GMAO)
Seb Eastham (Harvard)
Jessica Morena (Dalhousie)
Brian Boys (Dalhousie) & GCST
Updates to gain computational speedup:
Mike Long (Harvard)
Bob Yantosca (GCST)
Fixed typo in INIT_WINDOW Bob Yantosca (GCST)
netCDF file I/O updates:
Chris Holmes (Florida State)
Andy Jacobson (NOAA)
GCST
Makefile and build sequence updates:
Jiawei Zhang (Harvard)
GCST
Bug fixes for running UCX in ESMF environment Christoph Keller (NASA GMAO)
Bug fixes for diagnostics:
Aaron van Donkelaar (Dalhousie)
Jenny Fisher (U. Wollongong)
GCST
GCST
Jenny Fisher (U. Wollongong)
Chris Holmes (Florida State)
Removal of obsolete variables: GCST
Version, resolution, met fields used: v11-02, GEOS-FP (72L), 4x5, July 2013
1-month benchmark finished on: Sun Apr 16 01:37:36 EDT 2017
Performance statistics:
  • Ran on 24 CPUs of holyjacob01.rc.fas.harvard.edu (Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50 GHz)
  • Wall time: 04:22:54
  • CPU time / wall time: 22.5896
  • % of ideal performance: 94.12%
  • Memory: 5.2038 GB
Compared to previous benchmark: v11-01 public release
This update will impact:
(select all that apply with boldface) 		Advection, BL Mixing, Convection, Met Fields, Dry Dep, Wet Dep, Stratosphere, Anthro Emiss, Biogenic Emiss, Biomass Emiss, Photolysis, Chemistry, Other (please specify):
Unit test results may be viewed at: http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02a/v11-02a.results.html
Plots may be viewed at: http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02a/
Metrics
Global mean OH (from log file): 12.3128108012973 x 105 molec/cm3
Methyl chloroform lifetime: 5.1159 years
Did either of these change by more than 5%? No. Mean OH changed by -2.57% and MCF lifetime changed by 2.94%. These changes were primarily caused by two updates:
  • The JPL 15-10 update changed mean OH by -1.03% and MCF lifetime by 0.58%.
  • The PAN updates changed mean OH by -1.62% and MCF lifetime by 2.41%
At the SURFACE, list all species that changed by 10% or more: NO, O3, PAN, CO, ALK4, ISOP, HNO3, H2O2, ACET, MEK, ALD2, RCHO, MVK, MACR, PMN, PPN, R4N2, PRPE, C3H8, CH2O, C2H6, N2O5, HNO4, MP, DMS, SO2, SO4, SO4s, MSA, NH3, NH4, NIT, NITs, BCPI, OCPI, BCPO, OCPO, Br2, Br, BrO, HOBr, HBr, BrNO2, BrNO3, CHBr3, MPN, ISOPND, ISOPNB, MOBA, PROPNN, HAC, GLYC, MVKN, MACRN, RIP, IEPOX, MAP, NO2, NO3, HNO2, BrCl, HCl, Cl, ClO, HOCl, ClNO3, ClOO, OClO, Cl2, Cl2O2, MTPA, LIMO, MTPO, TSOG1, TSOG2, TSOG2, TSOG0, TSOA1, TSOA2, TSOA3, TSOA0, ISOG1, ISOG2, ISOG3, ISOA1, ISOA2, ISOA3, BENZ, TOLU, XYLE, ASOG1, ASOG2, ASO3G3, ASOAN, ASOA1, ASOA2, ASOA3, OH, HO2
Comments on SURFACE differences:

Below we summarize the notable changes caused by specific updates.

Update chemistry rate constants based on JPL Publication 15-10

  • Notable changes caused by this update:
    • >10% increase in MVK, PRPE, N2O5, Cl2O2
    • >10% decrease in MACR, PMN, CHBr3
  • Matt Evans wrote
    The only major changes [from JPL 15-10] are to do with the MVK / MACR + OH changes we expected, changes in the PRPE +OH and changes in the N2O5 / HO2NO2 formation / loss rates. Unless I’ve missed something I think we are happy with all of this.

PAN updates

Monthly mean NEI2011 emissions

  • This update caused numerical noise differences over the US in NO, ALK4, RCHO, N2O5, DMS, SO2, NH3, NIT, BCPO, OCPO, NO2, NO3, HNO2, BENZ, TOLU, XYLE.

Bug fixes in the GEOS-Chem sulfate module:

  • The bug fix in SEASALT_CHEM caused a decrease in SO4s and NITs globally.
  • The bug fix in CHEM_NIT did not cause any changes that amount to >10%.
  • The fix for sulfate production in HET_DROP_CHEM caused an increase in SO2, NH4, and NITs over the oceans and a decrease in SO4, SO4s, NH3, and NITs over the oceans
  • The bug fixes in sulfate chemistry routines caused changes in SO2, SO4, SO4s, NH3, NH4, NIT, and NITs.
    • Viral Shah wrote:
      The effects of my proposed fixes are a bit more pronounced during the NH winter. The plots show significant increases in sulfate concentrations over large areas of the NH. The SO4 increases are accompanied by decreases in NIT in some areas. The increase in SO4 in the cold areas of NH is mostly because of the correction to HNO3 effective Henry's law coefficient that leads to higher cloud water pH (between 4 and 5) and higher production by in-cloud oxidation by O3. The lower SO4 concentrations in the SH are probably due to the fix to ∆H/R parameter. I expect the effects [of the LWC units fix] to be relatively small.

Fix bug in dry deposition aerodynamic resistance

  • This update caused slight decreases in many species at the surface. Species changed by >10% from this update include HNO3, H2O2, N2O5, and NIT.

The following updates impacted species concentrations slightly, but the changes did not amount to >10%.

At 500 hPa, list all species that changed by 10% or more: NO, PAN, CO, ALK4, ISOP, HNO3, H2O2, ACET, MEK, ALD2, RCHO, MVK, MACR, PMN, PPN, R4N2, PRPE, C3H8, CH2O, C2H6, N2O5, HNO4, MP, DMS, SO2, SO4, SO4s, MSA, NH3, NH4, NIT, NITs, BCPI, OCPI, BCPO, OCPO, Br2, Br, BrO, HOBr, HBr, BrNO2, BrNO3, CHBr3, MPN, ISOPND, ISOPNB, MOBA, PROPNN, HAC, GLYC, MVKN, MACRN, RIP, IEPOX, MAP, NO2, NO3, HNO2, BrCl, Cl, ClO, HOCl, ClNO3, ClOO, OClO, Cl2, Cl2O2, MTPA, LIMO, MTPO, TSOG1, TSOG2, TSOG2, TSOG0, TSOA1, TSOA2, TSOA3, TSOA0, ISOG1, ISOG2, ISOG3, ISOA1, ISOA2, ISOA3, BENZ, TOLU, XYLE, ASOG1, ASOG2, ASO3G3, ASOAN, ASOA1, ASOA2, ASOA3, OH, HO2
Comments on 500 hPa differences: See comments on surface differences.
In the ZONAL MEAN differences, list all species that changed by 10% or more: NO, PAN, ALK4, ISOP, HNO3, H2O2, MEK, ALD2, RCHO, MVK, MACR, PMN, PPN, PRPE, C3H8, CH2O, N2O5, HNO4, MP, DMS, SO2, SO4, SO4s, NH3, NH4, NIT, NITs, BCPI, OCPI, BCPO, OCPO, Br2, Br, BrO, HOBr, HBr, BrNO2, CHBr3, MPN, ISOPND, ISOPNB, MOBA, HAC, GLYC, MVKN, MACRN, RIP, IEPOX, MAP, NO2, NO3, HNO2, BrCl, Cl, ClO, HOCl, ClNO3, ClOO, OClO, Cl2, Cl2O2, MTPA, LIMO, MTPO, TSOG1, TSOG2, TSOG3, TSOG0, TSOA1, TSOA2, TSOA3, TSOA0, ISOG1, ISOG2, ISOG3, ISOA1, ISOA2, ISOA3, BENZ, TOLU, XYLE, ASOG1, ASOG2, ASOG3, ASOAN, ASOA1, ASOA2, ASOA3, OH, HO2
Comments on ZONAL MEAN differences: See comments on surface differences.
In the EMISSION RATIO maps, list all species that changed by 10% or more:
  • NO: Anthropogenic+biofuel, biomass burning, and fertilizer
Comments on EMISSION RATIO differences:
  • The differences in NO emissions can be explained by the following updates:
    • NO biomass burning emissions decrease because NO is partitioned directly to 40% PAN and 20% HNO3.
    • Changes in NO anthropogenic emissions are isolated to the oceans, indicating a change in ship emissions. Changes in ship emissions may result from slight changes in NO or NO2 concentrations.
    • Differences in fertilizer emissions are caused by changes in NOx concentrations.
  • Anthropogenic emissions over US change slightly (<10%) because of the change from hourly NEI2011 emissions to monthly mean NEI2011 emissions. This impacts the following species: ALD2, ALK4, BC, C2H6, C3H8, CH2O, CO, MEK, NH3, NO, OC, PRPE, SO2, SO4.
  • Slight (<10%) differences can be seen in the biomass burning emission difference plots. This is numerical noise caused by injecting biomass burning emissions at different levels (65% to the boundary layer, 35% to the free troposphere).
  • ACET ocean source decreases slightly (<10%) because of the fix for acetone parameterization in hcox_seaflux_mod.F90.
Additional or summary comments: Removal of biomass burning updates

After the 1-month benchmark for v11-02a, the GCSC decided we should remove the code updates for vertically distributed biomass burning emissions (see this wiki post for more details). In addition, after the 1-year benchmark v11-02a-Run0, the GCSC asked that we comment out the partitioning of NOx biomass burning emissions directly to PAN and HNO3 (see this wiki post for more details). An unofficial 1-month benchmark was run for v11-02a to evaluate the model changes with these two biomass burning updates removed. The plots for that run may be viewed at:

http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02a_final/

Results from our v11-02a benchmark using the GNU Fortran Compiler

In order to evaluate the performance of GEOS-Chem using the the free and open source GNU Fortran compiler, we performed an additional 1-month benchmark for v11-02a. For this additional benchamark, we used GNU Fortran v6.2.0 to compile GEOS-Chem instead of our usual Intel Fortran Compiler version 11.1.069.

For more details, see this post on our GNU Fortran compiler wiki page.

Summary: The benchmark using GNU Fortran yielded essentially identical results to the benchmark using Intel Fortran. This is very encouraging, as it will allow GEOS-Chem development to take place on computational platforms that do not have proprietary compilers (such as Intel Fortran or PGI Fortran), which can be prohibitively expensive to purchase.

Approval
Requires further investigation: Yes, a 1-year benchmark simulation
Approved by: Emily Fischer, Prasad Kasibhatla, Brian Boys, Daniel Jacob
Date of approval: 24 Apr 2017

--Melissa Sulprizio (talk) 19:27, 20 April 2017 (UTC)

1-year Rn-Pb-Be benchmarks

TBD

High Performance Option benchmarks

v11-02b HP 1-month

Assessment form for 1-month benchmark simulation using GEOS-Chem v11-02b with the high performance option (GCHP):

Description
New features added:

As this is the first benchmark, new features include the high performance model capability in its entirety.

GEOS-Chem version, GCHP version tag: v11-02, v1.1.0
Meteorology fields: GEOS-FP (72L), 2x2.5, July 2013
Cubed-sphere resolution: c24
1-month benchmark finished on: Thurs June 07 23:08:02 EDT 2017 (placeholder)
Performance statistics:
  • Ran on 24 CPUs of holyjacob01.rc.fas.harvard.edu (Intel(R) Xeon(R) CPU E5-2680 v3 @ 2.50 GHz)
  • Wall time: 04:25:40 (placeholder)
  • CPU time / wall time: 22.5723 (placeholder)
  • % of ideal performance: 94.05% (placeholder)
  • Memory: 5.0852 GB (placeholder)
Compared to benchmark: GEOS-Chem v11-02b with "classic" capability
Additional comments:

We compared GEOS-Chem with and without the high performance option (GCHP and GCC respectively) for two different scenarios:

  1. Out-of-the-box
  2. Custom

The out-of-the-box comparison uses as-is code and run directory settings for both GCHP and GCC. Both use restart files regridded from higher resolution 2x2.5. GCC is compiled with netcdf diagnostics turned on and binary diagnostics turned off.

The custom comparison includes additional modifications to the GCC code base and run directory to remove certain remaining known differences between GCHP and GCC. These include the following changes to GCC:

  1. Turn off OTD-LIS factors and set OTD-LIS scaling to 0.5568042545
  2. Turn off all 2D initial mixing ratios except for stratospheric H2O
  3. Turn off usage of the HEMCO restart file
  4. Force cos(SZA) at timestep mid-point to equal cos(SZA) calculated at timestep start
Plots may be viewed at: http://ftp.as.harvard.edu/gcgrid/geos-chem/1mo_benchmarks/v11-02/v11-02b/gchp_comparisons/
Metrics
Global mean OH: Diagnostic not yet implemented
Methyl chloroform lifetime: Diagnostic not yet implemented
Did either of these change by more than 5%? N/A
At the SURFACE, list all species that changed by 10% or more:
Comments on SURFACE differences:
At 500 hPa, list all species that changed by 10% or more:
Comments on 500 hPa differences:
In the ZONAL MEAN differences, list all species that changed by 10% or more:
Comments on ZONAL MEAN differences:
Additional or summary comments:
Approval
Requires further investigation: TBD
Approved by: TBD
Date of approval: TBD

--Lizzie Lundgren (talk) 20:55, 9 June 2017 (UTC)

Retrieved from "https://wiki.seas.harvard.edu/geos-chem/index.php?title=GEOS-Chem_v11-02_benchmark_history&oldid=33232"