Difference between revisions of "GEOS-Chem benchmarking"
(→Update 2005/2009 CMDL CO data to 2013/2014 GMD CO data) |
(→Procedure) |
||
(8 intermediate revisions by the same user not shown) | |||
Line 9: | Line 9: | ||
== Procedure == | == Procedure == | ||
− | The GEOS-Chem benchmarking procedure is described below. | + | The GEOS-Chem benchmarking procedure is described below. GEOS-Chem uses [[GEOS-Chem_version_numbering_system#Numeric_versioning_system|semantic versioning (i.e. '''X.Y.Z''' version labels)]]. |
+ | |||
+ | === 1-hour benchmarks === | ||
+ | |||
+ | 1-hour benchmarks They are triggered when: | ||
+ | |||
+ | #A commit is pushed to any development branch<sup>1</sup> in the [https://github.com/geoschem/GCClassic <tt>geoschem/GCClassic</tt> superproject repository]; and/or | ||
+ | #A commit is puhsed to any development branch<sup>1</sup> in the [https://github.com/geoschem/GCHP <tt>geoschem/GCHP</tt> "superproject" repository]. | ||
+ | |||
+ | Evaluation tables are posted to [https://gc-dashboard.org <tt>gc-dashboard.org</tt>] upon successful completion of each 1-hour benchmark simulation. The evaluation tables include information on OH metrics, emissions totals, global mass, and a summary table. | ||
+ | |||
+ | The 1-hour benchmarks are important "sanity checks" and are often used to verify zero difference between two successive commits | ||
+ | |||
+ | |||
+ | #Development branches are '''dev/X.Y.Z''' or '''dev/no-diff-to-benchmark''' | ||
+ | |||
+ | === 1-month benchmarks === | ||
+ | |||
+ | # A '''1-month benchmark''' will run automatically on the AWS cloud whenever a tag is pushed to a '''development branch''' in the [https://github.com/geoschem/GCClassic geoschem/GCClassic] or [https://github.com/geoschem/GCHP geoschem/GCHP] superproject repositories. | ||
+ | #*Tags are | ||
+ | #*Development branches are '''dev/X.Y.Z''' or '''dev/no-diff-to-benchmark'''<br><br> | ||
+ | |||
+ | === etc === | ||
#Any update to the GEOS-Chem source code or run directories will change the [[GEOS-Chem_version_numbering_system#Numeric_versioning_system|GEOS-Chem version number ('''X.Y.Z''')]]. | #Any update to the GEOS-Chem source code or run directories will change the [[GEOS-Chem_version_numbering_system#Numeric_versioning_system|GEOS-Chem version number ('''X.Y.Z''')]]. | ||
Line 30: | Line 52: | ||
{| border=1 cellpadding=5 cellspacing=0 | {| border=1 cellpadding=5 cellspacing=0 | ||
|-bgcolor="#CCCCCC" | |-bgcolor="#CCCCCC" | ||
− | !width=" | + | !width="550px"|Directory |
− | !width=" | + | !width="300px"|Description |
|-valign="top" | |-valign="top" | ||
− | |<tt> | + | |<tt>https://gc-dashboard.org/search?searchString=&1Hr=1Hr&GCHP=GCHP&GCC=GCC</tt> |
+ | |Contains the following data from the [[GEOS-Chem_benchmarking#1-hour_benchmark|1-hour benchmarks]] used to evaluate GEOS-Chem: | ||
+ | *Evaluation plots & tables | ||
+ | *Run log | ||
+ | *Run directory (tarball) | ||
+ | *Diagnostic files (tarball) | ||
+ | *Restart Files (tarball) | ||
+ | |||
+ | |-valign="top" | ||
+ | |<tt>https://gc-dashboard.org/search?searchString=&1Mon=1Mon&GCHP=GCHP&GCC=GCC</tt> | ||
|Contains the following data from the [[GEOS-Chem_benchmarking#1-month_benchmark|1-month benchmarks]] used to evaluate GEOS-Chem: | |Contains the following data from the [[GEOS-Chem_benchmarking#1-month_benchmark|1-month benchmarks]] used to evaluate GEOS-Chem: | ||
− | * | + | *Evaluation plots & tables |
− | * | + | *Run log |
− | * | + | *Run directory (tarball) |
− | * | + | *Diagnostic files (tarball) |
− | * | + | *Restart Files (tarball) |
|-valign="top" | |-valign="top" | ||
|<tt>http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/</tt> | |<tt>http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/</tt> | ||
|Contains the following data from the [[GEOS-Chem_benchmarking#1-year_benchmark|1-year benchmarks]] used to evaluate GEOS-Chem: | |Contains the following data from the [[GEOS-Chem_benchmarking#1-year_benchmark|1-year benchmarks]] used to evaluate GEOS-Chem: | ||
− | |||
− | |||
− | |||
− | |||
*Evaluation plots | *Evaluation plots | ||
− | + | *Restart files (tarball) | |
+ | *Model output (tarball) | ||
+ | *Log files (tarball) | ||
+ | *Input files (tarball) | ||
+ | |-valign="top" | ||
+ | |<tt>http://ftp.as.harvard.edu/gcgrid/geos-chem/10yr_benchmarks/</tt> | ||
+ | |Contains the following data from the [[GEOS-Chem_benchmarking#10-year_benchmark|10-year benchmarks]] used to evaluate GEOS-Chem: | ||
+ | *Evaluation plots & tables | ||
+ | *Restart files (tarball) | ||
+ | *Model output (tarball) | ||
+ | *Log files (tarball) | ||
+ | *Input files (tarball) | ||
− | + | |} | |
− | + | NOTE: "tarball" refers to a <tt>*.tar.gz</tt> file. This is an archive of files & folders created with <tt>tar cvzf</tt> and can be extracted with <tt>tar xzvf</tt>. | |
− | == | + | == Benchmark plotting routines == |
− | The | + | The benchmark plotting routines are included with [https://github.com/geoschem/gcpy GCPy], a Python took kit available for GEOS-Chem. |
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + | ||
− | + |
Revision as of 17:58, 22 May 2024
Contents
Objectives
Benchmarking supports the maintenance of GEOS-Chem as a robust state-of-the-science facility with a nimble grass-roots approach and strong version control. Benchmarking has four main objectives:
- Document a consistent GEOS-Chem model configuration, and the expected characteristics of that configuration.
- Support version control through traceability, and by confirming the expected behavior of model developments submitted by the community.
- Track the evolution of the model over the years.
- Promote scientific transparency of GEOS-Chem.
Procedure
The GEOS-Chem benchmarking procedure is described below. GEOS-Chem uses semantic versioning (i.e. X.Y.Z version labels).
1-hour benchmarks
1-hour benchmarks They are triggered when:
- A commit is pushed to any development branch1 in the geoschem/GCClassic superproject repository; and/or
- A commit is puhsed to any development branch1 in the geoschem/GCHP "superproject" repository.
Evaluation tables are posted to gc-dashboard.org upon successful completion of each 1-hour benchmark simulation. The evaluation tables include information on OH metrics, emissions totals, global mass, and a summary table.
The 1-hour benchmarks are important "sanity checks" and are often used to verify zero difference between two successive commits
- Development branches are dev/X.Y.Z or dev/no-diff-to-benchmark
1-month benchmarks
- A 1-month benchmark will run automatically on the AWS cloud whenever a tag is pushed to a development branch in the geoschem/GCClassic or geoschem/GCHP superproject repositories.
- Tags are
- Development branches are dev/X.Y.Z or dev/no-diff-to-benchmark
etc
- Any update to the GEOS-Chem source code or run directories will change the GEOS-Chem version number (X.Y.Z).
- Z versions will be released at intervals determined by the GEOS-Chem Support Team (GCST) and may include bug fixes or updates that do not impact the full-chemistry simulation.
- Any change impacting the standard full-chemistry simulation will require a Y version change and a dedicated 1-month benchmark. The benchmark results will be posted on the wiki and an email will be sent to the developer(s) and the GEOS-Chem Steering Committee (GCSC).
- The developer(s) and GCSC will assess the benchmark results and review a benchmark assessment form on the wiki. If there are any concerns about the benchmark results, the GCST will be notified and further investigation and/or benchmarking may be required.
- If the update is for a specialty simulation (e.g. CO2, CH4, Hg), then a further benchmark may be conducted by the appropriate Working Group.
- Once the developer is satisfied with the changes in the 1-month benchmark, GEOS-Chem Model Scientist Daniel Jacob will review the results and approve the new internal version.
- 1-year full-chemistry and/or transport tracer benchmarks for Y versions will be conducted only if justifiably requested by the developer or by GEOS-Chem Steering Committee members.
- Each new major version release (i.e. X version) will be subject to a 1-year benchmark to be inspected by the GEOS-Chem Steering Committee before approval.
List of GEOS-Chem benchmarks
Links to past 1-month and 1-year benchmark simulations can be found on the GEOS-Chem versions wiki page.
Benchmark output archive
Output files and evaluation plots for 1-month and 1-year benchmark simulations are archived at Harvard as summarized below. GEOS-Chem users may utilize these output for comparisons against their own simulations.
Directory | Description |
---|---|
https://gc-dashboard.org/search?searchString=&1Hr=1Hr&GCHP=GCHP&GCC=GCC | Contains the following data from the 1-hour benchmarks used to evaluate GEOS-Chem:
|
https://gc-dashboard.org/search?searchString=&1Mon=1Mon&GCHP=GCHP&GCC=GCC | Contains the following data from the 1-month benchmarks used to evaluate GEOS-Chem:
|
http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/ | Contains the following data from the 1-year benchmarks used to evaluate GEOS-Chem:
|
http://ftp.as.harvard.edu/gcgrid/geos-chem/10yr_benchmarks/ | Contains the following data from the 10-year benchmarks used to evaluate GEOS-Chem:
|
NOTE: "tarball" refers to a *.tar.gz file. This is an archive of files & folders created with tar cvzf and can be extracted with tar xzvf.
Benchmark plotting routines
The benchmark plotting routines are included with GCPy, a Python took kit available for GEOS-Chem.