Difference between revisions of "GEOS-Chem benchmarking"
(→1-hour benchmarks) |
(→1-hour benchmarks) |
||
| Line 13: | Line 13: | ||
=== 1-hour benchmarks === | === 1-hour benchmarks === | ||
| − | 1-hour benchmarks | + | 1-hour benchmarks are automatically triggered when: |
#A commit is pushed to any development branch<sup>1</sup> in the [https://github.com/geoschem/GCClassic <tt>geoschem/GCClassic</tt> superproject repository] | #A commit is pushed to any development branch<sup>1</sup> in the [https://github.com/geoschem/GCClassic <tt>geoschem/GCClassic</tt> superproject repository] | ||
| Line 22: | Line 22: | ||
The 1-hour benchmarks are important "sanity checks" and are often used to verify zero difference between two successive commits | The 1-hour benchmarks are important "sanity checks" and are often used to verify zero difference between two successive commits | ||
| − | + | NOTES: | |
| + | # Development branches are '''dev/X.Y.Z''' or '''dev/no-diff-to-benchmark'''. | ||
=== 1-month benchmarks === | === 1-month benchmarks === | ||
Revision as of 18:00, 22 May 2024
Contents
Objectives
Benchmarking supports the maintenance of GEOS-Chem as a robust state-of-the-science facility with a nimble grass-roots approach and strong version control. Benchmarking has four main objectives:
- Document a consistent GEOS-Chem model configuration, and the expected characteristics of that configuration.
- Support version control through traceability, and by confirming the expected behavior of model developments submitted by the community.
- Track the evolution of the model over the years.
- Promote scientific transparency of GEOS-Chem.
Procedure
The GEOS-Chem benchmarking procedure is described below. GEOS-Chem uses semantic versioning (i.e. X.Y.Z version labels).
1-hour benchmarks
1-hour benchmarks are automatically triggered when:
- A commit is pushed to any development branch1 in the geoschem/GCClassic superproject repository
- A commit is puhsed to any development branch1 in the geoschem/GCHP "superproject" repository.
Evaluation tables are posted to gc-dashboard.org upon successful completion of each 1-hour benchmark simulation. The evaluation tables include information on OH metrics, emissions totals, global mass, and a summary table.
The 1-hour benchmarks are important "sanity checks" and are often used to verify zero difference between two successive commits
NOTES:
- Development branches are dev/X.Y.Z or dev/no-diff-to-benchmark.
1-month benchmarks
- A 1-month benchmark will run automatically on the AWS cloud whenever a tag is pushed to a development branch in the geoschem/GCClassic or geoschem/GCHP superproject repositories.
- Tags are
- Development branches are dev/X.Y.Z or dev/no-diff-to-benchmark
etc
- Any update to the GEOS-Chem source code or run directories will change the GEOS-Chem version number (X.Y.Z).
- Z versions will be released at intervals determined by the GEOS-Chem Support Team (GCST) and may include bug fixes or updates that do not impact the full-chemistry simulation.
- Any change impacting the standard full-chemistry simulation will require a Y version change and a dedicated 1-month benchmark. The benchmark results will be posted on the wiki and an email will be sent to the developer(s) and the GEOS-Chem Steering Committee (GCSC).
- The developer(s) and GCSC will assess the benchmark results and review a benchmark assessment form on the wiki. If there are any concerns about the benchmark results, the GCST will be notified and further investigation and/or benchmarking may be required.
- If the update is for a specialty simulation (e.g. CO2, CH4, Hg), then a further benchmark may be conducted by the appropriate Working Group.
- Once the developer is satisfied with the changes in the 1-month benchmark, GEOS-Chem Model Scientist Daniel Jacob will review the results and approve the new internal version.
- 1-year full-chemistry and/or transport tracer benchmarks for Y versions will be conducted only if justifiably requested by the developer or by GEOS-Chem Steering Committee members.
- Each new major version release (i.e. X version) will be subject to a 1-year benchmark to be inspected by the GEOS-Chem Steering Committee before approval.
List of GEOS-Chem benchmarks
Links to past 1-month and 1-year benchmark simulations can be found on the GEOS-Chem versions wiki page.
Benchmark output archive
Output files and evaluation plots for 1-month and 1-year benchmark simulations are archived at Harvard as summarized below. GEOS-Chem users may utilize these output for comparisons against their own simulations.
| Directory | Description |
|---|---|
| https://gc-dashboard.org/search?searchString=&1Hr=1Hr&GCHP=GCHP&GCC=GCC | Contains the following data from the 1-hour benchmarks used to evaluate GEOS-Chem:
|
| https://gc-dashboard.org/search?searchString=&1Mon=1Mon&GCHP=GCHP&GCC=GCC | Contains the following data from the 1-month benchmarks used to evaluate GEOS-Chem:
|
| http://ftp.as.harvard.edu/gcgrid/geos-chem/1yr_benchmarks/ | Contains the following data from the 1-year benchmarks used to evaluate GEOS-Chem:
|
| http://ftp.as.harvard.edu/gcgrid/geos-chem/10yr_benchmarks/ | Contains the following data from the 10-year benchmarks used to evaluate GEOS-Chem:
|
NOTE: "tarball" refers to a *.tar.gz file. This is an archive of files & folders created with tar cvzf and can be extracted with tar xzvf.
Benchmark plotting routines
The benchmark plotting routines are included with GCPy, a Python took kit available for GEOS-Chem.
