Difference between revisions of "Submitting updates for inclusion in GEOS-Chem"
(→Checklist for data submissions) |
|||
(23 intermediate revisions by 3 users not shown) | |||
Line 9: | Line 9: | ||
== Guidelines == | == Guidelines == | ||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
− | |||
# Contact your [http://acmg.seas.harvard.edu/geos/geos_working_groups.html Working Group] leaders to request that your changes be included in the standard code. | # Contact your [http://acmg.seas.harvard.edu/geos/geos_working_groups.html Working Group] leaders to request that your changes be included in the standard code. | ||
+ | #*The GEOS-Chem Steering Committee will meet every three months and set the [[GEOS-Chem model development priorities]] (i.e. decide on the order in which updates will be added to GEOS-Chem).<br><br> | ||
+ | # Create or log into your [https://github.com/ GitHub] account.<br><br> | ||
+ | # [https://help.github.com/articles/fork-a-repo/ Fork the GEOS-Chem repositories]. Navigate to each appropriate [https://github.com/geoschem GEOS-Chem repository on GitHub] and click '''Fork''' in the top-right corner of the page.<br><br> | ||
+ | # Clone your forks of the GEOS-Chem repositories to your system: | ||
+ | #*<tt>git clone https://github.com/YOUR-USERNAME/geos-chem Code.12.0.0</tt> | ||
+ | #*<tt>git clone https://github.com/YOUR-USERNAME/geos-chem-unittest UT.12.0.0</tt>.<br><br> | ||
+ | # Add your modifications into a [https://git-scm.com/book/en/v2/Git-Branching-Branches-in-a-Nutshell new branch] off of the master branch.<br><br> | ||
+ | # Test your update thoroughly and make sure that it works. | ||
+ | #*You can use the [[GEOS-Chem Unit Tester]] to make sure your code updates work for a given combination of met fields, resolutions, and simulation types. | ||
+ | #*A unit test looks for [[Parallelizing GEOS-Chem|parallelization errors]], [[Floating point math issues|floating point math issues]], [[Common GEOS-Chem error messages#Array-out-of-bounds error|array-out-of-bounds errors]], and [[Passing array arguments efficiently in GEOS-Chem|creation of array temporaries]]. | ||
+ | #*For structural updates, we recommend running [[Performing_Difference_Tests_with_GEOS-Chem|difference tests]] often to ensure your updates don't impact model output.<br><br> | ||
+ | # Review the [[#Checklist for code update submissions|code update checklist]] and [[#Checklist for data submissions|data checklist]] below.<br><br> | ||
# Create a [https://help.github.com/articles/creating-a-pull-request/ pull request in GitHub] (recommended) or a [https://www.devroom.io/2009/10/26/how-to-create-and-apply-a-patch-with-git/ Git patch file]. | # Create a [https://help.github.com/articles/creating-a-pull-request/ pull request in GitHub] (recommended) or a [https://www.devroom.io/2009/10/26/how-to-create-and-apply-a-patch-with-git/ Git patch file]. | ||
− | # | + | #* See our [https://github.com/geoschem/geos-chem/blob/dev/12.6.0/.github/PULL_REQUEST_TEMPLATE/create-a-geos-chem-pull-request.md pull request template on Github] for information to include.<br><br> |
− | + | #The GEOS-Chem Support Team will add your changes to the standard code. | |
− | #The GEOS-Chem Support Team will add your changes to the standard code. Your update may be bundled with other code updates and the [[GEOS-Chem_version_numbering_system#Numeric_versioning_system|GEOS-Chem version number ('''X.Y.Z''')]] will be changed. The update will be validated following the [[GEOS-Chem_benchmarking|GEOS-Chem benchmarking procedure]]. | + | #*Your update may be bundled with other code updates and the [[GEOS-Chem_version_numbering_system#Numeric_versioning_system|GEOS-Chem version number ('''X.Y.Z''')]] will be changed. |
+ | #*The update will be validated following the [[GEOS-Chem_benchmarking|GEOS-Chem benchmarking procedure]]. | ||
=== Checklist for code update submissions === | === Checklist for code update submissions === | ||
− | + | #Use F90 free format whenever possible | |
− | + | #Include thorough comments throughout code updates | |
− | + | #Remove extraneous code updates (e.g. testing options, other science) | |
− | + | #Include full citations at the top of relevant source code modules | |
− | + | #Complete [[Developing_GCHP#GCHP_Updates_Required_with_GEOS-Chem_Classic_Updates|GCHP updates required with GEOS-Chem Classic updates]] | |
=== Checklist for data submissions === | === Checklist for data submissions === | ||
− | *Make sure data files are COARDS-compliant | + | #Choose a final file naming convention for all data files that are to be submitted. |
− | + | #*Requesting files to be renamed after the benchmark process has begun can cause confusion and prevent reproducibility.<br><br> | |
− | *Make sure files have been [[Working_with_netCDF_data_files#Chunking_and_deflating_a_netCDF_file_to_improve_I.2FO|chunked and deflated]]. | + | #Make sure netCDF data files are [[The_COARDS_netCDF_conventions_for_earth_science_data|COARDS-compliant]] |
− | + | #*Use [[Working_with_netCDF_data_files#Determining_if_a_netCDF_file_is_COARDS-compliant|<tt>isCoards</tt> script]] and include output in submission.<br><br> | |
− | + | #[[Working_with_netCDF_data_files#Concatenating_netCDF_files|Concatenate individual netCDF files together]] so as to reduce tne number of files that need to be opened. | |
− | *Include updated HEMCO Configuration file (if the data will be read via HEMCO). Submit this as a pull request in the [https://github.com/geoschem/geos-chem-unittest geos-chem-unittest repository on GitHub]. | + | #*e.g. If the data is a multi-year archive of monthly-mean data, create one file per year containing all data variables. Each data variable would have 12 timestamps, one for each month. |
− | + | #*Reducing the number of files that have to be opened results in more efficient GEOS-Chem simulations.<br><br> | |
− | + | #Make sure files have been [[Working_with_netCDF_data_files#Chunking_and_deflating_a_netCDF_file_to_improve_I.2FO|chunked and deflated]]. | |
− | + | #*Deflation level 1 is usually sufficient for most files. | |
+ | #*e.g. <tt>./nc_chunk.pl myfile.nc 1</tt><br><br> | ||
+ | #Follow these additional steps to ensure data can also be read by GCHP: | ||
+ | #*All data variables should be of type <tt>float</tt> (aka <tt>REAL*4</tt>) or <tt>double</tt> (aka <tt>REAL*8</tt>). The MAPL library used by GCHP cannot read integer data from netCDF files. The time dimension is permitted to be integer. | ||
+ | #*Use a recent reference date, if possible, for the <tt>time:units</tt> attribute. MAPL may have issues interpreting reference times such as 1750 which is prior to the Julian-to-Gregorian calendar switch. | ||
+ | #*Make sure the time units attribute is of format <tt>"x since YYYY-MM-DD HH:ss:MM"</tt> where x is a time unit such as hours or days, and YYYY, MM, DD, HH, ss, and MM are reference year, month, day, hour, second, and minute. | ||
+ | #*The first time value in each file should be 0, corresponding with the reference time.<br><br> | ||
+ | #Include updated HEMCO Configuration file (if the data will be read via HEMCO). | ||
+ | #*Submit this as a pull request in the [https://github.com/geoschem/geos-chem-unittest geos-chem-unittest repository on GitHub].<br><br> | ||
+ | #Include README file detailing data source, contents, etc.<br><br> | ||
+ | #Include script(s) used to process original data<br><br> | ||
+ | #Include a summary or description of the expected results (e.g. emission totals for each species) | ||
== Further reading == | == Further reading == |
Latest revision as of 16:39, 13 July 2020
Contents
Overview
The GEOS-Chem Steering Committee (GCSC) encourages updates to the GEOS-Chem model. Updates to the GEOS-Chem model benefit both the developer and the entire GEOS-Chem community. The developer benefits through coauthorship and citations. Priority development needs are identified at GEOS-Chem users' meetings with updates between meetings based on GCSC input through working groups.
When should you submit updates to the GEOS-Chem code? Bug fixes should be submitted as soon as possible and be given high priority. Code related to model developments should be submitted when it is mature (i.e. a paper has been submitted). Your working group chair can offer guidance on the timing of submitting code to the GCST.
The practical aspects of submitting updates to the GEOS-Chem Support Team are outlined below.
Guidelines
- Contact your Working Group leaders to request that your changes be included in the standard code.
- The GEOS-Chem Steering Committee will meet every three months and set the GEOS-Chem model development priorities (i.e. decide on the order in which updates will be added to GEOS-Chem).
- The GEOS-Chem Steering Committee will meet every three months and set the GEOS-Chem model development priorities (i.e. decide on the order in which updates will be added to GEOS-Chem).
- Create or log into your GitHub account.
- Fork the GEOS-Chem repositories. Navigate to each appropriate GEOS-Chem repository on GitHub and click Fork in the top-right corner of the page.
- Clone your forks of the GEOS-Chem repositories to your system:
- git clone https://github.com/YOUR-USERNAME/geos-chem Code.12.0.0
- git clone https://github.com/YOUR-USERNAME/geos-chem-unittest UT.12.0.0.
- Add your modifications into a new branch off of the master branch.
- Test your update thoroughly and make sure that it works.
- You can use the GEOS-Chem Unit Tester to make sure your code updates work for a given combination of met fields, resolutions, and simulation types.
- A unit test looks for parallelization errors, floating point math issues, array-out-of-bounds errors, and creation of array temporaries.
- For structural updates, we recommend running difference tests often to ensure your updates don't impact model output.
- Review the code update checklist and data checklist below.
- Create a pull request in GitHub (recommended) or a Git patch file.
- See our pull request template on Github for information to include.
- See our pull request template on Github for information to include.
- The GEOS-Chem Support Team will add your changes to the standard code.
- Your update may be bundled with other code updates and the GEOS-Chem version number (X.Y.Z) will be changed.
- The update will be validated following the GEOS-Chem benchmarking procedure.
Checklist for code update submissions
- Use F90 free format whenever possible
- Include thorough comments throughout code updates
- Remove extraneous code updates (e.g. testing options, other science)
- Include full citations at the top of relevant source code modules
- Complete GCHP updates required with GEOS-Chem Classic updates
Checklist for data submissions
- Choose a final file naming convention for all data files that are to be submitted.
- Requesting files to be renamed after the benchmark process has begun can cause confusion and prevent reproducibility.
- Requesting files to be renamed after the benchmark process has begun can cause confusion and prevent reproducibility.
- Make sure netCDF data files are COARDS-compliant
- Use isCoards script and include output in submission.
- Use isCoards script and include output in submission.
- Concatenate individual netCDF files together so as to reduce tne number of files that need to be opened.
- e.g. If the data is a multi-year archive of monthly-mean data, create one file per year containing all data variables. Each data variable would have 12 timestamps, one for each month.
- Reducing the number of files that have to be opened results in more efficient GEOS-Chem simulations.
- Make sure files have been chunked and deflated.
- Deflation level 1 is usually sufficient for most files.
- e.g. ./nc_chunk.pl myfile.nc 1
- Follow these additional steps to ensure data can also be read by GCHP:
- All data variables should be of type float (aka REAL*4) or double (aka REAL*8). The MAPL library used by GCHP cannot read integer data from netCDF files. The time dimension is permitted to be integer.
- Use a recent reference date, if possible, for the time:units attribute. MAPL may have issues interpreting reference times such as 1750 which is prior to the Julian-to-Gregorian calendar switch.
- Make sure the time units attribute is of format "x since YYYY-MM-DD HH:ss:MM" where x is a time unit such as hours or days, and YYYY, MM, DD, HH, ss, and MM are reference year, month, day, hour, second, and minute.
- The first time value in each file should be 0, corresponding with the reference time.
- Include updated HEMCO Configuration file (if the data will be read via HEMCO).
- Submit this as a pull request in the geos-chem-unittest repository on GitHub.
- Submit this as a pull request in the geos-chem-unittest repository on GitHub.
- Include README file detailing data source, contents, etc.
- Include script(s) used to process original data
- Include a summary or description of the expected results (e.g. emission totals for each species)