Difference between revisions of "Creating GEOS-Chem run directories"

From Geos-chem
Jump to: navigation, search
(17 intermediate revisions by 3 users not shown)
Line 1: Line 1:
 
__FORCETOC__
 
__FORCETOC__
'''''[[Downloading GEOS-Chem data directories|Previous]] | [[GEOS-Chem input files|Next]] | [[Getting Started with GEOS-Chem]]'''''
+
'''''[[Downloading_data_from_Amazon_Web_Services_cloud_storage|Previous]] | [[Creating run directories for GEOS-Chem 13.0.0 and later|Next]] | [[Getting Started with GEOS-Chem]]'''''
 
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
 
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
 +
#[[Installing required software]]
 +
#[[Configuring your computational environment]]
 
#[[Downloading GEOS-Chem source code|Downloading source code]]
 
#[[Downloading GEOS-Chem source code|Downloading source code]]
 
#[[Downloading GEOS-Chem data directories|Downloading data directories]]
 
#[[Downloading GEOS-Chem data directories|Downloading data directories]]
 
#<span style="color:blue">'''Creating run directories'''</span>
 
#<span style="color:blue">'''Creating run directories'''</span>
#[[GEOS-Chem input files|Configuring runs]]
+
#*[[Creating run directories for GEOS-Chem 13.0.0 and later|... for GEOS-Chem 13.0.0 and later versions]]
 +
#*[[Creating run directories for GEOS-Chem 12.9.3 and prior|... for GEOS-Chem 12.9.3 and prior versions]]
 +
#[[GEOS-Chem configuration files|Configuring runs]]
 
#[[Compiling GEOS-Chem|Compiling]]
 
#[[Compiling GEOS-Chem|Compiling]]
 
#[[Running GEOS-Chem|Running]]
 
#[[Running GEOS-Chem|Running]]
 
#[[GEOS-Chem output files|Output files]]
 
#[[GEOS-Chem output files|Output files]]
#[[Guide_to_visualization_and_analysis_tools_for_GEOS-Chem|Visualizing and processing output]]
+
#[[Python tools for use with GEOS-Chem]]
 
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
 
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
 
#[[GEOS-Chem_overview#Further_reading|Further reading]]
 
#[[GEOS-Chem_overview#Further_reading|Further reading]]
Line 16: Line 20:
 
== Overview ==
 
== Overview ==
  
On this page we show you how to use the [[GEOS-Chem Unit Tester]] to generate and use fresh copies of run directories for your GEOS-Chem simulations. 
+
The method that you will use to download run directories changed in [[GEOS-Chem 13.0.0]]. Detailed instructions for downloading GEOS-Chem source code may be found in the links below:
 
+
== Downloading the GEOS-Chem Unit Tester ==
+
 
+
First, make sure that your system [[Unit_Tester_for_GEOS-Chem_12#Requirements|has these software packages installed]].  (Most of these come standard with your Unix-based operating system.)
+
 
+
Next, clone the [[GEOS-Chem Unit Tester]] package with these commands:
+
 
+
git clone <nowiki>https://github.com/geoschem/geos-chem-unittest</nowiki> UT
+
cd UT
+
git checkout master
+
 
+
This will download a fresh copy of the <tt>GEOS-Chem-UnitTest</tt> repository into a directory named <tt>UT</tt>, and place you in the '''master''' branch.  The '''master''' branch always corresponds to the last benchmarked GEOS-Chem version.
+
 
+
If you would like to check out the Unit Tester that corresponds to a specific numbered GEOS-Chem version you can type one of the following commands:
+
 
+
git checkout 12.0.0
+
git checkout v11-02-rc
+
git checkout v11-02e
+
 
+
etc.
+
 
+
''NOTE: The Git clone process may take a few minutes to complete depending on your connection speed.''
+
 
+
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 21:10, 21 June 2018 (UTC)
+
 
+
== Available run directories ==
+
 
+
=== Benchmark run directory ===
+
 
+
The [[GEOS-Chem Unit Tester]] includes a run directory called '''4x5_standard''' (GEOS-Chem v11-01 and later) or '''4x5_benchmark''' (GEOS-Chem v10-01). The standard [http://acmg.seas.harvard.edu/geos/geos_benchmark.html GEOS-Chem benchmark simulation] utilizes the [[GEOS-FP]] met fields ([[GEOS-Chem_horizontal_grids#GMAO_4_x_5_grid|4&deg; x 5 &deg;]], [[GEOS-Chem_vertical_grids#72-layer_vertical_grid|72 levels]]) with the [[#UCX chemistry mechanism|UCX tropospheric-stratospheric chemistry mechanism]] and [[Secondary organic aerosols|secondary organic aerosols]] included.
+
 
+
To reproduce our latest 1-month benchmark simulation, you can edit the <tt>CopyRunDirs.input</tt> file so that the following line is uncommented under <tt>RUNS</tt>:
+
 
+
  geosfp  4x5        -      benchmark    2016070100  2016080100  -
+
 
+
Make sure to check the start and end date so that your simulation will run for July 2016. You can then create the GEOS-Chem benchmark run directory by typing <tt>./gcCopyRunDirs</tt>. Navigate to your the newly created <tt>geosfp_4x5_benchmark</tt> run directory. To compile the GEOS-Chem source code, type:
+
 
+
make -j4 mpbuild
+
 
+
That will create a <tt>geos.mp</tt> executable file that you can use to submit your GEOS-Chem benchmark simulation to a queue system.
+
 
+
=== Other run directories ===
+
 
+
Run directories are provided for a combination of met field, grid, and simulation type and are named accordingly. For more details on the simulation types, please see our [[GEOS-Chem chemistry mechanisms|''GEOS-Chem chemistry mechanism'' wiki page]].
+
 
+
== Editing the CopyRunDirs.input file ==
+
 
+
Once you have downloaded the GEOS-Chem Unit Tester to your disk space, switch to the <tt>perl/</tt> directory:
+
 
+
cd UT/perl
+
 
+
In this directory there is a Perl script named <tt>gcCopyRunDirs</tt> that you will use to generate fresh copies of GEOS-Chem run directories.  This script uses an input file named <tt>CopyRunDirs.input</tt>, which is also located in the <tt>perl</tt> directory. 
+
 
+
<span style="color:red">'''IMPORTANT:''' The <tt>CopyRunDirs.input</tt> file that comes with the Unit Tester provides only a small sample of the possible run directories that you can create. You can add as many entries to the <tt>RUNS</tt> section as you wish. See the <tt>UnitTest.input</tt> file for a complete list of all possible run directories that you can add to <tt>CopyRunDirs.input</tt>.</span>
+
 
+
Your <tt>CopyRunDirs.input</tt> file will look something like this:
+
 
+
#------------------------------------------------------------------------------
+
#                  GEOS-Chem Global Chemical Transport Model                  !
+
#------------------------------------------------------------------------------
+
#BOP
+
#
+
# !DESCRIPTION: Input file that specifies configuration for creating and
+
#  copying a run directory from the UnitTester.
+
#\\
+
#\\
+
# !REMARKS:
+
#  For a complete description of how to customize the settings in the
+
#  INPUTS and RUNS sections, see the following wiki posts:
+
#
+
#  wiki.geos-chem.org/Creating_GEOS-Chem_run_directories#Section_1:_INPUTS
+
#  wiki.geos-chem.org/Creating_GEOS-Chem_run_directories#Section_2:_RUNS
+
#
+
# !REVISION HISTORY:
+
#  18 Mar 2015 - R. Yantosca - Initial version
+
#  19 Mar 2015 - E. Lundgren - Simplify content for only copying run dirs
+
#  19 May 2015 - R. Yantosca - Now can specify VERBOSE and WARNINGS options
+
#EOP
+
#------------------------------------------------------------------------------
+
#
+
# !INPUTS:
+
#
+
# %%% ID tags %%%
+
#
+
    VERSION        : v11-01
+
    DESCRIPTION    : Create run directory from UnitTest
+
#
+
# %%% Data path and HEMCO settings %%%
+
#
+
    DATA_ROOT      : /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData
+
    HEMCO_ROOT    : {DATAROOT}/HEMCO
+
    VERBOSE        : 0
+
    WARNINGS      : 1
+
#
+
# %%% Unit tester path names %%%
+
#
+
    UNIT_TEST_ROOT : {HOME}/UT
+
    RUN_ROOT      : {UTROOT}/runs
+
    RUN_DIR        : {RUNROOT}/{RUNDIR}
+
    PERL_DIR      : {UTROOT}/perl
+
#
+
# %%% Target directory and copy command %%%
+
#
+
    COPY_PATH      : {HOME}/GC/rundirs
+
    COPY_CMD      : cp -rfL
+
#
+
# !RUNS:
+
#  Specify the runs directories that you want to copy below.
+
#  Here we provide a few examples, but you may copy additional entries from
+
#  UnitTest.input and modify the dates as needed. You can deactivate copying
+
#  run certain directories by commenting them out with "#".
+
#
+
#--------|-----------|------|------------|------------|------------|---------|
+
# MET    | GRID      | NEST | SIMULATION | START DATE | END DATE  | EXTRA?  |
+
#--------|-----------|------|------------|------------|------------|---------|
+
  geosfp  4x5        -      standard    2013070100  2013080100  -
+
# geosfp  4x5        -      gc_timing    2013070100  2013070800  -
+
# geosfp  4x5        -      tropchem    2013070100  2013070101  -
+
# geosfp  4x5        -      soa          2013070100  2013070101  -
+
# geosfp  4x5        -      soa_svpoa    2013070100  2013070101  -
+
# geosfp  4x5        -      aciduptake  2013070100  2013070101  -
+
# geosfp  4x5        -      UCX          2013070100  2013070101  -
+
# geosfp  4x5        -      RRTMG        2013070100  2013070101  -
+
# geosfp  4x5        -      RnPbBe      2013070100  2013070101  -
+
# geosfp  4x5        -      Hg          2013010100  2013010101  -
+
# geosfp  4x5        -      POPs        2013070100  2013070101  -
+
# geosfp  4x5        -      TOMAS40      2013070100  2013070101  -
+
# geosfp  4x5        -      CH4          2013070100  2013070101  -
+
# geosfp  4x5        -      tagO3        2013070100  2013070101  -
+
# geosfp  4x5        -      tagCO        2013070100  2013070101  -
+
# geosfp  2x25        -      CO2          2013070100  201307010030 -
+
# geosfp  4x5        -      aerosol      2013070100  2013070101  -
+
# geosfp  025x03125  ch    tropchem    2013070100  201307010010 -
+
# geosfp  025x03125  na    tropchem    2013070100  201307010010 -
+
# gchp    4x5        -      tropchem    2013070100  2013070101  -
+
!END OF RUNS:
+
#EOP
+
#------------------------------------------------------------------------------
+
 
+
''NOTE: Lines starting with a <tt>#</tt> character will be treated as comments.''
+
 
+
The <tt>CopyRunDirs.input</tt> file has a layout that is very similar to the [[Debugging_with_the_GEOS-Chem_unit_tester#Specifying_unit_test_options_with_an_input_file|GEOS-Chem Unit Tester input files]] (<tt>UnitTest.input</tt> located within this directory). Like the GEOS-Chem Unit Tester input files, <tt>CopyRunDirs.input</tt> is composed of an <tt>INPUTS</tt> section and a <tt>RUNS</tt> section, which are described below.
+
 
+
=== Section 1: INPUTS ===
+
 
+
Under the <tt>INPUTS</tt> section, you can customize the directory paths and other options for your system. Each configurable input is described in the table below.
+
  
 
{| border=1 cellspacing=0 cellpadding=5  
 
{| border=1 cellspacing=0 cellpadding=5  
|-bgcolor="#CCCCCC"
+
|- bgcolor="#CCCCCC"
!width="100px"|Option
+
!width="300px"|If you are using
!width="825px"|Description
+
!width="500px"|Then please read this chapter
  
 
|-valign="top"
 
|-valign="top"
|<tt>VERSION</tt>
+
|[[GEOS-Chem 13.0.0]] or later
|An ID tag that will be added to all log files and output files.
+
|''[[Creating run directories for GEOS-Chem 13.0.0 and later]]''
  
 
|-valign="top"
 
|-valign="top"
|<tt>DESCRIPTION</tt>
+
|[[GEOS-Chem 12#12.9.3|GEOS-Chem 12.9.3]] or earlier
|A short (1-sentence) description of the purpose of this specific file (optional). This may be used to differentiate different input files, such as if you pre-configure several for future re-use.
+
|''[[Creating run directories for GEOS-Chem 12.9.3 and prior]]''
 
+
|-valign="top"
+
|<tt>DATA_ROOT</tt>
+
|Specifies the path for your root-level data directory. 
+
 
+
|-valign="top"
+
|<tt>HEMCO_ROOT</tt>
+
|Specifies the top-level path for the [[HEMCO data directories|HEMCO data directory tree]].
+
*The <tt>{DATAROOT}</tt> token in <tt>HEMCO_ROOT</tt> will be replaced with the value you specify for <tt>RUN_ROOT</tt> option. 
+
<b>Leave this as-is.</b>
+
 
+
|-valign="top"
+
|<tt>VERBOSE</tt>
+
|Specifies the level of debug output that will be sent to the HEMCO log file. (0=no debug output; 3=max debug output)
+
*Recommended setting: <tt>0</tt>
+
 
+
|-valign="top"
+
|<tt>WARNINGS</tt>
+
|Specifies the level of warning messages that will be sent to the HEMCO log file. (0=no warnings; 3=max warnings)
+
*Recommended setting: <tt>1</tt>
+
 
+
|-valign="top"
+
|<tt>UNIT_TEST_ROOT</tt>
+
|Specifies the path to the GEOS-Chem Unit Tester.
+
 
+
|-valign="top"
+
|<tt>RUN_ROOT</tt>
+
|Specifies the top-level unit test run directories.
+
<b>Leave this as-is.</b>
+
 
+
|-valign="top"
+
|<tt>RUN_DIR</tt>
+
|Specifies the run directory subdirectory.
+
<b>Leave this as-is.</b>
+
 
+
|-valign="top"
+
|<tt>PERL_DIR</tt>
+
|Specifies the directory where the unit test Perl scripts are found.
+
<b>Leave this as-is.</b>
+
 
+
|-valign="top"
+
|<tt>COPY_PATH</tt>
+
|Specifies the directory on your disk server where copies of the GEOS-Chem run directories will be created.
+
 
+
|-valign="top"
+
|<tt>COPY_CMD</tt>
+
|Specifies the command used to copy run directories from the GEOS-Chem Unit Tester to <tt>COPY_PATH</tt>.
+
*The default setting is <tt>cp -rfL</tt>. This will create a new copy of the directory, even if the prior copy exists.
+
*The <tt>-L</tt> option to the <tt>cp</tt> will create "hard" copies of files that are symbolic links.  This command may differ slightly depending on the flavor of your Unix-based Operating system.
+
  
 
|}
 
|}
  
=== Section 2: RUNS ===
 
 
The layout of the <tt>RUNS</tt> section is identical to the <tt>RUNS</tt> section in the GEOS-Chem Unit Tester input file. This enables copying and pasting simulation settings text from <tt>UnitTest.input</tt> into <tt>CopyRunDirs.input</tt>.
 
 
For example, the following line:
 
 
#
 
# !RUNS:
 
#  Specify the debugging runs that you want to perform below.
 
#  You can deactivate runs by commenting them out with "#".
 
#
 
#--------|-----------|------|------------|------------|------------|---------|
 
# MET    | GRID      | NEST | SIMULATION | START DATE | END DATE  | EXTRA?  |
 
#--------|-----------|------|------------|------------|------------|---------|
 
  geosfp  4x5        -      standard    2013070100  2013080100  -
 
 
will tell the <tt>gcCopyRunDirs</tt> script to create a run directory for a GEOS-Chem simulation with the following settings:
 
 
*Using [[GEOS-FP]] met fields
 
*On the [[GEOS-Chem_horizontal_grids#GMAO_4_x_5_grid|4&deg; x 5&deg; GEOS-Chem horizontal grid]]
 
*For the [[NOx-Ox-HC-aerosol|GEOS-Chem standard full-chemistry simulation]]
 
*Starting at 00:00 GMT on 2013/07/01
 
*Ending at 00:00 GMT on 2013/08/01
 
 
The date range will be used to initialize the <tt>input.geos</tt> file that is read during a GEOS-Chem simulation. Once the run directory is created, you may edit these dates within the <tt>input.geos</tt> file. Note, however, that time ranges must remain within the time range covered by the <tt>MET</tt> field you are using. You can check the <tt>MET</tt> field temporal coverage on the [[Overview_of_GMAO_met_data_products|GMAO met data products wiki page]]. '''Also note that we recommend using the restart files to spin up GEOS-Chem for at least one year prior to your simulation start date. You may use the resulting output restart files as initial values for your simulation.'''
 
 
You can add as many entries to the RUNS section as you wish. See the <tt>UnitTest.input</tt> file for a complete list of all possible run directories entries that you can add to <tt>CopyRunDirs.input</tt>.
 
 
== Generating a GEOS-Chem Run Directory ==
 
 
Once you have edited the <tt>CopyRunDirs.input</tt> script to your liking, you can use that to generate fresh copies of GEOS-Chem run directories.  Make sure you are in the <tt>perl</tt> directory, and then type:
 
 
./gcCopyRunDirs
 
 
If you do not pass a file name to <tt>gcCopyRunDirs</tt>, then the <tt>gcCopyRunDirs</tt> script will use the <tt>CopyRunDirs.input</tt> file that you just modified. 
 
 
If you wish, you can create many customized copies of <tt>CopyRunDirs.input</tt>.  For example, suppose you edit <tt>CopyRunDirs.input</tt> to generate a full-chemistry run directory. You can then save it as a separate file and use it explicitly with <tt>gcCopyRunDirs</tt>.
 
 
cp CopyRunDirs.input CopyRunDirs.fullchem  # Input file set up to only copy the full-chemistry run directories
 
 
 
gcCopyRunDirs CopyRunDirs.Hg               
 
 
Executing <tt>gcCopyRunDirs</tt> will create a new GEOS-Chem run directory corresponding to each entry that you specified in the input file <tt>RUNS</tt> section.  Each run directory will be created as a subdirectory of <tt>COPY_PATH</tt> that you specified in the input file <tt>INPUTS</tt> section.
 
 
Let's examine the contents of a sample <tt>geosfp_4x5_standard</tt> run directory.  Navigate to your newly created run directory and tssue the following commands:
 
 
make fileclean                      # Remove any files left over from previous unit test runs
 
 
ls -1                                # Get directory listing
 
 
And you will see this directory listing:
 
 
benchmark.run
 
GEOSChem.Restart.20160701_0000z.nc4
 
getRunInfo
 
HEMCO_Config.rc
 
HEMCO_Diagn.rc
 
HISTORY.rc
 
input.geos
 
Makefile
 
output/
 
README
 
validate
 
 
''NOTE: Run directories for other simulations may contain other files not pictured here.''
 
 
The <tt>input.geos</tt> and <tt>HEMCO_Config.rc</tt> files have been customized for this particular simulation.  They were created from the corresponding template files <tt>input.geos.template</tt> and <tt>HEMCO_Config.template</tt> in the Unit Tester. The Perl script <tt>getRunInfo</tt> is used by the Makefile to extract information about the simulation from <tt>input.geos</tt>. HEMCO and tracer restart files are also included in every run directory but care must be taken when using them. See the section below for more information about restart files.
 
  
 
----
 
----
'''''[[Downloading GEOS-Chem data directories|Previous]] | [[GEOS-Chem input files|Next]] | [[Getting Started with GEOS-Chem]]'''''
+
'''''[[Downloading_data_from_Amazon_Web_Services_cloud_storage|Previous]] | [[Creating run directories for GEOS-Chem 13.0.0 and later|Next]] | [[Getting Started with GEOS-Chem]]'''''

Revision as of 21:30, 8 December 2020

Previous | Next | Getting Started with GEOS-Chem

  1. Minimum system requirements
  2. Installing required software
  3. Configuring your computational environment
  4. Downloading source code
  5. Downloading data directories
  6. Creating run directories
  7. Configuring runs
  8. Compiling
  9. Running
  10. Output files
  11. Python tools for use with GEOS-Chem
  12. Coding and debugging
  13. Further reading


Contents

Overview

The method that you will use to download run directories changed in GEOS-Chem 13.0.0. Detailed instructions for downloading GEOS-Chem source code may be found in the links below:

If you are using Then please read this chapter
GEOS-Chem 13.0.0 or later Creating run directories for GEOS-Chem 13.0.0 and later
GEOS-Chem 12.9.3 or earlier Creating run directories for GEOS-Chem 12.9.3 and prior



Previous | Next | Getting Started with GEOS-Chem