|
|
(29 intermediate revisions by 3 users not shown) |
Line 1: |
Line 1: |
| __FORCETOC__ | | __FORCETOC__ |
− | '''''[[Downloading GEOS-Chem data directories|Previous]] | [[GEOS-Chem input files|Next]] | [[Getting_Started_with_GEOS-Chem|User Manual Home]] | [[Main_Page|GEOS-Chem Main Page]]''''' | + | '''''[[Downloading_data_from_Amazon_Web_Services_cloud_storage|Previous]] | [[Creating run directories for GEOS-Chem 13.0.0 and later|Next]] | [[Getting Started with GEOS-Chem]]''''' |
| #[[Minimum system requirements for GEOS-Chem|Minimum system requirements]] | | #[[Minimum system requirements for GEOS-Chem|Minimum system requirements]] |
| + | #[[Installing required software]] |
| + | #[[Configuring your computational environment]] |
| #[[Downloading GEOS-Chem source code|Downloading source code]] | | #[[Downloading GEOS-Chem source code|Downloading source code]] |
| #[[Downloading GEOS-Chem data directories|Downloading data directories]] | | #[[Downloading GEOS-Chem data directories|Downloading data directories]] |
| #<span style="color:blue">'''Creating run directories'''</span> | | #<span style="color:blue">'''Creating run directories'''</span> |
− | #[[GEOS-Chem input files|Configuring runs]] | + | #*[[Creating run directories for GEOS-Chem 13.0.0 and later|... for GEOS-Chem 13.0.0 and later versions]] |
| + | #*[[Creating run directories for GEOS-Chem 12.9.3 and prior|... for GEOS-Chem 12.9.3 and prior versions]] |
| + | #[[GEOS-Chem configuration files|Configuring runs]] |
| #[[Compiling GEOS-Chem|Compiling]] | | #[[Compiling GEOS-Chem|Compiling]] |
| #[[Running GEOS-Chem|Running]] | | #[[Running GEOS-Chem|Running]] |
| #[[GEOS-Chem output files|Output files]] | | #[[GEOS-Chem output files|Output files]] |
| + | #[[Python tools for use with GEOS-Chem]] |
| + | #[[GEOS-Chem_coding_and_debugging|Coding and debugging]] |
| + | #[[GEOS-Chem_overview#Further_reading|Further reading]] |
| | | |
| | | |
| == Overview == | | == Overview == |
| | | |
− | On this page we show you how to use the [[GEOS-Chem Unit Tester]] to generate and use fresh copies of run directories for your GEOS-Chem simulations.
| + | The method that you will use to download run directories changed in [[GEOS-Chem 13.0.0]]. Detailed instructions for downloading GEOS-Chem source code may be found in the links below: |
− | | + | |
− | == Downloading the GEOS-Chem Unit Tester ==
| + | |
− | | + | |
− | <span style="color:red">'''''IMPORTANT NOTE! As of June 2018, we have migrated the GEOS-Chem Unit Tester repository from Bitbucket back to Github. Going forward, please make sure to clone or pull code updates ONLY from the Github repository. Using Github will allow us to define a DOI for each GEOS-Chem version.'''''</span>
| + | |
− | | + | |
− | First, make sure that your system [[Debugging_with_the_GEOS-Chem_unit_tester#Requirements|has these software packages installed]]. (Most of these come standard with your Unix-based operating system.)
| + | |
− | | + | |
− | Next, clone the [[GEOS-Chem Unit Tester]] package with these commands:
| + | |
− | | + | |
− | git clone <nowiki>https://github.com/geoschem/geos-chem-unittest</nowiki> UT
| + | |
− | cd UT
| + | |
− | git checkout master
| + | |
− | | + | |
− | This will download a fresh copy of the <tt>GEOS-Chem-UnitTest</tt> repository into a directory named <tt>UT</tt>, and place you in the '''master''' branch. The '''master''' branch always corresponds to the last benchmarked GEOS-Chem version.
| + | |
− | | + | |
− | If you would like to check out the Unit Tester that corresponds to a specific numbered GEOS-Chem version you can type one of the following commands:
| + | |
− | | + | |
− | git checkout 12.0.0
| + | |
− | git checkout v11-02-rc
| + | |
− | git checkout v11-02e
| + | |
− | | + | |
− | etc.
| + | |
− | | + | |
− | ''NOTE: The Git clone process may take a few minutes to complete depending on your connection speed.''
| + | |
− | | + | |
− | --[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 21:10, 21 June 2018 (UTC)
| + | |
− | | + | |
− | == Available run directories ==
| + | |
− | | + | |
− | === Benchmark run directory ===
| + | |
− | | + | |
− | The [[GEOS-Chem Unit Tester]] includes a run directory called '''4x5_standard''' (GEOS-Chem v11-01 and later) or '''4x5_benchmark''' (GEOS-Chem v10-01). The standard [http://acmg.seas.harvard.edu/geos/geos_benchmark.html GEOS-Chem benchmark simulation] utilizes the [[GEOS-FP]] met fields ([[GEOS-Chem_horizontal_grids#GMAO_4_x_5_grid|4° x 5 °]], [[GEOS-Chem_vertical_grids#72-layer_vertical_grid|72 levels]]) with the [[#UCX chemistry mechanism|UCX tropospheric-stratospheric chemistry mechanism]] and [[Secondary organic aerosols|secondary organic aerosols]] included.
| + | |
− | | + | |
− | To reproduce our latest 1-month benchmark simulation, you can edit the <tt>CopyRunDirs.input</tt> file so that the following line is uncommented under <tt>RUNS</tt>:
| + | |
− | | + | |
− | geosfp 4x5 - benchmark 2016070100 2016080100 -
| + | |
− | | + | |
− | Make sure to check the start and end date so that your simulation will run for July 2016. You can then create the GEOS-Chem benchmark run directory by typing <tt>./gcCopyRunDirs</tt>. Navigate to your the newly created <tt>geosfp_4x5_benchmark</tt> run directory. To compile the GEOS-Chem source code, type:
| + | |
− | | + | |
− | make -j4 mpbuild
| + | |
− | | + | |
− | That will create a <tt>geos.mp</tt> executable file that you can use to submit your GEOS-Chem benchmark simulation to a queue system.
| + | |
− | | + | |
− | <span style="color:darkorange">'''''NOTE: If you are compiling GEOS-Chem within the code directory, and not within a run directory created from the GEOS-Chem Unit Tester, you will need to pass the <tt>UCX=y</tt> option in your <tt>make</tt> command.'''''</span>
| + | |
− | | + | |
− | === Other run directories ===
| + | |
− | | + | |
− | Run directories are provided for a combination of met field, grid, and simulation type and are named accordingly. For more details on the simulation types, please see our [[GEOS-Chem chemistry mechanisms|''GEOS-Chem chemistry mechanism'' wiki page]].
| + | |
− | | + | |
− | == Editing the CopyRunDirs.input file ==
| + | |
− | | + | |
− | Once you have downloaded the GEOS-Chem Unit Tester to your disk space, switch to the <tt>perl/</tt> directory:
| + | |
− | | + | |
− | cd UT/perl
| + | |
− | | + | |
− | In this directory there is a Perl script named <tt>gcCopyRunDirs</tt> that you will use to generate fresh copies of GEOS-Chem run directories. This script uses an input file named <tt>CopyRunDirs.input</tt>, which is also located in the <tt>perl</tt> directory.
| + | |
− | | + | |
− | <span style="color:red">'''IMPORTANT:''' The <tt>CopyRunDirs.input</tt> file that comes with the Unit Tester provides only a small sample of the possible run directories that you can create. You can add as many entries to the <tt>RUNS</tt> section as you wish. See the <tt>UnitTest.input</tt> file for a complete list of all possible run directories that you can add to <tt>CopyRunDirs.input</tt>.</span>
| + | |
− | | + | |
− | Your <tt>CopyRunDirs.input</tt> file will look something like this:
| + | |
− | | + | |
− | #------------------------------------------------------------------------------
| + | |
− | # GEOS-Chem Global Chemical Transport Model !
| + | |
− | #------------------------------------------------------------------------------
| + | |
− | #BOP
| + | |
− | #
| + | |
− | # !DESCRIPTION: Input file that specifies configuration for creating and
| + | |
− | # copying a run directory from the UnitTester.
| + | |
− | #\\
| + | |
− | #\\
| + | |
− | # !REMARKS:
| + | |
− | # For a complete description of how to customize the settings in the
| + | |
− | # INPUTS and RUNS sections, see the following wiki posts:
| + | |
− | #
| + | |
− | # wiki.geos-chem.org/Creating_GEOS-Chem_run_directories#Section_1:_INPUTS
| + | |
− | # wiki.geos-chem.org/Creating_GEOS-Chem_run_directories#Section_2:_RUNS
| + | |
− | #
| + | |
− | # !REVISION HISTORY:
| + | |
− | # 18 Mar 2015 - R. Yantosca - Initial version
| + | |
− | # 19 Mar 2015 - E. Lundgren - Simplify content for only copying run dirs
| + | |
− | # 19 May 2015 - R. Yantosca - Now can specify VERBOSE and WARNINGS options
| + | |
− | #EOP
| + | |
− | #------------------------------------------------------------------------------
| + | |
− | #
| + | |
− | # !INPUTS:
| + | |
− | #
| + | |
− | # %%% ID tags %%%
| + | |
− | #
| + | |
− | VERSION : v11-01
| + | |
− | DESCRIPTION : Create run directory from UnitTest
| + | |
− | #
| + | |
− | # %%% Data path and HEMCO settings %%%
| + | |
− | #
| + | |
− | DATA_ROOT : /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData
| + | |
− | HEMCO_ROOT : {DATAROOT}/HEMCO
| + | |
− | VERBOSE : 0
| + | |
− | WARNINGS : 1
| + | |
− | #
| + | |
− | # %%% Unit tester path names %%%
| + | |
− | #
| + | |
− | UNIT_TEST_ROOT : {HOME}/UT
| + | |
− | RUN_ROOT : {UTROOT}/runs
| + | |
− | RUN_DIR : {RUNROOT}/{RUNDIR}
| + | |
− | PERL_DIR : {UTROOT}/perl
| + | |
− | #
| + | |
− | # %%% Target directory and copy command %%%
| + | |
− | #
| + | |
− | COPY_PATH : {HOME}/GC/rundirs
| + | |
− | COPY_CMD : cp -rfL
| + | |
− | #
| + | |
− | # !RUNS:
| + | |
− | # Specify the runs directories that you want to copy below.
| + | |
− | # Here we provide a few examples, but you may copy additional entries from
| + | |
− | # UnitTest.input and modify the dates as needed. You can deactivate copying
| + | |
− | # run certain directories by commenting them out with "#".
| + | |
− | #
| + | |
− | #--------|-----------|------|------------|------------|------------|---------|
| + | |
− | # MET | GRID | NEST | SIMULATION | START DATE | END DATE | EXTRA? |
| + | |
− | #--------|-----------|------|------------|------------|------------|---------|
| + | |
− | geosfp 4x5 - standard 2013070100 2013080100 -
| + | |
− | # geosfp 4x5 - gc_timing 2013070100 2013070800 -
| + | |
− | # geosfp 4x5 - tropchem 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - soa 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - soa_svpoa 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - aciduptake 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - UCX 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - RRTMG 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - RnPbBe 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - Hg 2013010100 2013010101 -
| + | |
− | # geosfp 4x5 - POPs 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - TOMAS40 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - CH4 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - tagO3 2013070100 2013070101 -
| + | |
− | # geosfp 4x5 - tagCO 2013070100 2013070101 -
| + | |
− | # geosfp 2x25 - CO2 2013070100 201307010030 -
| + | |
− | # geosfp 4x5 - aerosol 2013070100 2013070101 -
| + | |
− | # geosfp 025x03125 ch tropchem 2013070100 201307010010 -
| + | |
− | # geosfp 025x03125 na tropchem 2013070100 201307010010 -
| + | |
− | # gchp 4x5 - tropchem 2013070100 2013070101 -
| + | |
− | !END OF RUNS:
| + | |
− | #EOP
| + | |
− | #------------------------------------------------------------------------------
| + | |
− | | + | |
− | ''NOTE: Lines starting with a <tt>#</tt> character will be treated as comments.''
| + | |
− | | + | |
− | The <tt>CopyRunDirs.input</tt> file has a layout that is very similar to the [[Debugging_with_the_GEOS-Chem_unit_tester#Specifying_unit_test_options_with_an_input_file|GEOS-Chem Unit Tester input files]] (<tt>UnitTest.input</tt> located within this directory). Like the GEOS-Chem Unit Tester input files, <tt>CopyRunDirs.input</tt> is composed of an <tt>INPUTS</tt> section and a <tt>RUNS</tt> section, which are described below.
| + | |
− | | + | |
− | === Section 1: INPUTS ===
| + | |
− | | + | |
− | Under the <tt>INPUTS</tt> section, you can customize the directory paths and other options for your system. Each configurable input is described in the table below.
| + | |
| | | |
| {| border=1 cellspacing=0 cellpadding=5 | | {| border=1 cellspacing=0 cellpadding=5 |
− | |-bgcolor="#CCCCCC" | + | |- bgcolor="#CCCCCC" |
− | !width="100px"|Option | + | !width="300px"|If you are using |
− | !width="825px"|Description | + | !width="500px"|Then please read this chapter |
| | | |
| |-valign="top" | | |-valign="top" |
− | |<tt>VERSION</tt> | + | |[[GEOS-Chem 13.0.0]] or later |
− | |An ID tag that will be added to all log files and output files. | + | |''[[Creating run directories for GEOS-Chem 13.0.0 and later]]'' |
| | | |
| |-valign="top" | | |-valign="top" |
− | |<tt>DESCRIPTION</tt> | + | |[[GEOS-Chem 12#12.9.3|GEOS-Chem 12.9.3]] or earlier |
− | |A short (1-sentence) description of the purpose of this specific file (optional). This may be used to differentiate different input files, such as if you pre-configure several for future re-use.
| + | |''[[Creating run directories for GEOS-Chem 12.9.3 and prior]]'' |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>DATA_ROOT</tt>
| + | |
− | |Specifies the path for your root-level data directory.
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>HEMCO_ROOT</tt>
| + | |
− | |Specifies the top-level path for the [[HEMCO data directories|HEMCO data directory tree]].
| + | |
− | *The <tt>{DATAROOT}</tt> token in <tt>HEMCO_ROOT</tt> will be replaced with the value you specify for <tt>RUN_ROOT</tt> option.
| + | |
− | <b>Leave this as-is.</b>
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>VERBOSE</tt>
| + | |
− | |Specifies the level of debug output that will be sent to the HEMCO log file. (0=no debug output; 3=max debug output)
| + | |
− | *Recommended setting: <tt>0</tt>
| + | |
− | | + | |
− | |-valign="top" | + | |
− | |<tt>WARNINGS</tt>
| + | |
− | |Specifies the level of warning messages that will be sent to the HEMCO log file. (0=no warnings; 3=max warnings)
| + | |
− | *Recommended setting: <tt>1</tt>
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>UNIT_TEST_ROOT</tt>
| + | |
− | |Specifies the path to the GEOS-Chem Unit Tester.
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>RUN_ROOT</tt>
| + | |
− | |Specifies the top-level unit test run directories.
| + | |
− | <b>Leave this as-is.</b>
| + | |
− | | + | |
− | |-valign="top" | + | |
− | |<tt>RUN_DIR</tt>
| + | |
− | |Specifies the run directory subdirectory.
| + | |
− | <b>Leave this as-is.</b>
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>PERL_DIR</tt>
| + | |
− | |Specifies the directory where the unit test Perl scripts are found.
| + | |
− | <b>Leave this as-is.</b>
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>COPY_PATH</tt>
| + | |
− | |Specifies the directory on your disk server where copies of the GEOS-Chem run directories will be created.
| + | |
− | | + | |
− | |-valign="top"
| + | |
− | |<tt>COPY_CMD</tt>
| + | |
− | |Specifies the command used to copy run directories from the GEOS-Chem Unit Tester to <tt>COPY_PATH</tt>.
| + | |
− | *The default setting is <tt>cp -rfL</tt>. This will create a new copy of the directory, even if the prior copy exists.
| + | |
− | *The <tt>-L</tt> option to the <tt>cp</tt> will create "hard" copies of files that are symbolic links. This command may differ slightly depending on the flavor of your Unix-based Operating system.
| + | |
| | | |
| |} | | |} |
| | | |
− | === Section 2: RUNS ===
| |
− |
| |
− | The layout of the <tt>RUNS</tt> section is identical to the <tt>RUNS</tt> section in the GEOS-Chem Unit Tester input file. This enables copying and pasting simulation settings text from <tt>UnitTest.input</tt> into <tt>CopyRunDirs.input</tt>.
| |
− |
| |
− | For example, the following line:
| |
− |
| |
− | #
| |
− | # !RUNS:
| |
− | # Specify the debugging runs that you want to perform below.
| |
− | # You can deactivate runs by commenting them out with "#".
| |
− | #
| |
− | #--------|-----------|------|------------|------------|------------|---------|
| |
− | # MET | GRID | NEST | SIMULATION | START DATE | END DATE | EXTRA? |
| |
− | #--------|-----------|------|------------|------------|------------|---------|
| |
− | geosfp 4x5 - standard 2013070100 2013080100 -
| |
− |
| |
− | will tell the <tt>gcCopyRunDirs</tt> script to create a run directory for a GEOS-Chem simulation with the following settings:
| |
− |
| |
− | *Using [[GEOS-FP]] met fields
| |
− | *On the [[GEOS-Chem_horizontal_grids#GMAO_4_x_5_grid|4° x 5° GEOS-Chem horizontal grid]]
| |
− | *For the [[NOx-Ox-HC-aerosol|GEOS-Chem standard full-chemistry simulation]]
| |
− | *Starting at 00:00 GMT on 2013/07/01
| |
− | *Ending at 00:00 GMT on 2013/08/01
| |
− |
| |
− | The date range will be used to initialize the <tt>input.geos</tt> file that is read during a GEOS-Chem simulation. Once the run directory is created, you may edit these dates within the <tt>input.geos</tt> file. Note, however, that time ranges must remain within the time range covered by the <tt>MET</tt> field you are using. You can check the <tt>MET</tt> field temporal coverage on the [[Overview_of_GMAO_met_data_products|GMAO met data products wiki page]]. '''Also note that we recommend using the restart files to spin up GEOS-Chem for at least one year prior to your simulation start date. You may use the resulting output restart files as initial values for your simulation.'''
| |
− |
| |
− | You can add as many entries to the RUNS section as you wish. See the <tt>UnitTest.input</tt> file for a complete list of all possible run directories entries that you can add to <tt>CopyRunDirs.input</tt>.
| |
− |
| |
− | == Generating a GEOS-Chem Run Directory ==
| |
− |
| |
− | Once you have edited the <tt>CopyRunDirs.input</tt> script to your liking, you can use that to generate fresh copies of GEOS-Chem run directories. Make sure you are in the <tt>perl</tt> directory, and then type:
| |
− |
| |
− | ./gcCopyRunDirs
| |
− |
| |
− | If you do not pass a file name to <tt>gcCopyRunDirs</tt>, then the <tt>gcCopyRunDirs</tt> script will use the <tt>CopyRunDirs.input</tt> file that you just modified.
| |
− |
| |
− | If you wish, you can create many customized copies of <tt>CopyRunDirs.input</tt>. For example, suppose you edit <tt>CopyRunDirs.input</tt> to generate a full-chemistry run directory. You can then save it as a separate file and use it explicitly with <tt>gcCopyRunDirs</tt>.
| |
− |
| |
− | cp CopyRunDirs.input CopyRunDirs.fullchem # Input file set up to only copy the full-chemistry run directories
| |
− |
| |
− | gcCopyRunDirs CopyRunDirs.Hg
| |
− |
| |
− | Executing <tt>gcCopyRunDirs</tt> will create a new GEOS-Chem run directory corresponding to each entry that you specified in the input file <tt>RUNS</tt> section. Each run directory will be created as a subdirectory of <tt>COPY_PATH</tt> that you specified in the input file <tt>INPUTS</tt> section.
| |
− |
| |
− | Let's examine the contents of a sample <tt>geosfp_4x5_standard</tt> run directory. Navigate to your newly created run directory and tssue the following commands:
| |
− |
| |
− | make fileclean # Remove any files left over from previous unit test runs
| |
− |
| |
− | ls -1 # Get directory listing
| |
− |
| |
− | And you will see this directory listing:
| |
− |
| |
− | brc.dat
| |
− | dust.dat
| |
− | FJX_j2j.dat
| |
− | FJX_spec.dat
| |
− | getRunInfo
| |
− | h2so4.dat
| |
− | HEMCO_Config.rc
| |
− | HEMCO_restart.201307010000.nc
| |
− | input.geos
| |
− | jv_spec_mie.dat
| |
− | Makefile
| |
− | org.dat
| |
− | output/
| |
− | README
| |
− | so4.dat
| |
− | soot.dat
| |
− | ssa.dat
| |
− | ssc.dat
| |
− | v11-01.run
| |
− | validate
| |
− |
| |
− | ''NOTE: Run directories for other simulations may contain other files not pictured here.''
| |
− |
| |
− | The <tt>input.geos</tt> and <tt>HEMCO_Config.rc</tt> files have been customized for this particular simulation. They were created from the corresponding template files <tt>input.geos.template</tt> and <tt>HEMCO_Config.template</tt> in the Unit Tester. The Perl script <tt>getRunInfo</tt> is used by the Makefile to extract information about the simulation from <tt>input.geos</tt>. HEMCO and tracer restart files are also included in every run directory but care must be taken when using them. See the section below for more information about restart files.
| |
− |
| |
− | == Important Notes About Restart Files ==
| |
− |
| |
− | Please see [[GEOS-Chem_basics#Restart_files|this post on the ''GEOS-Chem basics'' wiki page]] for more information about how to obtain sample restart files for your simulations.
| |
− |
| |
− | == Compiling GEOS-Chem ==
| |
− |
| |
− | Starting in v11-01, you can now compile GEOS-Chem directory from the run directory. See the[[GEOS-Chem_Makefile_Structure|GEOS-Chem Makefile Structure wiki page]] wiki page for more information about compiling GEOS-Chem and using the run directory Makefile.
| |
− |
| |
− | == Running GEOS-Chem ==
| |
− |
| |
− | We generally recommend users [[GEOS-Chem_Makefile_Structure#Compiling_in_a_run_directory|compile GEOS-Chem from within their run directory]] to create the <tt>geos</tt> (or <tt>geos.mp</tt>) executable. GEOS-Chem simulations can then be submitted to a local cluster's queue using a simple script. For example:
| |
− |
| |
− | #!/bin/bash
| |
− |
| |
− | ./geos.mp >> GC.log
| |
− |
| |
− | exit 0
| |
− |
| |
− | == GEOS-Chem Output Files ==
| |
| | | |
− | Once the GEOS-Chem simulation is complete, you will see several new files in the run directory. Some of these are generated for all simulations while others may be simulation-dependent. For more information, see our [[GEOS-Chem Output Files|''GEOS-Chem Output Files'' wiki page]].
| + | ---- |
| + | '''''[[Downloading_data_from_Amazon_Web_Services_cloud_storage|Previous]] | [[Creating run directories for GEOS-Chem 13.0.0 and later|Next]] | [[Getting Started with GEOS-Chem]]''''' |