Difference between revisions of "Creating GEOS-Chem run directories"
(→Generating a Run Directory) |
(→Compiling and Running GEOS-Chem) |
||
| Line 238: | Line 238: | ||
== Compiling and Running GEOS-Chem == | == Compiling and Running GEOS-Chem == | ||
| − | Once you have created one or more GEOS-Chem run directories, you can use them for your GEOS-Chem simulations. First, | + | Once you have created one or more GEOS-Chem run directories, you can use them for your GEOS-Chem simulations. First, double-check that the settings in the <tt>input.geos</tt> and <tt>HEMCO_Config.rc</tt> files are correct for the simulation that you are trying to run. See the [http://manual.geos-chem.org/ GEOS-Chem Manual] for more information about setting simulation options. |
| + | |||
| + | Next, open the Makefile in a text editor and define the variables CODE_DIR, LOG_DIR, and VERSION specific to your simulation. CODE_DIR is the path to the GEOS-Chem source code you wish to use. LOG_DIR is the path where you wish to store your log files. VERSION is text that you wish to prefix your log filename. Each of these three variables can also be passed manually to the make command, overwriting the definitions stored in the Makefile. | ||
To start a GEOS-Chem simulation, type the following commands. (We'll use the example of the <tt>geosfp_4x5_fullchem</tt> simulation, from the previous sections.) | To start a GEOS-Chem simulation, type the following commands. (We'll use the example of the <tt>geosfp_4x5_fullchem</tt> simulation, from the previous sections.) | ||
| − | make -j4 | + | make -j4 TRACEBACK=y mp |
| − | The above command specifies | + | The above command specifies whether to show traceback for debugging (<tt>TRACEBACK</tt>) and invokes the <tt>mp</tt> target in the Makefile. This results in compiling the source code at path CODE_DIR with [[Parallelizing_GEOS-Chem|OpenMP parallelization]] and then running the resultant GEOS-Chem executable using the simulation settings in the configuration files. Output files will have suffix <tt>.mp</tt> to indicate the OpenMP setting used. |
| − | + | There are many other targets that may be used with Make For example, you can specify <tt>sp</tt> to compile and run code using a single processor. This will result in output files having the <tt>.sp</tt> rather than <tt>.mp</tt> suffix. Alternatively, you build GEOS-Chem source code without running a simulation (<tt>mpbuild</tt>) or run a simulation without rebuilding (<tt>mprun</tt>). There are also several housekeeping targets that clean the run directory and source code (e.g. <tt>mpdataclean</tt>, <tt>mplogclean</tt>, and <tt>mpclean</tt>). See the <tt>Makefile</tt> for all make options and [http://acmg.seas.harvard.edu/geos/doc/man/chapter_3.html#RunDir Section 3.6.2 of the GEOS-Chem Manual] for more information about makefiles. | |
== GEOS-Chem Output Files == | == GEOS-Chem Output Files == | ||
Revision as of 16:01, 15 April 2015
On this page we show you how to use the GEOS-Chem Unit Tester to generate and use fresh copies of run directories for your GEOS-Chem simulations.
Contents
Downloading the GEOS-Chem Unit Tester
First, make sure that your system has these software packages installed. (Most of these come standard with your Unix-based operating system.)
Next, clone the GEOS-Chem unit tester package with the command:
git clone -b v10-01-public git://git.as.harvard.edu/bmy/GEOS-Chem-UnitTest UT
This will create a copy of the GEOS-Chem Unit Tester package in a directory named UT for short.
The option
-b v10-01-public
specifies which branch of the GEOS-Chem Unit Tester that you would like to download. Branches in the GEOS-Chem unit tester correspond to each internal version of GEOS-Chem (i.e. the states of the code for which we run 1-month benchmark simulations), as well as the publicly-released versions. Most likely, you will want to download the v10-01-public branch of the GEOS-Chem Unit Tester, which corresponds to the public release of GEOS-Chem v10-01.
--Bob Y. 13:36, 19 March 2015 (EDT)
Editing the CopyRunDirs.input file
Once you have downloaded the GEOS-Chem Unit Tester to your disk space, switch to the perl/ directory:
cd UT/perl
In this directory there is a Perl script named gcCopyRunDirs that you will use to generate fresh copies of GEOS-Chem run directories. This script uses an input file named CopyRunDirs.input, which is also located in the perl directory.
Your CopyRunDirs.input file will look something like this:
#------------------------------------------------------------------------------
# GEOS-Chem Global Chemical Transport Model !
#------------------------------------------------------------------------------
#BOP
#
# !DESCRIPTION: Input file that specifies configuration for creating and
# copying a run directory from the UnitTester.
#\\
#\\
# !REMARKS:
# Customize the run directory for your system by specifying these values:
# -------------------------------------------------------------------
# VERSION : A tag used to identify this Unit Test (e.g. v10-01h)
# DESCRIPTION : A short description of this file's run dir copy configuration
# COPY_PATH : Local path where run directory will be copied to
# DATA_ROOT : Root GEOS-Chem data directory
# HEMCO_ROOT : Root directory where HEMCO emissions data files are stored
# RUN_ROOT : Unit test run directories are subdirectories of RUN_ROOT
# RUN_DIR : Individual unit test run directory path
# PERL_DIR : Unit Test perl script directory (i.e. this directory)
# COPY_CMD : Unix copy command with optional tags
#
# !REVISION HISTORY:
# 18 Mar 2015 - R. Yantosca - Initial version
# 19 Mar 2015 - E. Lundgren - Simplify content for only copying run dirs
#EOP
#------------------------------------------------------------------------------
#
# !INPUTS:
#
VERSION : v10-01
DESCRIPTION : Create run directory from UnitTest
COPY_PATH : /home/{USER}/GC/rundirs
DATA_ROOT : /as/data/geos/ExtData
HEMCO_ROOT : {DATAROOT}/HEMCO
RUN_ROOT : /home/{USER}/UT/runs
RUN_DIR : {RUNROOT}/{RUNDIR}
PERL_DIR : /home/{USER}/UT/perl
COPY_CMD : cp -rfL
#
# !RUNS:
# Specify the debugging runs that you want to perform below.
# Here we provide a few examples, but you may copy additional entries from
# UnitTest.input and modify the dates as needed. You can deactivate copying
# run certain directories by commenting them out with "#".
#
#--------|-----------|------|------------|------------|------------|---------|
# MET | GRID | NEST | SIMULATION | START DATE | END DATE | EXTRA? |
#--------|-----------|------|------------|------------|------------|---------|
geosfp 4x5 - fullchem 2013070100 2013070101 -
geosfp 4x5 - soa 2013070100 2013070101 -
geosfp 4x5 - soa_svpoa 2013070100 2013070101 -
geosfp 4x5 - UCX 2013070100 2013070101 UCX=y
geosfp 4x5 - RRTMG 2013070100 2013070101 RRTMG=y
geosfp 4x5 - RnPbBe 2013070100 2013070101 -
geosfp 4x5 - Hg 2013070100 2013070101 -
geosfp 4x5 - POPs 2013070100 2013070101 -
geosfp 4x5 - CH4 2013070100 2013070101 -
geosfp 4x5 - tagO3 2013070100 2013070101 -
geosfp 4x5 - tagCO 2013070100 2013070101 -
geosfp 2x25 - CO2 2013070100 201307010030 -
geosfp 4x5 - aerosol 2013070100 2013070101 -
geosfp 025x03125 ch fullchem 2013070100 201307010010 -
geosfp 025x03125 na fullchem 2013070100 201307010010 -
!END OF RUNS:
#EOP
#------------------------------------------------------------------------------
NOTE: Lines starting with a # character will be treated as comments.
The CopyRunDirs.input file has a layout that is very similar to the GEOS-Chem Unit Tester input files (UnitTest.input located within this directory). Like the GEOS-Chem Unit Tester input files, CopyRunDirs.input is composed of an INPUTS<\tt> section and a <tt>RUNS<\tt> section, which are described below.
Section 1: INPUTS
Under the <tt>INPUTS section, you can customize the directory paths and other options for your system. Each configurable input is described in the table below.
| Option | Description |
|---|---|
| VERSION | An ID tag that will be added to all log files and output files. |
| DESCRIPTION | A short (1-sentence) description of the purpose of this specific file (optional). This may be used to differentiate different input files, such as if you pre-configure several for future re-use. |
| COPY_PATH | Specifies the root directory on your disk server where copies of the GEOS-Chem run directories will be created. |
| DATA_ROOT | Specifies the path for your root-level data directory. |
| HEMCO_ROOT | Specifies the top-level path for the HEMCO data directory tree.
Leave this as-is. |
| RUN_ROOT | Specifies the top-level unit test run directories.
Leave this as-is. |
| PERL_DIR | Specifies the directory where the unit test Perl scripts are found.
Leave this as-is. |
| COPY_CMD | Specifies the command used to copy run directories from the GEOS-Chem Unit Tester to COPY_PATH.
|
--Bob Y. 14:20, 19 March 2015 (EDT)
Section 2: RUNS
The layout of the RUNS section is identical to the RUNS section in the GEOS-Chem Unit Tester input file. This enables copying and pasting simulation settings text from UnitTest.input into CopyRunDirs.input.
For example, the following line:
# # !RUNS: # Specify the debugging runs that you want to perform below. # You can deactivate runs by commenting them out with "#". # #--------|-----------|------|------------|------------|------------|---------| # MET | GRID | NEST | SIMULATION | START DATE | END DATE | EXTRA? | #--------|-----------|------|------------|------------|------------|---------| geosfp 4x5 - fullchem 2013070100 2013070101 -
will tell the gcCopyRunDirs script to create a run directory for a GEOS-Chem simulation using:
- GEOS-FP met fields
- On the 4° x 5° GEOS-Chem horizontal grid
- For the GEOS-Chem standard full-chemistry simulation
- Starting at 00:00 GMT on 2013/07/01
- Ending at 01:00 GMT on 2013/07/01
The date range will be used to initialize the input.geos file that is read during a GEOS-Chem simulation. Once the run directory is created, you may edit these dates within the input.geos file. Note, however, that time ranges must remain within the time range covered by the MET field you are using. You can check the MET field temporal coverage on the GMAO met data products wiki page. Also note that we recommend using the restart files to spin up GEOS-Chem for at least one year prior to your simulation start date. You may use the resulting output restart files as initial values for your simulation.
You can add as many entries to the RUNS section as you wish. Simply comment out the lines containing runs you may wish to copy in the future.
Generating a Run Directory
Once you have edited the CopyRunDirs.input script to your liking, you can use that to generate fresh copies of GEOS-Chem run directories. Make sure you are in the perl directory, and then type:
gcCopyRunDirs
If you do not pass a file name to gcCopyRunDirs, then the gcCopyRunDirs script will use the CopyRunDirs.input file that you just created.
If you wish, you can create many customized copies of CopyRunDirs.input. For example, suppose you edit CopyRunDirs.input to generate a full-chemistry run directory. You can then save it as a separate file and use it explicitly with gcCopyRunDirs.
cp CopyRunDirs.input CopyRunDirs.fullchem # Input file set up to only copy the full-chemistry run directories gcCopyRunDirs CopyRunDirs.Hg
Executing gcCopyRunDirs will create a new GEOS-Chem run directory corresponding to each entry that you specified in the input file RUNS. Each run directory will be created as a subdirectory of COPY_PATH that you specified in the input file INPUTS section.
Let's examine the contents of a sample geosfp_4x5_fullchem run directory. Issue the following commands:
cd ~/GC/rundirs/geosfp_4x5_fullchem # Change to geosfp_4x5_fullchem run dir make fileclean # Remove any files left over from previous unit test runs ls -l # Get detailed directory listing
And you will see this directory listing:
-rw-r--r-- 1 mpayer mpayer 7862 2015-04-03 15:19 FJX_j2j.dat -rw-r--r-- 1 mpayer mpayer 44808 2015-04-03 15:19 FJX_spec.dat -rw-r--r-- 1 mpayer mpayer 186359 2015-04-03 15:19 HEMCO_Config.rc -rw-r--r-- 1 mpayer mpayer 4872 2015-04-03 15:19 Makefile -rw-r--r-- 1 mpayer mpayer 783 2015-04-03 15:19 README -rw-r--r-- 1 mpayer mpayer 610 2015-04-03 15:19 chemga.dat -rw-r--r-- 1 mpayer mpayer 42186 2015-04-03 15:19 dust.dat -rwxr-xr-x 1 mpayer mpayer 1458 2015-04-03 15:19 getEnd.pl* -rwxr-xr-x 1 mpayer mpayer 1374 2015-04-03 15:19 getStart.pl* -rw-r--r-- 1 mpayer mpayer 199632 2015-04-03 15:19 globchem.dat -rw-r--r-- 1 mpayer mpayer 401972 2015-04-03 15:19 initial_hemco_rst.geosfp_4x5_fullchem.nc -rw-r--r-- 1 mpayer mpayer 41110480 2015-04-03 15:19 initial_trac_rst.geosfp_4x5_fullchem -rw-r--r-- 1 mpayer mpayer 15913 2015-04-03 15:19 input.geos -rw-r--r-- 1 mpayer mpayer 26194 2015-04-03 15:19 jv_spec_mie.dat -rw-r--r-- 1 mpayer mpayer 905 2015-04-03 15:19 mglob.dat -rw-r--r-- 1 mpayer mpayer 42185 2015-04-03 15:19 org.dat -rw-r--r-- 1 mpayer mpayer 42185 2015-04-03 15:19 so4.dat -rw-r--r-- 1 mpayer mpayer 42186 2015-04-03 15:19 soot.dat -rw-r--r-- 1 mpayer mpayer 42185 2015-04-03 15:19 ssa.dat -rw-r--r-- 1 mpayer mpayer 42185 2015-04-03 15:19 ssc.dat -rwxr-xr-x 1 mpayer mpayer 7303 2015-04-03 15:19 validate*
NOTE: Run directories for other simulations may contain other files not pictured here.
The input.geos and HEMCO_Config.rc files have been customized for this particular simulation. They were created from the corresponding template files input.geos.template and HEMCO_Config.template in the Unit Tester.
The Perl script getRunInfo is used by the Makefile to extract information about the simulation from input.geos.
--Lizzie Lundgren 10:50, 15 April 2015 (EDT)
Compiling and Running GEOS-Chem
Once you have created one or more GEOS-Chem run directories, you can use them for your GEOS-Chem simulations. First, double-check that the settings in the input.geos and HEMCO_Config.rc files are correct for the simulation that you are trying to run. See the GEOS-Chem Manual for more information about setting simulation options.
Next, open the Makefile in a text editor and define the variables CODE_DIR, LOG_DIR, and VERSION specific to your simulation. CODE_DIR is the path to the GEOS-Chem source code you wish to use. LOG_DIR is the path where you wish to store your log files. VERSION is text that you wish to prefix your log filename. Each of these three variables can also be passed manually to the make command, overwriting the definitions stored in the Makefile.
To start a GEOS-Chem simulation, type the following commands. (We'll use the example of the geosfp_4x5_fullchem simulation, from the previous sections.)
make -j4 TRACEBACK=y mp
The above command specifies whether to show traceback for debugging (TRACEBACK) and invokes the mp target in the Makefile. This results in compiling the source code at path CODE_DIR with OpenMP parallelization and then running the resultant GEOS-Chem executable using the simulation settings in the configuration files. Output files will have suffix .mp to indicate the OpenMP setting used.
There are many other targets that may be used with Make For example, you can specify sp to compile and run code using a single processor. This will result in output files having the .sp rather than .mp suffix. Alternatively, you build GEOS-Chem source code without running a simulation (mpbuild) or run a simulation without rebuilding (mprun). There are also several housekeeping targets that clean the run directory and source code (e.g. mpdataclean, mplogclean, and mpclean). See the Makefile for all make options and Section 3.6.2 of the GEOS-Chem Manual for more information about makefiles.
GEOS-Chem Output Files
Once the GEOS-Chem simulation is complete, you will see several new files in the run directory. Some of these are generated for all simulations while others may be simulation-dependent. For the case of our full-chemistry simulation example, the following files are output:
| File | Description |
|---|---|
| trac_avg.geosfp_4x5_fullchem.YYYYMMDDhh.mp or trac_avg.geosfp_4x5_fullchem.YYYYMMDDhh.sp |
Diagnostic output file that contains time-averaged output for diagnostics enabled in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| trac_rst.geosfp_4x5_fullchem.YYYYMMDDhhmm.mp or trac_rst.geosfp_4x5_fullchem.YYYYMMDDhhmm.sp |
Tracer restart file that contains instantaneous tracer concentrations at simulation end time. This file saves instantaneous concentrations of all transported tracers (listed in the Tracer Menu of input.geos) on all levels and may be used to initialize another GEOS-Chem simulation.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| HEMCO_restart.YYYYMMDDhh.nc.mp or HEMCO_restart.YYYYMMDDhh.nc.sp |
HEMCO restart file containing several quantities tracked by HEMCO for use in initializing another GEOS-Chem simulation. This file includes data used to calculate soil NOx emissions, PARANOX ship plume chemistry, and MEGAN biogenic emissions.
Data in this file are in netcdf format. |
| v10-01.geosfp_4x5_fullchem.mp or v10-01.geosfp_4x5_fullchem.sp |
GEOS-Chem simulation log file.
Open this file in a text editor to diagnose compile and run errors. |
| HEMCO.log.mp or HEMCO.log.sp |
HEMCO log file containing detailed information about the emissions read from disk into HEMCO.
Open this file in a text editor to diagnose HEMCO errors. |
| smv2.log | SMVGEAR II log file containing information about reactions and species used by the ND65 prod-loss diagnostic. This file is only produced when SMVGEAR, and thus it will not be created for the specialty simulations.
Open this file in a text editor to see if SMVGEAR read the globchem.dat file properly. |
| diaginfo.dat | File generated by GEOS-Chem that contains diagnostic quantities for use with GAMAP. |
| tracerinfo.dat | File generated by GEOS-Chem that contains tracer name metadata for use with GAMAP. |
Additional files may be created by GEOS-Chem, depending on your simulation type and the options specified in the input.geos file. Other optional GEOS-Chem output files include:
| File | Description |
|---|---|
| spec_rst.geosfp_4x5_fullchem.YYYYMMDDhh | Species restart file that contains instantaneous species concentrations at simulation end time. This file saves instantaneous concentrations of all chemical species (listed in globchem.dat) on all levels and may be used to initialize another GEOS-Chem simulation.
NOTE: If you are going to be running a very long GEOS–Chem simulation and must split your simulation into several stages (i.e. in order to stay within the computational time limits of your system), then you should set LSVCSPEC to T in the Chemistry Menu of input.geos. That will make sure that the chemical species concentrations are preserved when the next run stage starts. Otherwise, GEOS-Chem use the default species concentrations specified in globchem.dat at the beginning of each run. Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| stations.YYYYMMDDhh | Diagnostic output file that contains ND48 station timeseries output as specified in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. For more information on working with ND48 timeseries output, see our timeseries tutorial. |
| tsYYYYMMDDhh.bpch | Diagnostic output file that contains ND49 instantaneous timeseries output as specified in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. For more information on working with ND49 timeseries output, see our timeseries tutorial. |
| ts_24hr_avg.YYYYMMDDhh.bpch | Diagnostic output file that contains ND50 24-hour average timeseries output as specified in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| ts_satellite.YYYYMMDDhh.bpch | Diagnostic output file that contains ND51 local-time-average (satellite) timeseries output as specified in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| paranox_ts.YYYYMMDDhh.bpch | Diagnostic output file that contains ND63 ship timeseries output as specified in input.geos.
Data in this file are in "binary punch" format and may be viewed and manipulated using GAMAP. |
| ocean_rst.YYYYMMDDhhmm.nc | Ocean mercury restart file containing several quantities pertaining to the ocean mercury module saved at the end of the GEOS-Chem simulation. Like the tracer restart file, this file may be used to initialize another GEOS-Chem mercury simulation.
Data in this file are in the in netcdf format. |
| plane.log.YYYYMMDD</tt?> | Diagnostic output file containing planeflight information scheduled via the <tt>Planeflight.dat file. This diagnostic is turned on in the Planeflight Menu of input.geos.
The data in this text file can be read and plotted using GAMAP routines CTM_READ_PLANEFLIGHT and PLANE_PLOT. |
--Melissa Sulprizio 15:42, 3 April 2015 (EDT)
Available run directories
Here we summarize the run directories that are available in the GEOS-Chem Unit Tester. Please choose the run directory or run directories that are most relevant for your research.
LEGEND
| Run directory is available | |
| Run directory is in need of updating | |
| Run directory is obsolete | |
| Run directory is not available |
--Melissa Sulprizio 17:52, 1 April 2015 (EDT)
Run directory tips and tricks
- In each run directory, we provide a GEOS-Chem restart file that you can use to initialize your simulation. However, it is highly recommended that you not use these restart files for any of your production runs. Rather, you will probably want to generate your own restart file by spinning up the model for at least a year.
- You can create a run directory for the nested grid simulations from a 4° x 5° or 2° x 2.5° run directory by following these steps:
- Modify the input.geos file for the nested grid simulation. Complete instructions can be found on this wiki page.
- Regrid the restart file(s) using the GAMAP regridding routines. For example, REGRIDH_RESTART can be used to regrid the tracer restart file from one horizontal resolution to another. You can then use CREATE_NESTED to crop the restart file to your nested domain (CH, EU, or NA).
- You can create a MERRA run directory from a GEOS-5 or GEOS-FP run directory, since all three met fields have the same horizontal and vertical resolution.
- If you would like to run a simulation and the run directory is not available in the Unit Tester, you can follow these steps:
- Modify the input.geos file for your simulation type following the relevant checklist in Chapter 6 of the GEOS-Chem Manual.
- Create a "fake" restart file using GAMAP routine MAKE_RESTART.
- Alternatively, you can send an email to the relevant Working Group to see if another member can provide you with a run directory or restart file.
- The GEOS-Chem Unit Tester can be used to create run directories for GEOS-Chem v10-01 and later versions. If you wish to download a run directory for an older version of GEOS-Chem, you can follow these instructions.
--Melissa Sulprizio 18:23, 1 April 2015 (EDT)
