Difference between revisions of "Running GEOS-Chem"

Revision as of 22:14, 8 December 2020

Previous | Next | Getting Started with GEOS-Chem

1. Minimum system requirements
2. Installing required software
3. Configuring your computational environment
4. Downloading source code
5. Downloading data directories
6. Creating run directories
7. Configuring runs
8. Compiling
9. Running
10. Output files
11. Python tools for use with GEOS-Chem
12. Coding and debugging
13. Further reading

Overview

This page presents the basic information needed to run GEOS-Chem as well as how to verify a successful run and reuse a run directory.

Important things to know before you submit your run

1. Please be aware of several options available for speeding up your GEOS-Chem simulations. See our Speeding up GEOS-Chem page for more information.
2. The initial restart files do not reflect the actual atmospheric state and should only be used to "spin up" the model. In other words, they should be used as initial values in an initialization simulation to generate more accurate initial conditions for your production runs. See this wiki post for more information.
3. Prior to running with GEOS-FP met fields, please be aware of several caveats regarding that data stream. See the GEOS-FP wiki page for more information.

Pre-run checklist

Prior to running GEOS-Chem, always run through the following checklist to ensure everything is set up properly.

1. You have properly configured your Unix environment
2. Your have run directory contains the geos executable
3. You have set the proper configurable settings for your simulation in:
   1. input.geos (general simulation settings)
   2. HEMCO_Config.rc (emissions settings)
   3. HEMCO_Diagn.rc
   4. HISTORY.rc and other diagnostic configuration files
4. You have a run script (see below)
5. Your GEOS-Chem "Classic" run script contains proper settings for OpenMP parallelization, namely:
   1. OMP_NUM_THREADS
   2. OMP_STACKSIZE
6. You have enough memory and computational cores available

How to run GEOS-Chem

You can run GEOS-Chem locally from within your run directory (interactively) or by submitting your run to your cluster's job scheduler.

Running Interactively

If your computer system allows interactive computational sessions, then you can run GEOS-Chem interactively. This is useful if you need to run short simulations for debugging or testing.

NOTE: If you are working with GEOS-Chem on the Amazon Web Services cloud computing platform, then you can always run GEOS-Chem interactively, because you will not be sharing your computational environment with other users. See cloud.geos-chem.org for more information.

To run GEOS-Chem interactively in your terminal window:

1. Make sure that you have set the proper Unix environment variables for GEOS-Chem in your interactive session.

2. Navigate to your run directory and type:

     ./geos

2a You may also send the GEOS-Chem output to a log file by using:

     ./geos > GC.log

2b You may send the GEOS-Chem output to a log file and to your screen using:

     ./geos 2>&1 | tee GC.log

NOTE: Prior to GEOS-Chem 12.6.0, the executable was named either geos.mp or geos.sp, depending on whether or not OpenMP parallelization was turned on or off.

Running as a Batch Job

Many shared computational clusters require that long simulations must be submitted as batch jobs via a scheduler (such as LSF, PBS, GridEngine, or SLURM). Here is a sample script that you can modify for your own particular scheduler.

NOTE: Ask your sysadmin or IT staff, or consult your IT documentation for more information about the scheduler used by your institution, and the commands that it takes.

#!/bin/bash

# NOTE: depending on your scheduler, there may be some tags at the
# top of the run script (e.g. #SBATCH, #PBS or #LSF).  These tell
# the scheduler how many cores to use, which queue to use, how
# much memory to use, etc.  Because these are very system-dependent,
# we have omitted these here.  See the benchmark.run script in the
# geosfp_4x5_benchmark run directory for an example using the SLURM
# scheduler.

# Load your environment settings
source ~/.bashrc

# Declare that we would like to run GEOS-Chem with 16 cores
# (change this for your system)
export OMP_NUM_THREADS=16

# Max out the stack memory for the simulation.
# If your system has less than 500 MB then it should still
# give you the max amount available, which should be sufficient.
export OMP_STACKSIZE=500m

# Run the GEOS-Chem executable, and pipe output to a log file
./geos >> GC.log

Run script samples

In GEOS-Chem 13.0.0 and later versions a folder named runScriptSamples is created in each GEOS-Chem run directory that you create. This folder has a few sample run scripts for the SLURM scheduler that you can use or modify

Verifying a Successful Run

There are several ways to verify that your run was successful.

1. The following output can be found at the end of your GEOS-Chem log file:

     **************   E N D   O F   G E O S -- C H E M   **************

2. NetCDF files (GEOSChem.*.nc4) are present in the OutputDir/ subfolder of the run directory.

3. Restart files (GEOSChem.Restart.*nc4, HEMCO_restart*.nc) for the ending date and time of the simulation are present in the run directory.

4. Your scheduler log (e.g. output from SLURM, LSF, PBS, etc.) does not contain any obvious errors.

If your run stopped with an error, please the following resources:

• Guide to GEOS-Chem error messages
• Tips on how to debug GEOS-Chem effectively
• Submitting GEOS-Chem support requests

Submitting consecutive runs

Please note: As of October 2019, the GCST is actively investigating differences in model output when running single vs multi-segmented runs. Any issues found will be addressed as soon as possible to minimize these differences.

Often, users will split long simulations into several smaller simulations to stay within their cluster's computational limits. When doing so, make sure you follow these guidelines to minimize differences in model output:

1. Turn off setting initial stratospheric H2O mixing ratios in input.geos for all runs after your initial run to ensure you use H2O concentrations (found in the restart file) from the previous run:

    => Set initial glob MRs:---
       => strat. H2O?      : F

2. Make sure GC_RESTART and HEMCO_RESTART options are set to true in HEMCO_Config.rc.

3. To ensure you restart files are read and species concentrations are properly initialized, you may check your GEOS-Chem log file for the following output:

     ===============================================================================
     R E S T A R T   F I L E   I N P U T

     Min and Max of each species in restart file [mol/mol]:
     Species   1,       NO: Min = 1.000000003E-30  Max = 1.560991691E-08
     Species   2,       O3: Min = 3.135925075E-09  Max = 9.816152669E-06
     Species   3,      PAN: Min = 3.435056848E-25  Max = 1.222619450E-09
     ...

Actual values may differ. If you see "Use background = ..." for most/all species, that suggests your restart file was not found. To avoid using the wrong restart file make sure to use time cycle flag EY in HEMCO_Config.rc to only use a restart file for the exact simulation date.

Further reading

1. SLURM manual
2. Convenient SLURM commands (Harvard)
3. LSF user manual (LLNL)
4. Learn to use PBS Pro job scheduler (LearnScientificProgramming.io)
5. Guide to GEOS-Chem performance

Previous | Next | Getting Started with GEOS-Chem