Running GCHP: Basics
Now that you have GCHP downloaded and compiled, you’re ready to start running things. First, however, it is a good idea to double-check that everything is setup properly. To start, check that your compilation was successful and you have an executable by looking for the geos file in your run directory. Next, check that your input meteorology resolution is set in ExtData.rc as you intend. Each meteorological data entry is collected at the top of the file and should end with a filename that includes resolution. For example, your file when using 2x25 inputs should include:
The resolution entry on each MetDir line in ExtData.rc must be consistent with the input Met resolution you would like to use. Edit ExtData.rc if they do not match, or, if you are an Odyssey user, repeat the steps to set up your run directory using the initialsetup.sh script provided.
Last, check that you have the proper modules loaded on your system. Forgetting to source your GCHP bashrc for a new session if running from the command line is a common error. For Odyssey users, you can check that you have sourced your GCHP bashrc by using the module list command at the prompt and checking that you have your preferred MPI implementation loaded (e.g. openmpi). If you intend to run the Makefile options or a SLURM run script, you can skip this option. Instead open the file and set your bashrc and other environment variables.
There are several options in the GCHP run directory Makefile for running GCHP interactively, or you may directly call the run command from the command line. If you already have an executable, you can use the following command to run interactively and send output to both the terminal window and a log file called MAPL.log.
If you want to recompile non-MAPL code and then run, you can use
Similarly, gchp_debug is available to recompile with floating point error and out-of-bounds checking turned on. See the Makefile for additional options.
Alternatively, you may submit your run to SLURM using the GCHP.run file if you are on the Odyssey compute cluster. Note that if you are using the GCHP Makefile or run script, open the file first and edit the configurable settings to reflect the environment you want to run in.
The rest of this section specifies how to run at the command line rather than use the Makefile. Once you understand the basics, you may switch to using the make options to streamline your work.
Quick start: 1-hr standard simulation
The default GCHP run directory is set for a 1-hour tropchem simulation at resolution c24 with 2x25 input Met resolution and 4x5 output concentration resolution. c24 is approximately the cubed sphere equivalent of 4x5. If you followed the above instructions for setting up your run directory, you should have specified 2x2.5 input resolution for meteorology fields for your initial devkit run and edited ExtData.rc to include paths that reflect this resolution. For Odyssey users, the ExtData.rc update occurred automatically while for all non-Odyssey users this required manual editing.
For this quick test, you will need an environment with the following:
- 6 CPUs (minimum - see model description)
- 1 node
- At least 2500 MB of memory per CPU
- Your compiler, NetCDF and MPI implementation loaded
Odyssey users should refer to the Harvard Odyssey Users Environment Setup section of this page for a refresher on how to set up your environment prior to running GCHP.
Once your run directory is all set up, start the simulation by typing the following:
mpirun -n 6 ./geos 2>&1 | tee 1hr_mapl.log
This command can be broken down as follows:
- mpirun executes an MPI-enabled executable and associates the necessary resources. This is a version-dependent executable; some MPI implementations use other commands, such as mpiexec.
- -n 6 specifies how many individual CPU cores are requested for the run. The number given here should always be the total number of cores, regardless of how many nodes they are spread over, and must be a multiple of 6 (at least one for each of the cubed sphere faces, and the same number for each face).
- ./geos is the local GEOS-Chem executable, as with GEOS-Chem Classic
- 2>&1 | tee 1hr_maple.log is a bash shell-specific means of collecting all MAPL output (standard and error) that is written to the screen into a file.
Note that the output collected to the log file specified when invoking mpirun is created by MAPL. The more traditional GEOS-Chem log output is automatically sent to a file defined in configuration file GCHP.rc. By default, its format is PET%%%%%.GEOSCHEMchem.log, where %%%%% is replaced during run-time with a processor id (typically 00000. PET stands for persistent execution thread. Unlike MAPL, which sends output to the log from ALL threads, GEOS-Chem only outputs from a single thread.
Once the simulation is complete, there should be two netcdf output files in the OutputDir sub-directory. To get started with manipulating output data, see GCHP output data. For more examples of running GCHP, see the Running GCHP wiki page.