Difference between revisions of "Compiling GCHP"

From Geos-chem
Jump to: navigation, search
(Checking for Successful Compilation)
 
(90 intermediate revisions by 3 users not shown)
Line 1: Line 1:
'''''[[Setting Up the GCHP Environment|Previous]] | [[GCHP_Basic_Example_Run|Next]] | [[Getting_Started_With_GCHP|Getting Started with GCHP]]'''''
+
----
 +
<span style="color:crimson;font-size:120%">'''The GCHP documentation has moved to https://gchp.readthedocs.io/.''' The GCHP documentation on http://wiki.seas.harvard.edu/ will stay online for several months, but it is outdated and no longer active!</span>
 +
----
 +
 
 +
'''''[[Downloading_GCHP|Previous]] | [[Obtaining_a_GCHP_Run_Directory|Next]] | [[Getting Started with GCHP]] | [[GCHP Main Page]]'''''
 
#[[GCHP_Hardware_and_Software_Requirements|Hardware and Software Requirements]]
 
#[[GCHP_Hardware_and_Software_Requirements|Hardware and Software Requirements]]
#[[Downloading_GCHP|Downloading Source Code]]
 
#[[Obtaining_a_GCHP_Run_Directory|Obtaining a Run Directory]]
 
 
#[[Setting_Up_the_GCHP_Environment|Setting Up the GCHP Environment]]
 
#[[Setting_Up_the_GCHP_Environment|Setting Up the GCHP Environment]]
 +
#[[Downloading_GCHP|Downloading Source Code and Data Directories]]
 
#<span style="color:blue">'''Compiling'''</span>
 
#<span style="color:blue">'''Compiling'''</span>
#[[GCHP_Basic_Example_Run|Basic Example Run]]
+
#[[Obtaining_a_GCHP_Run_Directory|Obtaining a Run Directory]]
 +
#[[Running_GCHP:_Basics|Running GCHP: Basics]]
 +
#[[Running_GCHP:_Configuration|Running GCHP: Configuration]]
 +
#[[GCHP_Output_Data|Output Data]]
 +
#[[Developing_GCHP|Developing GCHP]]
 
#[[GCHP_Run_Configuration_Files|Run Configuration Files]]
 
#[[GCHP_Run_Configuration_Files|Run Configuration Files]]
#[[Running_GCHP|Advanced Run Examples]]
 
#[[GEOS-Chem_HP_Output_Data|Output Data]]
 
#[[Developing_GCHP|Developing GCHP]]
 
 
<br>
 
<br>
  
== Overview ==
+
Compiling GCHP is the same procedure as compiling GEOS-Chem Classic. Please see [http://wiki.seas.harvard.edu/geos-chem/index.php/Compiling_with_CMake Compiling with CMake] for instructions.
 
+
You are ready to compile GCHP after you have obtained the GCHP source code, have acquired and set up your run directory, and have loaded your environment. First time compilation can take a while (30 minutes or more) because it requires building large modules. GCHP is made up of several different modules, all communicating via NASA's Modeling, Analysis and Prediciton Layer ([https://geos5.org/wiki/index.php?title=MAPL_software MAPL]). MAPL is built on the Earth System Modeling Framework ([https://www.earthsystemcog.org/projects/esmf/ ESMF]), and both systems are highly flexible but highly complex, with significant compilation times. Fortunately, MAPL, ESMF, and a third external component for advection ([https://www.gfdl.noaa.gov/fv3/ FV3 dycore]), do not need to be recompiled unless parts of their code have been modified. Subsequent compile times will therefore be significantly shorter, on the same order of magnitude as compiling GEOS-Chem Classic.  
+
 
+
This section provides instructions on how to perform a full ("clean") compilation, including MAPL, ESMF, and FV3 dycore, as well as a partial ("standard") compilation when you want to keep your existing MAPL, ESMF, and advection builds as is. The "clean" compilation '''MUST''' be done the first time you use GCHP since it cleans and compiles all modules. The "standard" compilation may be used for subsequent recompilation after updating GEOS-Chem code. Be careful not to wipe out the entire GCHP build when cleaning unless you intentionally want to do so!
+
 
+
== Initial Compilation ==
+
 
+
To compile GCHP from scratch, type the following within the run directory:
+
 
+
make compile_clean
+
 
+
This command will execute <tt>./build.sh compile_clean</tt> (described in the advanced topics section below) and send all log output to both the terminal window and to a log file called <tt>compile.log</tt>. Git information for both source code repositories at compile time are recorded in a separate file called <tt>lastbuild</tt> for future reference. Upon successful compilation, the following empty files are created in your GCHP source code directories:
+
* <tt>GCHP/ESMF/esmf.install</tt>
+
* <tt>GCHP/Shared/mapl.install</tt>
+
* <tt>GCHP/FVdycoreCubed_GridComp/fvdycore.install</tt>
+
 
+
These files indicate successful compilation per module and their presence prompts the GCHP source code <tt>Makefile</tt> to skip compiling each component during future builds so as to save time. The <tt>*.install</tt> files are deleted when you clean MAPL, ESMF, and FVdycore source code such as when running <tt>make compile_clean</tt> or <tt>make superclean</tt>. If you are ever uncertain on whether your ESMF, MAPL, and FV3 dycore builds have been wiped out, check if these files exist.
+
 
+
== Subsequent Compilation ==
+
 
+
For subsequent GCHP compilations where you only need to rebuild the GEOS-Chem code, you can do the following (see also alias "mcs" in sample bashrc files):
+
 
+
make compile_standard
+
 
+
If you updated MAPL code only, you can still do a partial compilation to avoid recompiling ESMF. Do this with:
+
 
+
make compile_mapl
+
 
+
== Additional Makefile Options ==
+
 
+
The GCHP run directory <tt>Makefile</tt> contains several options for cleaning and compiling GCHP using <tt>make</tt>, as well as options for other functionality. Options are printed out by typing the following at the command line:
+
 
+
make help
+
 
+
This will output something like this to your screen.
+
 
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
%%%    GCHP Run Directory Makefile Options    %%%
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+
+
Usage: make TARGET [OPTIONAL-FLAGS]
+
+
Default threads for compiling is hard-coded as 6
+
+
----------------------------------------------------------
+
TARGET may be one of the following:
+
----------------------------------------------------------
+
+
To print settings to screen:
+
  printbuildinfo  Print out git info for GC and GCHP repos
+
  help            If you got this far, you know this one
+
+
To clean source code:
+
  clean_all        Clean all source code using build.sh
+
  clean_gc        Clean GEOS-Chem classic only using build.sh
+
  clean_mapl      Clean MAPL and FVdyCore using build.sh
+
  clean_nuclear    Clean everything using build.sh
+
  superclean      Remove exe, logs, output, and clean ALL code
+
+
To compile:
+
  compile_clean    Include everything
+
  compile_standard  Skip MAPL, FvDyCore, and ESMF; debug off
+
  compile_debug    Skip MAPL, FvDyCore, and ESMF; debug on
+
  compile_mapl      Skip ESMF
+
+
To remove run directory files:
+
  cleanup_output  Remove output data, log files, and executable
+
  cleanup_data    Remove output data
+
  cleanup_logs    Remove log files
+
  cleanup_exe      Remove executable
+
+
To run interactively:
+
  run_interactive
+
+
To clean, build, and run interactively:
+
  gchp_clean      Include everything
+
  gchp_standard    Skip MAPL, FvDyCore, and ESMF; debug off
+
  gchp_debug      Skip MAPL, FvDyCore, and ESMF; debug on
+
  gchp_mapl        Skip ESMF
+
 
+
Some very useful options include the following:
+
 
+
*'''''<tt>make printbuildinfo</tt>:''''' prints git information such as branch and last commit for the GEOS-Chem and GCHP repositories your <tt>CodeDir</tt> symbolic link points to.
+
*'''''<tt>make cleanup_output</tt>:''''' cleans your run directory to remove all traces of your last run, including log and output files, but will not remove your executable or last build information.
+
 
+
== Checking for Successful Compilation ==
+
 
+
If GCHP compiled without issues, you should see:
+
 
+
###################################
+
### GCHP compiled successfully! ###
+
###################################
+
 
+
 
+
Compiling GCHP produces a GCHP executable called <tt>geos</tt> that is stored in the <tt>bin/</tt> sub-directory of your GEOS-Chem source code directory. Compiling within the run directory makes a copy of the executable locally for easy use.
+
 
+
Compiler output, including warnings and error messages, will be stored in a log file called <tt>compile.log</tt>. If you inspect your log file or watch the compile output real-time, you will notice that there are many warnings and error messages. Many of these errors are due to leftover components in MAPL which are no longer used but which still try to compile. As long as the compilation continues, these errors can almost always be ignored and they will have no effect on the operation of GCHP.
+
 
+
Beware that once you have an executable you must not rename your run directory or your source code directory or the executable will no longer work. This is because the GCHP executable will include paths to ESMF libraries that incorporate the <tt>CodeDir</tt> symbolic link in the run directory path. However, you can copy the executable elsewhere for use. If you do this then it is also a good idea to copy <tt>lastbuild</tt> and <tt>compile.log</tt> to your new run directory to keep track of where your executable came from. If you encounter an error while running GCHP that an ESMF shared library is missing, check that the exact path in the error message is still valid.
+
 
+
If you run into problems please contact the GEOS-Chem Support Team for help.
+
 
+
== Advanced Compilation Topics ==
+
 
+
The <tt>Makefile</tt> in the run directory invokes a bash shell script called <tt>build.sh</tt> that includes changing into the source code directory and invoking source code Makefiles. This section details the capabilities of <tt>build.sh</tt> and source code <tt>make</tt> options for advanced users who wish to better understand how GCHP works. You may skip this section and return to it at a later date when you want to dive deeper into how GCHP is compiled.
+
 
+
=== build.sh ===
+
 
+
Whenever <tt>build.sh</tt> is executed it checks if you have set up your environment by querying if the environment variable <tt>MPI_ROOT</tt> is defined. If it is not, you will be prompted to choose a <tt>.bashrc</tt>, source it, and export the <tt>BASHRC</tt> variable. Instructions on how to do this are printed to the terminal and are also described on the [[Setting_Up_the_GCHP_Environment|''Setting up the GCHP environment'' wiki page]]. Note that the script gives you the option to choose your <tt>.bashrc</tt> file from a list of files it finds in your run directory. If none of them are appropriate, for example if you are not working on the Harvard Odyssey or Compute Canada Glooscap compute clusters, then you will need to create a <tt>.bashrc</tt> file using these sample files as an example.
+
 
+
If you have multiple terminal windows and cannot remember if you have loaded your environment, a quick check is execute the <tt>help</tt> option of <tt>build.sh</tt>.
+
 
+
./build.sh help
+
 
+
Clean and compile options of <tt>build.sh</tt> will also print for your reference. For example, when working on the Harvard Odyssey compute cluster you output will look as follows:
+
 
+
MPI_ROOT is to /n/sw/fasrcsw/apps/Comp/intel/15.0.0-fasrc01/mvapich2/2.2-fasrc01
+
Script name:
+
  build.sh
+
Arguments:
+
  Accepts single argument indicating clean and/or compile settings.
+
  Currently implemented arguments include:
+
      clean_gc        - classic only
+
      clean_nuclear    - GCHP, ESMF, MAPL, FVdycore (be careful!)
+
      clean_all        - classic, GCHP, ESMF, MAPL, FVdycore (be careful!)
+
      clean_mapl      - mapl and fvdycore only
+
      compile_debug    - turns on debug flags, no cleaning
+
      compile_standard - no cleaning
+
      compile_mapl    - includes fvdycore
+
      compile_clean    - cleans and compiles everything (be careful!)
+
Example usage:
+
  ./build.sh compile_standard
+
 
+
The default compile commands for <tt>compile_clean</tt>, <tt>compile_standard</tt>, and <tt>compile_mapl</tt> are as follows:
+
    make -j${SLURM_NTASKS} NC_DIAG=y  CHEM=tropchem EXTERNAL_GRID=y  \
+
                          DEBUG=y    DEVEL=y      TRACEBACK=y      \
+
                          MET=geos-fp GRID=4x5      NO_REDUCED=y      \
+
                          UCX=n      hpc
+
 
+
The compile command for <tt>compile_debug</tt> is the same as above with <tt>BOUNDS=yes</tt> and <tt>FPEX=yes</tt> added for debugging purposes.
+
 
+
If you have only changed GEOS-Chem code and have not touched the GCHP structural files (e.g. MAPL), then you can recompile with <tt>./build.sh compile_standard</tt> to save time. Running this will recompile only changed files (excluding MAPL, FVdycore, and ESMF). To run it and also output stderr and stdout to log file <tt>compile.log</tt>, run the following <tt>make</tt> command:
+
 
+
<nowiki>make compile_standard</nowiki>
+
 
+
If you find that you need to fully recompile GEOS-Chem, however, you can still avoid recompiling the MAPL framework code (as well as components such as the FV3 dycore). You can do this simply by calling <tt>make realclean</tt> within your top-level source code directory prior to running <tt>make compile_standard</tt>. This will perform a clean build of the core GEOS-Chem components without necessitating a full recompilation of the more static components of GCHP.
+
 
+
=== Source Code Makefiles ===
+
 
+
At its core, building GCHP simply requires the inclusion of an additional option, <tt>HPC=yes</tt>, in the source code directory make command beyond what is typically included in GEOS-Chem compilation. Appending <tt>HPC=yes</tt> causes the make procedure to look for the contents of GCHP, and bypasses much of the serial GEOS-Chem code. For example, base GEOS-Chem classic source code is cleaned using the following command:
+
 
+
<nowiki> make HPC=yes realclean </nowiki>
+
 
+
After the process completes, the <tt>GCHP/</tt> subdirectory is cleaned by changing into that directory and invoking:
+
 
+
<nowiki> make EXTERNAL_GRID=yes DEVEL=yes the_nuclear_option </nowiki>
+
 
+
Calling <tt>make</tt> with <tt>the_nuclear_option</tt> defined in <tt>GCHP/Makefile</tt> removes ''all'' compiled code in the directory. As implied by the name, this is not to be used lightly! The nuclear option is an alias, calling three commands in turn: <tt>wipeout_esmf</tt>, <tt>wipeout_mapl</tt>, and <tt>wipeout_fvdycore</tt>. Each of these components can take a long time (30 minutes or more) to compile, and all are structural components that are not affected by small changes in GEOS-Chem.
+
 
+
If you are interested in seeing the core <tt>make</tt> commands used for all options provided in the run directory <tt>Makefile</tt>, see the commands defined in <tt>build.sh</tt>. Note that calling <tt>make</tt> always invokes the <tt>Makefile</tt> in the directory you are in and changing into the GEOS-Chem and GCHP source code directories occurs within <tt>build.sh</tt>. Also note that the <tt>GRID=4x5</tt> argument has no effect on the resolution at which the model runs since GCHP functions on the cubed-sphere grid. However, passing <tt>GRID</tt> is required so that components of GEOS-Chem Classic are functional.
+
  
 
--------------------------------------
 
--------------------------------------
  
'''''[[Setting Up the GCHP Environment|Previous]] | [[GCHP_Basic_Example_Run|Next]] | [[GEOS-Chem_HP|GCHP Home]]'''''
+
'''''[[Downloading_GCHP|Previous]] | [[Obtaining_a_GCHP_Run_Directory|Next]] | [[Getting Started with GCHP]] | [[GCHP Main Page]]'''''

Latest revision as of 15:40, 8 December 2020


The GCHP documentation has moved to https://gchp.readthedocs.io/. The GCHP documentation on http://wiki.seas.harvard.edu/ will stay online for several months, but it is outdated and no longer active!


Previous | Next | Getting Started with GCHP | GCHP Main Page

  1. Hardware and Software Requirements
  2. Setting Up the GCHP Environment
  3. Downloading Source Code and Data Directories
  4. Compiling
  5. Obtaining a Run Directory
  6. Running GCHP: Basics
  7. Running GCHP: Configuration
  8. Output Data
  9. Developing GCHP
  10. Run Configuration Files


Compiling GCHP is the same procedure as compiling GEOS-Chem Classic. Please see Compiling with CMake for instructions.


Previous | Next | Getting Started with GCHP | GCHP Main Page