Difference between revisions of "Compiling with GNU Make"

From Geos-chem
Jump to: navigation, search
(Compiling GEOS-Chem "Classic" versions 12.4.0 to 12.9.0)
(GNU Make will be retired in GEOS-Chem 13.0.0)
(38 intermediate revisions by one other user not shown)
Line 2: Line 2:
 
'''''[[Compiling with CMake|Previous]] | [[Running GEOS-Chem|Next]] | [[Getting Started with GEOS-Chem]]'''''
 
'''''[[Compiling with CMake|Previous]] | [[Running GEOS-Chem|Next]] | [[Getting Started with GEOS-Chem]]'''''
 
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
 
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
 +
#[[Installing required software]]
 
#[[Configuring your computational environment]]
 
#[[Configuring your computational environment]]
 
#[[Downloading GEOS-Chem source code|Downloading source code]]
 
#[[Downloading GEOS-Chem source code|Downloading source code]]
Line 12: Line 13:
 
#[[Running GEOS-Chem|Running]]
 
#[[Running GEOS-Chem|Running]]
 
#[[GEOS-Chem output files|Output files]]
 
#[[GEOS-Chem output files|Output files]]
#[[Guide_to_visualization_and_analysis_tools_for_GEOS-Chem|Visualizing and processing output]]
+
#[[Python tools for use with GEOS-Chem]]
 
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
 
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
 
#[[GEOS-Chem_overview#Further_reading|Further reading]]
 
#[[GEOS-Chem_overview#Further_reading|Further reading]]
  
  
[[Image: Page is under construction.jpg]]
+
On this page, we describe how to compile GEOS-Chem using the GNU Make build system.
  
 
== GNU Make will be retired in GEOS-Chem 13.0.0 ==
 
== GNU Make will be retired in GEOS-Chem 13.0.0 ==
  
The GNU Make build system will be retired in both GEOS-Chem and GCHP 13.0.0.  This is necessary in order to keep pace with software engineering developments that are taking place at NASA/GMAO and elsewhere in the GEOS-Chem modeling community.
+
<span style="color:red">'''The GNU Make build system has been retired in both GEOS-Chem and GCHP 13.0.0.'''</span> This is necessary in order to keep pace with software engineering developments that are taking place at NASA/GMAO and elsewhere in the GEOS-Chem modeling community.
  
 
GEOS-Chem "Classic" versions 12.6.0 through 12.9.* are compatible with both GNU Make and CMake.
 
GEOS-Chem "Classic" versions 12.6.0 through 12.9.* are compatible with both GNU Make and CMake.
  
== Compiling GEOS-Chem "Classic" versions 12.4.0 to 12.9.0 ==
+
== Compiling GEOS-Chem "Classic" ==
  
[[Creating_GEOS-Chem_run_directories|All run directories that you create]] contain a Makefile that enables you to easily compile and run GEOS-Chem from the run directory. Using this feature enables you to only select those options that are relevant to the GEOS-Chem simulation that you are using.
+
[[Creating_GEOS-Chem_run_directories|Each run directory that you create]] contains a Makefile that enables you to easily compile and run GEOS-Chem from the run directory. Using this feature enables you to only select those options that are relevant to the GEOS-Chem simulation that you are using.
  
 
In this section, we list the steps that are needed to compile [[GEOS-Chem 12#12.4.0|GEOS-Chem 12.4.0]] to 12.9.* with GNU Make.  If you are using an earlier GEOS-Chem version, then please see the next section.
 
In this section, we list the steps that are needed to compile [[GEOS-Chem 12#12.4.0|GEOS-Chem 12.4.0]] to 12.9.* with GNU Make.  If you are using an earlier GEOS-Chem version, then please see the next section.
Line 35: Line 36:
 
The following files and folders are part of a typical run directory.  This example is for the <tt>geosfp_4x5_standard</tt> simulation.
 
The following files and folders are part of a typical run directory.  This example is for the <tt>geosfp_4x5_standard</tt> simulation.
  
  CodeDir@           getRunInfo*      HEMCO_Diagn.rc input.geos OutputDir/ validate.pl*
+
  CodeDir@                             getRunInfo*      HISTORY.rc  OutputDir/
  download_data.py* HEMCO_Config.rc  HISTORY.rc     Makefile*  README
+
  download_data.py*                     HEMCO_Config.rc  input.geos  README
 +
GEOSChem.Restart.YYYYMMDD_hhmmz.nc4@  HEMCO_Diagn.rc   Makefile*  validate.pl*
  
 
#<tt>CodeDir</tt> is a symbolic link to the code directory that you specified when you created the run directory.   
 
#<tt>CodeDir</tt> is a symbolic link to the code directory that you specified when you created the run directory.   
#<tt>Makefile</tt> is the run-directory Makefile.  This controls the build sequence.  The Makefile relies on the <tt>getRunInfo</tt> script to pass the ver
+
#<tt>Makefile</tt> is the run-directory Makefile.  This controls the build sequence.  The Makefile relies on the <tt>getRunInfo</tt> script to obtain necessary information.
 
#<tt>OutputDir</tt> is where diagnostic files will be placed.
 
#<tt>OutputDir</tt> is where diagnostic files will be placed.
#<tt>input.geos</tt> and <tt>*.rc</tt> files are [[GEOS-Chem_configuration_files|configuration files]] that allow you to specify settings for the GEOS-Chem simulation.
+
#<tt>input.geos</tt> and the <tt>*.rc</tt> files are [[GEOS-Chem_configuration_files|configuration files]] that allow you to specify settings for the GEOS-Chem simulation.
 
#<tt>download_data.py</tt> is the data download script for the GEOS-Chem dry-run simulation (available in [[GEOS-Chem 12#12.7.0|12.7.0]] and later versions).
 
#<tt>download_data.py</tt> is the data download script for the GEOS-Chem dry-run simulation (available in [[GEOS-Chem 12#12.7.0|12.7.0]] and later versions).
 
#<tt>validate.pl</tt> is only used for GEOS-Chem unit tests and difference tests.  You can usually ignore this.
 
#<tt>validate.pl</tt> is only used for GEOS-Chem unit tests and difference tests.  You can usually ignore this.
  
=== Determining available compilation options ===
+
Earlier versions of GEOS-Chem might not have all of these files in the run directory.  For example, the <tt>CodeDir</tt> symbolic link was only added in [[#GEOS-Chem 12#12.3.0|GEOS-Chem 12.3.0]].
  
The compiler flags required for compiling GEOS-Chem, specifically <tt>MET</tt> and <tt>GRID</tt>, and in some cases <tt>NEST</tt>, <tt>TOMAS</tt>, <tt>RRTMG</tt>, or <tt>UCX</tt>, are automatically passed when you compile using the run directory Makefile. The flags are automatically set bu tje <tt>getRunInfo</tt> script, which parses the <tt>input.geos</tt> configuration file.
+
=== Determining available compilation options ===
  
The GEOS-Chem Makefiles allow you to choose from several different options. The best way to learn about these options is to type:  
+
The GEOS-Chem Makefiles allow you to choose from several different user-definable options. The best way to learn about these options is to type:  
  
 
  make help
 
  make help
  
at the Unix prompt.  Because the options might change from version to version, you should always use <tt>make help</tt> to find the specific options for the GEOS-Chem version that you are using.
+
at the Unix prompt.  Near the bottom of the printout you will see this section:
 +
 
 +
----------------------------------------------------------
 +
OPTIONAL-FLAGS may be one of the following:
 +
----------------------------------------------------------
 +
DEBUG=y          Compiles GEOS-Chem with various debugging options
 +
BOUNDS=y          Compiles GEOS-Chem with out-of-bounds error checks
 +
FPE=y            Compiles GEOS-Chem with floating-point math error checks
 +
TRACEBACK=y      Compiles GEOS-Chem with traceback error printout (this is the default)
 +
TIMERS=1          Compiles GEOS-Chem with timers turned on
 +
 
 +
Because the options might change from version to version, you should always use <tt>make help</tt> to find the specific options for the GEOS-Chem version that you are using.
 +
 
 +
Please see the [[#Examples|Examples]] section below for some common compilation commands.
  
 
=== Clean up files in the run directory ===
 
=== Clean up files in the run directory ===
Line 59: Line 74:
 
If you have created a fresh run directory, then there is no need to clean it prior to starting a simulation.
 
If you have created a fresh run directory, then there is no need to clean it prior to starting a simulation.
  
If you are reusing the same run directory to start a new GEOS-Chem simulation (i.e. for debugging or testing purposes), then we recommend that you remove any files that were generated by a prior simulation.  This can be done by the following options:
+
If you are reusing the same run directory to start a new GEOS-Chem simulation (i.e. for debugging or testing purposes), then we recommend that you remove any files that were generated by a prior simulation.   
  
make logclean    # Removes all log files
+
If you are using a recent version of GEOS-Chem (say [[GEOS-Chem 12#12.6.0|12.6.0]] or newer), you can clean up the run directory with:
make dataclean  # Removes
+
make fileclean  # fileclean + dataclean
+
make superclean  # fileclean + dataclean + cleans up the GEOS-Chem source code (in ../CodeDir)
+
  
=== Build the GEOS-Chem or HEMCO standalone executable ===
+
make cleanup_output
 +
 
 +
But if you are using an older version (prior to 12.6.0), use this command instead:
 +
 
 +
make fileclean
 +
 
 +
If you also want to remove all object (*.o), module (*.mod) and library (*.a) files in the GEOS-Chem code directory (all GEOS-Chem versions), type:
 +
 
 +
make superclean
 +
 
 +
=== Build the GEOS-Chem executable ===
  
 
To create the GEOS-Chem executable file, type:
 
To create the GEOS-Chem executable file, type:
  
  make -j4 mpbuild
+
  make -j4 build [OPTIONS]
  
 
This will build the <tt>geos</tt> executable file and send all text output to the <tt>compile.log</tt> file.
 
This will build the <tt>geos</tt> executable file and send all text output to the <tt>compile.log</tt> file.
  
NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type <tt>make -j8 ...</tt>, etc.
+
If you are using a GEOS-Chem version prior to 12.4.0, use this command instead:
 +
 
 +
make -j4 mpbuild [OPTIONS]
 +
 
 +
<tt>[OPTIONS]</tt> denotes additional Makefile options that you wish to apply to the build.  These are displayed with <tt>make help</tt>, as described above.
 +
 
 +
''NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type <tt>make -j8 ...</tt>, etc.''
 +
 
 +
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 19:24, 20 December 2019 (UTC)
  
 
=== Build the HEMCO standalone executable ===
 
=== Build the HEMCO standalone executable ===
Line 80: Line 110:
 
To build the HEMCO standalone executable (if you are in a HEMCO standalone run directory), type:
 
To build the HEMCO standalone executable (if you are in a HEMCO standalone run directory), type:
  
  make -j4 hcobuild
+
  make -j4 hcobuild [OPTIONS]
  
 
This will build the <tt>hemco_standalone.x</tt> executable file.
 
This will build the <tt>hemco_standalone.x</tt> executable file.
  
NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type <tt>make -j8 ...</tt>, etc.
+
''NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type <tt>make -j8 ...</tt>, etc.''
 +
 
 +
=== Log files created by the compilation ===
 +
 
 +
GEOS-Chem version 12.6.0 and later will produce these compilation log files:
 +
 
 +
#[[GEOS-Chem_log_files#compile.log_file|<tt>compile.log</tt>]]
 +
#[[GEOS-Chem_log_files#The lastbuild file|<tt>lastbuild</tt>]]
 +
 
 +
Versions prior to this sent the compilation output (i.e. what would go into <tt>compile.log</tt>) into the GEOS-Chem log file.  The lastbuild file would have been named either:
 +
 
 +
#<tt>lastbuild.mp</tt>: Compilation output when <tt>make -j4 mpbuild</tt> was used (i.e. for multi-processor)
 +
#<tt>lastbuild.sp</tt>: Compilation output when <tt>make -j4 spbuild</tt> was used (i.e. for single-processor)
 +
 
 +
=== Examples ===
 +
 
 +
Here are a few examples of the most common GEOS-Chem compilation commands with GNU Make:
 +
 
 +
==== Build GEOS-Chem with debugging options ====
 +
 
 +
To build GEOS-Chem with debugging flags turned on, use the command below.  This will allow you to run GEOS-Chem in a debugger such as gdb:
 +
 
 +
make -j4 build DEBUG=y BOUNDS=y FPEX=y
 +
 
 +
==== Build GEOS-Chem with timers ====
 +
 
 +
To turn on GEOS-Chem timers, use:
 +
 
 +
make -j4 build TIMERS=1
 +
 
 +
etc.
 +
 
 +
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 16:31, 2 January 2020 (UTC)
 +
 
 +
== Compiling GCHP (prior to 13.0.0) ==
  
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 18:31, 20 December 2019 (UTC)
+
GCHP versions prior to 13.0.0 use GNU Make as well.  But the compilation process is more complex than for GEOS-Chem "Classic".  For more information, please see [[Compiling GCHP|our ''Compiling GCHP'' wiki page]].
  
=== Information from the last time GEOS-Chem was compiled ===
+
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 19:25, 20 December 2019 (UTC)
  
When compiling from the run directory Makefile, a [[GEOS-Chem_log_files#The_lastbuild_filelog file called <tt>lastbuild</tt>]] will be created.  This file contains the settings that were used to compile GEOS-Chem, for your reference.
+
== Further reading ==
  
== Compiling from a run directory (in versions prior to 12.4.0) ==
+
#[https://www.gnu.org/software/make/ GNU Make home page]
 +
#[https://www.gnu.org/software/make/manual/ GNU Make manual]
 +
#[https://opensourceforu.com/2012/06/gnu-make-in-detail-for-beginners/ ''GNU Make in detail for beginners'' (Open Source for you)]
 +
#[[GEOS-Chem coding and debugging]]
 +
#''[[Guide to GEOS-Chem error messages]]''
  
  
 
----
 
----
 
'''''[[Compiling with CMake|Previous]] | [[Running GEOS-Chem|Next]] | [[Getting Started with GEOS-Chem]]'''''
 
'''''[[Compiling with CMake|Previous]] | [[Running GEOS-Chem|Next]] | [[Getting Started with GEOS-Chem]]'''''

Revision as of 16:55, 14 December 2020

Previous | Next | Getting Started with GEOS-Chem

  1. Minimum system requirements
  2. Installing required software
  3. Configuring your computational environment
  4. Downloading source code
  5. Downloading data directories
  6. Creating run directories
  7. Configuring runs
  8. Compiling GEOS-Chem
  9. Running
  10. Output files
  11. Python tools for use with GEOS-Chem
  12. Coding and debugging
  13. Further reading


On this page, we describe how to compile GEOS-Chem using the GNU Make build system.

GNU Make will be retired in GEOS-Chem 13.0.0

The GNU Make build system has been retired in both GEOS-Chem and GCHP 13.0.0. This is necessary in order to keep pace with software engineering developments that are taking place at NASA/GMAO and elsewhere in the GEOS-Chem modeling community.

GEOS-Chem "Classic" versions 12.6.0 through 12.9.* are compatible with both GNU Make and CMake.

Compiling GEOS-Chem "Classic"

Each run directory that you create contains a Makefile that enables you to easily compile and run GEOS-Chem from the run directory. Using this feature enables you to only select those options that are relevant to the GEOS-Chem simulation that you are using.

In this section, we list the steps that are needed to compile GEOS-Chem 12.4.0 to 12.9.* with GNU Make. If you are using an earlier GEOS-Chem version, then please see the next section.

Run directory contents

The following files and folders are part of a typical run directory. This example is for the geosfp_4x5_standard simulation.

CodeDir@                              getRunInfo*      HISTORY.rc  OutputDir/
download_data.py*                     HEMCO_Config.rc  input.geos  README
GEOSChem.Restart.YYYYMMDD_hhmmz.nc4@  HEMCO_Diagn.rc   Makefile*   validate.pl*
  1. CodeDir is a symbolic link to the code directory that you specified when you created the run directory.
  2. Makefile is the run-directory Makefile. This controls the build sequence. The Makefile relies on the getRunInfo script to obtain necessary information.
  3. OutputDir is where diagnostic files will be placed.
  4. input.geos and the *.rc files are configuration files that allow you to specify settings for the GEOS-Chem simulation.
  5. download_data.py is the data download script for the GEOS-Chem dry-run simulation (available in 12.7.0 and later versions).
  6. validate.pl is only used for GEOS-Chem unit tests and difference tests. You can usually ignore this.

Earlier versions of GEOS-Chem might not have all of these files in the run directory. For example, the CodeDir symbolic link was only added in GEOS-Chem 12.3.0.

Determining available compilation options

The GEOS-Chem Makefiles allow you to choose from several different user-definable options. The best way to learn about these options is to type:

make help

at the Unix prompt. Near the bottom of the printout you will see this section:

----------------------------------------------------------
OPTIONAL-FLAGS may be one of the following:
----------------------------------------------------------
DEBUG=y           Compiles GEOS-Chem with various debugging options
BOUNDS=y          Compiles GEOS-Chem with out-of-bounds error checks
FPE=y             Compiles GEOS-Chem with floating-point math error checks
TRACEBACK=y       Compiles GEOS-Chem with traceback error printout (this is the default)
TIMERS=1          Compiles GEOS-Chem with timers turned on

Because the options might change from version to version, you should always use make help to find the specific options for the GEOS-Chem version that you are using.

Please see the Examples section below for some common compilation commands.

Clean up files in the run directory

If you have created a fresh run directory, then there is no need to clean it prior to starting a simulation.

If you are reusing the same run directory to start a new GEOS-Chem simulation (i.e. for debugging or testing purposes), then we recommend that you remove any files that were generated by a prior simulation.

If you are using a recent version of GEOS-Chem (say 12.6.0 or newer), you can clean up the run directory with:

make cleanup_output

But if you are using an older version (prior to 12.6.0), use this command instead:

make fileclean

If you also want to remove all object (*.o), module (*.mod) and library (*.a) files in the GEOS-Chem code directory (all GEOS-Chem versions), type:

make superclean

Build the GEOS-Chem executable

To create the GEOS-Chem executable file, type:

make -j4 build [OPTIONS]

This will build the geos executable file and send all text output to the compile.log file.

If you are using a GEOS-Chem version prior to 12.4.0, use this command instead:

make -j4 mpbuild [OPTIONS]

[OPTIONS] denotes additional Makefile options that you wish to apply to the build. These are displayed with make help, as described above.

NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type make -j8 ..., etc.

--Bob Yantosca (talk) 19:24, 20 December 2019 (UTC)

Build the HEMCO standalone executable

To build the HEMCO standalone executable (if you are in a HEMCO standalone run directory), type:

make -j4 hcobuild [OPTIONS]

This will build the hemco_standalone.x executable file.

NOTE: the -jN option will compile N files at a time. If you have 8 cores available to you, you can type make -j8 ..., etc.

Log files created by the compilation

GEOS-Chem version 12.6.0 and later will produce these compilation log files:

  1. compile.log
  2. lastbuild

Versions prior to this sent the compilation output (i.e. what would go into compile.log) into the GEOS-Chem log file. The lastbuild file would have been named either:

  1. lastbuild.mp: Compilation output when make -j4 mpbuild was used (i.e. for multi-processor)
  2. lastbuild.sp: Compilation output when make -j4 spbuild was used (i.e. for single-processor)

Examples

Here are a few examples of the most common GEOS-Chem compilation commands with GNU Make:

Build GEOS-Chem with debugging options

To build GEOS-Chem with debugging flags turned on, use the command below. This will allow you to run GEOS-Chem in a debugger such as gdb:

make -j4 build DEBUG=y BOUNDS=y FPEX=y

Build GEOS-Chem with timers

To turn on GEOS-Chem timers, use:

make -j4 build TIMERS=1

etc.

--Bob Yantosca (talk) 16:31, 2 January 2020 (UTC)

Compiling GCHP (prior to 13.0.0)

GCHP versions prior to 13.0.0 use GNU Make as well. But the compilation process is more complex than for GEOS-Chem "Classic". For more information, please see our Compiling GCHP wiki page.

--Bob Yantosca (talk) 19:25, 20 December 2019 (UTC)

Further reading

  1. GNU Make home page
  2. GNU Make manual
  3. GNU Make in detail for beginners (Open Source for you)
  4. GEOS-Chem coding and debugging
  5. Guide to GEOS-Chem error messages



Previous | Next | Getting Started with GEOS-Chem