Difference between revisions of "Installing libraries for GEOS-Chem"

From Geos-chem
Jump to: navigation, search
(New page: On this page we provide instructions on how to install the netCDF-4 library and related libraries for GEOS-Chem. Alternate instructions for installing netCDF-3 are also given. == Overvie...)
 
(Downloading the GEOS-Chem-Libraries installer package)
Line 36: Line 36:
 
<blockquote>
 
<blockquote>
 
;<tt>README</tt>: File describing the contents of <tt>GEOS-Chem-Libraries</tt> and how to build the netCDF libraries
 
;<tt>README</tt>: File describing the contents of <tt>GEOS-Chem-Libraries</tt> and how to build the netCDF libraries
;<tt>GNUmakefile</tt>: Main-level makefile.  This calls down to the master Makefile in the <tt>src/</tt> sub-directory.
+
;<tt>GNUmakefile</tt>: Main-level makefile.  This calls down to the master GNUmakefile in the <tt>src/</tt> sub-directory.
 
;<tt>opt/</tt>: Default directory where libraries will be installed (if you do not choose an alternate directory)
 
;<tt>opt/</tt>: Default directory where libraries will be installed (if you do not choose an alternate directory)
 
;<tt>src/</tt>: Directory containing netCDF and related library distributions.
 
;<tt>src/</tt>: Directory containing netCDF and related library distributions.

Revision as of 18:49, 1 May 2012

On this page we provide instructions on how to install the netCDF-4 library and related libraries for GEOS-Chem. Alternate instructions for installing netCDF-3 are also given.

Overview

GEOS-Chem v9-01-03 (and higher versions) require that you install netCDF on your computer system. In order to facilitate building the netCDF library, the GEOS-Chem Support Team has constructed the GEOS-Chem-Libraries installer package. With GEOS-Chem-Libraries, you can build one of the following sets of libraries:

Either:

  • netCDF-4.0
  • ZLIB-1.2.3 (netCDF-4 depends on this)
  • HDF5-1.8.1 (netCDF-4 depends on this)

Or:

  • netCDF-3.6.3

We recommend that you try to build the netCDF-4 library first, since this is the most recent netCDF version. The netCDF-4 library is 100% backwards compatible with netCDF-3 files. Also, many data files (e.g. GEOS-5.7.2 "raw" data) are now starting to be distributed in netCDF-4 format. If you have problems building netCDF-4 then you can try to build netCDF-3.6.3, which has a simpler installation process.

Any version of netCDF may be used with GEOS-Chem. If you already have a version of netCDF installed on your system, then you will not have to use the GEOS-Chem-Libraries package to install netCDF. Ask your local IT guru(s) if you already have a build of netCDF on your system. If you already have a build of netCDF on your system, you may skip ahead to the Linking to the netCDF libraries section.

--Bob Y. 17:38, 30 April 2012 (EDT)

Downloading the GEOS-Chem-Libraries installer package

You can use the Git source-code management software to download the GEOS-Chem-Libraries installer package onto your system. Type:

git clone git://git.as.harvard.edu/bmy/GEOS-Chem-Libraries

and then change into the GEOS-Chem-Libraries directory:

cd GEOS-Chem-Libraries

In this directory you will find the following files and subdirectories

README
File describing the contents of GEOS-Chem-Libraries and how to build the netCDF libraries
GNUmakefile
Main-level makefile. This calls down to the master GNUmakefile in the src/ sub-directory.
opt/
Default directory where libraries will be installed (if you do not choose an alternate directory)
src/
Directory containing netCDF and related library distributions.

--Bob Y. 12:07, 30 April 2012 (EDT)

Building the netCDF-4 library

netCDF-4 installation process

To build the netCDF-4 library (and related libraries) using the Intel Fortran Compiler ("IFORT"), type:

  make netcdf4 PREFIX=root_library_dir

To build the libraries for the PGI compiler, type:

  make netcdf4 COMPILER=pgi PREFIX=root_library_dir

Makefile options:

COMPILER
This option is used to pick the compiler that you wish to use to build netCDF-4 and the related libraries. At present, you may select either ifort or pgi. If you omit the COMPILER option, the default value of ifort will be selected.
PREFIX

This option is used to set root_library_dir, which is the full path name of the directory into which the netCDF-4 and related libraries will be installed. If you omit the PREFIX option, then root_library_dir will be set to one of these default values:

GEOS-Chem-Libraries/opt/ifort  # If COMPILER=ifort

GEOS-Chem-Libraries/opt/pgi   # If COMPILER=pgi

If you do not have sysadmin privileges, then you can use the PREFIX option to set root_library_dir to a directory in your home space, such as /home/YOUR_USER_ID/lib (or you may install them in one of the default paths listed above). You can then ask your sysadmin to move the libraries to a common space on your machine (or cluster) where all users may access them.

--Bob Y. 12:13, 30 April 2012 (EDT)

netCDF-4 library paths

The netCDF-4 (and related libraries) will be installed into the following paths. As described above, you may use the PREFIX option to set the value of root_library_dir.

For ZLIB-1.2.3:
   root_library_dir/zlib/include       # Include files (*.h)
   root_library_dir/zlib/lib           # Library file (libz.a)
   root_library_dir/zlib/share         # Manual pages & documentation

For HDF5-1.8.1:
   root_library_dir/HDF5/bin           # Utility programs (can ignore)
   root_library_dir/HDF5/include       # Include files (*.h)
   root_library_dir/HDF5/lib           # Library files (*.a)

For netCDF-4.0:
   root_library_dir/netcdf4/bin        # ncdump, ncgen programs
   root_library_dir/netcdf4/include    # Include files (*.h)
   root_library_dir/netcdf4/lib        # Library files (*.a)
   root_library_dir/netcdf4/share      # Manual pages & documentation

--Bob Y. 12:21, 30 April 2012 (EDT)

Telling GEOS-Chem where to find the netCDF-4 libraries

GEOS-Chem uses several environment variables to point to the netCDF-4 library and related libraries. These can be set as follows:

For csh or tcsh

If you use C-shell or T-shell, then set the following environment variables in your ~/.cshrc file:

setenv ROOT_LIBRARY_DIR  "same directory as root_library_dir above"
setenv INC_NETCDF        "$(ROOT_LIBRARY_DIR)/netcdf4/include"
setenv LIB_NETCDF        "$(ROOT_LIBRARY_DIR)/netcdf4/lib"
setenv INC_HDF5          "$(ROOT_LIBRARY_DIR)/hdf5/include"
setenv LIB_HDF5          "$(ROOT_LIBRARY_DIR)/hdf5/lib"
setenv LIB_ZLIB          "$(ROOT_LIBRARY_DIR)/zlib/lib"  

To accept the changes, type at the Unix prompt:

source ~/.cshrc

For sh or bash

If you use sh (Bourne-shell), then set the following environment variables in your .profile file. If you use bash (Bourne-again shell), then set these variables in your ~/.bashrc file.

ROOT_LIBRARY_DIR="same directory as root_library_dir above"
INC_NETCDF=$(ROOT_LIBRARY_DIR)/netcdf4/include
LIB_NETCDF=$(ROOT_LIBRARY_DIR)/netcdf4/lib
INC_HDF5=$(ROOT_LIBRARY_DIR)/hdf5/include
LIB_HDF5=$(ROOT_LIBRARY_DIR)/hdf5/lib
LIB_ZLIB=$(ROOT_LIBRARY_DIR)/zlib/lib   
export INC_NETCDF
export LIB_NETCDF
export INC_HDF5
export LIB_HDF5
export LIB_ZLIB

To accept the changes, type one of the following commands at the Unix prompt:

source ~/.profile   # If you use sh

source ~/.bashrc    # If you use bash

--Bob Y. 12:26, 30 April 2012 (EDT)

Building the netCDF-3 library

netCDF-3 installation process

To build the netCDF-3 library (and related libraries) using the Intel Fortran Compiler ("IFORT"), type:

  make netcdf3 PREFIX=root_library_dir

To build the libraries for the PGI compiler, type:

  make netcdf3 COMPILER=pgi PREFIX=root_library_dir

Makefile options:

COMPILER
This option is used to pick the compiler that you wish to use to build netCDF-4 and the related libraries. At present, you may select either ifort or pgi. If you omit the COMPILER option, the default value of ifort will be selected.
PREFIX

This option is used to set root_library_dir, which is the full path name of the directory into which the netCDF-4 and related libraries will be installed. If you omit the PREFIX option, then root_library_dir will be set to one of these default values:

GEOS-Chem-Libraries/opt/ifort  # If COMPILER=ifort

GEOS-Chem-Libraries/opt/pgi   # If COMPILER=pgi

If you do not have sysadmin privileges, then you can use the PREFIX option to set root_library_dir to a directory in your home space, such as /home/YOUR_USER_ID/lib (or you may install them in one of the default paths listed above). You can then ask your sysadmin to move the libraries to a common space on your machine (or cluster) where all users may access them.

--Bob Y. 12:27, 30 April 2012 (EDT)

netCDF-3 library paths

The netCDF-4 (and related libraries) will be installed into the following paths. As described above, you may use the PREFIX option to set the value of root_library_dir.

For netCDF-3.6.3:
   root_library_dir/netcdf3/bin        # ncdump, ncgen programs
   root_library_dir/netcdf3/include    # Include files (*.h)
   root_library_dir/netcdf3/lib        # Library files (*.a)
   root_library_dir/netcdf3/share      # Manual pages & documentation

--Bob Y. 12:15, 30 April 2012 (EDT)

Telling GEOS-Chem where to find the netCDF-3 libraries

GEOS-Chem uses several environment variables to point to the netCDF-4 library and related libraries. These can be set as follows:

Using csh or tcsh

If you use C-shell or T-shell, then set the following environment variables in your ~/.cshrc file:

setenv ROOT_LIBRARY_DIR  "same directory as root_library_dir above"
setenv INC_NETCDF        "$(ROOT_LIBRARY_DIR)/netcdf3/include
setenv LIB_NETCDF        "$(ROOT_LIBRARY_DIR)/netcdf3/lib
setenv INC_HDF5          ""
setenv LIB_HDF5          ""
setenv LIB_ZLIB          ""

To accept the changes, type the following command at the Unix prompt:

source ~/.cshrc

Using sh or bash

If you use sh (Bourne-shell), then set the following environment variables in your .profile file. If you use bash (Bourne-again shell), then set these variables in your ~/.bashrc file.

ROOT_LIBRARY_DIR="same directory as root_library_dir above"
INC_NETCDF=$(ROOT_LIBRARY_DIR)/netcdf4/include
LIB_NETCDF=$(ROOT_LIBRARY_DIR)/netcdf4/lib
INC_HDF5=""
LIB_HDF5=""
LIB_ZLIB=""
export INC_NETCDF
export LIB_NETCDF
export INC_HDF5
export LIB_HDF5
export LIB_ZLIB

To accept the changes, type one of the following commands at the Unix prompt:

source ~/.profile   # If you use sh

source ~/.bashrc    # If you use bash

--Bob Y. 12:30, 30 April 2012 (EDT)

Linking to the netCDF libraries

Now that you have installed the Zlib, HDF5, and netCDF-4 libraries (or the netCDF-3 library), you must make let the compiler know where these files can be found when you are compiling your source code. Otherwise you will get errors during the compilation process.

From GEOS-Chem

In GEOS-Chem v9-01-03 and higher versions, you will see the following IF statement in Makefile_header.mk (in the top-level source code directory):

# Define the link & include paths for either netCDF-3 or netCDF-4
ifeq ($(NETCDF3),yes)
NCI       := -I$(INC_NETCDF)
NCL       := -L$(LIB_NETCDF) -lnetcdf
else
NCI       := -I$(INC_NETCDF) -I$(INC_HDF5)
NCL       := -L$(LIB_NETCDF) -lnetcdf \
             -L$(LIB_HDF5) -lhdf5_hl \
             -L$(LIB_HDF5) -lhdf5 \
             -L$(LIB_ZLIB) -lz
endif

Notwithstanding the very terse GNU-Make syntax, this is a simple IF statement that does the following:

  1. If you wanted to link GEOS-Chem to a netCDF-3 library build, then
    1. Define the path where the netCDF-3 include files (*.h, *.mod) can be found
    2. Define the path where the netCDF-3 library files (libnetcdf.a) can be found
    3. Tell GEOS-Chem to link to the netCDF-3 library
  2. Otherwise, assume that you wanted to link GEOS-Chem to a netCDF-4 library build
    1. Define the path where the netCDF-4 and HDF5 include files (*.h, .*.mod) can be found
    2. Define the path where the netCDF-4 library file (libnetcdf.a) can be found
    3. Tell GEOS-Chem to link to the netCDF library
    4. Define the path where the HDF-5 library files (libhdf5_hl.a, libhdf5.a) can be found
    5. Tell GEOS-Chem to link to the HDF-5 library
    6. Define the path where the ZLIB library files (libz.a) can be found
    7. Tell GEOS-Chem to link to the ZLIB library

Note that GEOS-Chem now assumes that it will link to a netCDF-4 library build by default. If you want to link to a netCDF-3 library build, simply compile the GEOS-Chem code with the following option:

make -j4 NETCDF3=yes 

You can also declare NETCDF3 as an environment variable either in your ~/.cshrc file:

setenv NETCDF3 yes

or ~/.bashrc file:

NETCDF3=yes
export NETCDF3

so that you don't have to type it every single time on the command line.

--Bob Y. 17:28, 30 April 2012 (EDT)

Detailed explanation of linker commands

Here is an in-depth explanation of each of these commands:

  1. The -I command tells the compiler to look for include (*.h) and module (*.mod) files in the specified directory.
    • Thus, the compiler will look for the netCDF include and module files in $(INC_NETCDF)/include,
    • And IFORT will look for the HDF5 include and module files (for use w/ netCDF-4) in $(INC_HDF5)/include
  2. The -L command tells the compiler to look for library files (*.a) in the specified directory
  3. The -lNAME command tells the compiler to link to the library file libNAME.a. Thus,
    • -L$(LIB_NETCDF) -lnetcdf will link to the library file $(LIB_NETCDF)/libnetcdf.a
    • -L$(LIB_HDF5) -lhdf5_hl will link to the library file $(LIB_HDF5)/libhdf5_hl.a
    • -L$(LIB_HDF5) -lhdf5 will link to the library file $(LIB_HDF5)/libhdf5.a
    • -L$(LIB_ZLIB) -lz will link to the library file $(LIB_ZLIB)/libz.a

Normally, the libraries have to be linked in reverse order. Because netCDF-4 depends on HDF5, and HDF5 depends on Zlib, we link to netCDF4 first, then to HDF5, then to Zlib. (The netCDF-3 library does not depend on any external libraries.)

If you are using a Makefile to compile your code, then this process can be simplified by creating Makefile variables, so that you don't have to explicity write out the names of the directories manually every single time.

--Bob Y. 17:21, 30 April 2012 (EDT)

Avoiding memory issues

If you are trying to work with extremely large data sets (such as GEOS-5.7.2 0.25° x 0.3125° nested grids), you may run into a memory error. To get around this limitation, do the following:

  1. Compile the netCDF, HDF5 and zlib libraries with the following compiler flags:
    1. -mcmodel=medium -i-dynamic  (for Intel Fortran Compiler)
    2. -mcmodel=medium  (for PGI Compiler)

These steps will already have been done for you if you used the GEOS-Chem-Libraries installer.

What is happening is that the compiler by default imposes a imposes maximum memory limit of ~2GB for static variables. If your code works with large data sets, then the combination of user memory + memory in the netCDF/HDF/Zlib libraries can exceed this limit. Using the -mcmodel=medium option tells the compiler to expect to use more than ~2GB of static memory.

--Bob Y. 17:25, 30 April 2012 (EDT)