HEMCO installation guide

From Geos-chem
Jump to: navigation, search

This wiki page is directed towards those of you who wish incorporate HEMCO into an atmospheric model other than GEOS-Chem, or who wish use HEMCO in standalone mode. The HEMCO source code is included as part of GEOS-Chem v10-01 and higher versions, and the HEMCO standalone executable is created along with the GEOS-Chem executable. If you are a GEOS-Chem user you will NOT need to download HEMCO separately.

Whether you use HEMCO as part of GEOS-Chem or not, you (or someone else at your institution) will need to download the HEMCO data directories to your local disk server.

— Bob Yantosca, 18 February 2015

On this page we provide instructions for downloading and installing the Harvard-NASA Emissions Component (HEMCO) source code.

Overview

Before you can use HEMCO, you must make sure that the following libraries and data directories are installed on your system:

Package Type Description
Git Version control
  • Git is used to download the HEMCO and NcdfUtilities packages.
  • A version of typicallly comes pre-installed on all builds of the the Linux, Ubuntu, Fedora, CentOS, etc. operating systems.
  • If your system doesn't have Git, ask your sysadmin or IT staff to install it for you. Git is free and open-source software.
GNU Make Building and installing code
  • A version of GNU Make ypically comes pre-installed all builds of the Linux, Ubuntu, Fedora, CentOS, etc. operating systems. You probably won't have to install this yourself.
Perl Scripting language
  • A version of Perl typically comes pre-installed on all builds of the Linux, Ubuntu, Fedora, Centos, etc. operating systems. You probably won't have to install this yourself.
Fortran Compiling source code
  • HEMCO currently supports the following compilers:
    • Intel Fortran Compiler (ifort), preferred
    • Portland Group Compiler (pgf90).
      • NOTE: There still may be issues with the Portland Group compiler. We are investigating.
netCDF I/O library
  • HEMCO reads data files in the netCDF format, and as such, needs to have a version of the netCDF libraries installed on your system.
  • netCDF-4 is preferred, but the older netCDF "classic" versions are OK too.
  • Check with your sysadmin to see if your system already has the netCDF library installed.
    • NOTE: If you need to install the netCDF libraries yourself, then you will also need to have C and C++ compilers on your system. Most versions of the Linux, Ubuntu, etc. operating systems will have the GNU C compilers gcc and g++ already pre-installed. See the Installing the netCDF libraries section below for more details.
HDF5 I/O library
  • All versions of netCDF-4.0 and higher require the HDF5 libraries.
  • But you do not need HDF5 if you are using earlier versions of netCDF.
  • Your system may already have a version of HDF5 pre-installed. Ask your sysadmin or IT staff to be sure.
NcdfUtilities I/O utility routines
  • This package contains Fortran utility routines for netCDF I/O.
  • It will be downloaded and installed as part of the HEMCO installation process.
HEMCO Emissions
  • This will be downloaded and installed as part of the HEMCO installation process.
HEMCO data directories Data files
  • This is the directory structure containing the various emissions data and scale factor files that HEMCO will read.
  • You can download these via anonymous FTP. See the Downloading the HEMCO emissions data section below.


The following sections discuss the HEMCO installation process in more detail.

--Bob Y. 16:52, 31 July 2014 (EDT)

Installing the netCDF libraries

This section describes how to install the netCDF library, which is required by HEMCO. But on many systems someone else may have already gone to the trouble of installing netCDF for you, thus sparing you the effort.

Check if your system already has netCDF installed

You can always ask your sysadmin or IT staff if netCDF has been installed on your system. But if your system uses the module command to load software packages, you can load a specific build of netCDF into your login environment. Follow these steps:

(1) First check to see if a netCDF library has already been loaded into your login environment. Type 

module list

and you will get the list of all software modules that you are currently using. It should look something like this: 

Currently Loaded Modulefiles:
  1) rocks/default-manpath     4) null                      7) compilers/pgi/7.0-7
  2) rocks/core                5) rocks/sge                 8) compilers/gcc/4.3.3
  3) seas/software             6) compilers/intel/11.1      9) mpi/openmpi/1.2.8/intel

As you can see from this example, your login environment isn't using a netCDF library installation. But this doesn't mean that netCDF is not installed on your system; all it means is that you haven't loaded it yet.

(2) To view a list of all of the available software packages on your system, type: 

module avail

at the Unix prompt. Focus on the libraries section of the list. You should see several entries such as: 

libraries/esmf/2.2.0rp1
libraries/ffmpeg/1.0
libraries/hdf5/1.8.6
libraries/libav/0.8.4
libraries/libvpx/1.1.0
libraries/mkl/10.0
libraries/mkl/11.0
libraries/mkl/11.1
libraries/netcdf/4.1.1

From this example, you see that the package libraries/netcdf/4.1.1—which is version 4.1.1 of the netCDF library—has already been installed and is available to use.

NOTE: All versions of netCDF version 4.0 and higher also require an HDF5 library installation. From the list above you can see that package libraries/hdf5/1.8.6 indicates that HDF5 version 1.8.6 is also available to use.

(3) Once you know that netCDF version 4.1.1 and HDF5 version 1.8.6 are available on your system, you can load them into your login environment. Type: 

module load libraries/netcdf/4.1.1
module load libraries/hdf5/1.8.6
module list

and your list of loaded modules will now look like this: 

Currently Loaded Modulefiles:
  1) rocks/default-manpath     5) rocks/sge                 9) mpi/openmpi/1.2.8/intel
  2) rocks/core                6) compilers/intel/11.1     10) libraries/netcdf/4.1.1
  3) seas/software             7) compilers/pgi/7.0-7      11) libraries/hdf5/1.8.6
  4) null                      8) compilers/gcc/4.3.3

You may also want to add the module load commands shown above into your .cshrc or .bashrc file, so that the netCDF library will be loaded automatically each time you log in to your system.

(4) Now that you have loaded the netCDF library into your login environment, you will need to know in which directory the library and include files live. Type: 

printenv | grep NETCDF

and you will see the root directory path 

NETCDFDIR=/software/linux/x86_64/netcdf-4.1.1

If you then type 

ls $NETCDFDIR

you will see the following directory structure: 

bin/  include/  lib/  share/

Which is arranged as follows:

  • The netCDF library files (libnetcdf.a, etc) are stored in $NETCDFDIR/lib
  • The netCDF include and module files (netcdf.inc, netcdf.mod) are stored in NETCDFDIR/include
  • The netCDF programs (nc-config, nf-config, ncdump, ncgen) are stored in NETCDFDIR/bin.

(5) In this example, because we are using netCDF-4, we also need to know where the HDF5 directory and library files are located. Type: 

printenv | grep HDF5

and you will see the root directory path 

 HDF5DIR=/software/linux/x86_64/hdf5-1.8.6

if you then type 

ls $HDF5DIR

you will see the following directory structure: 

bin/  include/  lib/  share/

Which is arranged as follows:

  • The HDF5 library files (libhdf5_hl.a libhdf5.a, etc) are stored in $HDF5DIR/lib
  • The HDF5 include and module files (*.h/tt>, <tt>*.mod) are stored in HDF5DIR/include
  • The HDF5 programs are stored in HDF5DIR/bin.

Now that you know where the netCDF (and also HDF5, if applicable) libraries are installed on your system, you can skip ahead to the Downloading the HEMCO emissions data section.

--Bob Y. 11:55, 18 July 2014 (EDT)

If you don't have already have netCDF, use the GEOS-Chem-Libraries installer

If your system does not have a version of the netCDF libraries installed, then you will have to install netCDF yourself. To simplify this process, we recommend using the GEOS-Chem-Libraries installation package. The GEOS-Chem-libraries package will download and install netCDF and its required libraries (HDF5, zlib, etc.) in one process. You can choose to install either netCDF-4.2 or the netCDF "classic" versions.

For complete instructions on how to use the GEOS-Chem-Libraries installer, please see our Installing libraries for GEOS-Chem wiki page.

NOTE: The GEOS-Chem-Libraries installer will place the netCDF and HDF library files into the same directory structure. We will need to keep this in mind when we set environment variables for HEMCO.

--Bob Y. 12:23, 18 July 2014 (EDT)

Setting environment variables for HEMCO installation

Once you have either loaded or built a version of netCDF, you must then set a few environment variables. These environment variables tell HEMCO where to look for the netCDF (and HDF5, if you are using netCDF-4) library files.

If netCDF and HDF5 were already installed on your system

Following the examples in the previous section, let's assume that netCDF and HDF5 has been installed into this directory: 

/software/linux/x86_64/netcdf-4.1.1
/software/linux/x86_64/hdf5-1.8.6

(Your actual installation paths may differ, depending on the setup that your sysadmin or IT staff have implemented.)

For csh or tcsh

If your login environment uses the csh or tcsh Unix shells, then add these lines to your ~/.cshrc file: 

setenv BIN_NETCDF /software/linux/x86_64/netcdf-4.1.1/bin
setenv INC_NETCDF /software/linux/x86_64/netcdf-4.1.1/include
setenv LIB_NETCDF /software/linux/x86_64/netcdf-4.1.1/lib
setenv BIN_HDF5   /software/linux/x86_64/hdf5-1.8.6/bin
setenv INC_HDF5   /software/linux/x86_64/hdf5-1.8.6/include
setenv LIB_HDF5   /software/linux/x86_64/hdf5-1.8.6/lib

and then type 

source ~/.cshrc

to accept the changes.

--Bob Y. 10:30, 1 August 2014 (EDT)

For bash

If, on the other hand, your login environment uses the bash Unix shell, then add these lines to your .bashrc file: 

export BIN_NETCDF=/software/linux/x86_64/netcdf-4.1.1/bin
export INC_NETCDF=/software/linux/x86_64/netcdf-4.1.1/include
export LIB_NETCDF=/software/linux/x86_64/netcdf-4.1.1/lib
export BIN_HDF5=/software/linux/x86_64/hdf5-1.8.6/bin
export INC_HDF5=/software/linux/x86_64/hdf5-1.8.6/include
export LIB_HDF5=/software/linux/x86_64/hdf5-1.8.6/lib

and type 

source ~/.bashrc

to accept the changes.

--Bob Y. 16:53, 31 July 2014 (EDT)

If you used the GEOS-Chem-Libraries installer to build netCDF and HDF5

If you used the GEOS-Chem-Libraries installer to build the netCDF and HDF5 libraries, you were directed to set the GC_BIN, GC_INCLUDE, and GC_LIB environment variables in your .cshrc or .bashrc file as described here. You can use these settings to define the environment variables for HEMCO.

For csh or tcsh

If your login environment uses the csh or tcsh Unix shells, then add these lines to your ~/.cshrc file: 

setenv BIN_NETCDF $GC_BIN
setenv INC_NETCDF $GC_INCLUDE
setenv LIB_NETCDF $GC_LIB
setenv BIN_HDF5   $GC_BIN
setenv INC_HDF5   $GC_INCLUDE
setenv LIB_HDF5   $GC_LIB

and then type 

source ~/.cshrc

to accept the changes.

--Bob Y. 10:30, 1 August 2014 (EDT)

For bash

If your login environment uses the bash Unix shell, then add these lines to your ~/.bashrc file: 

export BIN_NETCDF=$GC_BIN
export INC_NETCDF=$GC_INCLUDE
export LIB_NETCDF=$GC_LIB
export BIN_HDF5=$GC_BIN
export INC_HDF5=$GC_INCLUDE
export LIB_HDF5=$GC_LIB
export HEMCO_DATA_ROOT=/data/emissions/HEMCO

and then type 

source ~/.bashrc

to accept the changes.

--Bob Y. 12:38, 18 July 2014 (EDT)

Installing HEMCO

Once you have installed the netCDF (and HDF5, if necessary) library files and defined the necessary environment variables, you may install the HEMCO source code.

You can download the latest stable version of HEMCO by typing the following at the command line: 

 git clone https://github.com/geoschem/HEMCO HEMCO

This will create an exact copy (or clone) of the official HEMCO repository to your local disk space. You may name the code directory whatever you like.

Note for advanced users

If you plan on developing HEMCO updates to submit for inclusion in the standard model we recommend that you fork the repository on Github and clone it from your own account. You can then make a pull request when you are ready to submit updates to the GEOS-Chem Support Team. See these instructions for more information.

Creating HEMCO run directories

Navigate to the the HEMCO/run/ directory, and at the command line type ./createRunDir.sh. Follow the instructions to set up your HEMCO standalone run directory.


Compiling the HEMCO standalone

The HEMCO standalone can be compiled with CMake. In your HEMCO standalone run directory, do: 

mkdir build
cd build
cmake ../CodeDir
make -j install

For more details on building HEMCO with CMake, please see Building GEOS-Chem with CMake, as the steps are the similar.


Where can I get more emissions data for use with HEMCO?

NOTE: If you are one of several HEMCO users at your institution, check first to see if someone has already downloaded the HEMCO data directory tree to your system.

We have collated data files (in COARDS-compliant netCDF format) from several emissions inventories and other atmospheric data sets that you may download. HEMCO uses data files at their native resolution whenever possible, rather than having a different set of files for several different grids.

Please see our HEMCO data directory wiki page for a complete list of all of the emissions inventories and other relevant data sets that can be used with HEMCO. There you will also find instructions for downloading these data to your local disk server.

--Bob Y. 16:50, 18 February 2015 (EST)

Retrieved from "http://wiki.seas.harvard.edu/geos-chem/index.php?title=HEMCO_installation_guide&oldid=47471"