View source for HEMCO installation guide

<div style="color: #aa0000; background: #eeeeee;border: 3px solid red; padding: 1em; margin: auto; width: 90%; ">'''<p>This wiki page is directed towards those of you who wish incorporate [[HEMCO]] into an atmospheric model other than GEOS-Chem, or who wish use HEMCO in standalone mode. The HEMCO source code is included as part of [[GEOS-Chem v10-01]] and higher versions, and the HEMCO standalone executable is created along with the GEOS-Chem executable. If you are a GEOS-Chem user you will NOT need to download HEMCO separately.</p><p>Whether you use HEMCO as part of GEOS-Chem or not, you (or someone else at your institution) will need to download the [[HEMCO data directories]] to your local disk server.</p><p>&mdash; Bob Yantosca, 18 February 2015</p>'''</div>

On this page we provide instructions for downloading and installing the [[HEMCO|Harvard-NASA Emissions Component (HEMCO)]] source code.

== Overview == 

Before you can use HEMCO, you must make sure that the following libraries and data directories are installed on your system:

{| border=1 cellspacing=0 cellpadding=5 
|-bgcolor="#CCCCCC"
!width="100px"|Package
!width="150px"|Type
!width="750px"|Description

|-valign="top"
|[http://www.git-scm.com Git]
|Version control
|
*Git is used to download the HEMCO and NcdfUtilities packages.
*A version of  typicallly comes pre-installed on all builds of the the Linux, Ubuntu, Fedora, CentOS, etc. operating systems.  
*If your system doesn't have Git, ask your sysadmin or IT staff to install it for you.  Git is free and open-source software.

|-valign="top"
|[http://www.gnu.org/software/make/ GNU Make]
|Building and installing code
|
*A version of GNU Make ypically comes pre-installed all builds of the Linux, Ubuntu, Fedora, CentOS, etc. operating systems.  You probably won't have to install this yourself.

|-valign="top"
|[http://www.perl.org Perl]
|Scripting language
|
*A version of Perl typically comes pre-installed on all builds of the Linux, Ubuntu, Fedora, Centos, etc. operating systems.  You probably won't have to install this yourself.

|-valign="top"
|Fortran
|Compiling source code
|
*HEMCO currently supports the following compilers:
**Intel Fortran Compiler (<tt>ifort</tt>), preferred 
**Portland Group Compiler (<tt>pgf90</tt>).  
***'''''NOTE: There still may be issues with the Portland Group compiler.  We are investigating.'''''

|-valign="top"
|[http://www.unidata.ucar.edu/software/netcdf/ netCDF]
|I/O library
|
*HEMCO reads data files in the netCDF format, and as such, needs to have a version of the netCDF libraries installed on your system.  
*netCDF-4 is preferred, but the older netCDF "classic" versions are OK too.  
*Check with your sysadmin to see if your system already has the netCDF library installed.
**NOTE: If you need to install the netCDF libraries yourself, then you will also need to have C and C++ compilers on your system.  Most versions of the Linux, Ubuntu, etc. operating systems will have the GNU C compilers <tt>gcc</tt> and <tt>g++</tt> already pre-installed.  See the [[#Installing the netCDF libraries|''Installing the netCDF libraries'']] section below for more details.

|-valign="top"
|[http://www.hdfgroup.org/HDF5/ HDF5]
|I/O library
|
*All versions of netCDF-4.0 and higher require the HDF5 libraries.  
*But you do not need HDF5 if you are using earlier versions of netCDF.
*Your system may already have a version of HDF5 pre-installed.  Ask your sysadmin or IT staff to be sure.

|-valign="top"
|NcdfUtilities
|I/O utility routines
|
*This package contains Fortran utility routines for netCDF I/O.  
*It will be downloaded and installed as part of the HEMCO installation process.

|-valign="top"
|[[HEMCO]]
|Emissions
|
*This will be downloaded and installed as part of the HEMCO installation process.

|-valign="top"
|HEMCO data directories
|Data files
|
*This is the directory structure containing the various emissions data and scale factor files that HEMCO will read.
*You can download these via anonymous FTP. See the [[#Downloading the HEMCO emissions data|''Downloading the HEMCO emissions data'']] section below.

|}


The following sections discuss the HEMCO installation process in more detail.

--[[User:Bmy|Bob Y.]] 16:52, 31 July 2014 (EDT)

== Installing the netCDF libraries ==

This section describes how to install the netCDF library, which is required by HEMCO.  But on many systems someone else may have already gone to the trouble of installing netCDF for you, thus sparing you the effort. 

=== Check if your system already has netCDF installed ===

You can always ask your sysadmin or IT staff if netCDF has been installed on your system.  But if your system uses the <tt>module</tt> command to load software packages, you can load a specific build of netCDF into your login environment.  Follow these steps:

'''(1)''' First check to see if a netCDF library has already been loaded into your login environment.  Type 

 module list

and you will get the list of all software modules that you are currently using.  It should look something like this:

 Currently Loaded Modulefiles:
   1) rocks/default-manpath     4) null                      7) compilers/pgi/7.0-7
   2) rocks/core                5) rocks/sge                 8) compilers/gcc/4.3.3
   3) seas/software             6) compilers/intel/11.1      9) mpi/openmpi/1.2.8/intel

As you can see from this example, your login environment isn't using a netCDF library installation.  But this doesn't mean that netCDF is not installed on your system; all it means is that you haven't loaded it yet. 

'''(2)''' To view a list of all of the available software packages on your system, type:

 module avail

at the Unix prompt.  Focus on the libraries section of the list.  You should see several entries such as: 

 libraries/esmf/2.2.0rp1
 libraries/ffmpeg/1.0
 libraries/hdf5/1.8.6
 libraries/libav/0.8.4
 libraries/libvpx/1.1.0
 libraries/mkl/10.0
 libraries/mkl/11.0
 libraries/mkl/11.1
 libraries/netcdf/4.1.1

From this example, you see that the package <tt>libraries/netcdf/4.1.1</tt>&mdash;which is version 4.1.1 of the netCDF library&mdash;has already been installed and is available to use.

NOTE: All versions of netCDF version 4.0 and higher also require an HDF5 library installation.  From the list above you can see that package <tt>libraries/hdf5/1.8.6</tt> indicates that HDF5 version 1.8.6 is also available to use.

'''(3)''' Once you know that netCDF version 4.1.1 and HDF5 version 1.8.6 are available on your system, you can load them into your login environment.  Type:

 module load libraries/netcdf/4.1.1
 module load libraries/hdf5/1.8.6
 module list

and your list of loaded modules will now look like this: 

 Currently Loaded Modulefiles:
   1) rocks/default-manpath     5) rocks/sge                 9) mpi/openmpi/1.2.8/intel
   2) rocks/core                6) compilers/intel/11.1     10) libraries/netcdf/4.1.1
   3) seas/software             7) compilers/pgi/7.0-7      11) libraries/hdf5/1.8.6
   4) null                      8) compilers/gcc/4.3.3

You may also want to add the <tt>module load</tt> commands shown above into your <tt>.cshrc</tt> or <tt>.bashrc</tt> file, so that the netCDF library will be loaded automatically each time you log in to your system.

'''(4)''' Now that you have loaded the netCDF library into your login environment, you will need to know in which directory the library and include files live.  Type:

 printenv | grep NETCDF

and you will see the root directory path 

 NETCDFDIR=/software/linux/x86_64/netcdf-4.1.1

If you then type

 ls $NETCDFDIR

you will see the following directory structure:

 bin/  include/  lib/  share/

Which is arranged as follows:

*The netCDF library files (<tt>libnetcdf.a</tt>, etc) are stored in <tt>$NETCDFDIR/lib</tt>
*The netCDF include and module files (<tt>netcdf.inc</tt>, <tt>netcdf.mod</tt>) are stored in <tt>NETCDFDIR/include</tt>
*The netCDF programs (<tt>nc-config</tt>, <tt>nf-config</tt>, <tt>ncdump</tt>, <tt>ncgen</tt>) are stored in <tt>NETCDFDIR/bin</tt>.

'''(5)''' In this example, because we are using netCDF-4, we also need to know where the HDF5 directory and library files are located.  Type:

 printenv | grep HDF5

and you will see the root directory path 

  HDF5DIR=/software/linux/x86_64/hdf5-1.8.6

if you then type

 ls $HDF5DIR

you will see the following directory structure:

 bin/  include/  lib/  share/

Which is arranged as follows:

*The HDF5 library files (<tt>libhdf5_hl.a</tt> <tt>libhdf5.a</tt>, etc) are stored in <tt>$HDF5DIR/lib</tt>
*The HDF5 include and module files (<tt>*.h/tt>, <tt>*.mod</tt>) are stored in <tt>HDF5DIR/include</tt>
*The HDF5 programs are stored in <tt>HDF5DIR/bin</tt>.

Now that you know where the netCDF (and also HDF5, if applicable) libraries are installed on your system, you can skip ahead to the [[#Downloading_the_HEMCO_emissions_data|Downloading the HEMCO emissions data]] section.

--[[User:Bmy|Bob Y.]] 11:55, 18 July 2014 (EDT)

=== If you don't have already have netCDF, use the GEOS-Chem-Libraries installer ===

If your system does not have a version of the netCDF libraries installed, then you will have to install netCDF yourself.  To simplify this process, we recommend using the <tt>GEOS-Chem-Libraries</tt> installation package.  The <tt>GEOS-Chem-libraries</tt> package will download and install netCDF and its required libraries (HDF5, zlib, etc.) in one process.  You can choose to install either netCDF-4.2 or the netCDF "classic" versions.

For complete instructions on how to use the <tt>GEOS-Chem-Libraries</tt> installer, please see our [[Installing libraries for GEOS-Chem|''Installing libraries for GEOS-Chem'']] wiki page.

NOTE: The <tt>GEOS-Chem-Libraries</tt> installer will place the netCDF and HDF library files into the same directory structure.  We will need to keep this in mind when we [[#If you used the GEOS-Chem-Libraries installer to build netCDF and HDF5|set environment variables for HEMCO]].  

--[[User:Bmy|Bob Y.]] 12:23, 18 July 2014 (EDT)

== Setting environment variables for HEMCO installation ==

Once you have either [[HEMCO_installation_guide#Check_if_your_system_already_has_netCDF_installed|loaded]] or [[HEMCO_installation_guide#If_you_don.27t_have_already_have_netCDF.2C_use_the_GEOS-Chem-Libraries_installer|built]] a version of netCDF, you must then set a few environment variables.  These environment variables tell HEMCO where to look for the netCDF (and HDF5, if you are using netCDF-4) library files.

=== If netCDF and HDF5 were already installed on your system ===

[[#Check if your system already has netCDF installed|Following the examples in the previous section]], let's assume that netCDF and HDF5 has been installed into this directory:

 /software/linux/x86_64/netcdf-4.1.1
 /software/linux/x86_64/hdf5-1.8.6

(Your actual installation paths may differ, depending on the setup that your sysadmin or IT staff have implemented.)

==== For csh or tcsh ====

If your login environment uses the <tt>csh</tt> or <tt>tcsh</tt> Unix shells, then add these lines to your <tt>~/.cshrc</tt> file:

 setenv BIN_NETCDF /software/linux/x86_64/netcdf-4.1.1/bin
 setenv INC_NETCDF /software/linux/x86_64/netcdf-4.1.1/include
 setenv LIB_NETCDF /software/linux/x86_64/netcdf-4.1.1/lib
 setenv BIN_HDF5   /software/linux/x86_64/hdf5-1.8.6/bin
 setenv INC_HDF5   /software/linux/x86_64/hdf5-1.8.6/include
 setenv LIB_HDF5   /software/linux/x86_64/hdf5-1.8.6/lib

and then type 

 source ~/.cshrc

to accept the changes.

--[[User:Bmy|Bob Y.]] 10:30, 1 August 2014 (EDT)

==== For bash ====

If, on the other hand, your login environment uses the <tt>bash</tt> Unix shell, then add these lines to your <tt>.bashrc</tt> file:

 export BIN_NETCDF=/software/linux/x86_64/netcdf-4.1.1/bin
 export INC_NETCDF=/software/linux/x86_64/netcdf-4.1.1/include
 export LIB_NETCDF=/software/linux/x86_64/netcdf-4.1.1/lib
 export BIN_HDF5=/software/linux/x86_64/hdf5-1.8.6/bin
 export INC_HDF5=/software/linux/x86_64/hdf5-1.8.6/include
 export LIB_HDF5=/software/linux/x86_64/hdf5-1.8.6/lib

and type

 source ~/.bashrc

to accept the changes.
 
--[[User:Bmy|Bob Y.]] 16:53, 31 July 2014 (EDT)

=== If you used the GEOS-Chem-Libraries installer to build netCDF and HDF5 ===

If you used the <tt>GEOS-Chem-Libraries</tt> installer to build the netCDF and HDF5 libraries, you were directed to set the <tt>GC_BIN</tt>, <tt>GC_INCLUDE</tt>, and <tt>GC_LIB</tt> environment variables in your <tt>.cshrc</tt> or <tt>.bashrc</tt> file [[Installing_libraries_for_GEOS-Chem#Linking_from_GEOS-Chem|as described here]].  You can use these settings to define the environment variables for HEMCO.

==== For csh or tcsh ====

If your login environment uses the <tt>csh</tt> or <tt>tcsh</tt> Unix shells, then add these lines to your <tt>~/.cshrc</tt> file:

 setenv BIN_NETCDF $GC_BIN
 setenv INC_NETCDF $GC_INCLUDE
 setenv LIB_NETCDF $GC_LIB
 setenv BIN_HDF5   $GC_BIN
 setenv INC_HDF5   $GC_INCLUDE
 setenv LIB_HDF5   $GC_LIB

and then type 

 source ~/.cshrc

to accept the changes.

--[[User:Bmy|Bob Y.]] 10:30, 1 August 2014 (EDT)

==== For bash ====

If your login environment uses the <tt>bash</tt> Unix shell, then add these lines to your <tt>~/.bashrc</tt> file:

 export BIN_NETCDF=$GC_BIN
 export INC_NETCDF=$GC_INCLUDE
 export LIB_NETCDF=$GC_LIB
 export BIN_HDF5=$GC_BIN
 export INC_HDF5=$GC_INCLUDE
 export LIB_HDF5=$GC_LIB
 export HEMCO_DATA_ROOT=/data/emissions/HEMCO

and then type 

 source ~/.bashrc

to accept the changes.

--[[User:Bmy|Bob Y.]] 12:38, 18 July 2014 (EDT)

== Installing HEMCO ==

Once you have [[#Installing the netCDF libraries|installed the netCDF (and HDF5, if necessary) library files]] and [[#Setting environment variables for HEMCO installation|defined the necessary environment variables]], you may install the HEMCO source code.

You can download the latest stable version of HEMCO by typing the following at the command line:

  git clone https://github.com/geoschem/HEMCO HEMCO

This will create an exact copy (or clone) of the official HEMCO repository to your local disk space. You may name the code directory whatever you like. 

===Note for advanced users===

If you plan on developing HEMCO updates to submit for inclusion in the standard model we recommend that you [https://help.github.com/articles/fork-a-repo/ fork the repository] on Github and clone it from your own account. You can then make a [https://help.github.com/articles/creating-a-pull-request/ pull request] when you are ready to submit updates to the [[GEOS-Chem Support Team]]. See [[Submitting_updates_for_inclusion_in_GEOS-Chem|these instructions]] for more information.

== Creating HEMCO run directories ==

Navigate to the the <tt>HEMCO/run/</tt> directory, and at the command line type <tt>./createRunDir.sh</tt>. Follow the instructions to set up your HEMCO standalone run directory.


== Compiling the HEMCO standalone ==

The HEMCO standalone can be compiled with CMake. In your HEMCO standalone run directory, do:

 mkdir build
 cd build
 cmake ../CodeDir
 make -j install

For more details on building HEMCO with CMake, please see [[Building GEOS-Chem with CMake]], as the steps are the similar.


== Where can I get more emissions data for use with HEMCO? ==

'''''NOTE: If you are one of several HEMCO users at your institution, check first to see if someone has already downloaded the HEMCO data directory tree to your system.'''''

We have collated data files (in [http://ferret.wrc.noaa.gov/noaa_coop/coop_cdf_profile.html COARDS-compliant] netCDF format) from several emissions inventories and other atmospheric data sets that you may download.  HEMCO uses data files at their native resolution whenever possible, rather than having a different set of files for several different grids.

Please see our [[HEMCO data directories|''HEMCO data directory'' wiki page]] for a complete list of all of the emissions inventories and other relevant data sets that can be used with HEMCO.  There you will also find [[HEMCO_data_directories#Downloading_the_HEMCO_data_directories|instructions for downloading these data to your local disk server]].

--[[User:Bmy|Bob Y.]] 16:50, 18 February 2015 (EST)