HEMCO data directories

On this page we describe the directory tree from which the HEMCO emissions component can read emissions inventories and other atmospheric data sets.

Overview

The HEMCO emissions component can read several types of emission inventories, as well as other types atmospheric data sets, such as production and loss rates, or concentration data. We have collated all of this data into a comprehensive directory tree structure. Each folder of the HEMCO data directory tree represents a particular emissions inventory or other data set.

At present, the HEMCO data directory tree resides on the disk servers at Harvard University (and soon at Dalhousie University). We have created a package that will let you download this directory tree to your local disk storage space. For more information, please see the Downloading the HEMCO data directories section below.

Please see our list of recommended default emission inventories, which has been compiled by the Emissions and Deposition Working Group.

--Bob Yantosca (talk) 22:27, 7 September 2017 (UTC)

HEMCO data directory structure

The sections below describe each of the data sets contained in the HEMCO directory tree, grouped by type.

In each of the tables below:

The Inventory column contains a link that describes each data set in more detail. Most of these links point to existing pages on the GEOS-Chem wiki.

The Data file info column points to the README files that are stored with each data set. Each README provides a list of the files contained within a given folder, information about the data contained in each file, and (often) a description of how the files were created.

The Path column shows the location of each data set, with respect to $ROOT, the top-level HEMCO directory. For example, on the Harvard disk server, $ROOT points to the directory /mnt/gcgrid/data/ExtData/HEMCO/.

The Status column describes the current status of each data set:

CURRENTLY USED	Denotes that the data set is part of the standard emissions configuration for GEOS-Chem, that is: It is used in the GEOS-Chem 1-month and 1-year full-chemistry benchmark simulations, or It is used in one or more of the GEOS-Chem specialty simulations.
OPTIONAL	Denotes that the data set is not part of the the standard GEOS-Chem emissions configuration. But you can still use the data set in your research applications if you wish.
TO BE ADDED SOON	Denotes that the data set is not yet ready for use with HEMCO, but will be added soon.
OBSOLETE	Denotes that the data set has been removed from the standard GEOS-Chem emissions configuration. In most cases, OBSOLETE means that a newer version of the data is available and should be used instead.
NOT USED	Denotes that the data set is not used in the standard GEOS-Chem emissions configuration, even if it is current. For example, another data source is being used in place of this data set for a particular reason.

If use GEOS-Chem, then we recommend that you download the data sets that are listed as CURRENTLY USED, as these form the standard GEOS-Chem emissions configuration. You are free, however, to use any of the data sets listed below. Your choice of data sets will depend on your particular research needs.

--Bob Y. 10:53, 18 March 2015 (EDT)

Aerosol emissions

The following subdirectories of the HEMCO directory tree contain aerosol emission inventories.

Inventory	Data file info	Path	Directory size	Status
AEROCOM volcanic emissions	README	$ROOT/VOLCANO/v2015-02	1.6 GB	CURRENTLY USED Default global inventory for volcanic SO2.
OMI-based volcanic emissions	README	$ROOT/VOLCANO/v2018-03	12 MB	TO BE ADDED SOON Will be introduced in GEOS-Chem v11-02 Coverage extends from 2005-2012. Will replace the AEROCOM volcanic emissions.
Tami Bond et al (2007) BC and OC emissions	README	$ROOT/BCOC_BOND/v2014-07	3.0 MB	CURRENTLY USED Default global inventory for black carbon (BC) and organic carbon (OC).
Secondary organic aerosols	README	$ROOT/SOA/2014-07	3.0 MB	CURRENTLY USED Contain various inputs for SOA simulations.
Cooke et al BC and OC emissions	README	$ROOT/BCOC_COOKE/v2014-07	744 KB	OPTIONAL Can be used in place of Bond et al (2007).

--Bob Y. 10:15, 18 March 2015 (EDT)

Anthropogenic and biofuel emissions

The following subdirectories of the HEMCO directory tree contain inventories of anthropogenic and biofuel emissions.

Please also see list of species included in each of these global inventories and regional inventories.

Inventory	Data file info	Path	Directory size	Status
Inventories that are activated by default in the HEMCO configuration file for the current GEOS-Chem public release (v11-01)
EDGAR v4.2 global anthropogenic	README	$ROOT/EDGARv42/v2015-02 $ROOT/EDGARv42/v2015-02/CO $ROOT/EDGARv42/v2015-02/NH3 $ROOT/EDGARv42/v2015-02/NO $ROOT/EDGARv42/v2015-02/SO2 $ROOT/EDGARv42/v2015-02/VOCv2	4.5 GB	CURRENTLY USED Default global inventory for NOx, CO, SO2, and NH3 (replaces EDGAR v3). The following sectors are not used: EDGAR v4.2 ship emissions EDGAR v4.2 soil NO emissions EDGAR v4.2 biomass emissions NAP is still taken from EDGAR v2 data.
GEIA NH3 (anthro, biofuel, natural source)	README	$ROOT/NH3/v2014-07	8.1 MB	CURRENTLY USED Default global inventory for biofuel & natural NH3. Anthro, biofuel NH3 is overwritten by EDGAR v4.2
RETRO VOC emissions	README	$ROOT/RETRO/v2014-07/orig_kgC	38 MB	CURRENTLY USED Default global inventory for VOC species. C2H6 is replaced by Y. Xiao et al emissions.
Yevich & Logan biofuels	README	$ROOT/BIOFUEL/v2014-07	1.1 MB	CURRENTLY USED Default global biofuel emissions inventory.
BRAVO regional anthropogenic	README	$ROOT/BRAVO/v2014-07	68 KB	CURRENTLY USED Overwrites CO, NO, and SO2 over Mexico.
CAC regional anthropogenic	README	$ROOT/CAC/v2016-09	29 MB	CURRENTLY USED This is the default CAC inventory in v11-01 and higher versions Overwrites CO, NO, NH3, and SOx over Canada. Includes data at 0.1° x 0.1° resolution
EMEP regional anthropogenic	README	$ROOT/EMEP/v2015-03	24 MB	CURRENTLY USED Overwrites CO, NH3, NO, SO2, PRPE, ALK4, C2H6, ALD2, and MEK over Europe.
MIX Asian anthropogenic emissions	README	$ROOT/MIX/v2015-03	623 MB	CURRENTLY USED Replaces David Streets et al emissions. The base years for MIX is 2008 and 2010, as opposed to 2006 for Streets.
NEI2005 regional anthro/biofuel	README	$ROOT/NEI2005/v2014-09	8.1 MB	CURRENTLY USED Overwrites ACET, ALK4, ALD2, BC, C2H6, C3H8, CO, CH2O, MEK, NH3, NIT, NO, OC, PRPE, SO2, and SO4 over the USA. Use NEI2011 for simulations covering the period 2006-2013; otherwise use NEI2005. NOTE: NEI2005 is eventually going to be retired.
NEI/VISTAS scale factors	README1 README2	$ROOT/VISTAS/v2014-07 $ROOT/VISTAS/v2014-07/VISTAS_NOX $ROOT/VISTAS/v2014-07/VISTAS_WEWD	46 MB	CURRENTLY USED Used to scale NO in the NEI2005 inventory.
NEI2011 regional anthro/biofuel (hourly data)	README	$ROOT/NEI2011/v2015-03	248 GB	CURRENTLY USED Overwrites CO, NO, NO2, HNO2, NH3, CH2O, RCHO, MACR, ACET, ALK4, PRPE, EOH, MOH, XYLE, TOLU, BENZ, SO2, SO4, C2H4, C2H6, C3H8, ALD2, BC, and OC over the USA. Use NEI2011 for simulations covering the period 2006-2013; otherwise use NEI2005.
NEI2011 agricultural emissions only	README	$ROOT/NEI2011_ag_only/v2015-03	648 MB	CURRENTLY USED
Yaping Xiao et al C2H6 and C3H8 anthropogenic	README	$ROOT/XIAO/v2014-09	308 KB	CURRENTLY USED Overwrites RETRO C2H6, which is problematic.
Inventories that are deactivated by default in the HEMCO configuration file for the current GEOS-Chem public release (v11-01). You can use any of these inventories for your own research needs.
MASAGE agricultural NH3	README	$ROOT/MASAGE_NH3/v2015-02	4.4 MB	OPTIONAL Adds to the anthropogenic NH3 category.
HTAP global anthro+biofuel	README	$ROOT/HTAP/v2015-03	4.5 GB	OPTIONAL Can be used instead of EDGAR v4.2 emissions. NOTE: HTAP includes some regional emissions from other inventories. If you select HTAP you might not need to select the other regional inventories in HEMCO. See this document for more information.
Streets regional anthro	README1 README2	$ROOT/STREETS/v2014-07	1.7 MB	OPTIONAL Streets is being retired in favor of MIX. We will keep Streets as a research option. You may use it if you wish.
NEI2008 regional anthro/biofuel	README	$ROOT/NEI2008/v2015-02	3.0 GB	OPTIONAL We shall keep this as a research option NEI2008 is superseded by NEI2011.
Inventories that will be introduced in the next GEOS-Chem public release (v11-02)
C2H6 global emissions	README	$ROOT/C2H6_2010/v2017-05	140 GB	TO BE ADDED SOON Will replace the C2H6 emissions from both RETRO and Y. Xiao inventories.
CEDS anthropogenic emissions	README	$ROOT/CEDS/v2018-04	140 GB	TO BE ADDED SOON Will become the default global inventory for v11-02 full-chemistry simulations
EDGAR v4.3 anthropogenic emissions	README	$ROOT/EDGARv43/v2016-11	6.2 GB	TO BE ADDED SOON Introduced in GEOS-Chem v11-02 Can be used instead of CEDS
DICE-Africa anthropogenic emissions	README	$ROOT/DICE_Africa/v2016-10	202 MB	TO BE ADDED SOON Will become available in v11-02
NEI2011 regional anthro/biofuel (monthly mean data)	README	$ROOT/NEI2011/v2017-02-MM	TBD	TO BE ADDED SOON New monthly mean data files have been created from the existing hourly data in NEI2011/v2015-03. Reading monthly mean data can substantially reduce the I/O burden in GEOS-Chem simulations. This will be the default in v11-02a and higher versions.
POET anthropogenic emissions	README	HEMCO/POET/v2017-03/	44 KB	TO BE ADDED SOON Introduced with the PAN updates in v11-02a.
Trash burning emissions	README	HEMCO/TrashEmis/v2015-03/	86 MB	TO BE ADDED SOON Will become available in v11-02
Inventories that have been superseded by newer developments. You can still use these for your own research needs, if necessary.
CAC regional anthropogenic (1° x 1° resolution)	README	$ROOT/CAC/v2014-07	384 KB	OBSOLETE This was the default CAC inventory for v10-01 and prior versions Superseded by v2016-09
GEIA global anthropogenic	README	$ROOT/GEIA/v2014-07	4.8 MB	OBSOLETE Slated for replacement by newer inventories
EDGAR v3 global anthropogenic	README	$ROOT/EDGAR/v2014-07	40 MB	OBSOLETE EDGAR v3 has been replaced by EDGAR v4.2.

--Bob Yantosca (talk) 17:39, 25 April 2017 (UTC)

Anthropogenic aircraft and ship emissions

The following subdirectories of the HEMCO directory tree contain inventories of anthropogenic aircraft and ship emissions.

Inventory	Data file info	Path	Directory size	Status
AEIC aircraft	README	$ROOT/AEIC/v2015-01	2.0 GB	CURRENTLY USED Default global aircraft emissions inventory. Contains fuel burned, NO, CO, and hydrocarbons.
ARCTAS ship emissions (SO2)	README	$ROOT/ARCTAS_SHIP/v2014-07	508 KB	CURRENTLY USED Default inventory for ship SO2. Overwritten by EMEP over Europe.
ICOADS ship (CO)	README	$ROOT/ICOADS_SHIP/v2014-07	4.2 MB	CURRENTLY USED Default inventory for ship CO. Overwritten by EMEP over Europe.
EMEP ship (CO, NO, SO2)	README	$ROOT/EMEP/v2015-03	24 MB	CURRENTLY USED Overwrites ship CO, NO, SO2 over Europe. Stored together w/ other EMEP data files.
Corbett et al ship emissions (SO2)	README	$ROOT/CORBETT_SHIP/v2014-07	1.8 MB	OPTIONAL Can be used to overwrite ARCTAS ship SO2.
HTAP ship (CO, NO, SO2)	README	$ROOT/HTAP/v2015-03	4.5 GB	OPTIONAL Stored together with other HTAP data files. See this document for more information.
NEI2011 ship (several species)	README	$ROOT/NEI2011/v2015-03	248 GB	OPTIONAL Can be used to overwrite the following ship emissions over North America: NO, NO2, HONO, CO, NH3, CH2O, RCHO, MACR, ACET, C3H8, MEK, ALK4, PRPE, EOH, MOH, XYLE, TOLU, BENZ, SO2, SO4, C2H4, C2H6, ALD2, BCPI, BCPO, OCPI, OCPO.
EDGAR v3 ship (CO)	README	$ROOT/EDGAR/v2014-07	40 MB	OBSOLETE
EDGAR v4.2 ship	README	$ROOT/EDGARv42/v2015-02	4.5 GB	NOT USED EDGAR v4.2 ship emissions are not used because they are lumped with other non-road transportation sectors (aircraft, rail, etc), and thus cannot be easily separated.

--Bob Y. 10:07, 18 March 2015 (EDT)

Biomass burning emissions

The following subdirectories of the HEMCO directory tree contain inventories of biomass burning emissions.

Inventory	Data file info	Path	Directory size	Status
GFED4 biomass burning	README	$ROOT/GFED4/v2015-10	692 MB	CURRENTLY USED This is GFED v4.1, which is the default in v11-01 and higher versions Monthly-mean data is available from 1998-2014 Uses daily scale factors from GFED3. Uses 3-hourly scale factors from GFED3.
	README	$ROOT/GFED4/v2015-06	115 MB	OBSOLETE This was the default in v10-01 and prior versions. Superseded by GFED v4.1
GFED3 biomass burning	README	$ROOT/GFED3/v2014-10	343 MB	CURRENTLY USED But will be retired in GEOS-Chem v11-02
FINN v1.6 biomass burning	README	$ROOT/FINN/v2018-04	1.1 GB	OPTIONAL Compatible with v11-02 and higher versions You may choose to replace GFED with FINN for research purposes. If you do not need FINN, you may choose not to download it, in order to save disk space.
FINN v1.5 biomass burning	README	$ROOT/FINN/v2015-02	689 MB	OPTIONAL Compatible with v11-01 You may choose to replace GFED with FINN for research purposes. If you do not need FINN, you may choose not to download it, in order to save disk space.
QFED biomass burning	README	$ROOT/QFED/v2014-09	57 GB	OPTIONAL You may choose to replace GFED with QFED for research purposes. If you do not need QFED, you may choose not to download it, in order to save disk space.
GFED2 biomass burning	README	$ROOT/GFED2/v2014-07	196 KB	OBSOLETE Superseded by GFED4.
Duncan et al biomass burning	README	$ROOT/BIOBURN/v2014-07	925 MB	OBSOLETE Superseded by GFED4.

--Bob Yantosca (talk) 15:57, 19 December 2016 (UTC)

Emissions implemented as HEMCO extensions

The following subdirectories of the HEMCO directory tree contain input data used by various HEMCO extensions. These HEMCO extensions compute emissions for quantities that depend on meteorological variables (e.g. emissions from lighting, biogenic processes, etc.).

NOTE: GFED3, GFED4 and FINNv1 are implemented as HEMCO extensions, but we have listed them with the other biomass burning inventories, for clarity.

Inventory	Data file info!	Path	Directory size	Status
Acetone ocean exchange	README	$ROOT/ACET/v2014-07	52 KB	CURRENTLY USED
DEAD dust model	README	$ROOT/DUST_DEAD/2014-07	712 KB	CURRENTLY USED Default dust mobilization scheme
DMS ocean exchange	README	$ROOT/DMS/v2015-07	3.0 MB	CURRENTLY USED This is the default in v11-01 and higher versions Uses Lana 2011 climatology
	README	$ROOT/DMS/v2014-07	2.2 MB	OBSOLETE Superseded by the Lana 2011 climatology
MEGAN biogenic emissions	README	$ROOT/MEGAN/v2015-02	2.0 MB	CURRENTLY USED Default global biogenic emissions inventory.
NO from lightning	README	$ROOT/LIGHTNOX/v2014-07	13 MB	CURRENTLY USED
NO from soils/fertilizers	README	$ROOT/SOILNOX/v2014-07	79 MB	CURRENTLY USED
PARANOX ship plume model	README	$ROOT/PARANOX/v2015-02	162 MB	CURRENTLY USED Computes the aging of emissions in ship exhaust plumes.
GINOUX dust model	README	$ROOT/DUST_GINOUX/2014-07	36 KB	OPTIONAL You can use this in place of the DEAD dust model.

--Bob Yantosca (talk) 15:59, 19 December 2016 (UTC)

Future and historical emissions

The following subdirectories of the HEMCO directory tree contain historical and future emissions inventories.

Inventory	Data file info	Path	Directory size	Status
APEI historical Canadian emissions	README	HEMCO/APEI/v2016-11/	368 MB	OPTIONAL Will be introduced in GEOS-Chem v11-02. Historical Canadian emissions are not part of the standard GEOS-Chem emissions configuration but may be used as a research option.
RCP future scenarios	README	$ROOT/RCP/v2015-02	368 MB	OPTIONAL RCP emissions are not part of the standard GEOS-Chem emissions configuration but may be used as a research option.

--Bob Y. 15:04, 17 March 2015 (EDT)

GEOS-Chem specialty simulation data

The following subdirectories of the HEMCO directory tree input data (emissions, oxidants, etc.) for use with the GEOS-Chem specialty simulations. If you do not regularly use these simulations, you may choose not to download these data directories.

Inventory	Data file info	Path	Directory size	Status
Aerosol-only simulation	README	$ROOT/OFFLINE_AEROSOL/v2014-09	165 MB	CURRENTLY USED
CH4 simulation	README	$ROOT/CH4/v2014-09	274 MB	CURRENTLY USED
CO2 simulation	README1 README2 README3 README4 README5 README6	$ROOT/CO2/v2015-04 $ROOT/CO2/v2015-04/BIO/ $ROOT/CO2/v2015-04/BIOFUEL/ $ROOT/CO2/v2015-04/CHEM/ $ROOT/CO2/v2015-04/FOSSIL/ $ROOT/CO2/v2015-04/OCEAN/	1033 MB	CURRENTLY USED
Mercury simulation	README1 README2 README3 README4 README5 README6 README7 README8 README9 README10	$ROOT/MERCURY/v2014-09 $ROOT/MERCURY/v2014-09/ARTISANAL $ROOT/MERCURY/v2014-09/BrOx $ROOT/MERCURY/v2014-09/Hg2_PARTITION $ROOT/MERCURY/v2014-09/JVALUES $ROOT/MERCURY/v2014-09/NATURAL $ROOT/MERCURY/v2014-09/NEI2005 $ROOT/MERCURY/v2014-09/OCEAN $ROOT/MERCURY/v2014-09/SOIL $ROOT/MERCURY/v2014-09/STREETS	342 MB	CURRENTLY USED
POPs simulation	README	$$ROOT/POPs/v2015-08	809 MB	CURRENTLY USED Data corresponds to the POPs simulation update in v11-01c
	README	$ROOT/POPs/v2014-09	622 MB	OBSOLETE Superseded by v2015-08
Tagged CO simulation	README	$ROOT/TAGGED_CO/v2014-08	260 KB	CURRENTLY USED
Tagged O3 simulation	README	$ROOT/TAGGED_O3/v2014-09	372 MB	CURRENTLY USED
O3 for offline simulations	README	$ROOT/O3/v2014-09	130 MB	CURRENTLY USED
OH for offline simulations	README1 README2	$ROOT/OH/v2014-09 $ROOT/OH/v2014-09/v5-07-08 $ROOT/OH/v2014-09/v7-02-03.GMI	148 MB	CURRENTLY USED
H2O2 for offline simulations	README	$ROOT/OXIDANTS/v2014-07	32 MB	CURRENTLY USED
Oceanic Chlorophyll-A for Hg simulations	README	$ROOT/CHLA/v2014-07	2.0 MB	CURRENTLY USED
CH3I simulation	README	$ROOT/CH3I/v2014-07	280 KB	OBSOLETE This simulation is no longer used in GEOS-Chem (awaiting revival)

--Bob Yantosca (talk) 16:02, 19 December 2016 (UTC)

Halogen emissions

The following subdirectories of the HEMCO directory tree contain inventories of bromine and iodine emissions.

NOTE: See our Non-emissions data section for information about stratospheric bromine concentration data.

Inventory	Data file info	Path	Directory size	Status
Liang et al VSL emissions	README	$ROOT/BROMINE/v2015-02	156 KB	CURRENTLY USED
Ordonez et al Iodine emissions	README	$ROOT/IODINE/v2017-03	156 KB	TO BE ADDED SOON Will become standard in v11-02

--Bob Yantosca (talk) 21:16, 5 March 2018 (UTC)

Natural emissions data

The following subdirectories of the HEMCO directory tree contain inventories natural emissions.

Inventory	Data file info	Path	Directory size	Status
NH3 emissions from arctic sea birds	README	HEMCO/NH3/v2018-04/	104 MB	TO BE ADDED SOON Will become available in v11-02

--Bob Yantosca (talk) 19:56, 17 April 2018 (UTC)

Non-emissions data

The following subdirectories of the HEMCO directory tree input data contains non-emissions data sets. Most of these are used as inputs to GEOS-Chem's chemistry modules (i.e. photolysis, stratospheric chemistry, etc.).

Inventory	Data file info	Path	Directory size	Status
GMI strat chem mechanism	README	$ROOT/GMI/v2015-02	16 GB	CURRENTLY USED Used to compute P & L of species in the stratosphere.
Stratospheric Bry from CCM	README	$ROOT/STRAT/v2015-01/Bry	385 MB	CURRENTLY USED Required for the for the full-chemistry simulations.
Timezone offsets from UTC	README	$ROOT/TIMEZONES/v2015-02	264 KB	CURRENTLY USED Used by HEMCO to compute emissions that depend on local time rather than on UTC time.
TOMS/SBUV O3 columns	README	$ROOT/TOMS_SBUV/v2016-11	21 MB	CURRENTLY USED This is default in v11-01 and higher versions. These files are the same data as in $ROOT/TOMS_SBUV/v2015-03, but were reprocessed by Barron Henderson in order in order to fix a strange cycle in OH output when running GEOS-Chem with GEOS-5 met. Ignored if you are using GEOS-FP or MERRA-2 meteorology.
	README	$ROOT/TOMS_SBUV/v2015-03	17 MB	OBSOLETE This is the default in v10-01 and prior versions Superseded by v2016-11
UCX data for linearized strat chem	README	$ROOT/UVALBEDO/v2015-03	6.6 GB	CURRENTLY USED Used as an input to the FAST-JX photolysis mechanism.
UV surface albedoes	README	$ROOT/UCX/v2015-03	468 KB	TO BE ADDED SOON Will supersede the GMI strat chem data in v11-02

--Bob Yantosca (talk) 19:56, 17 April 2018 (UTC)

Oceanic emissions

The following folders contain data used to compute oceanic emissions of GEOS-Chem species.

Inventory	Data file info	Path	Directory size	Status
Fields to compute ALD2 emissions Seawater concentration of acetaldehyde Heterotrophic respiration rates, used to compute biogenic emissions of ALD2 and EOH	README	$ROOT/ALD2/v2017-03	0.5 MB	TO BE ADDED SOON Introduced with the PAN updates in v11-02a.

--Bob Yantosca (talk) 16:59, 25 April 2017 (UTC)

Other inputs for HEMCO

The following subdirectories of the HEMCO directory tree input data for various HEMCO functions. These include regional masks, emission scale factors, and grid information.

Inventory	Data file info	Path	Directory size	Status
Annual scale factors	README	$ROOT/AnnualScalar/v2014-07	3.0 MB	CURRENTLY USED Used to scale NEI2005 and other emissions to years other than the inventory base year.
ID codes for each country	README	$ROOT/COUNTRY_ID/v2015-04	15 MB	CURRENTLY USED ISO ALPHA-3 standard country codes are contained on an 0.1° x 0.1° grid. HEMCO can use these ID's to apply country-level scale factors to any emissions field.
Mask files for regional emissions	README	$ROOT/MASKS/v2014-07	51 MB	CURRENTLY USED Masks define the geographical regions where the various regional emissions (e.g. BRAVO, CAC, NEI) are to be applied.
MAP_A2A regridding data	README	$ROOT/MAP_A2A/v2014-07	152 KB	CURRENTLY USED
Weekly scale factors	README	$ROOT/WEEKSCALE/v2014-07	4.0 KB	OBSOLETE Not currently in use

--Bob Y. 15:08, 17 March 2015 (EDT)

Downloading the HEMCO data directories

The GEOS-Chem Support Team has created a package called hemco_data_download. With this package, you can download the various emissions inventories and related data files for HEMCO to your own disk server. Furthermore, you can specify which data directories that you would like to download (as well as those you would like to ignore) via a configuration file.

Obtaining the hemco_data_download package

To obtain the hemco_data_download package, use Git to clone this repository

git clone https://bitbucket.org/gcst/hemco_data_download.git

This will create a directory named hemco_data_download, in which you should see theR following files:

File	Description
README	File with an overall description of the directory contents
hemcoDataDownload.pl	Perl script to download HEMCO data directories
hemcoDataDownload.rc	Configuration file for the hemcoDataDownload.pl script. In this file you can specify which HEMCO data directories you would like to download and which you would like to omit.
forTesting.rc	A configuration file that you can use for testing or debugging. This will tell hemcoDataDownload.pl only to download a couple of emissions inventories with files that do not take up much disk space.

--Bob Yantosca (talk) 17:02, 5 December 2016 (UTC)

Setting up the configuration file

The configuration files (i.e. hemcoDataDownload.rc and forTesting.rc) are pretty much self-explanatory.

At the top of the configuration file you will see this section:

###############################################################################
#                                                                             #
#  Specify the remote and local HEMCO data paths, plus other options.         #
#                                                                             #
###############################################################################

Remote HEMCO data path | ftp://ftp.as.harvard.edu/gcgrid/data/ExtData/HEMCO
Your HEMCO data path   | /as/scratch/bmy/HEMCO
Verbose output         | yes
Dryrun only?           | no

Path	Description
Remote HEMCO data path	Location on the FTP server from which you are going to download the data. This can be from either Harvard or from Dalhousie. (For now we will use the Harvard server). You can edit this accordingly.
Your HEMCO data path	The root-level directory for HEMCO data on your own disk space. If you are not sure where to place this, then ask your sysadmin.
Verbose output	Lets you specify if you want to print out extra output during the download process. This can be set to either "yes" or "no".
Dryrun only	Allows you to print out the data download commands without actually downloading the data. This is useful for debugging. This can be set to either "yes" or "no".

In the next section you specify all of the HEMCO inventories that you want to download. You will see this header:

###############################################################################
#                                                                             #
#  THE FOLLOWING DATA DIRECTORIES WILL BE DOWNLOADED.                         #
#                                                                             #
#  These data directories comprise the recommended emissions configuration    #
#  for typical GEOS-Chem full-chemistry and specialty simulations.            #
#                                                                             #
#  NOTE: In most cases, you only have to specify the top-level folder.        #
#  All subfolders will be downloaded automatically.                           #
#                                                                             #
###############################################################################

#=============================+================================================
# AEROSOLS                    | Directory paths
#=============================+================================================
AEROCOM volcano emissions     | $ROOT/VOLCANO/v2014-10
Bond et al BC/OC              | $ROOT/BCOC_BOND/v2014-07
Cooke et al BC/OC             | $ROOT/BCOC_COOKE/v2014-07
Secondary organic aerosols    | $ROOT/SOA/v2014-07
... etc ...

Each line specifies the name of a HEMCO emissions inventory and the data path where it can be found on disk, relative to the root data path. NOTE: The script will replace the $ROOT token with the value you gave to the "HEMCO remote data path" above. (Lines starting with the comment character # will be ignored.)

Any inventory found in this section will be downloaded. To prevent an inventory from being downloaded you can either comment it out (i.e. place a # in the first column) or move the inventory to the next section.

The final section specifies HEMCO emission inventories that you do not wish to download. The section looks like this:

###############################################################################
#                                                                             #
#  THE FOLLOWING DATA DIRECTORIES WILL NOT BE DOWNLOADED.                     #
#                                                                             #
#  These data directories contain are optional emissions inventories that     #
#  are not used in typical GEOS-Chem simulations.  If you wish to download    #
#  any of these inventories, simply move the corresponding entry for each     #
#  inventory to the previous section.                                         #
#                                                                             #
###############################################################################

CH3I simulation (obsolete)    | $ROOT/CH3I/v2014-07
Chlorophyll A                 | $ROOT/CHLA/v2014-07
... etc ...

--Bob Y. 13:55, 12 February 2015 (EST)

Downloading the data

Once you have set up your configuration file, you can run the hemcoDataDownload.pl script to start downloading the HEMCO data to your local server. To run the script you can type:

hemcoDataDownload.pl

If you do not specify a configuration file name, then the hemcoDataDownload.pl scriptwill read the default configuration file hemcoDataDownload.rc configuration file. If you wish to specify a different configuration file name, simply pass that as an argument to the script, e.g.

hemcoDataDownload.pl myNewConfigFile.rc

Before you start downloading GB's of data, we recommend that you run a short test to make sure that the data directories are being copied to the proper locations on your disk server. For this purpose, we have provided a configuration file named forTesting.rc. Typing

hemcoDataDownload.pl forTesting.pl

will only download a couple of data inventories that do not take up much disk space. This allows you to ensure that the data transfer is sucessful without making you wait a long time.

--Bob Y. 13:57, 12 February 2015 (EST)

New features for the GEOS-Chem v10-01 public release

For the GEOS-Chem v10-01 public release, we modified the default download options in the hemcoDataDownload.pl script. We changed the default wget options from:

wget r -nH -q ... 

to

wget -r -nH -c -N -q ...

This will tell wget to only download new or modified files, instead of trying to download the entire data archive from scratch. This should hopefully subsequent data download processes faster.

--Bob Y. (talk) 18:35, 16 June 2015 (UTC)

Submitting new data for use with HEMCO

If you have a new emissions inventory or other atmospheric data set that you would like to add to the HEMCO data directories, then please contact the GEOS-Chem Support Team. Data files must be in COARDS-compliant or CF-compliant netCDF format. Please see our Preparing data files for use with HEMCO wiki page for more information.

--Bob Y. 11:49, 5 March 2015 (EST)