View source for Downloading GEOS-Chem data directories

__FORCETOC__
'''''[[Downloading GEOS-Chem source code (13.0.0 and later versions)|Previous]] | [[Downloading data with the GEOS-Chem dry-run option|Next]] | [[Getting Started with GEOS-Chem]]'''''
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
#[[Installing required software]]
#[[Configuring your computational environment]]
#[[Downloading GEOS-Chem source code|Downloading source code]]
#<span style="color:blue">'''Downloading data directories'''</span>
#*[[Downloading data with the GEOS-Chem dry-run option|... with the GEOS-Chem dry-run option]]
#*[[Downloading data from Compute Canada|... from Compute Canada]]
#*[[Downloading data from Amazon Web Services cloud storage|... from Amazon Web Services cloud storage]]
#[[Creating GEOS-Chem run directories|Creating run directories]]
#[[GEOS-Chem input files|Configuring runs]]
#[[Compiling GEOS-Chem|Compiling]]
#[[Running GEOS-Chem|Running]]
#[[GEOS-Chem output files|Output files]]
#[[Python tools for use with GEOS-Chem]]
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
#[[GEOS-Chem_overview#Further_reading|Further reading]]


This page describes where you can obtain the GEOS-Chem source code and required data files.

== Overview ==

=== What are the GEOS-Chem shared data directories? ===

In addition to the [[GEOS-Chem_configuration_files|configuration files]] that ship with GEOS-Chem run directories, GEOS-Chem also needs to access data directories containing:

* [[Overview of GMAO met data products|'''Meteorological data''']] (a.k.a. the "met fields") used to drive GEOS–Chem
* [[HEMCO data directories|'''Emissions inventories''']] used by GEOS-Chem
* [[Scale_factors_for_anthropogenic_emissions|'''Scale factors''']] used to scale emissions from a base year to a given year
* [[GEOS-Chem basics#Restart files|Sample '''restart files''']] that you can use to spin up your GEOS-Chem simulations
* '''Oxidant (OH, O3) concentrations''' for both full-chemistry and offline simulations
* Other GEOS–Chem specific data files.

These files are often too large to store in a single user's disk space.  Therefore, they are meant to be stored in shared disk space where all GEOS-Chem users in your group can have access to them.

=== Do I really need to download ALL of this data? ===

Maybe not!  If you are located at an [http://acmg.seas.harvard.edu/geos/geos_people.html institution that has multiple GEOS-Chem users], then your computer system might already have a copy of the GEOS-Chem shared data directories.  If this is the case, you will not have to download any data (unless you need e.g. met field data for 2020 and your system only has the data up to 2019, etc.)  If you are unsure whether or not the shared data directories are available to you, ask your sysadmin or IT staff.

Also, starting with GEOS-Chem 12.7.0, you can use a [[Downloading data with the GEOS-Chem dry-run option|GEOS-Chem dry-run]] to download only the data files you need for a specific GEOS-Chem simulation.  This can drastically reduce the number of data files that you need to download.

=== What if I am running GEOS-Chem on the AWS cloud? ===

A copy of the GEOS-Chem data directories is synced from the Harvard University FTP site (<tt>ftp.as.harvard.edu</tt>) to the Amazon Web Services <tt>s3://gcgrid</tt> bucket.  You can easily download the data files you need from <tt>s3://gcgrid</tt> to the Elastic Block Storage (EBS) volume that is attached to your cloud instance.  This is described in our cloud-computing tutorial [http://cloud.geos-chem.org '''cloud.geos-chem.org''']

To simplify matters even further, we recommend that you use a [[Downloading data with the GEOS-Chem dry-run option|GEOS-Chem dry-run]] to download data from <tt>s3://gcgrid</tt> to your EBS volume.

=== I am located in China and data download speeds are slow.  What can I do? ===

At present we are working on a better solution for our Chinese GEOS-Chem users.  This will probably involve a point person located in China who can oversee and/or centralize data download activities.  Stay tuned for more information.

--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 18:17, 6 January 2020 (UTC)

== Shared data directory archives ==

The GEOS–Chem shared data directories may be downloaded from the following locations:

{| border=1 cellpadding=5 cellspacing=0
|-bgcolor="#CCCCCC"                         
!width="125px"|Archive
!width="300px"|Location     
!width="475px"|Description
!width="250px"|How to download?

|-valign="top"
|[[Downloading_data_from_Compute_Canada|Washington University in St. Louis]] 
|<tt><nowiki>http://geoschemdata.wustl.edu</nowiki></tt>
|This is soon-to-be the main GEOS-Chem data archive. The Compute Canada server is being phased out, and the WUSTL server is its long term replacement.

*<span style="color:red">'''Use this archive to download data to your local computer system.'''</span>
|
#[[Downloading_data_with_the_GEOS-Chem_dry-run_option|GEOS-Chem dry-run]]
#*<span style="color:red">'''Our preferred method'''</span>
#*<span style="color:red">'''Available in 12.7.0 or later'''</span>
#[[Downloading data from Compute Canada|or by manual download]]

|-valign="top"
|[[Downloading_data_from_Compute_Canada|Compute Canada]] 
|<tt><nowiki>http://geoschemdata.computecanada.ca</nowiki></tt>
|This archive is still active but it's being phased out. The Compute Canada archive is being replaced by http://geoschemdata.wustl.edu.

|
#[[Downloading_data_with_the_GEOS-Chem_dry-run_option|GEOS-Chem dry-run]]
#*<span style="color:red">'''Our preferred method'''</span>
#*<span style="color:red">'''Available in 12.7.0 or later'''</span>
#[[Downloading data from Compute Canada|or by manual download]]

|-valign="top"
|[[Downloading data from Amazon Web Services cloud storage|Amazon Web Services S3 storage]]
|<tt><nowiki>s3://gcgrid</nowiki></tt>
|This is an AWS S3 bucket containing a mirror of the Harvard University storage server. It will not contain the complete record of met fields, but additional data may be added by submitting a request to the [[GCST]]. 

See our cloud computing tutorial ('''[http://cloud.geos-chem.org cloud.geos-chem.org]''') for more information.

*<span style="color:red">'''Use this archive for download data to your AWS cloud instance.'''</span>
*<span style="color:red">'''NOTE: Downloading data from this archive to your local computer system will incur an egress fee.  Use with caution!'''</span>
|
#[[Downloading_data_with_the_GEOS-Chem_dry-run_option|GEOS-Chem dry-run]]
#*<span style="color:red">'''Our preferred method'''</span>
#*<span style="color:red">'''Available in 12.7.0 or later'''</span>
#[[Downloading_data_from_Amazon_Web_Services_cloud_storage|or by manual download]]

|}

--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 18:09, 13 December 2019 (UTC)


----
'''''[[Downloading GEOS-Chem source code (13.0.0 and later versions)|Previous]] | [[Downloading data with the GEOS-Chem dry-run option|Next]] | [[Getting Started with GEOS-Chem]]'''''