|
|
| (316 intermediate revisions by 7 users not shown) |
| Line 1: |
Line 1: |
| − | This page describes where you can obtain the GEOS-Chem source code and required data files.
| + | __FORCETOC__ |
| | + | '''''[[GEOS-Chem directory structure|Previous]] | [[Creating GEOS-Chem run directories|Next]] | [[Getting Started with GEOS-Chem]]''''' |
| | + | #[[Minimum system requirements for GEOS-Chem|Minimum system requirements (and software installation)]] |
| | + | #[[Configuring your computational environment]] |
| | + | #[[Downloading GEOS-Chem source code|Downloading source code]] |
| | + | #<span style="color:blue">'''Downloading data directories'''</span> |
| | + | #[[Creating GEOS-Chem run directories|Creating run directories]] |
| | + | #[[GEOS-Chem input files|Configuring runs]] |
| | + | #[[Compiling GEOS-Chem|Compiling]] |
| | + | #[[Running GEOS-Chem|Running]] |
| | + | #[[GEOS-Chem output files|Output files]] |
| | + | #[[Python tools for use with GEOS-Chem]] |
| | + | #[[GEOS-Chem_coding_and_debugging|Coding and debugging]] |
| | + | #[[GEOS-Chem_overview#Further_reading|Further reading]] |
| | | | |
| − | == What you need to download before you can run GEOS-Chem ==
| |
| | | | |
| − | When setting up GEOS–Chem on your system, you will need install the following components:
| + | This content has been migrated to the [https://geos-chem.readthedocs.io/en/latest/gcclassic-user-guide/download-data.html '''Download input data''' chapter of <tt>geos-chem.readthedocs.io</tt>]. |
| | | | |
| − | #[http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.2 '''The GEOS–Chem source code directory''']. This is the directory where the Fortran-90 source code files (i.e. *.f, *.f90 files) and Makefiles reside. Your Fortran compiler will create an executable from these source code files.
| |
| − | #[http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.3 '''A GEOS–Chem run directory''']. Here is where you will run the compiled GEOS–Chem executable. Each run directory contains:
| |
| − | ##Various input files that you can modify in order to select different options for your GEOS–Chem simulation
| |
| − | ##Files that define the GEOS–Chem's chemical and photolysis mechanisms
| |
| − | ##"Restart files" that hold the initial conditions for your GEOS–Chem simulation
| |
| − | #[http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.4 '''The GEOS–Chem shared data directories''']. This is the directory tree where the following types of data are stored:
| |
| − | ##Meteorological data (a.k.a. the "met fields) used to drive GEOS–Chem
| |
| − | ##Emissions inventories for GEOS–Chem
| |
| − | ##Scale factors to used to scale emissions from a base year to a given year
| |
| − | ##Oxidant (OH, O3) concentrations for both full-chemistry and offline simulations
| |
| − | ##IPCC future scenarios (for GCAP simulatons)
| |
| − | ##Other GEOS–Chem specific data files.
| |
| | | | |
| − | The GEOS–Chem source code and run directories are small enough to download directly to your own disk space in your Unix account. You can download these with the [[Version control with Git|Git version control software]].
| + | ---- |
| − | | + | '''''[[GEOS-Chem directory structure|Previous]] | [[Creating GEOS-Chem run directories|Next]] | [[Getting Started with GEOS-Chem]]''''' |
| − | On the other hand, the GEOS–Chem shared data directories contain many large files that probably cannot all fit into your own personal disk quota. Therefore, you (or your sysadmin) should download the shared data directories to a common disk space where all GEOS–Chem users in your group can access them. The volume of data contained in the shared data directories precludes using Git; you must instead download these files via FTP, [[#Using wget to download files|wget]], or similar file transfer programs.
| + | |
| − | | + | |
| − | For complete downloading instructions, please see:
| + | |
| − | | + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.2 Chapter 2.2, Downloading the GEOS-Chem source code]
| + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.3 Chapter 2.3, Downloading the GEOS-Chem run directories]
| + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_2.html#2.4 Chapter 2.4, Downloading the GEOS-Chem shared data directories]
| + | |
| − | | + | |
| − | For more information on the files contained in each of these directories, please see:
| + | |
| − | | + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_3.html Chapter 3, Compiling the GEOS-Chem source code]
| + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_4.html Chapter 4, The GEOS-Chem shared data directories]
| + | |
| − | # GEOS-Chem User's Guide: [http://acmg.seas.harvard.edu/geos/doc/man/chapter_5.html Chapter 5, The GEOS-Chem run directories]
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 11:14, 10 February 2011 (EST)
| + | |
| − | | + | |
| − | == TARBALL downloads are no longer supported ==
| + | |
| − | | + | |
| − | In the past, the GEOS–Chem source code and run directories were distributed to the user community as a series of TARBALL (i.e. <tt>*.tar.gz</tt>) files via anonymous FTP. The advantage of this method was that one would only have to download a single file. However, as the number of GEOS–Chem users (and submitted source code modifications) grew, this method became unwieldy. For example, if only a single file needed to be updated, the entire TARBALL file would have to be regenerated. This often became a source of confusion and error.
| + | |
| − | | + | |
| − | Given the large number of user code submissions, robust source code management techniques must be employed in order to ensure the integrity of the GEOS–Chem code. Therefore, the GEOS–Chem Support Team has selected the Git version control software for managing the GEOS–Chem source code and run directories.
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 11:14, 10 February 2011 (EST)
| + | |
| − | | + | |
| − | == Data Directory Access ==
| + | |
| − | | + | |
| − | The [http://acmg.seas.harvard.edu/geos/doc/man/chapter_4.html GEOS–Chem shared data directories] contain the various met fields, emissions, and other data that GEOS–Chem will read in during the course of a simulation. You must download the shared data directories via FTP or a similar utility (e.g. wget, FireFTP, SecureFX, etc.) The large volume of data makes it impossible to track this directory structure with Git.
| + | |
| − | | + | |
| − | If there are several GEOS–Chem users in your research group, then chances are someone (perhaps your sysadmin) has already downloaded the GEOS–Chem shared directory structure to a common disk space on your servers. Nevertheless, you should still check to make sure that the most recent subdirectories are present. See Chapter 4 for complete details about the contents of the GEOS–Chem shared data directories.
| + | |
| − | | + | |
| − | There are currently two data archives from which you may download the GEOS–Chem shared data directories:
| + | |
| − | | + | |
| − | #[[#Primary download site|Harvard archive]] (ftp.as.harvard.edu)
| + | |
| − | #[[#Alternative download site|Dalhousie archive]] (rain.ucis.dal.ca)
| + | |
| − | | + | |
| − | The Dalhousie archive is not as comprehensive as the Harvard archive. However, the Dalhousie archive stores data files for the various nested grids and global 1° x 1.25° grids that are not available on the Harvard archive.
| + | |
| − | | + | |
| − | You may access either of these archives via the following methods:
| + | |
| − | | + | |
| − | # [[#Using wget to download files|wget]] (Recommended)
| + | |
| − | # Anonymous FTP
| + | |
| − | | + | |
| − | === Primary download site ===
| + | |
| − | | + | |
| − | The primary GEOS-Chem data download site is located at:
| + | |
| − | | + | |
| − | ftp ftp.as.harvard.edu
| + | |
| − | cd pub/geos-chem
| + | |
| − | | + | |
| − | We recommend that you use the [[#Using wget to download files|wget utility]] to download these directories instead of anonymous FTP. Wget allows you to download multiple directories at once.
| + | |
| − | | + | |
| − | ==== Downtime ====
| + | |
| − | | + | |
| − | The Harvard FTP archive will be unavailable during the weekly maintenance period every Monday between 7-10 AM ET. Please plan your data downloads accordingly.
| + | |
| − | | + | |
| − | ==== Directory structure ====
| + | |
| − | | + | |
| − | The <tt>pub/geos-chem</tt> directory is further divided into the following subdirectories:
| + | |
| − | | + | |
| − | 1month_plots/
| + | |
| − | 1yr_benchmarks/
| + | |
| − | NRT/
| + | |
| − | NRT-ARCTAS/
| + | |
| − | NRT_archive/
| + | |
| − | beta_releases/
| + | |
| − | dao/
| + | |
| − | data/
| + | |
| − | downloads/
| + | |
| − | mean_OH/
| + | |
| − | patches/
| + | |
| − | public_releases/
| + | |
| − | | + | |
| − | The <tt>data/</tt> subdirectory is the most important. This is the root under which the [http://acmg.seas.harvard.edu/geos/doc/man/chapter_4.html GEOS–Chem shared data directories] reside. This is where you will find the GEOS-Chem met fields and emissions data.
| + | |
| − | | + | |
| − | Here is a quick look at the contents of these subdirectories of <tt>pub/geos-chem/</tt>
| + | |
| − | | + | |
| − | {| border=1 cellpadding=5 cellspacing=0
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !Data Directories under pub/geos-chem/
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>data/</tt>
| + | |
| − | |Root Data Directory
| + | |
| − | |-
| + | |
| − | |<tt>data/aerosol_optics/</tt>
| + | |
| − | |Contains files which specify the [[Aerosol optical properties|aerosol optical properties]] for the [[Photolysis mechanism|FAST-J photolysis mechanism]].
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_MEAN/</tt>
| + | |
| − | |Contains P(O3), L(O3) and mean OH data for offline simulations.
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !0.5° x 0.666° Data Directories
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_0.5x0.666_CH/</tt>
| + | |
| − | |Emissions etc. files for the China/SE Asia 0.5° x 0.666° [[GEOS-Chem nested grid simulations|nested-grid simulation]]
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_0.5x0.666_CH/GEOS_5/YYYY/MM/</tt>
| + | |
| − | |GEOS-5 met data for the China/SE Asia 0.5° x 0.666° nested grid simulation
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_0.5x0.666_NA/</tt>
| + | |
| − | |Emissions etc. files for the North American 0.5° x 0.666° nested grid simulation
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_0.5x0.666_NA/GEOS_5/YYYY/MM/</tt>
| + | |
| − | |GEOS-5 met data for the North American 0.5° x 0.666° nested grid simulation
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !1° x 1° Data Directories
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_1x1/</tt>
| + | |
| − | |1° x 1° emissions etc. data files for use with GEOS-Chem global simulations
| + | |
| − | |-
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !2° x 2.5° Data Directories
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_2x2.5/</tt>
| + | |
| − | |Emissions etc. data for GEOS-chem 2° x 2.5° global simulations
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_2x2.5/GEOS_3/YYYY/MM/</tt>
| + | |
| − | |GEOS-3 met data for 2° x 2.5° global simulations
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_2x2.5/GEOS_4_v4/YYYY/MM/</tt>
| + | |
| − | |GEOS-4 met data for 2° x 2.5° global simulations (late-look)
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_2x2.5/GEOS_5/YYYY/MM</tt>
| + | |
| − | |GEOS-5 met data for 2° x 2.5° global simulations
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !4° x 5° Data Directories
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/</tt>
| + | |
| − | |Emissions etc. data for GEOS-chem 4° x 5° global simulations
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/GEOS_3/YYYY/MM/</tt>
| + | |
| − | |GEOS-3 met data for 4° x 5° global simulations
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/GEOS_4_v4/YYYY/MM/</tt>
| + | |
| − | |GEOS-4 met data for 4° x 5° global simulations (late-look)
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/GEOS_5/YYYY/MM/</tt>
| + | |
| − | |GEOS-5 met data for 4° x 5° global simulations
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/MERRA/YYYY/MM/</tt>
| + | |
| − | |MERRA met data for 4° x 5° global simulations
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !Other Subdirectories under pub/geos-chem/
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>1yr_benchmarks/</tt>
| + | |
| − | |Contains the following types of data from the [http://acmg.seas.harvard.edu/geos/geos_1yr_plots.html 1-year benchmarks] used to evaluate GEOS-Chem.
| + | |
| − | *Restart files
| + | |
| − | *Model output (bpch and netCDF formats)
| + | |
| − | *Log files
| + | |
| − | *Input files
| + | |
| − | *Evaluation plots
| + | |
| − | |-
| + | |
| − | |<tt>dao/</tt>
| + | |
| − | |Internal use only
| + | |
| − | |-
| + | |
| − | |<tt>mean_OH/</tt>
| + | |
| − | |Directory containing 3-D mean OH fields archived from previous GEOS-Chem simulations.
| + | |
| − | |}
| + | |
| − | | + | |
| − | Please view the [http://acmg.seas.harvard.edu/geos/geos_sim.html catalog of met data at the Harvard archive] to determine if the data period you wish to download is available.
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 15:39, 10 February 2011 (EST)
| + | |
| − | | + | |
| − | ==== Obsolete data directories ====
| + | |
| − | | + | |
| − | The following data directories contain iles that are now considered obsolete.
| + | |
| − | | + | |
| − | {| border=1 cellpadding=5 cellspacing=0
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !Obsolete directories under pub/geos-chem/
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | |<tt>1month_plots/</tt>
| + | |
| − | |Formerly used to store plots from 1-month benchmark simulations. We now use Git to manage the benchmark run directories.
| + | |
| − | |- | + | |
| − | |<tt>/beta_releases</tt>
| + | |
| − | |Contains TARBALL files with source code and run directories for GEOS-Chem beta versions prior to v8-03-01. We now use Git to manage the GEOS-Chem source code & run directories.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_1x1_CH/</tt>
| + | |
| − | |Emissions etc. data for the China/SE Asia 1° x 1° simulation. This simulation has been superseded by the GEOS-5 0.5° x 0.666° nested simulation.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_1x1_CH/GEOS_3/YYYY/MM/</tt>
| + | |
| − | |GEOS-3 met data for the China/SE Asia 1° x 1° nested grid simulation This simulation has been superseded by the GEOS-5 0.5° x 0.666° nested simulation.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_1x1_NA/</tt>
| + | |
| − | |Emissions etc. data for the North American 1° x 1° nested grid simulation. This simulation has been superseded by the GEOS-5 0.5° x 0.666° nested simulation.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_1x1_NA/GEOS_3/YYYY/MM/</tt>
| + | |
| − | |GEOS-3 met data for the North American 1° x 1° nested grid simulation. This simulation has been superseded by the GEOS-5 0.5° x 0.666° nested simulation.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_2x2.5/GEOS_4_flk/YYYY/MM/</tt>
| + | |
| − | |GEOS-4 met data for 2° x 2.5° global simulations (1st-look). These data were only used for the ITCT/2k2 campaing. They have been replaced by the data in GEOS_4_v4.
| + | |
| − | |-
| + | |
| − | |<tt>data/GEOS_4x5/GEOS_4_flk/YYYY/MM/</tt>
| + | |
| − | |GEOS-4 met data for 4° x 5° global simulations (1st-look). These data were only used for the ITCT/2k2 campaing. They have been replaced by the data in GEOS_4_v4.
| + | |
| − | |-
| + | |
| − | |<tt>downloads/</tt>
| + | |
| − | |Formerly used to contain the following code packages:
| + | |
| − | *Code to read HDF4 and HDF4-EOS data files
| + | |
| − | *Code to read HDF5 and HDF5-EOS data files
| + | |
| − | *Code to read netCDF data files
| + | |
| − | *Code to process GEOS-5 met data
| + | |
| − | These are now maintained as Git repositories. Contact [mailto:yantosca@seas.harvard.edu Bob Yantosca] for more information.
| + | |
| − | |-
| + | |
| − | |<tt>NRT/</tt>
| + | |
| − | |Contains some data from the GEOS-Chem near-real-time simulations for ICARTT/ITCT 2k2. We maintain this directory for archival purposes. Not all data have been preserved.
| + | |
| − | |-
| + | |
| − | |<tt>NRT-ARCTAS/</tt>
| + | |
| − | |Contains output from the [http://coco.atmos.washington.edu/cgi-bin/ion-p?page=arctas.ion GEOS-Chem Near-Real-Time simulations for ARCTAS]. We maintain this directory for archival purposes.
| + | |
| − | |- | + | |
| − | |<tt>patches/</tt>
| + | |
| − | |Directory that formerly contained bug-fix software patches (if necessary). We now use Git to issue GEOS-Chem software patches.
| + | |
| − | |-
| + | |
| − | |<tt>public_releases/</tt>
| + | |
| − | |Contains TARBALL files with source code and run directories for GEOS-Chem public versions prior to v8-03-01. We now use Git to manage the GEOS-Chem source code & run directories.
| + | |
| − | |}
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 15:39, 10 February 2011 (EST)
| + | |
| − | | + | |
| − | === Alternative download site ===
| + | |
| − | | + | |
| − | The GEOS-Chem data and meteorological fields used by Dalhousie University are also available via anonymous FTP from:
| + | |
| − | | + | |
| − | ftp rain.ucis.dal.ca
| + | |
| − | | + | |
| − | We recommend that you use the [[#Using wget to download files|wget utility]] to download these directories instead of anonymous FTP. Wget allows you to download multiple directories at once.
| + | |
| − | | + | |
| − | ==== Directory structure ====
| + | |
| − | | + | |
| − | The Dalhousie archive overlaps with many of the above directories from the Harvard site, but it is not as extensive. This site, however, additionally hosts the following unique datasets:
| + | |
| − | | + | |
| − | {| border=1 cellpadding=5 cellspacing=0
| + | |
| − | |- bgcolor="#CCCCCC"
| + | |
| − | ! 0.5° x 0.666° Data Directories
| + | |
| − | ! Description
| + | |
| − | |-
| + | |
| − | |<tt>/GEOS_0.5x0.666_EU/</tt>
| + | |
| − | |1/2 x 2/3 European nested grid emission etc files
| + | |
| − | |-
| + | |
| − | |<tt>/GEOS_0.5x0.666_EU.d/</tt>
| + | |
| − | |1/2 x 2/3 European nested grid met fields (GEOS-5)
| + | |
| − | |-
| + | |
| − | |<tt>/GEOS_0.5x0.666_NA/</tt>
| + | |
| − | |1/2 x 2/3 North American nested grid emission etc files
| + | |
| − | |-
| + | |
| − | |<tt>/GEOS_0.5x0.666_NA.d/</tt>
| + | |
| − | |1/2 x 2/3 North American nested grid met fields (GEOS-5)
| + | |
| − | |-bgcolor="#CCCCCC"
| + | |
| − | !1° x 1.25° Data Directories
| + | |
| − | !Description
| + | |
| − | |-
| + | |
| − | | <tt>/GEOS_1x1.25/</tt>
| + | |
| − | | 1 x 1.25 Global GEOS4 emission etc files
| + | |
| − | |-
| + | |
| − | | <tt>/GEOS_1x1.25.d/</tt>
| + | |
| − | | 1 x 1.25 Global GEOS4 met fields
| + | |
| − | |}
| + | |
| − | | + | |
| − | A catalog of available data may be found [http://fizz.phys.dal.ca/~atmos/g47.swf?jump=6.2 HERE].
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 11:49, 10 February 2011 (EST)
| + | |
| − | | + | |
| − | === For GCAP users ===
| + | |
| − | | + | |
| − | For those users who wish to run GEOS-Chem with the [http://acmg.seas.harvard.edu/gcap/ GISS/GCAP] met fields, please contact [mailto:mickley@fas.harvard.edu Loretta Mickley].
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 09:52, 8 March 2010 (EST)
| + | |
| − | | + | |
| − | === Question about directory structure ===
| + | |
| − | | + | |
| − | '''''[mailto:shaked@umich.edu Shanna Shaked] wrote:''''' | + | |
| − | | + | |
| − | :We are working again on trying to run GEOS-Chem. However, we are encountering some errors that may be due to the directory structure. We find a discrepancy between the directory structure described in the GEOS-Chem manual and that available on the ftp site.
| + | |
| − | | + | |
| − | :The GEOS-Chem manual describes a directory structure of:
| + | |
| − |
| + | |
| − | data/GEOS_4x5/GEOS_5/YYYY/MM
| + | |
| − | | + | |
| − | :However, on the ftp site, we find a directory structure with an extra '.d':
| + | |
| − | | + | |
| − | data/GEOS_4x5.d/GEOS_5/YYYY/MM
| + | |
| − | | + | |
| − | :(the <tt>GEOS_5</tt> folder is in <tt>GEOS_4x5.d</tt> rather than <tt>GEOS_4x5</tt>). There does exist a <tt>GEOS_4x5</tt> that contains many of the emissions data, but does not contain <tt>GEOS_5</tt>.
| + | |
| − | | + | |
| − | :If we leave the structure as is, and enter <tt>../data/GEOS_4x5/</tt> as our root data directory in <tt>input.geos</tt>, we get a file not found error when it looks for <tt>GEOS_5</tt> within this directory (obviously).
| + | |
| − | | + | |
| − | :If we instead enter <tt>../data/GEOS_4x5.d</tt> as our root data directory, we get a file not found error when the program looks for emissions within this directory (lightning NOx emissions, in this case).
| + | |
| − | | + | |
| − | :QUESTION: To solve this problem, we have moved the <tt>GEOS_5</tt> folder into the <tt>GEOS_4x5</tt> directory. [Is this] okay?
| + | |
| − | | + | |
| − | '''''[mailto:yantosca@seas.harvard.edu Bob Yantosca] replied:'''''
| + | |
| − | | + | |
| − | :The only difference on our system between e.g. <tt>GEOS_4x5</tt> and <tt>GEOS_4x5.d</tt> is that our sysadmin (Jack Yatteau) set up the ".d" directories separately so that they only contain met data (which is much larger than the emissions etc. data). That way he could separate the disks that just had met data from the disk that have the emissions data to facilitate our configuration here. There are symbolic links from <tt>GEOS_4x5</tt> to <tt>GEOS_4x5.d</tt> etc. (i.e. the directory <tt>GEOS_4x5/GEOS_5</tt> is actually a symbolic link to the corresponding directory in <tt>GEOS_4x5.d/GEOS_5/</tt> and etc. for the other met field resolutions & directories).
| + | |
| − | | + | |
| − | :You don't necessarily have to do this on your end, but this is what we did here. You can just make the <tt>GEOS_4x5/GEOS_5</tt> etc. real subdirectories and not symbolic links and store the data there. The solution you picked above is OK.
| + | |
| − | | + | |
| − | :Also to facilitate FTP file transfer, you could do the following:
| + | |
| − | :* Write a script or an FTP macro
| + | |
| − | :* Use a 3rd-party GUI program like FireFTP in Mozilla Firefox.
| + | |
| − | :* Or even better yet, use the Unix <tt>wget</tt> utility (see below)
| + | |
| − | | + | |
| − | :Each user is responsible for their own file transfers.
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 11:04, 5 February 2009 (EST)
| + | |
| − | | + | |
| − | == Using wget to download files ==
| + | |
| − | | + | |
| − | Probably the simplest way to download the GEOS-Chem emissions and met field data is to use the Unix <tt>wget</tt> utility. This allows you to download multiple files and directories at a time.
| + | |
| − | | + | |
| − | The <tt>wget</tt> utility is free and open-source (published by GNU), and comes standard with pretty much all builds of *nix (Linux, Ubuntu, Fedora, Centos, etc.). You can check out the [http://www.editcorp.com/Personal/Lars_Appel/wget/wget_1.html user manual] for more information.
| + | |
| − | | + | |
| − | === Syntax ===
| + | |
| − | | + | |
| − | Most of the time, the syntax you will use to download multiple directories is as follows:
| + | |
| − | | + | |
| − | Downloading data from Harvard:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://ftp.as.harvard.edu/pub/geos-chem/data/DIRECTORY_NAME/"</nowiki>
| + | |
| − | | + | |
| − | Downloading data from Dalhousie:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://rain.ucis.dal.ca/DIRECTORY_NAME/"</nowiki>
| + | |
| − | | + | |
| − | The options to <tt>wget</tt> are as follows:
| + | |
| − | | + | |
| − | -r Specifies recursive directory transfer (i.e. will download all subdirectories)
| + | |
| − | -nH Will store all directories and subdirectories in DIRECTORY_NAME, not ftp.as.harvard.edu/DIRECTORY_NAME
| + | |
| − | | + | |
| − | If you wish to trim the name of the downloaded directory (i.e., so it downloads as DIRECTORY_NAME, not pub/geos-chem/data/DIRECTORY_NAME), then use the <tt>--cut-dirs</tt> option:
| + | |
| − |
| + | |
| − | <nowiki>wget -r -nH --cut-dirs=X "ftp://ftp.as.harvard.edu/pub/geos-chem/data/DIRECTORY_NAME/"</nowiki>
| + | |
| − | | + | |
| − | where <tt>X</tt> is the number of directories to trim.
| + | |
| − | | + | |
| − | '''''NOTE: The URL must be enclosed in quotes for file transfer to occur. If you omit the quotes then <tt>wget</tt> will just return a directory listing in a file named <tt>index.html</tt> without any files being downloaded.'''''
| + | |
| − | | + | |
| − | | + | |
| − | | + | |
| − | | + | |
| − | === Examples ===
| + | |
| − | | + | |
| − | 1. Download all emissions files in the <tt>GEOS_2x2.5/</tt> data directory structure:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://ftp.as.harvard.edu/pub/geos-chem/data/GEOS_2x2.5/" &</nowiki>
| + | |
| − | | + | |
| − | The <tt>&</tt> character will make sure the file transfer happens in the Unix background.
| + | |
| − | | + | |
| − | | + | |
| − | 2. Download all available GEOS-Chem 2° x 2.5° met field data files in the <tt>GEOS_2x2.5.d</tt> directory structure:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://ftp.as.harvard.edu/pub/geos-chem/data/GEOS_2x2.5.d/" &</nowiki>
| + | |
| − | | + | |
| − | NOTE: Due to the huge volume of data involved, this is not recommended, as the file downloads may swamp your system. It's better to do this:
| + | |
| − | | + | |
| − | | + | |
| − | 3. Download all GEOS-5 met data at 2° x 2.5° resolution:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://ftp.as.harvard.edu/pub/geos-chem/data/GEOS_2x2.5.d/GEOS_5/" &</nowiki>
| + | |
| − | | + | |
| − | And it may even be better to download one year at a time:
| + | |
| − | | + | |
| − | | + | |
| − | 4. Download all GEOS-5 met data at 2° x 2.5° resolution for 2008:
| + | |
| − | | + | |
| − | <nowiki>wget -r -nH "ftp://ftp.as.harvard.edu/pub/geos-chem/data/GEOS_2x2.5.d/GEOS_5/2008/" &</nowiki>
| + | |
| − | | + | |
| − | | + | |
| − | 5. Download the <tt>lightning_NOx_201101</tt> folder to an existing local data directory <tt>/data/gc/GEOS_4x5</tt> using the <tt>--cut</tt> option:
| + | |
| − | | + | |
| − | <nowiki>cd /data/gc/GEOS_4x5</nowiki>
| + | |
| − | <nowiki>wget -r -nH --cut-dirs=4 "ftp://ftp.as.harvard.edu/pub/geos-chem/data/GEOS_4x5/lightning_NOx_201101"</nowiki>
| + | |
| − | etc.
| + | |
| − | | + | |
| − | --[[User:Bmy|Bob Y.]] 14:27, 8 September 2010 (EDT)
| + | |
| − | --[[User:Daven|Daven]] 17:35, 16 February 2011 (EST)
| + | |
