Difference between revisions of "GEOS-Chem required disk space"

From Geos-chem
Jump to: navigation, search
(Data generated by GEOS-Chem)
Line 115: Line 115:
 
=== Monthly mean output ===
 
=== Monthly mean output ===
  
For the [[GEOS-Chem_chemistry_mechanisms|full-chemistry simulations]], we can look to the GEOS-Chem benchmarks as a rough upper limit of how much disk space is needed for diagnostic output.  The [[Benchmark/GEOS-Chem 12.4.0|GEOS-Chem 12.4.0 benchmark simulation]] generates approximately '''760 MB/month''' of files (monthly-mean output).  This includes netCDF-format diagnostic output and restart files only, but excludes binary punch output (which is being phased out).
+
For the [[GEOS-Chem_chemistry_mechanisms|full-chemistry simulations]], we can look to the GEOS-Chem benchmarks as a rough upper limit of how much disk space is needed for diagnostic output.  The [[Benchmark/GEOS-Chem 13.0.0-beta.0|GEOS-Chem 13.0.0= benchmark simulation]] generates approximately '''837 MB/month''' of monthly-mean diagnostic output (646 MB) and restart file output (191 MB).
  
 
This would be an upper limit because in the benchmark simulations, we archive the "kitchen sink"—all species concentrations, various aerosol diagnostics, convective fluxes, dry dep fluxes and velocities, J-values, various chemical and meteorological quantities, transport fluxes,  wet deposition diagnostics, and emissions diagnostics.  Most GEOS-Chem users would not need to archive this much output.
 
This would be an upper limit because in the benchmark simulations, we archive the "kitchen sink"—all species concentrations, various aerosol diagnostics, convective fluxes, dry dep fluxes and velocities, J-values, various chemical and meteorological quantities, transport fluxes,  wet deposition diagnostics, and emissions diagnostics.  Most GEOS-Chem users would not need to archive this much output.
 +
 +
The GEOS-Chem specialty simulations—simulations for species with first-order loss by prescribed oxidant fields (i.e. Hg, CH4, CO2, CO)—will produce much less output than the benchmark simulations.  This is because these simulations typically only have a few species.
 +
 +
=== Reducing output file sizes ===
  
 
In [[GEOS-Chem 12#12.6.0|GEOS-Chem 12.6.0]] (released 18 Oct 2019), we introduced the capability of [[List_of_diagnostics_archived_to_netCDF_format#Sample_HISTORY.rc_diagnostic_input_file|horizontal and vertical subsetting for diagnostics being archived to netCDF output]].  This will let you save only a sub-region of the globe, or a subset of vertical levels (or both) in case you do not wish to archive diagnostics for the entire globe.  This can help to further reduce the amount of diagnostic output being sent to disk.
 
In [[GEOS-Chem 12#12.6.0|GEOS-Chem 12.6.0]] (released 18 Oct 2019), we introduced the capability of [[List_of_diagnostics_archived_to_netCDF_format#Sample_HISTORY.rc_diagnostic_input_file|horizontal and vertical subsetting for diagnostics being archived to netCDF output]].  This will let you save only a sub-region of the globe, or a subset of vertical levels (or both) in case you do not wish to archive diagnostics for the entire globe.  This can help to further reduce the amount of diagnostic output being sent to disk.
  
The GEOS-Chem specialty simulations—simulations for species with first-order loss by prescribed oxidant fields (i.e. Hg, CH4, CO2, CO)—will produce much less output than the benchmark simulations.  This is because these simulations typically only have a few species.
+
Furthermore, in [[GEOS-Chem 13.0.0]], we have modified the diagnostic code so that diagnostic arrays are only dimensioned with enough elements necessary to save out the required output.  For example, if you only wish to output the <tt>SpeciesConc_O3</tt> diagnostic, GEOS-Chem will dimension the relevant array with <tt>(NX,NY,NZ,1)</tt> elements (1 because we are only archiving 1 species).  This can drastically reduce the amount of memory that your simulation wil require.
  
 
=== Timeseries output ===
 
=== Timeseries output ===

Revision as of 15:33, 23 November 2020

Previous | Next | Getting Started with GEOS-Chem

  1. Minimum system requirements for GEOS-Chem
  2. Installing required software
  3. Configuring your computational environment
  4. Downloading source code
  5. Downloading data directories
  6. Creating run directories
  7. Configuring runs
  8. Compiling
  9. Running
  10. Output files
  11. Visualizing and processing output
  12. Coding and debugging
  13. Further reading


Data for input into GEOS-Chem

The following sections will help you assess how much disk space you need to run GEOS-Chem.

Emissions fields

Please see our HEMCO data directories wiki page for a list of disk space requirements for each of the emissions inventories read into GEOS-Chem by the HEMCO emissions component.

In most cases, you will not have to download the entire set of HEMCO data directories. You can use a GEOS-Chem "dry-run" simulation to only download as many emissions inventories and met field data files as your simulation needs. For more information, please see Downloading data with the GEOS-Chem dry-run option.

--Bob Yantosca (talk) 20:42, 20 November 2020 (UTC)

Meterorology fields

The amount of disk space that you will need depends on two things:

  1. Which type of met data you will use, and
  2. How many years of met data you will download

Typical disk space requirements are:

Met field Resolution File type Size
MERRA-2 4° x 5° COARDS-compliant netCDF (compressed) ~ 30 GB/yr
MERRA-2 2° x 2.5° COARDS-compliant netCDF (compressed) ~ 110 GB/yr
MERRA-2 0.5° x 0.625 Asia ("AS") nested grid COARDS-compliant netCDF (compressed) ~ 115 GB/yr
MERRA-2 0.5° x 0.625 Europe ("EU") nested grid COARDS-compliant netCDF (compressed) ~ 58 GB/yr
MERRA-2 0.5° x 0.625 North America ("NA") nested grid COARDS-compliant netCDF (compressed) ~ 110 GB/yr
GEOS-FP 4° x 5° COARDS-compliant netCDF (compressed) ~ 30 GB/yr
GEOS-FP 2° x 2.5° COARDS-compliant netCDF (compressed) ~ 120 GB/yr
GEOS-FP 0.25° x 0.3125° China ("CH") nested grid COARDS-compliant netCDF (compressed) ~ 175 GB/yr
GEOS-FP 0.25° x 0.3125° Europe ("EU") nested grid COARDS-compliant netCDF (compressed) ~ 58 GB/yr
GEOS-FP 0.25° x 0.3125° North America ("NA") nested grid COARDS-compliant netCDF (compressed) ~ 226 GB/yr

--Bob Yantosca (talk) 20:45, 20 November 2020 (UTC)

Data generated by GEOS-Chem

Monthly mean output

For the full-chemistry simulations, we can look to the GEOS-Chem benchmarks as a rough upper limit of how much disk space is needed for diagnostic output. The GEOS-Chem 13.0.0= benchmark simulation generates approximately 837 MB/month of monthly-mean diagnostic output (646 MB) and restart file output (191 MB).

This would be an upper limit because in the benchmark simulations, we archive the "kitchen sink"—all species concentrations, various aerosol diagnostics, convective fluxes, dry dep fluxes and velocities, J-values, various chemical and meteorological quantities, transport fluxes, wet deposition diagnostics, and emissions diagnostics. Most GEOS-Chem users would not need to archive this much output.

The GEOS-Chem specialty simulations—simulations for species with first-order loss by prescribed oxidant fields (i.e. Hg, CH4, CO2, CO)—will produce much less output than the benchmark simulations. This is because these simulations typically only have a few species.

Reducing output file sizes

In GEOS-Chem 12.6.0 (released 18 Oct 2019), we introduced the capability of horizontal and vertical subsetting for diagnostics being archived to netCDF output. This will let you save only a sub-region of the globe, or a subset of vertical levels (or both) in case you do not wish to archive diagnostics for the entire globe. This can help to further reduce the amount of diagnostic output being sent to disk.

Furthermore, in GEOS-Chem 13.0.0, we have modified the diagnostic code so that diagnostic arrays are only dimensioned with enough elements necessary to save out the required output. For example, if you only wish to output the SpeciesConc_O3 diagnostic, GEOS-Chem will dimension the relevant array with (NX,NY,NZ,1) elements (1 because we are only archiving 1 species). This can drastically reduce the amount of memory that your simulation wil require.

Timeseries output

Archiving hourly or daily timeseries output would require much more disk space than the monthly-mean output. The disk space actually used will depend on how many quantities are archived and what the archival frequency is.

--Bob Yantosca (talk) 20:45, 20 November 2020 (UTC)



Previous | Next | Getting Started with GEOS-Chem