Difference between revisions of "FlexGrid"

From Geos-chem
Jump to: navigation, search
(FlexGrid for different types of users)
(Implementation)
 
(70 intermediate revisions by 3 users not shown)
Line 1: Line 1:
<span style="color:red">'''''NOTE: This page is being actively edited right now and the information might be incomplete.'''''</span>
+
On this page we provide information about the FlexGrid, which was introduced in [[GEOS-Chem 12#12.4.0|GEOS-Chem 12.4.0]].
 
+
FlexGrid documentation by Jiawei Zhuang (jiaweizhuang@g.harvard.edu)
+
  
 
== Overview ==
 
== Overview ==
  
FlexGrid is a new functionality in [[GEOS-Chem v11-02]] to unify all metfield I/O routines by [[HEMCO]].  
+
FlexGrid is a new functionality that will allow users to define the grid for their GEOS-Chem simulation at run time. The major features include:
 +
#[[FlexGrid#Read_in_meteorology_fields_via_HEMCO|Reading in meteorology fields via HEMCO]]
 +
#[[FlexGrid#Define_meteorology_field_and_grid_parameters_in_input.geos|Defining meteorology field, horizontal grid, and vertical grid at run time]]
 +
#[[FlexGrid#Save_out_boundary_conditions_in_HISTORY.rc|Saving out boundary conditions via the HISTORY component]]
 +
#[[FlexGrid#Read_in_boundary_conditions_via_HEMCO|Reading in boundary conditions via HEMCO]]
  
HEMCO's I/O interface and regridding&cropping capability allow users to
+
== Developers ==
* Set up a nested simulation over any custom domains without changing model's source code.
+
* Explicitly control each meteorological variable in run-time configuration files to perform metfield experiments (an interface resembling [[GCHP]]).
+
  
=== The idea of FlexGrid  ===
+
*[https://scholar.harvard.edu/zhuangjw Jiawei Zhuang] (Harvard)
 +
*[http://www.phy.pku.edu.cn/~acm/ Jintai Lin] (PKU)
 +
*[http://people.seas.harvard.edu/~mpayer/ Melissa Sulprizio] (Harvard)
  
In theory, setting up a new nested domain simply requires
+
== Implementation ==
* New metfields over that domain
+
* New regional emission data sets if necessary
+
* Tunning several domain-dependent or resolution-dependent parameters.
+
  
There's no reason that one has to touch the model source code. However, in previous versions, all nested domains were hard-coded so users need to do a lot of tedious coding in order to set up a new domain. The major trouble was metfield I/O, as we assumed that the model grid should exactly match the metfield grid. The solution is HEMCO, which has already been used for regridding emission data sets at multiple grids to the same model grid. Here we just do the reverse -- regrid the same metfield data set to different user-specified domains.
+
The table below shows the implementation plan for FlexGrid in GEOS-Chem:
 
+
=== FlexGrid implementation plan ===
+
 
+
The table below shows the implementation plan for FlexGrid:
+
  
 
{| border=1 cellspacing=0 cellpadding=5  
 
{| border=1 cellspacing=0 cellpadding=5  
 
|-bgcolor="#CCCCCC"
 
|-bgcolor="#CCCCCC"
!width="225px"|Stage
+
!Update
!width="150px"|Detail
+
!Description
!width="150px"|Status
+
!Status
  
 
|-valign="top"
 
|-valign="top"
|Stage 1: Read metfields by HEMCO
+
|Stage 1: Read met fields by HEMCO
|Enable the compile option "MET=flexgrid" to read any NetCDF-format metfields through HEMCO. This unifies "MET=geosfp" and "MET=merra2"
+
 
|
 
|
*Will be included in v11-02
+
*Add entries to read netCDF meteorology files in <tt>HEMCO_Config.rc</tt>
 +
*Unify code for GEOS-FP and MERRA-2 meteorology by creating a new <tt>flexgrid_read_mod.F90</tt>
 +
|
 +
*<span style="color:green">'''''This update was included in [[GEOS-Chem 12#12.1.0|GEOS-Chem 12.1.0]], which was released on 26 Nov 2018.'''''</span>
  
 
|-valign="top"
 
|-valign="top"
|Stage 2: Implement custom nested simulations
+
|Stage 2: Add capability to define custom grids
|Enable the compile option "NEST=cu" (custom) to allow custom nested simulations.
+
 
|
 
|
*Will be included in v11-02
+
*Define meteorology field, grid resolution, and grid domain at run time in <tt>input.geos</tt>
 +
*Add new <tt>State_Grid</tt> derived type object to pass grid parameters from top level routines down
 +
*Save out boundary conditions in <tt>HISTORY.rc</tt> and read in via HEMCO
 +
|
 +
*<span style="color:green">'''''This update was included in [[GEOS-Chem 12#12.4.0|GEOS-Chem 12.4.0]], which was released on 05 Aug 2019.'''''</span>
  
|-valign="top"
 
|Stage 3: Grid-independent GEOS-Chem classic
 
|Unify compile options "GRID=xxx" by moving all grid size parameters and resolution-dependent parameters to run-time configuration files.
 
|No specific plan right now. Need to be traded off against the nested-GCHP development.
 
 
|}
 
|}
  
=== FlexGrid for different types of users ===
+
=== Read in meteorology fields via HEMCO ===
 +
 
 +
FlexGrid takes advantage of HEMCO's I/O interface and regridding & cropping capability. By reading the meteorology fields through HEMCO, the fields are automatically processed for the correct grid resolution and domain at run time. These fields are then passed to <tt>State_Met</tt> in the various subroutines in <tt>flexgrid_read_mod.F90</tt>. Reading meteorology fields in HEMCO also allows users to explicitly control each meteorological variable in run-time configuration files to perform meteorology sensitivity tests (an interface resembling [[GCHP]]).
 +
 
 +
The following lines in <span style="color:green">green</span> were added to <tt>HEMCO_Config.rc</tt>:
 +
 
 +
###############################################################################
 +
### BEGIN SECTION SETTINGS
 +
###############################################################################
 +
 +
ROOT:                        /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/HEMCO
 +
<span style="color:green">METDIR:                      /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/GEOS_4x5/GEOS_FP</span>
 +
...
 +
 +
###############################################################################
 +
### NON-EMISSIONS DATA (subsection of BASE EMISSIONS SECTION)
 +
###
 +
### Non-emissions data. The following fields are read through HEMCO but do
 +
### not contain emissions data. The extension number is set to wildcard
 +
### character denoting that these fields will not be considered for emission
 +
### calculation. A given entry is only read if the assigned species name is
 +
### an HEMCO species.
 +
###############################################################################
 +
 +
<span style="color:green">#==============================================================================
 +
# --- Meteorology fields for FlexGrid ---
 +
#==============================================================================
 +
 +
# --- CN fields ---
 +
* FRLAKE    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAKE  */1/1/0              C xy  1  * -  1 1
 +
* FRLAND    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAND  */1/1/0              C xy  1  * -  1 1
 +
* FRLANDIC  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLANDIC */1/1/0              C xy  1  * -  1 1
 +
* FROCEAN  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FROCEAN  */1/1/0              C xy  1  * -  1 1
 +
* PHIS      $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        PHIS    */1/1/0              C xy  1  * -  1 1
 +
 +
# --- A1 fields ---
 +
* ALBEDO    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC    ALBEDO  1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
 +
* CLDTOT    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC    CLDTOT  1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
 +
* EFLUX    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC    EFLUX    1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
 +
* EVAP      $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC    EVAP    1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1</span>
 +
...
 +
 
 +
=== Define meteorology field and grid parameters in input.geos ===
 +
 
 +
Prior to FlexGrid, users would define the meteorology field and grid resolution at compile time by passing the <tt>MET=</tt> and <tt>GRID=</tt> options at compile time. Changing meteorology field or grid resolution would require recompiling the source code. In addition, many of the grid parameters were hardcoded in <tt>CMN_SIZE_mod.F</tt> or stored in module-level arrays (e.g. <tt>XMID, YMID, AREA_M2</tt> in <tt>gc_grid_mod.F90</tt>). To implement a new meteorology field or nested grid, users would have to process meteorology files offline and update several areas of the GEOS-Chem source code to recognize the new grid.
 +
 
 +
FlexGrid introduces the capability to define the meteorology field in the Simulation Menu of <tt>input.geos</tt> and the grid resolution and domain in the Grid Menu of <tt>input.geos</tt>. The new derived type object [[Derived_type_objects_used_by_GEOS-Chem#The_Grid_State_object|<tt>State_Grid</tt>]] holds the user-defined grid parameters as well as those computed by GEOS-Chem at run time. This new capability greatly simplifies the process of running GEOS-Chem with other meteorology fields and on other grids. It also allows for the use of the same <tt>geos</tt> executable for different meteorology and grid settings.
 +
 
 +
The following lines in <span style="color:green">green</span> were added to <tt>input.geos</tt>:
 +
 
 +
%%% SIMULATION MENU %%% :
 +
Start YYYYMMDD, hhmmss  : 20160701 000000
 +
End  YYYYMMDD, hhmmss  : 20160703 000000
 +
Run directory          : ./
 +
Root data directory    : /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData
 +
<span style="color:green">Met field              : GEOS-FP
 +
------------------------+------------------------------------------------------
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 4.0x5.0
 +
Longitude min/max      : -180.0 180.0
 +
Latitude  min/max      : -90.0  90.0
 +
  Half-sized polar boxes?: T
 +
Number of levels        : 47
 +
Nested grid simulation? : F
 +
  Buffer zone (N S E W ) :  3  3  3  3</span>
 +
 
 +
The meteorology fields currently supported are <tt>[[GEOS-FP]]</tt> and <tt>[[MERRA-2]]</tt>. As of implementation, an error check in <tt>GeosCore/input_mod.F</tt> will stop the simulation if the meteorology type is not recognized, but other met fields may be added in the future to improve flexibility.
 +
 
 +
Users can now define the grid resolution and domain for their simulation at run time in the Grid Menu.
 +
 
 +
<blockquote>
 +
'''Grid Resolution'''<br>
 +
The grid resolutions currently supported are <tt>4.0x5.0</tt>, <tt>2.0x2.5</tt>, <tt>0.5x0.625</tt>, and <tt>0.25x0.3125</tt> (real numbers, not integers, must be used). As of implementation, an error check in <tt>GeosCore/input_mod.F</tt> will stop the simulation if the meteorology type is not recognized, but this error check may be removed or other grid resolutions may be added in the future to improve flexibility. Theoretically, any grid resolution should work, but this has not been validated.
 +
 
 +
'''Grid Domain'''<br>
 +
Users can choose to use either global grids or nested/regional grids. To use a global grid, simply specify longitude and latitude bounds of <tt>-180.0, 180.0</tt> and <tt>-90.0, 90.0</tt>. All global grids currently supported use half-sized polar boxes, but a switch is included in the Grid Menu for flexibility with other grids.
 +
 
 +
'''Vertical Grid'''<br>
 +
Users can also specify the vertical grid in number of levels. The vertical resolutions currently supported are <tt>47</tt> and <tt>72</tt> levels, formerly known as the reduced vertical grid and native-resolution vertical grid. As of implementation, an error check in <tt>GeosCore/input_mod.F</tt> will stop the simulation if the vertical resolution is not recognized, but this error check may be removed or other grid resolutions may be added in the future to improve flexibility.
 +
 
 +
'''Nested Grid'''<br>
 +
If using a grid other than a global grid, users must set the <tt>Nested grid simulation</tt> option to True and specify the number of grid cells along each border that define the buffer region (default is <tt>3 3 3 3</tt>. The buffer region is the region where boundary conditions from a coarse-resolution simulation are applied.
 +
</blockquote>
 +
 
 +
From the settings specified in the Grid Menu, the number of grid boxes in the X, Y, and Z directions will be computed and stored in <tt>State_Grid%NX</tt>, <tt>State_Grid%NY</tt>, and <tt>State_Grid%NZ</tt>, which will replace the old <tt>IIPAR</tt>, <tt>JJPAR</tt>, and <tt>LLPAR</tt> parameters from <tt>CMN_SIZE_mod.F</tt>. Several additional fields will be computed at run time in <tt>GeosUtil/gc_grid_mod.F90</tt>. These fields include arrays of longitude centers (<tt>State_Grid%XMid</tt>, longitude edges (<tt>XEdge</tt>), latitude centers (<tt>State_Grid%YMid</tt>), latitude edges (<tt>State_Grid%YEdge</tt>), grid box areas (<tt>State_Grid%Area_M2</tt>), etc.
 +
 
 +
With this update, the following C-preprocessor switches have been removed from the code and are replaced with IF statements to check the relevant field in either <tt>State_Met</tt> or <tt>State_Grid</tt>.
 +
*<tt>GEOS_FP, MERRA2</tt>
 +
*<tt>GRID_4x5, GRID_2x25, GRID_05x0625, GRID025x03125</tt>
 +
*<tt>NESTED_AS, NESTED_CH, NESTED_EU, NESTED_NA</tt>
 +
*<tt>GRIDREDUCED</tt>
 +
 
 +
=== Save out boundary conditions in HISTORY.rc ===
 +
 
 +
A new <tt>BoundaryConditions</tt> collection allows users to save out instantaneous species concentration fields (v/v) at specified intervals (default is 3-hourly boundary conditions saved to daily files). For the initial implementation, global boundary condition fields will be saved out, and users can utilize these fields as boundary conditions for any custom nested-grid simulation. Eventually, the goal is to add the capability to define a specific region for saving out boundary conditions to save disk space.
 +
 
 +
The following lines in <span style="color:green">green</span> were added to <tt>HISTORY.rc</tt>:
 +
 
 +
COLLECTIONS: 'Restart',
 +
              <span style="color:green">'BoundaryConditions',</span>
 +
              ...
 +
 +
<span style="color:green">#==============================================================================
 +
# %%%%% THE BoundaryConditions COLLECTION %%%%%
 +
#
 +
# GEOS-Chem boundary conditions for use in nested grid simulations
 +
#
 +
# Available for all simulations
 +
#==============================================================================
 +
  BoundaryConditions.template:  '%y4%m2%d2_%h2%n2z.nc4',
 +
  BoundaryConditions.format:    'CFIO',
 +
  BoundaryConditions.frequency:  00000000 030000
 +
  BoundaryConditions.duration:  00000001 000000
 +
  BoundaryConditions.mode:      'instantaneous'
 +
  BoundaryConditions.LON_RANGE:  -130.0 -60.0,
 +
  BoundaryConditions.LAT_RANGE:  10.0 60.0,
 +
  BoundaryConditions.fields:    'SpeciesBC_?ADV?            ', 'GIGCchem',
 +
::</span>
 +
 
 +
'''NOTE: The <tt>.LON_RANGE</tt> and <tt>.LAT_RANGE</tt> options were added in [[GEOS-Chem 12#12.6.0|GEOS-Chem 12.6.0]]. For subsetting with versions 12.4.0 and 12.5.0 used <tt>BoundaryConditions.subset : -130.0 -60.0 10.0 60.0</tt>.'''
 +
 
 +
=== Read in boundary conditions via HEMCO ===
 +
 
 +
The I/O routines in HEMCO will automatically regrid and crop the boundary conditions to the grid defined in <tt>input.geos</tt>. The boundary conditions will be applied to the buffer zone, also defined in <tt>input.geos</tt>. These fields are obtained from HEMCO and applied to the species concentrations (<tt>State_Chm%Species</tt>) in <tt>GeosCore/hcoi_gc_main_mod.F90</tt>.
 +
 
 +
The following lines in <span style="color:green">green</span> were added to <tt>HEMCO_Config.rc</tt>:
 +
 
 +
# ExtNr ExtName                on/off  Species
 +
0      Base                  : on    *
 +
    --> GC_RESTART            :      true   
 +
    <span style="color:green">--> GC_BCs                :      true</span>
 +
    ...
 +
 +
<span style="color:green">#==============================================================================
 +
# --- GEOS-Chem boundary condition file ---
 +
#==============================================================================
 +
(((GC_BCs
 +
* BC_          ./GEOSChem.BoundaryConditions.$YYYY$MM$DD_$HH$MNz.nc4 SpeciesBC_?ADV?  1980-2019/1-12/1-31/0-23 RY xyz 1 * - 1 1
 +
)))GC_BCs</span>
 +
 
 +
'''IMPORTANT:''' As of implementation, the boundary condition files must have 8 time slices (e.g. hours 0, 3, 6, 9, 12, 15, 18, 21). When generating boundary conditions make sure you start your coarse-resolution simulation at least one day before the start of your nested grid simulation to ensure the boundary condition file for the first day includes hour 0. When hour 0 is missing, HEMCO sees 7 time slices and identifies the dataset as weekly data -- this warrants further investigation.'''
 +
 
 +
== FlexGrid for different types of users ==
  
 
According to the availability of meteorological data, users can get different benefits from FlexGrid:
 
According to the availability of meteorological data, users can get different benefits from FlexGrid:
Line 54: Line 192:
 
{| border=1 cellspacing=0 cellpadding=5  
 
{| border=1 cellspacing=0 cellpadding=5  
 
|-bgcolor="#CCCCCC"
 
|-bgcolor="#CCCCCC"
!width="200px"|User type
+
!User type
!width="400px"|benefits
+
!Benefits
  
 
|-valign="top"
 
|-valign="top"
|Only have standard metfields
+
|Only have standard met fields
 
(e.g. global 4x5, 0.25x0.3125 nested NA)
 
(e.g. global 4x5, 0.25x0.3125 nested NA)
 
|
 
|
* Be able to explicitly control each met variable in HEMCO_Config.rc (like Extdata.rc for GCHP). You can freeze certain variables, apply scaling factors, or read user-generated met data.
+
* Be able to explicitly control each met variable in <tt>HEMCO_Config.rc</tt> (like <tt>Extdata.rc</tt> for [[GCHP]]). You can freeze certain variables, apply scaling factors, or read user-generated met data.
* Set up nested simulations over any sub-domains of the metfield you have. For example, you can set up an east-US simulation in no time if you have the nested North American metfield data.
+
* Be able to set up nested simulations over any sub-domains of the met field you have. For example, you can set up an east-US simulation in no time if you have the nested North American met field data.
* If a new nested domain is needed, you can skip the source code modification and just prepare the metfield input.
+
* If a new nested domain is needed, you can skip the source code modification and just prepare the met field input.
  
 
|-valign="top"
 
|-valign="top"
|Have global native metfields
+
|Have global native met fields
(For example, GCHP users might store them for global high-resolution runs. We can re-use them for nested simulations.)
+
(For example, GCHP users might store them for global high-resolution runs. We can reuse them for nested simulations.)
 
|
 
|
 
* Be able to run nested simulations over any domains in the world.
 
* Be able to run nested simulations over any domains in the world.
 
|}
 
|}
  
== FlexGrid Stage 1 User's Guide ==
+
== FlexGrid examples ==
 +
 
 +
<tt>HEMCO_Config.rc</tt> is the place to explicitly control all meteorological input variables (similar to <tt>Extdata.rc</tt> in [[GCHP]]). Each variable is listed in one line, just like other emission data entries in that file:
 +
 
 +
# --- CN fields ---
 +
* FRLAKE    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAKE  */1/1/0              C xy  1  * -  1 1
 +
* FRLAND    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAND  */1/1/0              C xy  1  * -  1  1
 +
* FRLANDIC  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLANDIC */1/1/0              C xy  1  * -  1 1
 +
* FROCEAN  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FROCEAN  */1/1/0              C xy  1  * -  1 1
 +
* PHIS      $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        PHIS    */1/1/0              C xy  1  * -  1 1
 +
...
 +
# --- A3dyn fields ---
 +
* DTRAIN    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  DTRAIN  1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
* OMEGA    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  OMEGA    1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
* RH        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  RH      1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
* U        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  U        1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
* V        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  V        1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
...
 +
 
 +
Most of the tricks that you apply to the emission data can also be applied here, such as
 +
*[[HEMCO_standalone#Provide_meteorological_data_for_extensions|Adding scaling factors]]
 +
*[[HEMCO_standalone#Provide_meteorological_data_for_extensions|Applying masks]]
 +
*[[#Change the met field path|Using different met field files]]
 +
*[[#Freeze certain variables|Holding met fields constant]]
 +
 
 +
==== Change the met field path ====
 +
 
 +
Say you've modified the default temperature data to test the sensitivity to the temperature, you can just tell HEMCO to read your new file instead of the default file, while keep all other variables untouched:
 +
 
 +
* TMPU1    <span style="color:red">path_to_new_file</span>    T        1980-2018/1-12/1-31/0-23          CY xyz  1  * -  1 1
 +
* TMPU2    <span style="color:red">path_to_new_file</span>    T        1980-2018/1-12/1-31/0-23/+3hour  CY xyz  1  * -  1 1
 +
 
 +
==== Freeze certain variables ====
 +
 
 +
You can force the model to keep reusing the wind in 2011 by restricting the time range (e.g. to see how does the trend of certain meteorological variables affects the simulation):
 +
 
 +
* U        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  U        <span style="color:red">2011</span>/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
* V        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  V        <span style="color:red">2011</span>/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
 +
 
 +
== Supported horizontal grid definitions ==
 +
 
 +
Examples of grid definitions in <tt>input.geos</tt> are provided below for users looking to recreate the [[GEOS-Chem_horizontal_grids|global and nested grids]] used in GEOS-Chem prior to the FlexGrid implementation. All examples below may be used with [[GEOS-FP]] met fields. [[MERRA-2]] met fields may be used with all examples below except for the 0.25&deg; x 0.3125&deg; grids.
 +
 
 +
=== 4&deg; x 5&deg; global grid ===
 +
 
 +
[[GEOS-Chem_horizontal_grids#GMAO_4_x_5_grid|See this link for grid details.]]
 +
 
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 4.0x5.0
 +
Longitude min/max      : -180.0 180.0
 +
Latitude  min/max      : -90.0  90.0
 +
  Half-sized polar boxes?: T
 +
Number of levels        : 47
 +
Nested grid simulation? : F
 +
  Buffer zone (N S E W ) :  3  3  3  3
 +
 
 +
=== 2&deg; x 2.5&deg; global grid ===
 +
 
 +
[[GEOS-Chem_horizontal_grids#GMAO_2_x_2.5_grid|See this link for grid details.]]
 +
 
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 2.0x2.5
 +
Longitude min/max      : -180.0 180.0
 +
Latitude  min/max      : -90.0  90.0
 +
  Half-sized polar boxes?: T
 +
Number of levels        : 47
 +
Nested grid simulation? : F
 +
  Buffer zone (N S E W ) :  3  3  3  3
 +
 
 +
=== 0.5&deg; x 0.625&deg; nested Asia ===
 +
 
 +
[[GEOS-Chem_horizontal_grids#0.5_x_0.625_AS_nested_grid|See this link for grid details.]]
 +
 
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.5x0.625
 +
Longitude min/max      :  60.0 150.0
 +
Latitude  min/max      : -11.0  55.0
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3
 +
 
 +
=== 0.5&deg; x 0.625&deg; nested Europe ===
 +
 
 +
[[GEOS-Chem_horizontal_grids#0.5_x_0.625_EU_nested_grid|See this link for grid details.]]
 +
 
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.5x0.625
 +
Longitude min/max      : -30.0 50.0
 +
Latitude  min/max      :  30.0 70.0
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3
 +
 
 +
=== 0.5&deg; x 0.625&deg; nested North America ===
 +
 
 +
[[GEOS-Chem_horizontal_grids#0.5_x_0.625_NA_nested_grid|See this link for grid details.]]
 +
 
 +
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.5x0.625
 +
Longitude min/max      : -140.0 -40.0
 +
Latitude  min/max      :  10.0  70.0
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3
  
=== Compile ===
+
=== 0.25&deg; x 0.3125&deg; nested China ===
  
=== Set up run directory ===
+
[[GEOS-Chem_horizontal_grids#0.25_x_0.3125_CH_nested_grid|See this link for grid details.]]
  
=== Use FlexGrid functionalities ===
+
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.25x0.3125
 +
Longitude min/max      : 70.0 140.0
 +
Latitude  min/max      : 15.0  55.0
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3
  
 +
=== 0.25&deg; x 0.3125&deg; nested Europe ===
  
== FlexGrid Stage 2 User's Guide ==
+
[[GEOS-Chem_horizontal_grids#0.25_x_0.3125_EU_nested_grid|See this link for grid details.]]
  
=== Issues to discuss ===
+
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.25x0.3125
 +
Longitude min/max      : -15.0 40.0
 +
Latitude  min/max      : 32.75 61.25
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3
  
== FlexGrid implementation details ==
+
=== 0.25&deg; x 0.3125&deg; nested North America ===
  
=== Source code availability ===
+
[[GEOS-Chem_horizontal_grids#0.25_x_0.3125_NA_nested_grid|See this link for grid details.]]
  
=== Utilities for compile flags ===
+
%%% GRID MENU %%%      :
 +
Grid resolution        : 0.25x0.3125
 +
Longitude min/max      : -130.0 -60.0
 +
Latitude  min/max      :  9.75  60.0
 +
  Half-sized polar boxes?: F
 +
Number of levels        : 47
 +
Nested grid simulation? : T
 +
  Buffer zone (N S E W ) :  3  3  3  3

Latest revision as of 13:32, 16 January 2020

On this page we provide information about the FlexGrid, which was introduced in GEOS-Chem 12.4.0.

Overview

FlexGrid is a new functionality that will allow users to define the grid for their GEOS-Chem simulation at run time. The major features include:

  1. Reading in meteorology fields via HEMCO
  2. Defining meteorology field, horizontal grid, and vertical grid at run time
  3. Saving out boundary conditions via the HISTORY component
  4. Reading in boundary conditions via HEMCO

Developers

Implementation

The table below shows the implementation plan for FlexGrid in GEOS-Chem:

Update Description Status
Stage 1: Read met fields by HEMCO
  • Add entries to read netCDF meteorology files in HEMCO_Config.rc
  • Unify code for GEOS-FP and MERRA-2 meteorology by creating a new flexgrid_read_mod.F90
  • This update was included in GEOS-Chem 12.1.0, which was released on 26 Nov 2018.
Stage 2: Add capability to define custom grids
  • Define meteorology field, grid resolution, and grid domain at run time in input.geos
  • Add new State_Grid derived type object to pass grid parameters from top level routines down
  • Save out boundary conditions in HISTORY.rc and read in via HEMCO
  • This update was included in GEOS-Chem 12.4.0, which was released on 05 Aug 2019.

Read in meteorology fields via HEMCO

FlexGrid takes advantage of HEMCO's I/O interface and regridding & cropping capability. By reading the meteorology fields through HEMCO, the fields are automatically processed for the correct grid resolution and domain at run time. These fields are then passed to State_Met in the various subroutines in flexgrid_read_mod.F90. Reading meteorology fields in HEMCO also allows users to explicitly control each meteorological variable in run-time configuration files to perform meteorology sensitivity tests (an interface resembling GCHP).

The following lines in green were added to HEMCO_Config.rc:

###############################################################################
### BEGIN SECTION SETTINGS
###############################################################################

ROOT:                        /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/HEMCO 
METDIR:                      /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/GEOS_4x5/GEOS_FP
...

###############################################################################
### NON-EMISSIONS DATA (subsection of BASE EMISSIONS SECTION)
###
### Non-emissions data. The following fields are read through HEMCO but do 
### not contain emissions data. The extension number is set to wildcard 
### character denoting that these fields will not be considered for emission 
### calculation. A given entry is only read if the assigned species name is 
### an HEMCO species.
###############################################################################

#==============================================================================
# --- Meteorology fields for FlexGrid ---
#==============================================================================

# --- CN fields ---
* FRLAKE    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAKE   */1/1/0               C xy  1  * -  1 1
* FRLAND    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAND   */1/1/0               C xy  1  * -  1 1
* FRLANDIC  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLANDIC */1/1/0               C xy  1  * -  1 1
* FROCEAN   $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FROCEAN  */1/1/0               C xy  1  * -  1 1
* PHIS      $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        PHIS     */1/1/0               C xy  1  * -  1 1

# --- A1 fields ---
* ALBEDO    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC     ALBEDO   1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
* CLDTOT    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC     CLDTOT   1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
* EFLUX     $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC     EFLUX    1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
* EVAP      $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A1.$RES.$NC     EVAP     1980-2018/1-12/1-31/0-23/+30minute CY xy  1  * -  1 1
...

Define meteorology field and grid parameters in input.geos

Prior to FlexGrid, users would define the meteorology field and grid resolution at compile time by passing the MET= and GRID= options at compile time. Changing meteorology field or grid resolution would require recompiling the source code. In addition, many of the grid parameters were hardcoded in CMN_SIZE_mod.F or stored in module-level arrays (e.g. XMID, YMID, AREA_M2 in gc_grid_mod.F90). To implement a new meteorology field or nested grid, users would have to process meteorology files offline and update several areas of the GEOS-Chem source code to recognize the new grid.

FlexGrid introduces the capability to define the meteorology field in the Simulation Menu of input.geos and the grid resolution and domain in the Grid Menu of input.geos. The new derived type object State_Grid holds the user-defined grid parameters as well as those computed by GEOS-Chem at run time. This new capability greatly simplifies the process of running GEOS-Chem with other meteorology fields and on other grids. It also allows for the use of the same geos executable for different meteorology and grid settings.

The following lines in green were added to input.geos:

%%% SIMULATION MENU %%% :
Start YYYYMMDD, hhmmss  : 20160701 000000
End   YYYYMMDD, hhmmss  : 20160703 000000
Run directory           : ./
Root data directory     : /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData
Met field               : GEOS-FP
------------------------+------------------------------------------------------
%%% GRID MENU %%%       :
Grid resolution         : 4.0x5.0
Longitude min/max       : -180.0 180.0
Latitude  min/max       : -90.0   90.0
 Half-sized polar boxes?: T
Number of levels        : 47
Nested grid simulation? : F
 Buffer zone (N S E W ) :  3  3  3  3

The meteorology fields currently supported are GEOS-FP and MERRA-2. As of implementation, an error check in GeosCore/input_mod.F will stop the simulation if the meteorology type is not recognized, but other met fields may be added in the future to improve flexibility.

Users can now define the grid resolution and domain for their simulation at run time in the Grid Menu.

Grid Resolution
The grid resolutions currently supported are 4.0x5.0, 2.0x2.5, 0.5x0.625, and 0.25x0.3125 (real numbers, not integers, must be used). As of implementation, an error check in GeosCore/input_mod.F will stop the simulation if the meteorology type is not recognized, but this error check may be removed or other grid resolutions may be added in the future to improve flexibility. Theoretically, any grid resolution should work, but this has not been validated.

Grid Domain
Users can choose to use either global grids or nested/regional grids. To use a global grid, simply specify longitude and latitude bounds of -180.0, 180.0 and -90.0, 90.0. All global grids currently supported use half-sized polar boxes, but a switch is included in the Grid Menu for flexibility with other grids.

Vertical Grid
Users can also specify the vertical grid in number of levels. The vertical resolutions currently supported are 47 and 72 levels, formerly known as the reduced vertical grid and native-resolution vertical grid. As of implementation, an error check in GeosCore/input_mod.F will stop the simulation if the vertical resolution is not recognized, but this error check may be removed or other grid resolutions may be added in the future to improve flexibility.

Nested Grid
If using a grid other than a global grid, users must set the Nested grid simulation option to True and specify the number of grid cells along each border that define the buffer region (default is 3 3 3 3. The buffer region is the region where boundary conditions from a coarse-resolution simulation are applied.

From the settings specified in the Grid Menu, the number of grid boxes in the X, Y, and Z directions will be computed and stored in State_Grid%NX, State_Grid%NY, and State_Grid%NZ, which will replace the old IIPAR, JJPAR, and LLPAR parameters from CMN_SIZE_mod.F. Several additional fields will be computed at run time in GeosUtil/gc_grid_mod.F90. These fields include arrays of longitude centers (State_Grid%XMid, longitude edges (XEdge), latitude centers (State_Grid%YMid), latitude edges (State_Grid%YEdge), grid box areas (State_Grid%Area_M2), etc.

With this update, the following C-preprocessor switches have been removed from the code and are replaced with IF statements to check the relevant field in either State_Met or State_Grid.

  • GEOS_FP, MERRA2
  • GRID_4x5, GRID_2x25, GRID_05x0625, GRID025x03125
  • NESTED_AS, NESTED_CH, NESTED_EU, NESTED_NA
  • GRIDREDUCED

Save out boundary conditions in HISTORY.rc

A new BoundaryConditions collection allows users to save out instantaneous species concentration fields (v/v) at specified intervals (default is 3-hourly boundary conditions saved to daily files). For the initial implementation, global boundary condition fields will be saved out, and users can utilize these fields as boundary conditions for any custom nested-grid simulation. Eventually, the goal is to add the capability to define a specific region for saving out boundary conditions to save disk space.

The following lines in green were added to HISTORY.rc:

COLLECTIONS: 'Restart',
             'BoundaryConditions',
             ...

#==============================================================================
# %%%%% THE BoundaryConditions COLLECTION %%%%%
#
# GEOS-Chem boundary conditions for use in nested grid simulations
#
# Available for all simulations
#==============================================================================
  BoundaryConditions.template:   '%y4%m2%d2_%h2%n2z.nc4',
  BoundaryConditions.format:     'CFIO',
  BoundaryConditions.frequency:  00000000 030000
  BoundaryConditions.duration:   00000001 000000
  BoundaryConditions.mode:       'instantaneous'
  BoundaryConditions.LON_RANGE:  -130.0 -60.0,
  BoundaryConditions.LAT_RANGE:  10.0 60.0,
  BoundaryConditions.fields:     'SpeciesBC_?ADV?             ', 'GIGCchem',
::

NOTE: The .LON_RANGE and .LAT_RANGE options were added in GEOS-Chem 12.6.0. For subsetting with versions 12.4.0 and 12.5.0 used BoundaryConditions.subset : -130.0 -60.0 10.0 60.0.

Read in boundary conditions via HEMCO

The I/O routines in HEMCO will automatically regrid and crop the boundary conditions to the grid defined in input.geos. The boundary conditions will be applied to the buffer zone, also defined in input.geos. These fields are obtained from HEMCO and applied to the species concentrations (State_Chm%Species) in GeosCore/hcoi_gc_main_mod.F90.

The following lines in green were added to HEMCO_Config.rc:

# ExtNr ExtName                on/off  Species 
0       Base                   : on    *
    --> GC_RESTART             :       true     
    --> GC_BCs                 :       true
    ...

#==============================================================================
# --- GEOS-Chem boundary condition file --- 
#==============================================================================
(((GC_BCs
* BC_           ./GEOSChem.BoundaryConditions.$YYYY$MM$DD_$HH$MNz.nc4 SpeciesBC_?ADV?  1980-2019/1-12/1-31/0-23 RY xyz 1 * - 1 1
)))GC_BCs

IMPORTANT: As of implementation, the boundary condition files must have 8 time slices (e.g. hours 0, 3, 6, 9, 12, 15, 18, 21). When generating boundary conditions make sure you start your coarse-resolution simulation at least one day before the start of your nested grid simulation to ensure the boundary condition file for the first day includes hour 0. When hour 0 is missing, HEMCO sees 7 time slices and identifies the dataset as weekly data -- this warrants further investigation.

FlexGrid for different types of users

According to the availability of meteorological data, users can get different benefits from FlexGrid:

User type Benefits
Only have standard met fields

(e.g. global 4x5, 0.25x0.3125 nested NA)

  • Be able to explicitly control each met variable in HEMCO_Config.rc (like Extdata.rc for GCHP). You can freeze certain variables, apply scaling factors, or read user-generated met data.
  • Be able to set up nested simulations over any sub-domains of the met field you have. For example, you can set up an east-US simulation in no time if you have the nested North American met field data.
  • If a new nested domain is needed, you can skip the source code modification and just prepare the met field input.
Have global native met fields

(For example, GCHP users might store them for global high-resolution runs. We can reuse them for nested simulations.)

  • Be able to run nested simulations over any domains in the world.

FlexGrid examples

HEMCO_Config.rc is the place to explicitly control all meteorological input variables (similar to Extdata.rc in GCHP). Each variable is listed in one line, just like other emission data entries in that file:

# --- CN fields ---
* FRLAKE    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAKE   */1/1/0               C xy  1  * -  1 1
* FRLAND    $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLAND   */1/1/0               C xy  1  * -  1  1
* FRLANDIC  $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FRLANDIC */1/1/0               C xy  1  * -  1 1 
* FROCEAN   $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        FROCEAN  */1/1/0               C xy  1  * -  1 1
* PHIS      $METDIR/$CNYR/01/$MET.$CNYR0101.CN.$RES.$NC        PHIS     */1/1/0               C xy  1  * -  1 1
...
# --- A3dyn fields ---
* DTRAIN    $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  DTRAIN   1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
* OMEGA     $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  OMEGA    1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
* RH        $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  RH       1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
* U         $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  U        1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
* V         $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  V        1980-2018/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
...

Most of the tricks that you apply to the emission data can also be applied here, such as

Change the met field path

Say you've modified the default temperature data to test the sensitivity to the temperature, you can just tell HEMCO to read your new file instead of the default file, while keep all other variables untouched:

* TMPU1     path_to_new_file    T        1980-2018/1-12/1-31/0-23          CY xyz  1  * -  1 1
* TMPU2     path_to_new_file    T        1980-2018/1-12/1-31/0-23/+3hour   CY xyz  1  * -  1 1

Freeze certain variables

You can force the model to keep reusing the wind in 2011 by restricting the time range (e.g. to see how does the trend of certain meteorological variables affects the simulation):

* U         $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  U        2011/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1
* V         $METDIR/$YYYY/$MM/$MET.$YYYY$MM$DD.A3dyn.$RES.$NC  V        2011/1-12/1-31/0-23/+90minute CY xyz 1  * -  1 1

Supported horizontal grid definitions

Examples of grid definitions in input.geos are provided below for users looking to recreate the global and nested grids used in GEOS-Chem prior to the FlexGrid implementation. All examples below may be used with GEOS-FP met fields. MERRA-2 met fields may be used with all examples below except for the 0.25° x 0.3125° grids.

4° x 5° global grid

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 4.0x5.0
Longitude min/max       : -180.0 180.0
Latitude  min/max       : -90.0   90.0
 Half-sized polar boxes?: T
Number of levels        : 47
Nested grid simulation? : F
 Buffer zone (N S E W ) :  3  3  3  3

2° x 2.5° global grid

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 2.0x2.5
Longitude min/max       : -180.0 180.0
Latitude  min/max       : -90.0   90.0
 Half-sized polar boxes?: T
Number of levels        : 47
Nested grid simulation? : F
 Buffer zone (N S E W ) :  3  3  3  3

0.5° x 0.625° nested Asia

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.5x0.625
Longitude min/max       :  60.0 150.0
Latitude  min/max       : -11.0  55.0
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3

0.5° x 0.625° nested Europe

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.5x0.625
Longitude min/max       : -30.0 50.0
Latitude  min/max       :  30.0 70.0
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3

0.5° x 0.625° nested North America

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.5x0.625
Longitude min/max       : -140.0 -40.0
Latitude  min/max       :   10.0  70.0
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3

0.25° x 0.3125° nested China

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.25x0.3125
Longitude min/max       : 70.0 140.0
Latitude  min/max       : 15.0  55.0
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3

0.25° x 0.3125° nested Europe

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.25x0.3125
Longitude min/max       : -15.0 40.0
Latitude  min/max       : 32.75 61.25
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3

0.25° x 0.3125° nested North America

See this link for grid details.

%%% GRID MENU %%%       :
Grid resolution         : 0.25x0.3125
Longitude min/max       : -130.0 -60.0
Latitude  min/max       :   9.75  60.0
 Half-sized polar boxes?: F
Number of levels        : 47
Nested grid simulation? : T
 Buffer zone (N S E W ) :  3  3  3  3