FlexGrid

From Geos-chem
Jump to: navigation, search

This page contains FlexGrid documentation written by Jiawei Zhuang.

Overview

FlexGrid is a new functionality that will be implemented in GEOS-Chem 12 to unify all met field I/O routines by HEMCO.

HEMCO's I/O interface and regridding & cropping capability allow users to

  • Set up a nested simulation over any custom domains without changing model's source code.
  • Explicitly control each meteorological variable in run-time configuration files to perform met field experiments (an interface resembling GCHP).

The idea of FlexGrid

In theory, setting up a new nested domain simply requires

  • New met fields over that domain
  • New regional emission data sets if necessary
  • Tuning several domain-dependent or resolution-dependent parameters.

There's no reason that one has to touch the model source code. However, in previous versions, all nested domains were hard-coded so users need to do a lot of tedious coding in order to set up a new domain. The major trouble was met field I/O, as we assumed that the model grid should exactly match the met field grid. The solution is HEMCO, which has already been used for regridding emission data sets at multiple grids to the same model grid. Here we just do the reverse -- regrid the same met field data set to different user-specified domains.

FlexGrid implementation plan

The table below shows the implementation plan for FlexGrid:

Stage Detail Status
Stage 1: Read met fields by HEMCO Enable the compile option MET=flexgrid to read any netCDF-format met fields through HEMCO. This unifies MET=geosfp and MET=merra2
  • This update was included in GEOS-Chem 12.1.0, which was released on 26 Nov 2018.
Stage 2: Implement custom nested simulations Enable the compile option NEST=cu (custom) to allow custom nested simulations.
Stage 3: Grid-independent GEOS-Chem classic Unify compile options GRID=xxx by moving all grid size parameters and resolution-dependent parameters to run-time configuration files. No specific plan right now. Need to be traded off against the nested-GCHP development.

FlexGrid for different types of users

According to the availability of meteorological data, users can get different benefits from FlexGrid:

User type Benefits
Only have standard met fields

(e.g. global 4x5, 0.25x0.3125 nested NA)

  • Be able to explicitly control each met variable in HEMCO_Config.rc (like Extdata.rc for GCHP). You can freeze certain variables, apply scaling factors, or read user-generated met data.
  • Be able to set up nested simulations over any sub-domains of the met field you have. For example, you can set up an east-US simulation in no time if you have the nested North American met field data.
  • If a new nested domain is needed, you can skip the source code modification and just prepare the met field input.
Have global native met fields

(For example, GCHP users might store them for global high-resolution runs. We can reuse them for nested simulations.)

  • Be able to run nested simulations over any domains in the world.

FlexGrid Stage 1 User's Guide

NOTE: Some of these instructions are out of date. We will update them shortly.

Compile

Just change the MET option to MET=flexgrid. Leave other options as-is. This works for any kinds of simulations (no matter global or nested, UCX or tropchem) that use NetCDF metfields.

For example, the original compile command for the 4x5 standard run is:

make -j4 UCX=yes CHEM=standard GRID=4x5 MET=geosfp

Now simply use

make -j4 UCX=yes CHEM=standard GRID=4x5 MET=flexgrid

Set up run directory

(Need to eventually add new rundirs to UT)

The executable generated in the above section works with the original run directory "geosfp_4x5_standard", with minimal modifications.

(1) Add this new config file FlexGrid_Config.rc to your run directory and include it in your HEMCO_Config.rc, preferably at the NON-EMISSIONS subsection:

###############################################################################
### NON-EMISSIONS DATA (subsection of BASE EMISSIONS SECTION)
...
###############################################################################

#==============================================================================
# --- metfields for FlexGrid ---
#==============================================================================
>>>include FlexGrid_Config.rc  

#==============================================================================
# --- Time zones (offset to UTC) ---
#==============================================================================
...

(2) Add user-defined tokens in HEMCO_Config.rc to complete the file paths in FlexGrid_Config.rc, for example

###############################################################################
### BEGIN SECTION SETTINGS
###############################################################################

ROOT:                        /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/HEMCO
METDIR:                      /n/holylfs/EXTERNAL_REPOS/GEOS-CHEM/gcgrid/data/ExtData/GEOS_4x5/GEOS_FP
METMODEL:                    GEOSFP
MET:                         geosfp
METRES:                      4x5 
Logfile:                     HEMCO.log

Note that the $MET token is not for met field I/O but is needed by FlexGrid. This is for the path of OTD-LIS scaling factor:

103 LIGHTNOX_OTDLIS $ROOT/LIGHTNOX/v2014-07/OTD-LIS-Local-Redist.CTH.v5.$MET.$RES.nc OTD $YYYY/1-12/1/0 C xy unitless NO - 1 1

$MET was once determined by the compile flag, but as long as a single compilation now works for both geosfp and merra2, the model will not know which one we are using unless we specify it.

(3) Add Lightning NOx and dust scaling factor to HEMCO_Config.rc. For 4x5 GEOSFP, they should be:

103     LightNOx             : on    NO
    --> OTD-LIS scaling      :       0.5568 
    --> OTD-LIS factors      :       true
    --> CDF table            :       $ROOT/LIGHTNOX/v2014-07/light_dist.ott2010.dat
104     SoilNOx              : on    NO
    --> Use fertilizer NOx   :       true
105     DustDead             : on    DST1/DST2/DST3/DST4
    --> Mass tuning factor   :       8.3286e-4 

We force users to set scaling factors explicitly because those processes are known to have strong resolution&domain dependence -- you should keep this in mind when building a new nested domain. (same for changing the resolution in GCHP)

The numbers are taken from hcox_dustdead_mod.F and hcox_lightnox_mod.F90. (Need a table summarizing all those factors)

(4) Now you should be able to run the model as usual.

Also note that the original met field path in input.geos has no effect now and will be removed.

 => GEOS-FP subdir      : GEOS_FP/YYYY/MM/

Use FlexGrid functionalities

FlexGrid_Config.rc is the place to explicitly control all meteorological input variables (similar to Extdata.rc in GCHP). Each variable is listed in one line, just like other emission data entries in HEMCO_Config.rc:

# --- CN     metfields   ---
* FRLAKE    $METDIR/2011/01/$METMODEL.20110101.CN.$METRES.nc FRLAKE    2011/1/1/0  C xy  1  * -  1 1
* FRLAND    $METDIR/2011/01/$METMODEL.20110101.CN.$METRES.nc FRLAND    2011/1/1/0  C xy  1  * -  1 1
* FRLANDIC  $METDIR/2011/01/$METMODEL.20110101.CN.$METRES.nc FRLANDIC  2011/1/1/0  C xy  1  * -  1 1
* FROCEAN   $METDIR/2011/01/$METMODEL.20110101.CN.$METRES.nc FROCEAN   2011/1/1/0  C xy  1  * -  1 1
* PHIS      $METDIR/2011/01/$METMODEL.20110101.CN.$METRES.nc PHIS      2011/1/1/0  C xy  1  * -  1 1
...
# --- A3dyn  metfields   ---
* DTRAIN    $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc DTRAIN   2011-2014/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1
* OMEGA     $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc OMEGA    2011-2014/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1
* RH        $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc RH       2011-2014/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1
* U         $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc U        2011-2014/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1
* V         $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc V        2011-2014/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1
...

Most of the tricks that you apply to the emission data can also be applied here, such as

Change the met field path

Say you've modified the default temperature data to test the sensitivity to the temperature, you can just tell HEMCO to read your new file instead of the default file, while keep all other variables untouched:

* TMPU1     path_to_new_file    T        2011-2014/1-12/1-31/0-23          C xyz  1  * -  1 1
* TMPU2     path_to_new_file    T        2011-2014/1-12/1-31/0-23/+3hour   C xyz  1  * -  1 1

Freeze certain variables

You can force the model to keep reusing the wind in 2011 by restricting the time range: (to see how does the trend of certain meteorological variables affects the simulation)

* U   $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc U   2011/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1 
* V   $METDIR/$YYYY/$MM/$METMODEL.$YYYY$MM$DD.A3dyn.$METRES.nc V   2011/1-12/1-31/0-23/+90minute C xyz  1  * -  1 1

FlexGrid Stage 2 User's Guide

This section describes the steps to set up a custom nested domain, assuming the knowledge of Stage 1.

Issues to discuss

Changing the nested domain becomes trivial once the metfield I/O is handled by HEMCO. However, we should decide how should the new configuration file (e.g. input.geos) look like. We need to read those hard-coded parameters from the configuration file:

  • IIPAR, JJPAR in Header/CMN_SIZE_mod.F
  • I1_BC, J1_BC, I2_BC, J2_BC in GeosCore/tpcore_bc_mod.F

along with "Global offsets I0, J0" in input.geos, those parameters fully describe the range of the nested domain.

FlexGrid implementation details

Source code availability

The source code is currently under development at my own repository. Will be soon merged to the standard repository.

Utilities for compile flags

Cleaning the #ifdef C-preprocessor statements in the source can be very tedious. There are some python scripts to check those statements and massively add new codes for FlexGrid.