Difference between revisions of "FlexChem"

From Geos-chem
Jump to: navigation, search
(Adding production & loss families)
Line 462: Line 462:
  
 
The family name must start with <tt>P</tt> or <tt>L</tt> to indicate whether KPP should calculate a production or a loss rate.
 
The family name must start with <tt>P</tt> or <tt>L</tt> to indicate whether KPP should calculate a production or a loss rate.
 +
 +
The maximum number of families allowed by KPP is currently set to 50. Depending on how many prod/loss families you add, you may need to increase that to a larger number to avoid errors in KPP. You can change the number for <tt>MAX_FAMILIES</tt> in <tt>KPP/kpp-2.2.3_01/src/gdata.h</tt> and then rebuild KPP.
 +
 +
#define MAX_EQN        800
 +
#define MAX_SPECIES    800
 +
#define MAX_SPNAME      30
 +
#define MAX_IVAL        40
 +
/* MAX_EQNTAG = max length of equation ID in eqn file */
 +
#define MAX_EQNTAG      12
 +
/* MAX_K = max length of rate expression in eqn file */
 +
#define MAX_K          150
 +
#define MAX_ATOMS        10
 +
#define MAX_ATNAME      10
 +
#define MAX_ATNR        250
 +
#define MAX_PATH        120
 +
#define MAX_FILES        20
 +
<span style="color:green">'''#define MAX_FAMILIES    50'''</span>
 +
#define MAX_MEMBERS    150
 +
#define MAX_EQNLEN      200
  
 
<span style="color:red">'''IMPORTANT:''' When adding a prod/loss family or changing any of the other settings in <tt>gckpp.kpp</tt>, the chemistry mechanism will need to be rebuilt with KPP as [[#Building a custom chemical mechanism|described above]].</span>
 
<span style="color:red">'''IMPORTANT:''' When adding a prod/loss family or changing any of the other settings in <tt>gckpp.kpp</tt>, the chemistry mechanism will need to be rebuilt with KPP as [[#Building a custom chemical mechanism|described above]].</span>

Revision as of 12:07, 1 August 2018



GEOS-Chem v11-02-final will also carry the designation GEOS-Chem 12.0.0. We are migrating to a purely numeric versioning system in order to adhere more closely to software development best practices. For a complete description of the new versioning system, please see our GEOS-Chem version numbering system wiki page.




On this page we provide information about the FlexChem chemical solver, which was introduced in GEOS-Chem v11-01g.

Overview

The clean and flexible reimplementation of the Kinetic PreProcessor package (aka KPP)—known as FlexChem—is nearing full integration into GEOS-Chem. Most of the remaining FlexChem development work will focus on replacing legacy infrastructure—which prevents GEOS-Chem from operating efficiently in high-performance computing (HPC) environments—with newer, more efficient algorithms.

Check back soon for the latest information!

--Bob Yantosca (talk) 19:58, 19 May 2016 (UTC)

Milestones

The following table shows several milestones that were achieved in the FlexChem implementation, development, as well as ongoing development tasks.

Task Developer Status
Add FlexChem into v11-01c Mike Long Completed 14 Dec 2015
Enable the "Tropchem" mechanism Mike Long Completed 14 Dec 2015
Restore the OH and HO2 diagnostics (ND43) Melissa Sulprizio Completed 18 Dec 2015
Remove CSPEC array and replaced with State_Chm%Species Melissa Sulprizio Completed 22 Dec 2015
Enable a temporary workaround for family tracers (ISOPN, MMN, CFCX, HCFCX) Mike Long Completed 25 Jan 2016
Enable FAST-JX photochemistry Mike Long Completed 25 Jan 2016
Enable the "Benchmark" (aka "Standard") chemistry mechanism Melissa Sulprizio Completed 29 Jan 2016
Restore the broken J-value diagnostic (ND22) Melissa Sulprizio Completed 15 Mar 2016
Parallelize the main KPP driver loop; fixed other minor issues Bob Yantosca Completed 30 Mar 2016
Enable the "SOA" and "SOA-SVPOA" mechanisms Lizzie Lundgren Completed 01 Apr 2016
Merge FlexChem code based on v11-01c with v11-01f Melissa Sulprizio Completed 20 Apr 2016
Add KPP repository to Bitbucket (https://bitbucket.org/gcst/kpp) Mike Long Completed 19 Apr 2016
Create new gckpp_*.F90 files from the updated KPP solver package Melissa Sulprizio Completed 22 Apr 2016
Enable the "UCX" mechanism Melissa Sulprizio Completed 26 Apr 2016
Introduce a new prod/loss diagnostic into the KPP solver package Mike Long Completed 29 Apr 2016
Complete unit tests and 1-month benchmarks for the Standard, Tropchem, UCX, SOA, and SOA-SVPOA simulations Melissa Sulprizio
Lizzie Lundgren
Completed 05 May 2016
Remove tracer indices of Rn, Pb, Be, Hg, and POPs simulations from tracerid_mod.F. (This paves the way for us to retire tracerid_mod.F.) Bob Yantosca Completed 02 May 2016
Add a fast species name lookup algorithm Bob Yantosca Code is in place as of 04 May 2016, but has not been implemented throughout GEOS-Chem yet.
Store the unique list of GEOS-Chem species in the GEOS-Chem species database object. (In other words, we combine the list of advected tracers with the list of KPP chemical species and remove duplicate entries). Bob Yantosca Completed 09 May 2016
Store the indices of each species in the KPP chemical reaction matrix in the GEOS-Chem species database object. Bob Yantosca Completed 09 May 2016
Merge FlexChem with other v11-01g updates and bug fixes Bob Yantosca Completed 17 May 2016
Remove family tracer fields of the Input_Opt object, namely: TRACER_N_CONST, TRACER_CONST, TRACER_COEFF, ID_EMITTED. These were only used by SMVGEAR and are now obsolete. Bob Yantosca Completed 18 May 2016
Create mapping vectors in the State_Chm object to store the ID numbers of species that are advected, dry-deposited, wet-deposited, and/or included in the KPP chemical mechanism Bob Yantosca Completed 19 May 2016
Remove all 1-D loop indexing variables (JLOOP, KLOOP, KTLOOP, JLOP, IXSAVE, IYSAVE, IZSAVE) that were used by SMVGEAR Melissa Sulprizio Completed 19 May 2016
Remove SMVGEAR input files (globchem.dat, mglob.dat) and associated subroutines (readchem.F, reader.F) Melissa Sulprizio Completed 30 Jun 2016
Remove tracerid_mod.F and related tracer ID flags (IDTxxxx, IDxxxx). Instead, use the GEOS-Chem species database to retrieve species index from species name. Jacob Group Code-A-Thon Completed 24 Jun 2016
Remove tracers from restart files and read/write restart file species only Lizzie Lundgren Completed 12 Jul 2016
Create unit conversion routines for species that mimic existing routines for tracers or convert to/from molecules/cm3 Lizzie Lundgren Completed 25 Jul 2016
Add State_Chm variable to track species units throughout GEOS-Chem (State_Chm%Spc_Units) Lizzie Lundgren Completed 25 Jul 2016
Remove the remaining family tracers (ISOPN, MMN, CFCX, HCFCX). Henceforth, FlexChem will only work with individual species. Melissa Sulprizio Completed 28 Jul 2016
Move the non-advected species initial unit conversion of background values from INIT_FLEXCHEM (called during chemistry) to READ_GC_RESTART (called during restart file read) Lizzie Lundgren Completed 4 Aug 2016
Store species background concentrations values previously stored in globchem.dat in the species database Lizzie Lundgren Completed 11 Aug 2016
Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms to use the new prod/loss functionality in KPP (i.e. FLUX on) Melissa Sulprizio Completed 13 Jul 2016
Attach the new KPP prod/loss output to the GEOS-Chem prod/loss diagnostic (ND65) Melissa Sulprizio Completed 11 Aug 2016
Remove State_Chm%Tracers and use species concentrations in State_Chm%Species instead GCST Completed 29 Aug 2016
Validate FlexChem with v11-01g benchmarks GCST 1-month benchmark:
Approved 14 Sep 2016

1-year benchmark:
Approved 28 Sep 2016

Fix P/L diagnostics to ignore cycling reactions Mike Long 08 Nov 2016
Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms with Kppa GCST TBD
Update the GEOS-Chem Makefiles so that a fresh version of KPP will be built (using a custom chemical mechanism that you specify) each time GEOS-Chem is compiled. GCST TBD

Adding chemical mechanisms in FlexChem

KPP source code

The KPP source code for FlexChem is currently available as a Git repository on Bitbucket. It can be obtained using:

git clone https://bitbucket.org/gcst/kpp
git checkout -b GC_updates origin/GC_updates

Once you have KPP on your system, you will need to build it:

cd kpp/kpp-2.2.3_01
make clean
make

If the build completed successfully, you should see the executable kpp in the kpp/kpp-2.2.3_01/bin/ directory. Next, add KPP to your PATH. For bash users, add the following lines the ~/.bashrc file:

export PATH=$PATH:/PATH_TO_KPP/KPP/kpp-2.2.3_01/bin/
export KPP_HOME=/PATH_TO_KPP/KPP/kpp-2.2.3_01

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Building a custom chemical mechanism

In GEOS-Chem v11-02

GEOS-Chem v11-02 has three pre-built chemical mechanisms: Standard, Tropchem, and SOA_SVPOA. These mechanisms are described on our GEOS-Chem chemistry mechanisms wiki page.

GEOS-Chem will eventually include the option to create a custom chemistry mechanism and automatically build that mechanism with KPP when GEOS-Chem is compiled. For now, users still need to build custom chemistry mechanisms manually using the following files that can be found in the Custom mechanism subdirectory (e.g Code.v11-02/KPP/Custom/):

 Custom.eqn
 gckpp.kpp
 build_mechanism.sh
 rename_f90.sh (used by build_mechanism.sh)

The .eqn file currently included in each of the KPP mechanism subdirectories defines the variable species (#DEFVAR), fixed species (#DEFFIX), and reactions (#EQUATIONS) to include in that mechanism. The equations section is further broken down into gas-phase reactions, heterogeneous reactions, and photolysis reactions. The default custom mechanism defined in Custom.eqn is based on the Standard (UCX-based) mechanism (found in KPP/Standard/Standard.eqn). The production and loss families in the custom gckpp.kpp are also built for the Standard (UCX-based) mechanism (found in KPP/Standard/gckpp.kpp ), and will not work with the Tropchem.eqn . If you would like to base your custom mechanism on the tropchem mechanism instead, simply copy the Tropchem.eqn and gckpp.kpp files over instead:

 cd KPP/Custom/
 cp ../Tropchem/Tropchem.eqn Custom.eqn
 cp ../Tropchem/gckpp.kpp gckpp.kpp
  1. Edit the Custom.eqn to add or modify species and reactions for your mechanism. If you are adding a heterogeneous reaction or photolysis reaction, those require additional modifications to other files. See the Adding a reaction to the mechanism section below.
  2. Modify the gckpp.kpp file if needed. This file defines the KPP integrator and tells KPP to use the specified .eqn file to build the mechanism. This file also defines the prod/loss families as described below.
  3. Rebuild the mechanism with KPP by typing ./build_mechanism.sh. The build_mechanism.sh script will call on KPP to build the necessary gckpp_*.F90 files.
  4. Compile GEOS-Chem for the new mechanism by including CHEM=Custom in your make command. NOTE: If your custom mechanism is a UCX-based mechanism then you should also include NO_REDUCED=y in your make command.

--Melissa Sulprizio (talk) 17:54, 6 February 2018 (UTC)

--Jared Brewer 20:30, 25 July 2018 (UTC)

In GEOS-Chem v11-01

GEOS-Chem v11-01g has five pre-built chemical mechanisms: Standard, Tropchem, SOA, SOA-SVPOA, and UCX. These mechanisms are described on our GEOS-Chem chemistry mechanisms wiki page.

GEOS-Chem will eventually include the option to create a custom chemistry mechanism and automatically build that mechanism with KPP when GEOS-Chem is compiled. For now, users still need to build custom chemistry mechanisms manually. The KPP Git repository includes the following files:

 geos2kpp_parser.pl
 gckpp.kpp
 rename_f90.sh
 copy_gckpp.sh

Note: You'll need to checkout the GC_updates branch in the KPP repository to access these files. This can be achieved with git checkout GC_updates.

These files can be used to build a custom chemistry mechanisms with KPP. Follow these steps:

  1. The geos2kpp_parser.pl script can be used to create globchem.eqn, globchem.spc, and globchem.def files from a globchem.dat file. You may also choose to start from the preexisting globchem.eqn, globchem.spc, and globchem.def files found in the KPP subdirectories of the GEOS-Chem source code (e.g. Code.v11-01/KPP/Standard/).
  2. If needed, edit globchem.eqn to add or modify reactions for your mechanism (see below) and edit globchem.spc to add/modify species to your mechanism. Put the final globchem.* files in the same directory as the gckpp.kpp file.
  3. Modify the gckpp.kpp file if needed. This file defines the KPP integrator and tells KPP to use the globchem.spc and globchem.eqn files to build the mechanism. This file also defines the prod/loss families as described below.
    IMPORTANT: The default gckpp.kpp file that is included with KPP does not define any prod/loss families. Therefore, you must make sure the ND65 prod/loss diagnostic is turned off in input.geos when you run GEOS-Chem. If you would like to use prod/loss families in your custom mechanism, you can see the gckpp.kpp files in the KPP subdirectories of the GEOS-Chem source code (e.g. Code.v11-01/KPP/Standard/gckpp.kpp) for examples.
  4. Execute KPP using kpp gckpp.kpp. This will create the necessary gckpp_*.f90 files.
  5. Run ./rename_f90.sh to rename gckpp_*.f90 to gckpp_*.F90.
  6. Open copy_gckpp.sh and modify the line for codedir. Save and run ./copy_gckpp.sh. This will copy the gckpp_*.F90 files to codedir/Custom/.
  7. Compile GEOS-Chem for the new mechanism by including CHEM=Custom in your make command.

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Adding a reaction to the mechanism

No matter what reaction is being added, the general procedure is the same. A new line must be added to .eqn of the following form:

{700} A + B = C + 2.000D : REQN(ARG_A,ARG_B);

The first number (700) is a KPP tag. The specific value has no effect on the reaction, but this should uniquely identify the reaction. The second section denotes the reactants (A and B) as well as the products (C and D) of the reaction. If exactly one molecule is consumed or produced, then the factor can be omitted; otherwise the number of molecules consumed or produced should be specified with at least 1 decimal place of accuracy. The final section, between the colon and semi-colon, specifies the function and arguments which will be used to calculate the reaction rate constant k. For a two-body reaction, the overall rate at which the reaction proceeds will be calculated as k[A][B].

Two-body reactions

In the case of two-body reactions proceeding according to the Arrhenius equation, calculation of k is straightforwardly implemented by calling the GCARR function. For example, the most recent JPL chemical data evaluation (as of February 2017) specifies that the reaction O3 + NO produces NO2 and O2, and its Arrhenius parameters are A = 3.0x10^-12 and E/R = 1500. The entry for this reaction therefore reads as follows (noting the different sign convention used for E/R)

{1} O3 + NO = NO2 + O2 : GCARR(3.00E-12,0,-1500); 

The exact implementation of the Arrhenius rate calculation can be found in gckpp.kpp, as can other rate functions such as that required for three-body, pressure-dependent reactions. Any rate function which is to be referenced in the .eqn file must be available in gckpp.kpp prior to building the reaction mechanism.

Photolysis reactions

Similarly, a photolysis reaction can be specified by giving the correct index of the PHOTOL array. This index can be determined by inspecting the file FJX_j2j.dat in your run directory. For example, one branch of the NO3 photolysis reaction is specified in the .eqn file as

{522} NO3 + hv = NO2 + O : PHOTOL(12) 

Referring back to FJX_j2j.dat shows that reaction 12, as specified by the left-most index, is indeed NO3 = NO2 + O:

 12 NO3       PHOTON    NO2       O                       0.886 /NO3   /

If your reaction is not already in FJX_j2j.dat, you may add it there. You may also need to modify FJX_spec.dat (also found in your run directory) to include cross-sections for your species.

Finally, you will need to update routine PHOTO_JX found in fast_jx_mod.F to archive the J-values for your new reaction to the ND22 diagnostic. The J-values are obtained from array ZPJ which uses the same index found in the FJX_j2j.dat file. For example:

     !---------------------------------------------------------------
     ! Archive J-values for ND22 diagnostic
     !---------------------------------------------------------------
     IF ( ND22 > 0 ) THEN

        ! Save J-values for 2PM diagnostic boxes
        IF ( LTJV(ILON,ILAT) > 0 ) THEN

           DO L = 1, LD22
              ! AD22 IDs 5, 6, and 15 (J-values for O3 and O2)
              ! are handled in routine PHOTRATE_ADJ
              ! Hardcode ZPJ indices based on valued from FJX_j2j.dat
              ! for now (mps, 3/15/16)
              AD22(ILON,ILAT,L, 1) = AD22(ILON,ILAT,L, 1) +  ! JNO2
    &                                ZPJ(L,11,ILON,ILAT)
              ...

Index 11 used for obtaining JNO2 correcponds to the index found FJX_j2j_mod.F90:

 11 NO2       PHOTON    NO        O                       1.000 /NO2   /

Heterogeneous reactions

Implementing new heterogeneous chemistry requires an additional step. For the reaction in question, a reaction should be added as usual, but this time the rate function should be given as an entry in the HET array. A simple example is uptake of HO2, specified as

{285} HO2 = O2 : HET(ind_HO2,1);

Note that the product in this case, O2, is actually a fixed species, so no O2 will actually be produced. O2 is used in this case only as a dummy product to satisfy the KPP requirement that all reactions have at least one product. Here, HET is simply an array of pre-calculated rate constants. The rate constants in HET are actually calculated in gckpp_HetRates.F90. To implement an additional heterogeneous reaction, the rate calculation must be added to this file. The following example illustrates a (fictional) heterogeneous mechanism which converts the species XYZ into CH2O. This reaction is assumed to take place on the surface of all aerosols, but not cloud droplets (this requires additional steps not shown here). Three steps would be required:

  • Add a new line to the .eqn file, such as {900} XYZ = CH2O : HET(ind_XYZ,1);
  • Add a new function to gckpp_HetRates.F90 designed to calculate the heterogeneous reaction rate. As a simple example, we can copy the function HETNO3 and rename it HETXYZ. This function accepts two arguments: molecular mass of the impinging gas-phase species, in this case XYZ, and the reaction's "sticking coefficient" - the probability that an incoming molecule will stick to the surface and undergo the reaction in question. In the case of HETNO3, it is assumed that all aerosols will have the same sticking coefficient, and the function returns a first-order rate constant based on the total available aerosol surface area and the frequency of collisions.
  • Add a new line to the function SET_HET in gckpp_HetRates.F90 which calls the new function with the appropriate arguments and passes the calculated constant to HET. Example: assuming a molar mass of 93 g/mol, and a sticking coefficient of 0.2, we would write HET(ind_XYZ, 1) = HETXYZ(9.30E1_fp, 2E-1_fp)

The function HETXYZ can then be specialized to distinguish between aerosol types, or extended to provide a second-order reaction rate, or whatever the user desires.

--Sebastian D. Eastham (talk) 22:45, 14 February 2017 (UTC)

Chemical species in FlexChem

A complete list of chemical species included in the pre-built (Standard, Tropchem, SOA, SOA-SVPOA, UCX) mechanisms can be viewed on our Species in GEOS-Chem wiki page.

Adding a chemical species

Adding chemical species to FlexChem can be achieved be following these simple steps:

  1. Add the species name(s) to the .spc file under the #DEFVAR or #DEFFIX section.
  2. Add or modify any reactions in the .eqn file under the #EQUATIONS section.
  3. Build the chemical mechanism with KPP by following the steps outlined above.

NOTE: If a species is defined by KPP, it will automatically be added to the species database with the default settings for non-advected chemical species when GEOS-Chem runs. However, we recommend also adding your species to the species database by following these instructions. Including a species in the species database is required if the new species is to be defined as an advected species in input.geos.

--Melissa Sulprizio (talk) 17:27, 22 November 2016 (UTC)

Removal of family tracers

In GEOS-Chem v9-02 we removed all family tracers from GEOS-Chem. For example, the NOx family was replaced by individual tracers NO, O3, NO3, and HNO2. Similarly, the Ox family was replaced by individual tracers O3, NO2, and NO3.

But certain updates that were made to GEOS-Chem after v9-02 introduced new family tracers into the various chemistry mechanisms:

The implementation of the FlexChem chemical solver package in GEOS-Chem v11-01 forced us to remove these family tracers, because the KPP chemical solver is designed to only include individual species in its mechanism. Removing these families also allows us to avoid some cumbersome sections of code that were used to construct each family from its constituent members.

Therefore, the following families have been removed from GEOS-Chem v11-01g, and replaced with individual species:

Removed Added as individual advected species Mechanisms in which these are used
ISOPN ISOPND ISOPNB Standard, UCX, TropChem, SOA, SOA-SVPOA
MVKN MACRN MVKN Standard, UCX, TropChem, SOA, SOA-SVPOA
CFCX CFC113 CFC114 CFC115 Standard, UCX
HCFCX HCFC22 HCFC141b HCFC142b Standard, UCX

The constituents of each former family are now included as advected species in the various GEOS-Chem mechanisms. To obtain diagnostic output for the former families, you can save out each of the individual species and combine the output in post-processing.

--Bob Yantosca (talk) 16:20, 16 August 2016 (UTC)

Production & loss diagnostic

Mike Long implemented the functionality for chemical prod/loss families in the KPP source code. With this option turned on, KPP will calculate the net prod/loss rates for user-defined chemical families.

Adding production & loss families

KPP prod/loss families are turned on by the #FAMILIES token in the gckpp.kpp file. For example:

  #INTEGRATOR rosenbrock
  #LANGUAGE Fortran90
  #DRIVER none
  
  #INCLUDE globchem.spc
  #INCLUDE globchem.eqn
  
  #FAMILIES
  POx : O3 + NO2 + 2NO3 + PAN + PMN + PPN + HNO4 + 3N2O5 + HNO3 + BrO + HOBr + BrNO2 + 2BrNO3 + MPN + ETHLN + ISN1 + ISOPNB + ISOPND + MACRN + MVKN + PROPNN + R4N2 + INPN + ISNP + INO2 + ISNOOA + ISNOOB + ISNOHOO + MAN2 + PRN1 + PRPN + R4N1 + PMNN + MACRNO2 + ClO + HOCl + ClNO2 + 2ClNO3 + 2Cl2O2 + 2OClO + O + O1D;
  LOx : O3 + NO2 + 2NO3 + PAN + PMN + PPN + HNO4 + 3N2O5 + HNO3 + BrO + HOBr + BrNO2 + 2BrNO3 + MPN + ETHLN + ISN1 + ISOPNB + ISOPND + MACRN + MVKN + PROPNN + R4N2 + INPN + ISNP + INO2 + ISNOOA + ISNOOB + ISNOHOO + MAN2 + PRN1 + PRPN + R4N1 + PMNN + MACRNO2 + ClO + HOCl + ClNO2 + 2ClNO3 + 2Cl2O2 + 2OClO + O + O1D;
  PCO : CO;
  LCO : CO;
  PSO4 : SO4;

To add a new prod/loss family, add a new line to the #FAMILIES section with the format

  FAM_NAME : MEMBER_1 + MEMBER_2 + ... + MEMBER_N;

The family name must start with P or L to indicate whether KPP should calculate a production or a loss rate.

The maximum number of families allowed by KPP is currently set to 50. Depending on how many prod/loss families you add, you may need to increase that to a larger number to avoid errors in KPP. You can change the number for MAX_FAMILIES in KPP/kpp-2.2.3_01/src/gdata.h and then rebuild KPP.

#define MAX_EQN         800
#define MAX_SPECIES     800
#define MAX_SPNAME       30
#define MAX_IVAL         40
/* MAX_EQNTAG = max length of equation ID in eqn file */
#define MAX_EQNTAG       12
/* MAX_K = max length of rate expression in eqn file */
#define MAX_K           150
#define MAX_ATOMS        10
#define MAX_ATNAME       10
#define MAX_ATNR        250 
#define MAX_PATH        120
#define MAX_FILES        20
#define MAX_FAMILIES    50
#define MAX_MEMBERS     150
#define MAX_EQNLEN      200

IMPORTANT: When adding a prod/loss family or changing any of the other settings in gckpp.kpp, the chemistry mechanism will need to be rebuilt with KPP as described above.

The pre-built chemistry mechanisms (Standard, Tropchem, SOA, SOA-SVPOA, and UCX) were built with the default prod/loss families listed in the example above. For the Tropchem, SOA, and SOA-SVPOA mechanisms several species (ClO + HOCl + ClNO2 + 2ClNO3 + 2Cl2O2 + 2OClO + O + O1D) were removed from the Ox family because those species are not defined in those mechanisms. The Ox and CO rates are used in GEOS-Chem for computing budgets in the 1-month benchmark simulations and PSO4 is required for simulations using TOMAS aerosol microphysics.

--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)

Tagging reactions

Reactions may be manually tagged so that reaction rates can be saved out to the prod/loss diagnostic. To do this, follow these steps:

1. Create dummy species in #DEFVAR section in the .eqn file

  #DEFVAR
  
  A3O2       = IGNORE; {CH3CH2CH2OO; Primary RO2 from C3H8}
  ACET       = IGNORE; {CH3C(O)CH3; Acetone}
  ACTA       = IGNORE; {CH3C(O)OH; Acetic acid}
  ...
  RXN1       = IGNORE; {Dummy species to tag reaction}

2. Add the dummy species as a product in desired reaction

  #EQUATIONS
  //
  // Gas-phase reactions
  //
  O3 + NO = NO2 + O2 + RXN1:                        GCARR(3.00E-12, 0.0E+00, -1500.0);

3. Add the dummy species to the #FAMILIES section in gckpp.kpp

  #FAMILIES
  POx : O3 + NO2 + 2NO3 + PAN + NPMN + PPN + HNO4 + 3N2O5 + ...;
  LOx : O3 + NO2 + 2NO3 + PAN + NPMN + PPN + HNO4 + 3N2O5 + ...;
  PCO : CO;
  LCO : CO;
  PSO4 : SO4
  PRXN1 : RXN1;

4. Rebuild the mechanism with KPP. Each mechanism subdirectory (e.g. Code.v11-02/KPP/Standard/) in GEOS-Chem v11-02 includes a build_mechanism.sh script that will call KPP and create the gckpp*.F90 files. To build the mechanism, type ./build_mechanism.sh.

5. Recompile and run GEOS-Chem, making sure you turn on the prod/loss diagnostics.

--Melissa Sulprizio (talk) 17:31, 6 February 2018 (UTC)

How it works

In GEOS-Chem v11-02 and later versions

This update was included in v11-02a and approved on 12 May 2017.

In GEOS-Chem v11-02, KPP has been updated to greatly simplify how the prod/loss rates are computed. In v11-02a, the chemical mechanisms were rebuilt with KPP at commit Fix to add coefficients to *_Monitor.F90 - MSL.. In the updated KPP, prod/loss families now produce one dummy species that is now named with the family name in the SPC_NAMES array. For example, POx is now computed using KPP species POx in the variable species array VAR instead of using several RR* species. This greatly reduces the number of dummy species down to the number of prod/loss families that are defined in the input file gckpp.kpp. KPP will tag a reaction with the prod/loss family name by processing all reactions to determine if they are net production or net loss for that family.

GEOS-Chem can now obtain the prod/loss rates directly from the VAR array in KPP. The following updates were made to routine Do_FlexChem in GeosCore/flexchem_mod.F90 in v11-02a to properly use the updated KPP prod/loss rates. Text green (red) indicates code that was added (removed).

1. Declare new variable KppID and remove obsolete variables for the prod/loss diagnostic
    !
    ! !LOCAL VARIABLES: 
    !
        INTEGER                :: I, J, L, N, F, SpcID, KppID
 
        ...

        ! For prod/loss diagnostic
        REAL(fp)               :: FAM(NFAM)
        INTEGER                :: IND
        INTEGER                :: COEF
        CHARACTER(LEN=14)      :: NAME
2. Declare KppID as !OMP PRIVATE
    !$OMP PRIVATE  ( SpcID, KppID,    F                       ) &
3. Initialize the prod/loss rates to zero in KPP each time Do_FlexChem is called
        !===========================================================
        ! Initialize species concentrations
        !===========================================================
        ! Loop over KPP Species
        DO N=1,NSPEC

           ! GEOS-Chem species ID
           SpcID = State_Chm%Map_KppSpc(N)

           ! Initialize KPP species concentration array
           IF ( SpcID .eq. 0) THEN
              C(N) = 0.0_dp
           ELSE
              C(N) = State_Chm%Species(I,J,L,SpcID)
           ENDIF

        ENDDO

        IF ( Input_Opt%DO_SAVE_PL ) THEN

           ! Loop over # prod/loss families
           DO F = 1, NFAM

              ! Determine dummy species index in KPP
              KppID = Ind_(TRIM(FAM_NAMES(F)),'K')

              ! Initialize prod/loss rates
              IF ( KppID > 0 ) C(KppID) = 0.0_dp

           ENDDO

        ENDIF
4. Update the prod/loss rate diagnostic code
           !==============================================================
           ! Obtain prod/loss rates from KPP [molec/cm3]
           !==============================================================
           IF ( Input_Opt%DO_SAVE_PL ) THEN

              ! Obtain prod/loss rates from KPP [molec/cm3]
              CALL ComputeFamilies( VAR, FAM )

              ! Loop over # prod/loss families
              DO F = 1, NFAM

                 ! Determine dummy species index in KPP
                 KppID = Ind_(TRIM(FAM_NAMES(F)),'K')

                 !--------------------------------------------------------
                 ! Add to AD65 array [molec/cm3/s]
                 !--------------------------------------------------------
                 AD65(I,J,L,F) = AD65(I,J,L,F) + FAM(F) / DT
                 IF ( KppID > 0 ) THEN
                    AD65(I,J,L,F) = AD65(I,J,L,F) + VAR(KppID) / DT
                 ENDIF

                 !--------------------------------------------------------
                 ! Save out P(Ox) and L(Ox) from the fullchem simulation
                 ! for a future tagged O3 run
                 !--------------------------------------------------------
                 IF ( Input_Opt%DO_SAVE_O3 ) THEN
                    IF ( TRIM(FAM_NAMES(F)) == 'POx' ) THEN
                       POx(I,J,L) = FAM(F) / DT
                       POx(I,J,L) = VAR(KppID) / DT
                    ENDIF
                    IF ( TRIM(FAM_NAMES(F)) == 'LOx' ) THEN
                       LOx(I,J,L) = FAM(F) / DT
                       LOx(I,J,L) = VAR(KppID) / DT
                    ENDIF
                 ENDIF

    #if defined( TOMAS )
                 !-------------------------------------------------------
                 ! FOR TOMAS MICROPHYSICS:
                 !
                 ! Obtain P/L with a unit [kg S] for tracing 
                 ! gas-phase sulfur species production (SO2, SO4, MSA)
                 ! (win, 8/4/09)
                 !-------------------------------------------------------

                 ! Calculate H2SO4 production rate [kg s-1] in each
                 ! time step (win, 8/4/09)
                 IF ( TRIM(FAM_NAMES(F)) == 'PSO4' ) THEN 
                    ! Hard-coded MW
                    H2SO4_RATE(I,J,L) = FAM(F) / AVO * 98.e-3_fp * &
                    H2SO4_RATE(I,J,L) = VAR(KppID) / AVO * 98.e-3_fp * &
                                        State_Met%AIRVOL(I,J,L)  * &
                                        1e+6_fp / DT
                 ENDIF
    #endif
              ENDDO

           ENDIF

--Melissa Sulprizio (talk) 20:12, 23 March 2017 (UTC)


The table below shows the speedup that is obtained with the improved prod/loss rate diagnostics.

Run name,
timesteps,
and submitter
Machine or Node
and Compiler
CPU vendor CPU model Speed [MHz] # of
CPUs
CPU time Wall time CPU / Wall
ratio
% of ideal
v11-01-public
(C20T10)
Bob Yantosca
regal16.rc.fas.harvard.edu
ifort 11.1
GenuineIntel Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz 2199.915 8 62554.07 s
17:22:34
9355.80 s
02:35:59
6.6861 83.58
v11-02a-P/L
(C20T10)
Melissa Sulprizio
regal17.rc.fas.harvard.edu
ifort 11.1
GenuineIntel Intel(R) Xeon(R) CPU E5-2660 0 @ 2.20GHz 2199.993 8 55853.62 s
15:30:54
8373.14 s
02:19:44
1.12X faster than v11-01-public
6.6706 83.38

--Bob Yantosca (talk) 15:58, 9 February 2017 (UTC)

In GEOS-Chem v11-01

NOTE: The way that KPP computes production and loss families was greatly simplified in GEOS-Chem v11-02a. See the post above for more information.

The number of prod/loss families is defined by NFAM in gckpp_Parameters.F90. KPP automatically sets this number according to the number of families listed in the gckpp.kpp file used to build the mechanism. The prod/loss family names are defined in array FAM_NAMES in gckpp_Monitor.F90. For example, for the default prod/loss families listed above we have:

  CHARACTER(LEN=15), PARAMETER, DIMENSION(5) :: FAM_NAMES = (/ &
     'POx            ','LOx            ','PCO            ', & ! index 1 - 3
     'LCO            ','PSO4           ' /)

KPP computes the prod/loss families in routine ComputeFamilies (found in gckpp_Util.F90). The FAM array in that routine contains the accumulated prod/loss rates for that family. For example, the POx family (family 1) is computed as:

  FAM(1) = V(13)+V(14)+V(15)+V(16)+V(17)+V(18)+V(19)+V(20)+V(21)+2*V(22)+...

The V array here corresponds to KPP's VAR array containing species concentrations in molec/cm3. The indices passed to V indicate which reaction rates are used to compute the prod/loss rate for that family. Here are the steps you can take to validate the reactions used for a given family:

  1. Select any index number for array V used in the computation
  2. Find the index number in SPC_NAMES (defined in gckpp_Monitor.F90)
  3. Find the dummy RR species in EQN_NAMES (defined in gckpp_Monitor.F90)
  4. Make sure the reaction identified in the previous step includes a family member as a product (for prod families) or a reactant (for loss families)

Following these steps for the POx family example, we have:

  1. Select V(13)
  2. Index 13 corresponds to RR10 in SPC_NAMES
  3. RR10 is found in HO2 + NO --> RR10 + NO2 + OH in EQN_NAMES
  4. NO2 is a member of the POx family and is a product in the above reaction ✔

--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)

Saving out production & loss rates

Binary punch format

The prod/loss rates from KPP are saved out in GEOS-Chem via the ND65 diagnostic. To activate this diagnostic, set the option for ND65 (stored in Input_Opt%DO_SAVE_PL) to T in the input.geos file:

  -------------------------------------------------------------------------------
  %%% PROD & LOSS MENU %%%:
  Turn on P/L (ND65) diag?: T
  # of levels for ND65    : 72
  Save O3 P/L (ND20)?     : F
  ------------------------+------------------------------------------------------

When Input_Opt%DO_SAVE_PL is true, FlexChem will call KPP routine ComputeFamilies (found in gckpp_Util.F90) to obtain the prod/loss rates stored in array FAM. The ND65 diagnostic is set up to automatically save out NFAM families to the output file and obtain the family names from FAM_NAMES, so no code modifications are required when adding prod/loss families to the chemical mechanism.

--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)

NetCDF format

GEOS-Chem v11-02 introduces the option to save GEOS-Chem diagnostics to netCDF format by compiling with NC_DIAG=y. To save out the prod/loss diagnostics to netCDF format, you can add the following field names to your defined collection(s) in HISTORY.rc:

  'Prod_?PRD?',                  'GIGCchem',  
  'Loss_?LOS?',                  'GIGCchem',  

where ?PRD? and ?LOS? are wildcards that will be placed with all production and loss species as defined in GEOS-Chem. NOTE: GCHP doesn't accept wildcards at this time, so each prod/loss species will need to be listed separately.

--Melissa Sulprizio (talk) 17:21, 6 February 2018 (UTC)

Validation

Benchmark simulations

FlexChem was implemented in v11-01g and validated with a 1-month benchmark and a 1-year benchmark. Please see the following links for more information:

  1. Approval form for 1-month benchmark simulation v11-01g
  2. Results for 1-year benchmark simulation v11-01g-Run0

--Melissa Sulprizio (talk) 18:38, 9 November 2016 (UTC)

Preliminary timing tests

Seven-day time tests were performed with the FlexChem code in order to evaluate model performance. The time tests all used 4° x 5° GEOS-FP met fields and were assigned 8 CPUs. Two model version were evaluated: v11-01f and v11-01g (which includes FlexChem). The results for the standard mechanism and tropchem mechanism are summarized below.


Standard Chemistry Mechanism

Model Version Wall Time CPU Time CPU Time / Wall Time Memory
v11-01f 1:43:15 10:55:22 6.3474 (79.34% ideal) 6.32 GB
v11-01g 2:10:37 14:19:33 6.5807 (82.26% ideal) 5.00 GB


Tropchem Chemistry Mechanism

Model Version Wall Time CPU Time CPU Time / Wall Time Memory
v11-01f 0:48:23 5:15:25 6.5191 (81.49% ideal) 3.57 GB
v11-01g 0:50:24 5:37:04 6.6878 (83.60% ideal) 2.70 GB

The GEOS-Chem Support Team will continue to evaluate and improve the performance of the FlexChem code for the GEOS-Chem v11-01 release.

--Melissa Sulprizio (talk) 16:26, 7 September 2016 (UTC)

Previous issues that are now resolved

FlexChem bug fix: do not zero ACTA, EOH, HCOOH

This fix will be included in GEOS-Chem 12.0.0.

Katie Travis wrote:

I am working on a VOC simulation, and noticed that in my copy of v11-02f, the following species are set to zero in two places:
     ! Zero certain species
     C(ind_ACTA) = 0.e0_dp
     C(ind_EOH) = 0.e0_dp
     C(ind_HCOOH) = 0.e0_dp
And
  C(ind_ACTA)  = 0.0_dp
  C(ind_HCOOH) = 0.0_dp
Since none of these species are fixed in Tropchem.eqn, shouldn’t they NOT be set to zero?

Mike Long wrote:

I think the code should be removed. This must have been a patch added to maintain parity with SMVGEAR w/o anticipating that the species would become active.

--Bob Yantosca (talk) 16:19, 17 May 2018 (UTC)

Fix for compiling with CHEM=Custom

This fix was included in GEOS-Chem v11-01 public release

Prasad Kasibhatla wrote:

I am trying to create a custom simulation with some chemistry added to the SOA_SVPOA simulation. I followed all the steps in the wiki to create the gckpp*F90 files and copy them to the Code.v11-01/KPP/Custom directory. I then compile from Code.v11-01 using the command
  make -j4 MET=geosfp GRID=4x5 CHEM=Custom
I noticed that during the compile, the KPP files being compiled are in the Standard KPP directory. So it looks like the CHEM=Custom option is not directing the compiler to the right place.
I solved the problem by adding the following to Makefile_header.mk in the Code directory:
  # %%%%% Test if Custom %%%%%
  REGEXP               :=(^[Cc][Uu][Ss][Tt][Oo][Mm])
  ifeq ($(shell [[ "$(CHEM)" =~ $(REGEXP) ]] && echo true),true)
     KPP_CHEM           :=Custom
     IS_CHEM_SET        :=1
  endif

--Melissa Sulprizio (talk) 22:43, 17 January 2017 (UTC)

Remove calls to UPDATE_SUN, UPDATE_RCONST from gckpp_Integrator.F90

This update was included in GEOS-Chem v11-01 provisional release

A slow down in GEOS-Chem run time was observed following the implementation of FlexChem in v11-01. To resolve this, a temporary workaround was implemented. This fix may be replaced with a more permanent solution in GEOS-Chem v11-01 public release.

Bob Yantosca wrote:

KPP automatically places calls to UPDATE_SUN and UPDATE_RCONST in the routines FunTemplate and JacTemplate (both in the gckpp_Integrator.F90 module). This assumes that you are not interfacing KPP into any other model, and that you will use KPP to compute the sun angles at each timestep.

We now call UPDATE_RCONST once per grid box before calling the KPP integrator. Also, because we use FAST-JX to get the photo rates, we no longer need to call UPDATE_SUN. These duplicate calls were causing a performance bottleneck, as UPDATE_RCONST was being called more than 7 million times per day.

We have removed the duplicate calls from the gckpp_Integrator.F90 modules in each of the chemistry mechanisms. But we will also need to make sure that when building KPP fresh from an equation file, that this step gets automatically added to the build sequence.

--Melissa Sulprizio (talk) 19:53, 9 January 2017 (UTC)

Incorrect species units used in routines UCX_NOX and UCX_H2SO4PHOT

This update was validated with the 1-month benchmark simulation v11-01j and approved on 03 Dec 2016

Seb Eastham reported an error in FlexChem_mod.F90 routine DO_FLEXCHEM, where UCX routines UCX_NOX and UCX_H2SO4PHOT are called when species concentrations are in the wrong units. Both routines expect species concentrations in units of kg but species are not converted from molec/cm3 to kg until after the calls.

Seb Eastham wrote:

The consequences of this error can actually be seen in the v11-01g benchmark zonal mean concentrations. N2O should be uniform in the trop and then lost in the strat, but if you check out the color bar you’ll see that there is a very difficult-to-see (but huge!) maximum in the mesosphere. This is why the concentration in the trop, which should be the maximum, is so low on the color scale.

To correct this issue, the UCX routines are now called after the species unit conversion to kg.

--Lizzie Lundgren (talk) 22:01, 14 November 2016 (UTC)

Known issues

TBD