FlexChem
On this page we provide information about the FlexChem chemical solver, which was introduced in GEOS-Chem v11-01g.
Contents
Overview
The clean and flexible reimplementation of the Kinetic PreProcessor package (aka KPP)—known as FlexChem—is nearing full integration into GEOS-Chem. Most of the remaining FlexChem development work will focus on replacing legacy infrastructure—which prevents GEOS-Chem from operating efficiently in high-performance computing (HPC) environments—with newer, more efficient algorithms.
Check back soon for the latest information!
--Bob Yantosca (talk) 19:58, 19 May 2016 (UTC)
Milestones
The following table shows several milestones that were achieved in the FlexChem implementation, development, as well as ongoing development tasks.
| Task | Developer | Status |
|---|---|---|
| Add FlexChem into v11-01c | Mike Long | Completed 14 Dec 2015 |
| Enable the "Tropchem" mechanism | Mike Long | Completed 14 Dec 2015 |
| Restore the OH and HO2 diagnostics (ND43) | Melissa Sulprizio | Completed 18 Dec 2015 |
| Remove CSPEC array and replaced with State_Chm%Species | Melissa Sulprizio | Completed 22 Dec 2015 |
| Enable a temporary workaround for family tracers (ISOPN, MMN, CFCX, HCFCX) | Mike Long | Completed 25 Jan 2016 |
| Enable FAST-JX photochemistry | Mike Long | Completed 25 Jan 2016 |
| Enable the "Benchmark" (aka "Standard") chemistry mechanism | Melissa Sulprizio | Completed 29 Jan 2016 |
| Restore the broken J-value diagnostic (ND22) | Melissa Sulprizio | Completed 15 Mar 2016 |
| Parallelize the main KPP driver loop; fixed other minor issues | Bob Yantosca | Completed 30 Mar 2016 |
| Enable the "SOA" and "SOA-SVPOA" mechanisms | Lizzie Lundgren | Completed 01 Apr 2016 |
| Merge FlexChem code based on v11-01c with v11-01f | Melissa Sulprizio | Completed 20 Apr 2016 |
| Add KPP repository to Bitbucket (https://bitbucket.org/gcst/kpp) | Mike Long | Completed 19 Apr 2016 |
| Create new gckpp_*.F90 files from the updated KPP solver package | Melissa Sulprizio | Completed 22 Apr 2016 |
| Enable the "UCX" mechanism | Melissa Sulprizio | Completed 26 Apr 2016 |
| Introduce a new prod/loss diagnostic into the KPP solver package | Mike Long | Completed 29 Apr 2016 |
| Complete unit tests and 1-month benchmarks for the Standard, Tropchem, UCX, SOA, and SOA-SVPOA simulations | Melissa Sulprizio Lizzie Lundgren |
Completed 05 May 2016 |
| Remove tracer indices of Rn, Pb, Be, Hg, and POPs simulations from tracerid_mod.F. (This paves the way for us to retire tracerid_mod.F.) | Bob Yantosca | Completed 02 May 2016 |
| Add a fast species name lookup algorithm | Bob Yantosca | Code is in place as of 04 May 2016, but has not been implemented throughout GEOS-Chem yet. |
| Store the unique list of GEOS-Chem species in the GEOS-Chem species database object. (In other words, we combine the list of advected tracers with the list of KPP chemical species and remove duplicate entries). | Bob Yantosca | Completed 09 May 2016 |
| Store the indices of each species in the KPP chemical reaction matrix in the GEOS-Chem species database object. | Bob Yantosca | Completed 09 May 2016 |
| Merge FlexChem with other v11-01g updates and bug fixes | Bob Yantosca | Completed 17 May 2016 |
| Remove family tracer fields of the Input_Opt object, namely: TRACER_N_CONST, TRACER_CONST, TRACER_COEFF, ID_EMITTED. These were only used by SMVGEAR and are now obsolete. | Bob Yantosca | Completed 18 May 2016 |
| Create mapping vectors in the State_Chm object to store the ID numbers of species that are advected, dry-deposited, wet-deposited, and/or included in the KPP chemical mechanism | Bob Yantosca | Completed 19 May 2016 |
| Remove all 1-D loop indexing variables (JLOOP, KLOOP, KTLOOP, JLOP, IXSAVE, IYSAVE, IZSAVE) that were used by SMVGEAR | Melissa Sulprizio | Completed 19 May 2016 |
| Remove SMVGEAR input files (globchem.dat, mglob.dat) and associated subroutines (readchem.F, reader.F) | Melissa Sulprizio | Completed 30 Jun 2016 |
| Remove tracerid_mod.F and related tracer ID flags (IDTxxxx, IDxxxx). Instead, use the GEOS-Chem species database to retrieve species index from species name. | Jacob Group Code-A-Thon | Completed 24 Jun 2016 |
| Remove tracers from restart files and read/write restart file species only | Lizzie Lundgren | Completed 12 Jul 2016 |
| Create unit conversion routines for species that mimic existing routines for tracers or convert to/from molecules/cm3 | Lizzie Lundgren | Completed 25 Jul 2016 |
| Add State_Chm variable to track species units throughout GEOS-Chem (State_Chm%Spc_Units) | Lizzie Lundgren | Completed 25 Jul 2016 |
| Remove the remaining family tracers (ISOPN, MMN, CFCX, HCFCX). Henceforth, FlexChem will only work with individual species. | Melissa Sulprizio | Completed 28 Jul 2016 |
| Move the non-advected species initial unit conversion of background values from INIT_FLEXCHEM (called during chemistry) to READ_GC_RESTART (called during restart file read) | Lizzie Lundgren | Completed 4 Aug 2016 |
| Store species background concentrations values previously stored in globchem.dat in the species database | Lizzie Lundgren | Completed 11 Aug 2016 |
| Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms to use the new prod/loss functionality in KPP (i.e. FLUX on) | Melissa Sulprizio | Completed 13 Jul 2016 |
| Attach the new KPP prod/loss output to the GEOS-Chem prod/loss diagnostic (ND65) | Melissa Sulprizio | Completed 11 Aug 2016 |
| Remove State_Chm%Tracers and use species concentrations in State_Chm%Species instead | GCST | Completed 29 Aug 2016 |
| Validate FlexChem with v11-01g benchmarks | GCST | 1-month benchmark: Approved 14 Sep 2016 1-year benchmark: |
| Fix P/L diagnostics to ignore cycling reactions | GCST | TBD |
| Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms with Kppa | GCST | TBD |
| Update the GEOS-Chem Makefiles so that a fresh version of KPP will be built (using a custom chemical mechanism that you specify) each time GEOS-Chem is compiled. | GCST | TBD |
| Update GEOS-Chem documentation and user manual for v11-01 | GCST | TBD |
Adding chemical mechanisms in FlexChem
KPP source code
The KPP source code for FlexChem is currently available as a Git repository on Bitbucket. It can be obtained using:
git clone https://bitbucket.org/gcst/kpp
Once you have KPP on your system, you will need to build it:
cd kpp/kpp-2.2.3_01 make clean make
If the build completed successfully, you should see the executable kpp in the kpp/kpp-2.2.3_01/bin/ directory. Next, add KPP to your PATH. For bash users, add the following lines the ~/.bashrc file:
export PATH=$PATH:/PATH_TO_KPP/KPP/kpp-2.2.3_01/bin/ export KPP_HOME=/PATH_TO_KPP/KPP/kpp-2.2.3_01
--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)
Building a custom chemical mechanism
GEOS-Chem v11-01g has five pre-built chemical mechanisms: Standard, Tropchem, SOA, SOA-SVPOA, and UCX. These mechanisms are described on our GEOS-Chem chemistry mechanisms wiki page.
GEOS-Chem will eventually include the option to create a custom chemistry mechanism and automatically build that mechanism with KPP when GEOS-Chem is compiled. For now, users still need to build custom chemistry mechanisms manually. The KPP Git repository includes the following files:
geos2kpp_parser.pl gckpp.kpp rename_f90.sh copy_gckpp.sh
Note: You'll need to checkout the GC_updates branch to access these files. This can be achieved with git checkout GC_updates.
These files can be used to build a custom chemistry mechanisms with KPP. Follow these steps:
- The geos2kpp_parser.pl script can be used to create globchem.eqn, globchem.spc, and globchem.def files from a globchem.dat file. You may also choose to start from the preexisting globchem.eqn, globchem.spc, and globchem.def files found in the KPP subdirectories of the GEOS-Chem source code (e.g. Code.v11-01/KPP/Standard/).
- If needed, edit globchem.eqn to add/modify reactions for your mechanism and edit globchem.spc to add/modify species to your mechanism. Put the final globchem.* files in the same directory as the gckpp.kpp file.
- Execute KPP using
kpp gckpp.kpp. This will create the necessary gckpp_*.f90 files. - Run
rename_f90.shto rename gckpp_*.f90 to gckpp_*.F90. - Open copy_gckpp.sh and modify the line for codedir. Save and run
copy_gckpp.sh. This will copy the gckpp_*.F90 files to codedir/Custom/. - Compile GEOS-Chem for the new mechanism by including CHEM=Custom in your make command.
--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)
Production & loss diagnostic
Mike Long implemented the functionality for chemical prod/loss families in the KPP source code. With this option turned on, KPP will calculate the net prod/loss rates for user-defined chemical families.
Adding production & loss families
KPP prod/loss families are turned on by the #FAMILIES token in the gckpp.kpp file. For example:
#INTEGRATOR rosenbrock
#LANGUAGE Fortran90
#DRIVER none
#INCLUDE globchem.spc
#INCLUDE globchem.eqn
#FAMILIES
POx : O3 + NO2 + 2NO3 + PAN + PMN + PPN + HNO4 + 3N2O5 + HNO3 + BrO + HOBr + BrNO2 + 2BrNO3 + MPN;
LOx : O3 + NO2 + 2NO3 + PAN + PMN + PPN + HNO4 + 3N2O5 + HNO3 + BrO + HOBr + BrNO2 + 2BrNO3 + MPN;
PCO : CO;
LCO : CO;
PSO4 : SO4;
To add a new prod/loss family, add a new line to the #FAMILIES section with the format
FAM_NAME : MEMBER_1 + MEMBER_2 + ... + MEMBER_N;
The family name must start with P or L to indicate whether KPP should calculate a production or a loss rate.
IMPORTANT: When adding a prod/loss family or changing any of the other settings in gckpp.kpp, the chemistry mechanism will need to be rebuilt with KPP as described above.
The pre-built chemistry mechanisms (Standard, Tropchem, SOA, SOA-SVPOA, and UCX) were built with the default prod/loss families listed in the example above. The Ox and CO rates are used for computing budgets in our standard 1-month benchmark simulations and PSO4 is required for simulations using TOMAS aerosol microphysics.
--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)
How it works
The number of prod/loss families is defined by NFAM in gckpp_Parameters.F90. KPP automatically sets this number according to the number of families listed in the gckpp.kpp file used to build the mechanism. The prod/loss family names are defined in array FAM_NAMES in gckpp_Monitor.F90. For example, for the default prod/loss families listed above we have:
CHARACTER(LEN=15), PARAMETER, DIMENSION(5) :: FAM_NAMES = (/ &
'POx ','LOx ','PCO ', & ! index 1 - 3
'LCO ','PSO4 ' /)
KPP computes the prod/loss families in routine ComputeFamilies (found in gckpp_Util.F90). The FAM array in that routine contains the accumulated prod/loss rates for that family. For example, the POx family (family 1) is computed as:
FAM(1) = V(13)+V(14)+V(15)+V(16)+V(17)+V(18)+V(19)+V(20)+V(21)+2*V(22)+...
The V array here corresponds to KPP's VAR array containing species concentrations in molec/cm3. The indices passed to V indicate which reaction rates are used to compute the prod/loss rate for that family. Here are the steps you can take to validate the reactions used for a given family:
- Select any index number for array V used in the computation
- Find the index number in SPC_NAMES (defined in gckpp_Monitor.F90)
- Find the dummy RR species in EQN_NAMES (defined in gckpp_Monitor.F90)
- Make sure the reaction identified in the previous step includes a family member as a product (for prod families) or a reactant (for loss families)
Following these steps for the POx family example, we have:
- Select V(13)
- Index 13 corresponds to RR10 in SPC_NAMES
- RR10 is found in HO2 + NO --> RR10 + NO2 + OH in EQN_NAMES
- NO2 is a member of the POx family and is a product in the above reaction ✔
--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)
Saving out production & loss rates
The prod/loss rates from KPP are saved out in GEOS-Chem via the ND65 diagnostic. To activate this diagnostic, set the option for ND65 (stored in Input_Opt%DO_SAVE_PL) to T in the input.geos file:
------------------------------------------------------------------------------- %%% PROD & LOSS MENU %%%: Turn on P/L (ND65) diag?: T # of levels for ND65 : 72 Save O3 P/L (ND20)? : F ------------------------+------------------------------------------------------
When Input_Opt%DO_SAVE_PL is true, FlexChem will call KPP routine ComputeFamilies (found in gckpp_Util.F90) to obtain the prod/loss rates stored in array FAM. The ND65 diagnostic is set up to automatically save out NFAM families to the output file and obtain the family names from FAM_NAMES, so no code modifications are required when adding prod/loss families to the chemical mechanism.
--Melissa Sulprizio (talk) 17:13, 9 November 2016 (UTC)
Removal of family tracers
In GEOS-Chem v9-02 we removed all family tracers from GEOS-Chem. For example, the NOx family was replaced by individual tracers NO, O3, NO3, and HNO2. Similarly, the Ox family was replaced by individual tracers O3, NO2, and NO3.
But certain updates that were made to GEOS-Chem after v9-02 introduced new family tracers into the various chemistry mechanisms:
- The Caltech isoprene scheme added the ISOPN and MVKN families
- The UCX chemistry mechanism added the CFCX and HCFCX families
The implementation of the FlexChem chemical solver package in GEOS-Chem v11-01 forced us to remove these family tracers, because the KPP chemical solver is designed to only include individual species in its mechanism. Removing these families also allows us to avoid some cumbersome sections of code that were used to construct each family from its constituent members.
Therefore, the following families have been removed from GEOS-Chem v11-01g, and replaced with individual species:
| Removed | Added as individual advected species | Mechanisms in which these are used | ||
|---|---|---|---|---|
| ISOPN | ISOPND | ISOPNB | Standard, UCX, TropChem, SOA, SOA-SVPOA | |
| MVKN | MACRN | MVKN | Standard, UCX, TropChem, SOA, SOA-SVPOA | |
| CFCX | CFC113 | CFC114 | CFC115 | Standard, UCX |
| HCFCX | HCFC22 | HCFC141b | HCFC142b | Standard, UCX |
The constituents of each former family are now included as advected species in the various GEOS-Chem mechanisms. To obtain diagnostic output for the former families, you can save out each of the individual species and combine the output in post-processing.
--Bob Yantosca (talk) 16:20, 16 August 2016 (UTC)
Chemical species in FlexChem
Please see our Species in GEOS-Chem wiki page for more information.
--Melissa Sulprizio (talk) 22:26, 9 November 2016 (UTC)
Validation
Benchmark simulations
FlexChem was implemented in GEOS-Chem v11-01#v11-01g and validated with a 1-month benchmark and a 1-year benchmark. Please see the following links for more information:
- Approval form for 1-month benchmark simulation v11-01g
- Results for 1-year benchmark simulation v11-01g-Run0
--Melissa Sulprizio (talk) 18:38, 9 November 2016 (UTC)
Preliminary timing tests
Seven-day time tests were performed with the FlexChem code in order to evaluate model performance. The time tests all used 4° x 5° GEOS-FP met fields and were assigned 8 CPUs. Two model version were evaluated: v11-01f and v11-01g (which includes FlexChem). The results for the standard mechanism and tropchem mechanism are summarized below.
Standard Chemistry Mechanism
| Model Version | Wall Time | CPU Time | CPU Time / Wall Time | Memory |
|---|---|---|---|---|
| v11-01f | 1:43:15 | 10:55:22 | 6.3474 (79.34% ideal) | 6.32 GB |
| v11-01g | 2:10:37 | 14:19:33 | 6.5807 (82.26% ideal) | 5.00 GB |
Tropchem Chemistry Mechanism
| Model Version | Wall Time | CPU Time | CPU Time / Wall Time | Memory |
|---|---|---|---|---|
| v11-01f | 0:48:23 | 5:15:25 | 6.5191 (81.49% ideal) | 3.57 GB |
| v11-01g | 0:50:24 | 5:37:04 | 6.6878 (83.60% ideal) | 2.70 GB |
The GEOS-Chem Support Team will continue to evaluate and improve the performance of the FlexChem code for the GEOS-Chem v11-01 release.
--Melissa Sulprizio (talk) 16:26, 7 September 2016 (UTC)
Previous issues that are now resolved
Incorrect species units used in routines UCX_NOX and UCX_H2SO4PHOT
This issue will be fixed in v11-01j.
Seb Eastham reported an error in FlexChem_mod.F90 routine DO_FLEXCHEM, where UCX routines UCX_NOX and UCX_H2SO4PHOT are called when species concentrations are in the wrong units. Both routines expect species concentrations in units of kg but species are not converted from molec/cm3 to kg until after the calls.
Seb Eastham wrote:
The consequences of this error can actually be seen in the v11-01g benchmark zonal mean concentrations. N2O should be uniform in the trop and then lost in the strat, but if you check out the color bar you’ll see that there is a very difficult-to-see (but huge!) maximum in the mesosphere. This is why the concentration in the trop, which should be the maximum, is so low on the color scale.
To correct this issue, the UCX routines are now called after the species unit conversion to kg.
--Lizzie Lundgren (talk) 22:01, 14 November 2016 (UTC)
Known issues
TBD
