FlexChem

On this page we provide information about the FlexChem chemical solver, which was introduced in GEOS-Chem v11-01g.

Overview

The clean and flexible reimplementation of the Kinetic PreProcessor package (aka KPP)—known as FlexChem—is nearing full integration into GEOS-Chem. Most of the remaining FlexChem development work will focus on replacing legacy infrastructure—which prevents GEOS-Chem from operating efficiently in high-performance computing (HPC) environments—with newer, more efficient algorithms.

Check back soon for the latest information!

--Bob Yantosca (talk) 19:58, 19 May 2016 (UTC)

Milestones

The following table shows several milestones that were achieved in the FlexChem implementation, development, as well as ongoing development tasks.

Task	Developer	Status
Add FlexChem into v11-01c	Mike Long	Completed 14 Dec 2015
Enable the "Tropchem" mechanism	Mike Long	Completed 14 Dec 2015
Restore the OH and HO2 diagnostics (ND43)	Melissa Sulprizio	Completed 18 Dec 2015
Remove CSPEC array and replaced with State_Chm%Species	Melissa Sulprizio	Completed 22 Dec 2015
Enable a temporary workaround for family tracers (ISOPN, MMN, CFCX, HCFCX)	Mike Long	Completed 25 Jan 2016
Enable FAST-JX photochemistry	Mike Long	Completed 25 Jan 2016
Enable the "Benchmark" (aka "Standard") chemistry mechanism	Melissa Sulprizio	Completed 29 Jan 2016
Restore the broken J-value diagnostic (ND22)	Melissa Sulprizio	Completed 15 Mar 2016
Parallelize the main KPP driver loop; fixed other minor issues	Bob Yantosca	Completed 30 Mar 2016
Enable the "SOA" and "SOA-SVPOA" mechanisms	Lizzie Lundgren	Completed 01 Apr 2016
Merge FlexChem code based on v11-01c with v11-01f	Melissa Sulprizio	Completed 20 Apr 2016
Add KPP repository to Bitbucket (https://bitbucket.org/gcst/kpp)	Mike Long	Completed 19 Apr 2016
Create new gckpp_*.F90 files from the updated KPP solver package	Melissa Sulprizio	Completed 22 Apr 2016
Enable the "UCX" mechanism	Melissa Sulprizio	Completed 26 Apr 2016
Introduce a new prod/loss diagnostic into the KPP solver package	Mike Long	Completed 29 Apr 2016
Complete unit tests and 1-month benchmarks for the Standard, Tropchem, UCX, SOA, and SOA-SVPOA simulations	Melissa Sulprizio Lizzie Lundgren	Completed 05 May 2016
Remove tracer indices of Rn, Pb, Be, Hg, and POPs simulations from tracerid_mod.F. (This paves the way for us to retire tracerid_mod.F.)	Bob Yantosca	Completed 02 May 2016
Add a fast species name lookup algorithm	Bob Yantosca	Code is in place as of 04 May 2016, but has not been implemented throughout GEOS-Chem yet.
Store the unique list of GEOS-Chem species in the GEOS-Chem species database object. (In other words, we combine the list of advected tracers with the list of KPP chemical species and remove duplicate entries).	Bob Yantosca	Completed 09 May 2016
Store the indices of each species in the KPP chemical reaction matrix in the GEOS-Chem species database object.	Bob Yantosca	Completed 09 May 2016
Merge FlexChem with other v11-01g updates and bug fixes	Bob Yantosca	Completed 17 May 2016
Remove family tracer fields of the Input_Opt object, namely: TRACER_N_CONST, TRACER_CONST, TRACER_COEFF, ID_EMITTED. These were only used by SMVGEAR and are now obsolete.	Bob Yantosca	Completed 18 May 2016
Create mapping vectors in the State_Chm object to store the ID numbers of species that are advected, dry-deposited, wet-deposited, and/or included in the KPP chemical mechanism	Bob Yantosca	Completed 19 May 2016
Remove all 1-D loop indexing variables (JLOOP, KLOOP, KTLOOP, JLOP, IXSAVE, IYSAVE, IZSAVE) that were used by SMVGEAR	Melissa Sulprizio	Completed 19 May 2016
Remove SMVGEAR input files (globchem.dat, mglob.dat) and associated subroutines (readchem.F, reader.F)	Melissa Sulprizio	Completed 30 Jun 2016
Remove tracerid_mod.F and related tracer ID flags (IDTxxxx, IDxxxx). Instead, use the GEOS-Chem species database to retrieve species index from species name.	Jacob Group Code-A-Thon	Completed 24 Jun 2016
Remove tracers from restart files and read/write restart file species only	Lizzie Lundgren	Completed 12 Jul 2016
Create unit conversion routines for species that mimic existing routines for tracers or convert to/from molecules/cm3	Lizzie Lundgren	Completed 25 Jul 2016
Add State_Chm variable to track species units throughout GEOS-Chem (State_Chm%Spc_Units)	Lizzie Lundgren	Completed 25 Jul 2016
Remove the remaining family tracers (ISOPN, MMN, CFCX, HCFCX). Henceforth, FlexChem will only work with individual species.	Melissa Sulprizio	Completed 28 Jul 2016
Move the non-advected species initial unit conversion of background values from INIT_FLEXCHEM (called during chemistry) to READ_GC_RESTART (called during restart file read)	Lizzie Lundgren	Completed 4 Aug 2016
Store species background concentrations values previously stored in globchem.dat in the species database	Lizzie Lundgren	Completed 11 Aug 2016
Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms to use the new prod/loss functionality in KPP (i.e. FLUX on)	Melissa Sulprizio	Completed 13 Jul 2016
Attach the new KPP prod/loss output to the GEOS-Chem prod/loss diagnostic (ND65)	Melissa Sulprizio	Completed 11 Aug 2016
Remove State_Chm%Tracers and use species concentrations in State_Chm%Species instead	GCST	Completed 29 Aug 2016
Validate FlexChem with v11-01g benchmarks	GCST	In progress
Fix P/L diagnostics to ignore cycling reactions	GCST	TBD
Rebuild Standard, Tropchem, UCX, SOA, SOA-SVPOA chemistry mechanisms with Kppa	GCST	TBD
Update the GEOS-Chem Makefiles so that a fresh version of KPP will be built (using a custom chemical mechanism that you specify) each time GEOS-Chem is compiled.	GCST	TBD
Update GEOS-Chem documentation and user manual for v11-01	GCST	TBD

Adding chemical mechanisms in FlexChem

KPP source code

The KPP source code for FlexChem is currently available as a Git repository on Bitbucket. It can be obtained using:

git clone https://bitbucket.org/gcst/kpp

Once you have KPP on your system, you will need to build it:

cd kpp/kpp-2.2.3_01
make clean
make

If the build completed successfully, you should see the executable kpp in the kpp/kpp-2.2.3_01/bin/ directory. Next, add KPP to your PATH. For bash users, add the following lines the ~/.bashrc file:

export PATH=$PATH:/PATH_TO_KPP/KPP/kpp-2.2.3_01/bin/
export KPP_HOME=/PATH_TO_KPP/KPP/kpp-2.2.3_01

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Building a custom chemical mechanism

GEOS-Chem v11-01g has five pre-built chemical mechanisms: Standard, Tropchem, SOA, SOA-SVPOA, and UCX. These mechanisms are described on our GEOS-Chem chemistry mechanisms wiki page.

GEOS-Chem will eventually include the option to create a custom chemistry mechanism and automatically build that mechanism with KPP when GEOS-Chem is compiled. For now, users still need to build custom chemistry mechanisms manually. The KPP Git repository includes the following files:

 geos2kpp_parser.pl
 gckpp.kpp
 rename_f90.sh
 copy_gckpp.sh

These files can be used to build a custom chemistry mechanisms with KPP. Follow these steps:

1. The geos2kpp_parser.pl script can be used to create globchem.eqn, globchem.spc, and globchem.def files from a globchem.dat file. You may also choose to start from the preexisting globchem.eqn, globchem.spc, and globchem.def files found in the KPP subdirectories of the GEOS-Chem source code (e.g. Code.v11-01/KPP/Standard/).
2. If needed, edit globchem.eqn to add/modify reactions for your mechanism and edit globchem.spc to add/modify species to your mechanism. Put the final globchem.* files in the same directory as the gckpp.kpp file.
3. Execute KPP using kpp gckpp.kpp. This will create the necessary gckpp_*.f90 files.
4. Run rename_f90.sh to rename gckpp_*.f90 to gckpp_*.F90.
5. Open copy_gckpp.sh and modify the line for codedir. Save and run copy_gckpp.sh. This will copy the gckpp_*.F90 files to codedir/Custom/.
6. Compile GEOS-Chem for the new mechanism by including CHEM=Custom in your make command.

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Production & loss diagnostic

Mike Long implemented prod/loss structure parsing in the KPP source code. This option can be turned on by adding the line #FLUX on to the gckpp.kpp file. For example:

  #INTEGRATOR rosenbrock
  #LANGUAGE Fortran90
  #DRIVER none
  #HESSIAN off
  #MEX off
  #STOICMAT off
  #FLUX on
  
  #INCLUDE globchem.spc
  #INCLUDE globchem.eqn

For the FLUX option to work properly, MEX and HESSIAN must be turned off as shown above.

When changing any of the settings in gckpp.kpp, the chemistry mechanism will need to be rebuilt with KPP as described above.

The pre-built chemistry mechanisms (Standard, Tropchem, SOA, SOA-SVPOA, and UCX) were built with the FLUX option turned on. The prod/loss rates from KPP can be saved out in GEOS-Chem via the ND65 diagnostic.

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Inefficiencies in prod/loss code

The current implementation of the prod/loss diagnostic output in KPP is somewhat inefficient. The code is set up so that KPP calculates the prod/loss of ALL species, even though a user may only want prod/loss rates for a few species. This likely adds considerable memory overhead because KPP has NREAC additional species to track. Making this code more efficient is a task for future development.

--Melissa Sulprizio (talk) 21:18, 10 August 2016 (UTC)

Removal of family tracers

In GEOS-Chem v9-02 we removed all family tracers from GEOS-Chem. For example, the NOx family was replaced by individual tracers NO, O3, NO3, and HNO2. Similarly, the Ox family was replaced by individual tracers O3, NO2, and NO3.

But certain updates that were made to GEOS-Chem after v9-02 introduced new family tracers into the various chemistry mechanisms:

• The Caltech isoprene scheme added the ISOPN and MVKN families
• The UCX chemistry mechanism added the CFCX and HCFCX families

The implementation of the FlexChem chemical solver package in GEOS-Chem v11-01 forced us to remove these family tracers, because the KPP chemical solver is designed to only include individual species in its mechanism. Removing these families also allows us to avoid some cumbersome sections of code that were used to construct each family from its constituent members.

Therefore, the following families have been removed from GEOS-Chem v11-01g, and replaced with individual species:

Removed	Added as individual advected species			Mechanisms in which these are used
ISOPN	ISOPND	ISOPNB		Standard, UCX, TropChem, SOA, SOA-SVPOA
MVKN	MACRN	MVKN		Standard, UCX, TropChem, SOA, SOA-SVPOA
CFCX	CFC113	CFC114	CFC115	Standard, UCX
HCFCX	HCFC22	HCFC141b	HCFC142b	Standard, UCX

The constituents of each former family are now included as advected species in the various GEOS-Chem mechanisms. To obtain diagnostic output for the former families, you can save out each of the individual species and combine the output in post-processing.

--Bob Yantosca (talk) 16:20, 16 August 2016 (UTC)

Validation

Preliminary timing tests

Several 7-day time tests were performed with the FlexChem code in order to evaluate model performance. The time tests all used 4° x 5° GEOS-FP met fields and were assigned 8 CPUs. Three model version were evaluated: v11-01f, FlexChem without production/loss diagnostics enabled in KPP, and FlexChem with production/loss diagnostics enabled in KPP. The results for the standard mechanism and tropchem mechanism are summarized below.

Standard Chemistry Mechanism

Model Version	Wall Time	CPU Time	CPU Time / Wall Time	Memory
v11-01f	1:43:15	10:55:22	6.3474 (79.34% ideal)	6.32 GB
v11-01g	3:01:44	20:29:35	6.7659 (84.57% ideal)	5.02 GB

Tropchem Chemistry Mechanism

Model Version	Wall Time	CPU Time	CPU Time / Wall Time	Memory
v11-01f	0:48:23	5:15:25	6.5191 (81.49% ideal)	3.57 GB
v11-01g	1:02:26	7:13:38	6.9455 (86.82% ideal)	2.70 GB

The GEOS-Chem Support Team will continue to evaluate and improve the performance of the FlexChem code for the GEOS-Chem v11-01 release.

--Melissa Sulprizio (talk) 16:26, 7 September 2016 (UTC)

Known issues

TBD