Coupling GEOS-Chem with RRTMG

---

---

GEOS-Chem v11-02-final will also carry the designation GEOS-Chem 12.0.0. We are migrating to a purely numeric versioning system in order to adhere more closely to software development best practices. For a complete description of the new versioning system, please see our GEOS-Chem version numbering system wiki page.

---

---

On this page we provide information about the coupling of GEOS-Chem with the RRTMG radiative transfer model (by AER, Inc.).

Overview

The GEOS-Chem model with online radiative transfer calculations (referred to as GCRT) was developed to allow GEOS-Chem users to produce gas and aerosol direct radiative effect (DRE) output for both the longwave and shortwave. This alternative to offline coupling allows better temporal resolution in the RT calculations and provides a consist platform for GEOS-Chem users with the widely used radiative transfer package RRTMG.

Most of the added code is 'transparent', therefore this version of the GEOS-Chem model can still be run with the radiation code switched off. The optical properties are calculated at multiple wavelengths so that the user is no longer restricted to 550nm as default, so there are associated changes regardless of whether the radiative code is invoked. However, these cause negligible slow down (the default model is actually slightly faster than the standard v9-01-03). Compiling with RRTMG=yes requires approximately double the amount of RAM (~15Gb for a 2x2.5 simulation) and takes between 40% and 100% longer depending on the settings used.

This wiki explains the key changes and requirements. If you want to dig deeper into the changes and assumptions made, or are looking for a quick-start guide to running the model, download the GCRT User Guide

GCRT User Groups

User Group	Personnel	Projects
Atmospheric Chemistry, Massachusetts Institute of Technology	Colette Heald David Ridley Xuan Wang	GCRT Publication Inter-annual variability and trends in African dust Modeling black and brown carbon
Add yours here

Obtaining the source code

GCRT was originally added into GEOS-Chem v10-01i, which was tested with 1-month benchmark simulations approved on 30 April 2015 and a 1-year benchmark simulation approved on 01 May 2015. It is included in the GEOS-Chem v10-01 public release.

--Melissa Sulprizio (talk) 22:15, 16 June 2015 (UTC)

The following bug fix for GCRT will be added into GEOS-Chem v11-01a. Interested users should update to this version once it is ready. Contact the GEOS-Chem Support Team for more information.

--Bob Y. (talk) 16:11, 26 June 2015 (UTC)

Authors and collaborators

• David Ridley (Civil and Environmental Engineering, MIT) -- Lead Developer
• Colette Heald (Civil and Environmental Engineering, MIT) -- Principal Investigator
• Steven Barrett (Aeronautics and Astronautics, MIT)
• Karen Cady-Pereira (AER)
• Matthew Alvarado (AER)

Questions regarding GCRT can be directed at David (e-mail linked above).

--David R. 18:00, 08 October 2013 (EST)

Key Changes

1. A new menu for Radiation exists in input.geos (see below). This also includes a wavelength selection for optical depth output that is independent of whether RRTMG is switched on (up to three optical depths can be output in the ND21 (OD-MAP-$) diagnostic, but only the first is used for the timeseries diagnostics ND48, ND49, ND50, ND51, and ND51b).
   
   
2. A new code folder, GeosRad/ is required for the RRTMG code. The module rrtmg_rad_transfer_mod.F is the driver code (found in GeosCore/) that interfaces with RRTMG.
   
   
3. The optics look-up tables are updated, containing multiple wavelengths and separated into files for each species (soot.dat, so4.dat, org.dat, dust.dat, ssa.dat, ssc.dat) that are stored in the run directory. To prevent discrepancies jv_spec.dat no longer contains optical properties for aerosol, Fast-J uses aerosol optics from from the same speciated look-up tables. The new files mean that jv_spec_aod.dat is obsolete and has been removed.
   
   
4. Several new input files are required. Surface albedo and emissivity climatologies have been generated and must be stored in modis_surf_201210/ within the root data directory (e.g. ExtData/CHEM_INPUTS/). Climatologies of gases (CH4, N2O, CFC-11, CFC-12, CFC-22 and CCl4) must be stored within the root data directory.
   
   
5. A new set of diagnostics (ND72) are available providing the change in radiative flux (DRE) for gases and aerosol, LW/SW, top of atmosphere (TOA) and surface, and clear-sky and all-sky conditions. These also include AOD, SSA and asymmetry parameter for each aerosol species at the requested wavelengths.

Running GCRT

As with the latest version of GEOS-Chem, NetCDF libraries must be installed (see Installing libraries for GEOS-Chem). The code should be compiled using the compile switch RRTMG, i.e. make RRTMG=yes. A fresh compilation after a make realclean will take 15-20 min (subsequent compiles are much faster).

Once the code is compiled the input.geos file will need configuring:

%%% RADIATION MENU %%%  :
AOD Wavelength (nm)     : 550 440 532 
Turn on RRTMG?          : T
Calculate LW fluxes?    : T
Calculate SW fluxes?    : T
Clear-sky flux?         : T
All-sky flux?           : T
Radiation Timestep [min]: 180
Species fluxes          : 0 0 1 0 0 0 0 0 0 1
[O3,ME,SU,NI,AM,BC,OA,SS,DU,PM]

The species for which DRE output is currently available are listed with two-letter identifiers. In the example the DRE is calculated for sulfate (SU) and all particulate matter (PM) - see the table below for all identifier descriptions. The recommended radiation time step is 180 min but can be set for any multiple of the chemistry time step. The baseline (BA) flux is always output by default and is the flux (W/m^2) with all aerosol and species included.

In this set up all timeseries diagnostics (ND48, ND49, ND50, ND51, ND51b) produce optical depth output at 550nm, but the main ND21 diagnostic output contains three copies of the optical depths (and SSA and asymmetry parameters if requested). The output variables have the wavelength appended to the variable name e.g. OPBC550.

The species available for output from the flux calculations are as follows:

Abbreviation	Species
O3 ME SU NI AM BC OA SS DU PM ST BA	Ozone Methane Sulfate Nitrate Ammonium Black carbon Organic aerosol Sea salt Mineral dust All particulate matter Stratospheric aerosol (UCX simulation only) Baseline flux

Please note:

• The radiative impacts of gases (ozone and methane) are relatively untested at this stage and should be interpreted with caution.
• If you are using the UCX chemistry mechanism, then you will need to add an extra entry (0 or 1) at the end of the species fluxes line for stratospheric aerosols for a total of 11 possible species.

Finally, the ND72 diagnostic must be set to 1 if RRTMG is switched on.

Once all the auxiliary files are in place (new optics data files, surface albedo climatologies, trace gas profile climatologies) and the input.geos set up, the model can be run in the usual way.

For more information please consult the GCRT User Guide.

Example Output

Below, example output fluxes and direct radiative effect (DRE) from GCRT are displayed for March 2013. The top row show the total flux with all aerosol and gases included, subsequent rows show the DRE for all particulate matter, black carbon, and sulfate. The left column shows top of atmosphere (TOA) and the right column shows surface DRE. All results are for all-sky conditions (i.e. including clouds). The identifier for each figure is |F|SW|PM|T|A| (F - denotes a flux output, SW/LW - shortwave or longwave, PM - species identifier, T/S - TOA or surface, A/C - All-sky or clear sky)

Previous issues that are now resolved

Replace nonstandard DLOG function with LOG

This fix was included in v11-01j and approved on 03 Dec 2016

In routine GeosRad/rrtmg_lw_setcoef.F, the GNU Fortran compiler detected an error using the specific REAL*8 DLOG function. We have now converted this to the standard LOG function.

We commented out the code in RED and added the code in GREEN:

!------------------------------------------------------------------------------
! Prior to 10/14/16: 
! Use the F90 standard LOG function instead of ALOG or DLOG.  (bmy, 10/14/16)
!!         plog = alog(pavel(lay))
!         plog = dlog(pavel(lay))
!------------------------------------------------------------------------------
         plog = LOG(pavel(lay))

--Bob Yantosca (talk) 19:38, 17 October 2016 (UTC)

Bug fixes in rrtmg_rad_transfer_mod.F

These fixes were validated with 1-month benchmark simulation v11-01b and 1-year benchmark simulation v11-01b-Run0. This version was approved on 19 Aug 2015.

David Ridley wrote:

Seb has kindly found and fixed a couple of bugs in the RRTMG code that have crept in recently. These do mess up the results and so will need implementing in the model as soon as possible. I’ll let anyone that I know currently using RRTMG to re-download the code once it has been fixed.

In GeosCore/rrtmg_rad_transfer_mod.F, around line 1011 change the following lines in red to the lines in green:

              IF (ASMAERDIAG(IJLOOP,L,IB_SW).GT.0.999d0) THEN
               ASMAER(IJLOOP,L,IB_SW) = 0.999d0
               ASMAERDIAG(IJLOOP,L,IB_SW) = 0.999d0
              ENDIF
!              IF (ASMAERDIAG(IJLOOP,L,IB_SW).LT.0.001d0) THEN
!               ASMAER(IJLOOP,L,IB_SW) = 0.001
!               ASMAERDIAG(IJLOOP,L,IB_SW) = 0.001
!              ENDIF
              IF ((SSAAERDIAG(IJLOOP,L,IB_SW).LT.0.001d0).OR.
     &            (SSAAERDIAG(IJLOOP,L,IB_SW).GT.1.0d0)) THEN
               SSAAER(IJLOOP,L,IB_SW) = 0.99
               SSAAERDIAG(IJLOOP,L,IB_SW) = 0.99
              ENDIF
              IF (TAUAERDIAG(IJLOOP,L,IB_SW).GT.1.0) THEN
               TAUAER_SW(IJLOOP,L,IB_SW) = 1.0
               TAUAERDIAG(IJLOOP,L,IB_SW) = 1.0
              ENDIF

In the same routine around line 1370, change the following line from:

            p_SSAAER     = TAUAER_LW (  IL,:,:)

to:

            p_SSAAER     = SSAAER    ( IL,:,:)

--Melissa Sulprizio (talk) 22:14, 21 July 2015 (UTC)

Fix bug that prevents storage of O3 and CH4 RFs

This fix was validated in the 1-month benchmark simulation v11-01a, which was approved on 07 Jul 2015.

Sebastian Eastham wrote:

(GEOS-Chem v10-01 contains) an issue in the RRTMG interface where there is a single-character typo in SET_SPECMASK. This typo results in dust, ozone and methane RFs being incorrectly stored - in fact it prevents O3 and CH4 RFs from being stored at all, instead storing dust RFs in their place (which in turn miss the last two dust bins and instead use either sea salt or stratospheric aerosol). This is because line 1609 in rrtmg_rad_transfer_mod reads

 NXTRA=NASPECRAD-16                                                

When it should read

 NXTRA=NSPECRAD-16                              

(note second character). This bug affects both the tropospheric and UCX version of the code. Since NXTRA is the offset used to account for whether or not stratospheric aerosols are inserted mid-way through the indexing, and NASPECRAD is always 2 smaller than NSPECRAD, everything index above 7 is set to a value 2 smaller than it should be in the mask.

This bug occurs with or without UCX. The offset is always incorrectly calculated, so dust, ozone and methane are never output correctly. The easiest test is to check the ozone or methane output (they will both look like large-diameter dust RF). Dust will look *almost* right, but will actually include stratospheric aerosols (if UCX=yes) or sea-salt (if UCX is not on).

--Melissa Sulprizio (talk) 20:27, 25 June 2015 (UTC)

Bug in RRTMG array sizes

This fix was validated with 1-month benchmark simulation v11-01b and 1-year benchmark simulation v11-01b-Run0. This version was approved on 19 Aug 2015.

Sebastian Eastham wrote:

I came across a bug in the RRTMG code, which looks like it originated with AER. In GeosRad/mcica_subcol_gen_lw.F90 and GeosRad/rrtmg_lw_rad.F90, a few arrays are incorrectly declared to be size (nbndlw,ncol,nlay) when they should instead by (ngptlw,ncol,nlay), or vice versa. This results in NaNs sometimes creeping into the arrays when OpenMP is turned on, or just outright failure.

The comments show that the arrays should have been declared differently in the first place, so I’ll be getting in touch with AER shortly to make sure they’re aware of the issue.

In GeosRad/mcica_subcol_gen_lw.F90, change the following code (in bold) to:

     real(kind=rb), intent(in) :: tauc(nbndlw,ncol,nlay)        ! in-cloud optical depth
                                                  !    Dimensions: (nbndlw,ncol,nlay)

In GeosRad/rrtmg_lw_rad.F90, change the following code (in bold) to:

     real(kind=rb), intent(in) :: cldfmcl(ngptlw,ncol,nlay)     ! Cloud fraction
                                                     !    Dimensions: (ngptlw,ncol,nlay)
     real(kind=rb), intent(in) :: ciwpmcl(ngptlw,ncol,nlay)     ! In-cloud ice water path (g/m2)
                                                     !    Dimensions: (ngptlw,ncol,nlay)
     real(kind=rb), intent(in) :: clwpmcl(ngptlw,ncol,nlay)     ! In-cloud liquid water path (g/m2)
                                                     !    Dimensions: (ngptlw,ncol,nlay)
     ...
     real(kind=rb), intent(in) :: taucmcl(ngptlw,ncol,nlay)     ! In-cloud optical depth
                                                     !    Dimensions: (ngptlw,ncol,nlay)

--Melissa Sulprizio (talk) 14:39, 6 July 2015 (UTC)

Known issues

Bug fixes for RRTMG in v11-02

This fix was included in GEOS-Chem 12.0.0.

Christopher Holmes wrote:

The RRTMG calculations have a bug in v11-01. The problem was probably introduced with the unit changes in v11-01. O3 and CH4 concentrations are specified incorrectly, so their radiative effects are badly wrong. Aerosol radiative effects are also affected due to spectral overlap with gases (whose concentrations are wrong), but the aerosol error is modest. Xinyi Dong (U Tennessee) tested my fix and found that the error in BC radiative effect is <1% in the global mean for a 1 month average, but locally up to 5%. Other aerosols likely have similar magnitude errors.

--Bob Yantosca (talk) 19:28, 14 May 2018 (UTC)

RRTMG does not compile with GNU Fortran

The GEOS-Chem Support Team has recently discovered that the RRTMG code in GEOS-Chem does not compile with the gfortran compiler. The compile error is included below for reference. While users still have the option to use the IFORT compilers, this is somewhat problematic for our movement towards using open-source software. For example, if users run GEOS-Chem on the Amazon Web Services cloud computing environment, we recommend that they use gfortran. We’ve also been using gfortran 7.1.0 for our unit tests because that compiler tends to be more strict.

We wanted to alert you to this unresolved issue in case anyone is interested in pursuing a fix with the folks at AER and/or updating RRTMG in GEOS-Chem. The GCST typically tries to remain hands-off with third party code (e.g. RRTMG, ISORROPIA) in GEOS-Chem. It appears that RRTMG v3.0 was implemented in GEOS-Chem. The latest RRTMG version is 3.3, so it may be that the issue has been resolved on AER’s end but that’s not clear.

gfortran -cpp -w -std=legacy -fautomatic -fno-align-commons -fconvert=big-endian -fno-range-check -O1 -override-limits -fopenmp -mcmodel=medium -fbacktrace -g -DLINUX_GFORTRAN -DBPCH_DIAG -DBPCH_TIMESER -DBPCH_TPBC -DRRTMG -DGEOS_FP -DGRIDREDUCED -DGRID4x5 -DNC_HAS_COMPRESSION -DUSE_REAL8 -J../mod -I/n/helmod/apps/centos7/MPI/gcc/7.1.0-fasrc01/openmpi/2.1.0-fasrc02/netcdf/4.3.2-fasrc05/include -I/n/helmod/apps/centos7/MPI/gcc/7.1.0-fasrc01/openmpi/2.1.0-fasrc02/netcdf-fortran/4.4.0-fasrc03/include -fdefault-real-8 -fdefault-double-8 -c -O1 -override-limits rrtmg_sw_rad.F90
rrtmg_lw_k_g.F90:4298:0:

       subroutine lw_kgb03

note: variable tracking size limit exceeded with -fvar-tracking-assignments, retrying without

ar crs librad.a rrsw_cld.o rrlw_kg01.o mcica_random_numbers.o rrlw_kg09.o mcica_subcol_gen_sw.o rrlw_wvn.o rrsw_kg28.o rrtmg_sw_setcoef.o rrsw_kg23.o parkind.o rrlw_cld.o rrlw_ncpar.o rrsw_kg29.o rrlw_tbl.o rrsw_kg17.o rrlw_kg11.o rrtmg_sw_spcvmc.o rrsw_tbl.o rrtmg_sw_init.o parrrtm.o rrtmg_sw_k_g.o rrlw_kg04.o rrsw_kg18.o rrlw_kg02.o rrtmg_lw_k_g.o rrtmg_lw_init.o rrtmg_lw_taumol.o rrsw_con.o rrtmg_lw_rtrnmc.o rrsw_kg19.o rrtmg_sw_rad.o rrsw_kg21.o rrsw_kg27.o rrlw_kg16.o rrsw_kg26.o rrsw_ref.o rrtmg_lw_setcoef.o rrtmg_sw_reftra.o rrsw_aer.o rrsw_vsn.o rrlw_kg10.o rrlw_ref.o test_arr_mult.o rrlw_kg12.o rrlw_con.o parrrsw.o rrsw_kg22.o rrsw_kg20.o rrlw_vsn.o rrlw_kg15.o rrtmg_sw_taumol.o rrsw_kg25.o rrsw_kg16.o rrlw_kg03.o rrtmg_lw_cldprmc.o rrlw_kg05.o rrlw_kg14.o rrtmg_sw_cldprmc.o rrlw_kg06.o rrtmg_sw_vrtqdr.o rrlw_kg13.o rrlw_kg07.o rrlw_kg08.o rrsw_wvn.o rrtmg_lw_rad.o rrsw_kg24.o mcica_subcol_gen_lw.o
ar: rrsw_cld.o: No such file or directory
make[4]: *** [lib] Error 1
make[4]: Leaving directory `/n/home05/msulprizio/GC/Code.Dev/GeosRad'
make[3]: *** [librad] Error 2
make[3]: Leaving directory `/n/home05/msulprizio/GC/Code.Dev/GeosCore'
make[2]: *** [lib] Error 2
make[2]: Leaving directory `/n/home05/msulprizio/GC/Code.Dev/GeosCore'
make[1]: *** [all] Error 2
make[1]: Leaving directory `/n/home05/msulprizio/GC/Code.Dev/GeosCore'
make: *** [all] Error 2

--Bob Yantosca (talk) 18:52, 3 August 2018 (UTC)

References

1. Heald, C.L., D.A. Ridley, J.H. Kroll, S.R.H. Barrett, K.E. Cady-Pereira, M.J. Alvarado, C.D. Holmes, Beyond Direct Radiative Forcing: The Case for Characterizing the Direct Radiative Effect of Aerosols, Atmos. Chem. Phys., 14, 5513-5527, doi:10.5194/acp-14-5513-2014, 2014. (Article)