- 1 Overview
- 2 Validation
- 3 Previous issues now resolved in GEOS-Chem v11-01
- 3.1 Added a test for mass conservation
- 3.2 Prevent negative tracer concentrations after advection
- 3.3 Update of PMN + O3 reaction products in globchem.dat file
- 3.4 Modifications to ND20 to archive afternoon P(O3) and L(O3)
- 3.5 Bug fixes for the ND21 diagnostic
- 3.6 Bug fix for custom grid in tpcore_bc_mod.F
- 4 Outstanding issues not yet resolved in GEOS-Chem v11-01
The table below shows the previous, current, and successive versions of GEOS-Chem:
|Previous version||This version||Next version|
|GEOS-Chem v10-01||GEOS-Chem v11-01||TBD|
|PUBLIC RELEASE 17 Jun 2015||TBD||TBD|
|View v10-01 benchmark history||View v11-01 benchmark history||TBD|
What's new in this version
NOTE: v11-01a, v11-02b, etc. denote 1-month benchmark simulations, which are designed to evaluate GEOS-Chem's performance at intermediate stages of development.
Please see the Benchmark approval form for GEOS-Chem v11-01a for complete information about the validation of GEOS-Chem v10-01a.
|Incorporate moisture into air quantities, tracer units, and tracer unit conversions||Science &
|Lizzie Lundgren (GEOS-Chem Support Team)||Approved 07 Jul 2015|
|Correct bug in below cloud average mixing ratio calculation in DO_MERRA_CONVECTION to improve mass balance||Bug fix||Lizzie Lundgren (GEOS-Chem Support Team)||" "|
|Added the capability to print out tracer masses every few hours, in order to check for mass conservation||Structural||Lizzie Lundgren (GEOS-Chem Support Team)||" "|
|Fix bug in RRTMG that prevents storage of O3 and CH4 RFs||Bug fix||David Ridley (MIT)
Sebastian Eastham (MIT)
|Prevent negative tracer concentrations at poles after advection||Bug fix||Lizzie Lundgren (GEOS-Chem Support Team)||" "|
Please see the the following links for complete information about the validation of GEOS-Chem v11-01b:
- Approval form for 1-month benchmark simulation v11-01b
- Results for 1-year benchmark simulation v11-01b-Run0
- Results for 1-year benchmark simulation v11-01b-RnPbBe
|Update DMS climatology to Lana||Science||Tom Breider (Harvard)||1-month benchmark:|
Approved on 04 Aug 2015
|Impaction scavenging for hydrophobic BC||Science||Qiaoqiao Wang (Max Planck Institute)||" "|
|Homogeneous IN removal||Science||Qiaoqiao Wang (Max Planck Institute)||" "|
|Density of OA update||Science||Melanie Hammer (Dalhousie)
Eloïse Marais (Harvard)
|Improved dust size distribution scheme||Science||Li Zhang (Colorado U.)
Daven Henze (Colorado U.)
|Addition of BrC UV absorption to address impacts on OH
||Science||Melanie Hammer (Dalhousie)||" "|
|Acid uptake on dust aerosols
||Science||T. Duncan Fairlie (NASA/LARC)||" "|
|Now treat DST2-DST4 as coarse mode in wet scavenging||Bug fix||T. Duncan Fairlie (NASA/LARC)||" "|
|Online emission of marine primary organic aerosol (POA)
||Science||Brett Gantt (NCSU)
Matthew Johnson (NASA Ames)
|Update of PMN + O3 reaction products in globchem.dat file||Bug fix||Dylan Millet (UMN)||" "|
|Bug fix in RRTMG array sizes||Bug fix||Sebastian Eastham (MIT)||" "|
|Bug fixes in rrtmg_rad_transfer_mod.F||Bug fix||Sebastian Eastham (MIT)
David Ridley (MIT)
|Bug fix for black carbon in ucx_mod.F||Bug fix||Sebastian Eastham (MIT)||" "|
|Bug fix for archiving P(Ox)/L(Ox) from full-chemistry simulations via the ND20 diagnostic||Bug fix||GEOS-Chem Support Team||" "|
|Bug fix for linking files when compiling with PGI||Bug fix||GEOS-Chem Support Team||" "|
Please see the Benchmark approval form for GEOS-Chem v11-01c for complete information about the validation of GEOS-Chem v10-01c.
|Updates to PAH code||Science||Carey Friedman (MIT)||Approved 14 Sept 2015|
|Hg Ocean MLD bug fixes||Bug fix||Amanda Giang (MIT)||" "|
|Hg Ocean rate coefficients (as an option, not default)||Science||Shaojie Song (MIT)||" "|
|Hg Arctic process updates||Science||Jenny Fisher (Wollongong)||" "|
|Hg emission updates (NEI2011, NPRI2011, UNEP2010 with adjustments for and emission controls)||Science||Yanxu Zhang (Harvard)
Amanda Giang (MIT)
|Update CO2 files for HEMCO||Science & Bug fix||Ray Nassar (Environment Canada)||" "|
|Bug fix for sea salt alkalinity in sulfate_mod.F||Bug fix||Johan Schmidt (Harvard)||" "|
|Correct problem in OTD-LIS local redistribution files for GEOS-5||Bug fix||Christoph Keller (Harvard)
Lu Hu (Harvard)
|Bug fixes for the ND21 diagnostic||Bug fix||Sebastian Eastham (MIT)||" "|
|Bug fix for custom nested grid in tpcore_bc_mod.F||Bug fix||Shannon Koplitz (Harvard), GCST||" "|
|Bug fix for monoterpenes in ND46 diagnostic||Bug fix||Jared Brewer (CSU)||" "|
|GEOS-Chem timers||Structural||Matt Yannetti (GCST)||" "|
|Initial modifications for running GEOS-Chem with the GMAO MERRA2 reanalysis met field product||Structural||Bob Yantosca (GCST)||
Please see the the following links below for complete information about the validation of GEOS-Chem v11-01d.
NOTE: The original v11-01d submitted for approval included a "quick fix" for low Pb tropospheric lifetime against deposition in GEOS-FP and MERRA2. Due to high impact on aerosols, the GEOS-Chem Steering Committee rejected the "quick fix", and we have subsequently removed it from the version and performed an additional 1-year benchmark simulation with the updated model.
Initial benchmark results for v11-01d including the "quick fix":
- Approval form for 1-month benchmark simulation v11-01d
- Results for 1-year benchmark simulation v11-01d-Run0
- Results for 1-year benchmark simulation v11-01d-RnPbBe
Updated benchmark results for v11-01d after removal of the "quick fix":
|CO2 direct effect on isoprene emissions||Science||Amos Tai (CUHK)||Pending benchmark approval|
|Criegee intermediates||Science||Dylan Millet (U. Minnesota)
Eloïse Marais (Harvard)
|Update biomass burning emissions to GFED4.1, includes:
||Science||Prasad Kasibhatla (Duke)
Christoph Keller (Harvard)
|Update implementation of optical properties for brown carbon||Science||Melanie Hammer (Dalhousie)||" "|
|Species Database Phase 1: Store physical properties of GEOS-Chem species in a centralized species database object||Structural||GEOS-Chem Support Team||" "|
|Resolve very high tracer concentrations in MERRA wet deposition||Bug fix||Viral Shah (UW)||" "|
|Bug fix for European grid range in tagged Ox simulation||Bug fix||GEOS-Chem Support Team||" "|
|Quick fix for low Pb tropospheric lifetime against deposition in GEOS-FP and MERRA2||Science||Bo Zhang, Hongyu Liu (NIA / NASA Langley)||This feature was removed following evaluation of the initial 1-year benchmark simulation due to a high magnitude change for aerosols. It has been added to the pipeline for incorporation into a later version after further testing.|
|Fix for reading hourly NEI2011 emissions||Bug fix||Viral Shah (UW)||" "|
|Bug fix for PBLTOP and MTTOP values in tagged Ox simulation||Bug fix||GEOS-Chem Support Team||" "|
|Parallelization bug fix in Online emission of marine primary organic aerosol (POA)||Bug fix||GEOS-Chem Support Team||" "|
|Bug fix for offline dust aerosols when UCX is on||Bug fix||Seb Eastham (Harvard)||" "|
NOTE: This version will require a 1-month benchmark simulation.
|Update HEMCO from version 1.1.005 to version 1.1.011||Structural||Christoph Keller (Harvard)||
|Update air quantities after humidity update in non-local PBL mixing||Bug fix||Lizzie Lundgren (GEOS-Chem Support Team)||
|Tracer Units Phase 1: Initial implementation of area-independent tracer units||Structural||Lizzie Lundgren (GEOS-Chem Support Team)||
|Species Database Phase 2a: Remove inconsistencies in the definitions of physical properties of GEOS-Chem species||Structural||Bob Yantosca (GEOS-Chem Support Team)||
|Species Database Phase 2b: Update the wetdep module to use the same Henry's law functions as are used in HEMCO, for consistency's sake||Structural||Bob Yantosca (GEOS-Chem Support Team)||
|Bug fixes for Arctic Hg updates in v11-01c||Bug Fix||Jenny Fisher (Wollongong)||
NOTE: This version will require a 1-month benchmark simulation and a 1-year benchmark simulation.
|PAN updates||Science||Emily Fischer (CSU)||
|Monthly mean surface methane distributions||Science||Lee Murray (NASA GISS/LDEO)||
|Ocean ammonia emission inventory||Science||Fabien Paulot (NOAA/GFDL)||
|Temporary fix to remove SO2 scavenging in convective updrafts||Science||Duncan Fairlie (NASA/LARC)||
|Bug fixes for aerosol effective radius and surface area density||Bug fix||Seb Eastham (Harvard)||
In the pipeline
The following features are slated for inclusion in GEOS-Chem v11-01:
|NetCDF output: Flexible diagnostic package and tracer restart file||Science||GEOS-Chem Support Team||TBD|
|Harmonizing Henry's Law coefficients across wet and dry deposition||Science||GEOS-Chem Support Team||
|Flexchem||Science||Mike Long (Harvard)||TBD|
|Uniformity of units||Science||Lizzie Lundgren (GEOS-Chem Support Team)||
|Metal catalyzed oxidation of SO2 (as a switch)||Science||Becky Alexander (UW)||
Becky Alexander wrote:
|Update SO2 scavenging in convective updrafts for consistency||Science||Duncan Fairlie (NASA/LARC)||TBD|
|Fix to direct/diffuse radiation||Science &
|Fix for NO3 + ISOP product||Science||Katie Travis||TBD|
|Update MODIS LAI fields through 2015||Science||Barron Henderson (UF)||TBD|
|Updated organic deposition||Science||Katie Travis (Harvard)
Jenny Fisher (U. Wollongong)
|Hg code updates (emissions, chemistry, ocean, land)||Science||Hg and POPs WG||TBD|
|EPA-derived BC/OA emissions over US (1990-2012)||Science||David Ridley (MIT)||TBD|
|Update density of BC to 1.8 and add absorption enhancement factor in input.geos||Science||Xuan Wang (MIT)||TBD|
|Updates to isoprene chemistry, includes:
||Science||Jenny Fisher (U. Wollongong)
Eloïse Marais (Harvard)
Kelvin Bates (Caltech)
Katie Travis (Harvard)
|Nighttime isoprene chemistry updates||Science||Rebecca Schwantes (Caltech)||TBD|
|Remove dependence of species drydep on HNO3 drydep||Science||Katie Travis (Harvard)
Jenny Fisher (U. Wollongong)
|A snow NOx source from deep snowpack||Science||Maria Zatko (UW)
Becky Alexander (UW)
|Land cover module in GEOS-Chem||Science||Jeffrey Geddes (MIT)
Michael Long (Harvard)
|Merge standard simulation with specialty simulations for OVOC, acids, aromatics, dicarbonyls, terpenes||Science||Dylan Millet (UMN)
Xin Chen (UMN)
|Automate redistribution of NOx in HEMCO||Science||Lee Murray (NASA GISS/LDEO)||TBD|
|Injection height for fire emissions||Science||Emily Fischer (CSU)||TBD|
|Improved lightning emission scheme||Science||Lee Murray (NASA GISS/LDEO)||TBD|
|Updated halogen chemistry||Science||Tomás Sherwen (York)
Johan Schmidt (Harvard)
Oxidants and Chemistry WG
|Sedimentation of stratospheric aerosols||Science||Sebastian Eastham (MIT)||TBD|
New data directories
The following new data directories have been added for GEOS-Chem v11-01. You will have to download the directories relevant to your simulation.
In this section we provide information about the benchmarks and tests that we have done to validate GEOS-Chem v11-01.
1-month and 1-year benchmarks
For complete information about the benchmark simulations used to validate GEOS-Chem v11-01, please see our GEOS-Chem v11-01 benchmark history wiki page.
--Melissa Sulprizio 10:30, 5 May 2015 (EDT)
Unit tests for GEOS-Chem v11-01
We performed a full set of unit tests on GEOS-Chem before submitting each 1-month benchmark. Click on the links below to see the results from each set of unit tests:
|v11-01a||02 Jul 2015||Unit test done prior to submitting v11-01a 1-month benchmark.|
|v11-01b||29 Jul 2015||Unit test done prior to submitting v11-01b 1-month benchmark.|
|v11-01c||10 Sep 2015||Unit test done prior to submitting v11-01c 1-month benchmark|
|v11-01d||Pending completion||Unit test done prior to submitting v11-01d 1-month benchmark|
Previous issues now resolved in GEOS-Chem v11-01
The following bugs and/or technical issues have now been resolved in GEOS-Chem v11-01.
Added a test for mass conservation
These updates were validated in the 1-month benchmark simulation v11-01a, which was approved on 07 Jul 2015.
In GEOS-Chem v11-01a, we have added a mass conservation check capability. This will let you see at a glance if GEOS-Chem is conserving mass.
In the GEOS-Chem Unit Tester, we have created two new run directories:
which are instances of the 2° x 2.5° CO2 simulation, but with zero emissions and with a constant (e.g. 370 ppm) initial condition field.
You can make a fresh copy of one (or both) of these directories with the gcCopyRunDirs script to a convenient location in your account. Once you have done that, then follow these steps:
- 1. Let's say you want to run the geos5_2x5_masscons test. First cd into the copied geos5_2x25_masscons directory.
- 2. Make sure you specify your desired start & end dates in your input.geos file. Also make sure to put a 3 on the last day of the simulation in the OUTPUT MENU section.
- 3. Then type this command:
make -j4 TRACEBACK=y NO_ISO=y mp
- this will start compiling and running the GEOS-5 mass conservation test. The Makefile in this run directory will automatically set a new Makefile option, MASSCONS=y, which disables output to the bpch, timeseries, and restart files. Instead, the mass of the total CO2 tracer will be printed out to a file named tracer_mass_kg.dat (you can rename this in the ND49 MENU section of input.geos). The output will look similar to this:
2.887072562747136E+015 : 2012/07/01 06:00 2.887072562747084E+015 : 2012/07/01 12:00 2.887072562747109E+015 : 2012/07/01 18:00 2.887072562747105E+015 : 2012/07/02 00:00 2.887072562747306E+015 : 2012/07/02 06:00 2.887072562747272E+015 : 2012/07/02 12:00 2.887072562747154E+015 : 2012/07/02 18:00 2.887072562746720E+015 : 2012/07/03 00:00
- The output shows the sum of CO2 tracer (in kg) over all longitudes, latitudes and levels, with a corresponding time stamp. As you can see from this example, mass is conserved to about the 11th decimal place, which indicates good mass conservation.
Prevent negative tracer concentrations after advection
These updates were validated in the 1-month benchmark simulation v11-01a, which was approved on 07 Jul 2015.
In GEOS-Chem v11-01a, we have included a fix for negative tracer concentrations following advection. Tracer concentrations may become negative within routine TPCORE_FVDAS at the poles with values on the order of -1e-10 v/v or smaller in v10-01. Negative values can cause GEOS-Chem to stop downstream if TOMAS is enabled due to a negative NK check within routine MNFIX in tomas_mod.F.
To fix this issue, we set all negative tracer concentration values to 1.0e-26 at the end of advection. The fix, inserted in routine TPCORE_FVDAS within tpcore_fvdas_mod.F90, is shown below. This code is not inserted in nested region advection routines because none of the nested regions overlap the poles.
!====================================================================== ! MODIFICATION by Harvard Atmospheric Chemistry Modeling Group ! ! Set tracer concentration to a small positive number if concentration ! is negative. Negative concentrations may occur at the poles. ! (ewl, 6/30/15) !====================================================================== !$OMP PARALLEL DO & !$OMP DEFAULT( SHARED )& !$OMP PRIVATE( I, J, K ) DO K = 1, KM DO J = 1, JM DO I = 1, IM IF ( q(I,J,K,IQ) < 0.0e0_fp ) THEN q(I,J,K,IQ) = 1.0e-26_fp ENDIF ENDDO ENDDO ENDDO !$OMP END PARALLEL DO
Note that the reasons for negative values at the poles is not currently understood and is listed in the Outstanding issues not yet resolved in GEOS-Chem v11-01 section of this wiki page.
Update of PMN + O3 reaction products in globchem.dat file
Dylan Millet wrote:
- I'd like to submit the following change for the globchem.dat file. The PMN + O3 reaction does not give the appropriate products. We should change:
A 505 8.20E-18 0.0E+00 0 0 0.00 0. 0. PMN + O3 =1.000NO2 +0.600CH2O +1.000HO2 + + + + + + + + + + + + +
A 505 8.20E-18 0.0E+00 0 0 0.00 0. 0. PMN + O3 =1.000CH2O +1.000MCO3 +1.000NO3 + + + + + + + + + + + + +
- It won't have a major impact, but it is currently is wrong, and my sense (Mat as well) is that this falls under 'bug fix' rather than science update.
Modifications to ND20 to archive afternoon P(O3) and L(O3)
Bob Yantosca wrote:
- I was able to make a couple of modifications that will let you just archive afternoon P(O3) and L(O3), since that may let you avoid the too-low-values at night. If you look in diag_pl_mod.F, in routine DIAG20, you will see these new lines:
! ! !DEFINED PARAMETERS: ! ! Local time limits (optional) REAL(fp), PARAMETER :: LT_START = 12.0e+0_fp REAL(fp), PARAMETER :: LT_END = 16.0e+0_fp
!$OMP+DEFAULT( SHARED ) !$OMP+PRIVATE( I, J, L, P_Ox, L_Ox, JLOOP, LT ) !$OMP+SCHEDULE( DYNAMIC ) DO L = 1, LD65 DO J = 1, JJPAR DO I = 1, IIPAR ! Zero P(Ox), L(Ox) variables for safety's sake P_Ox = 0e+0_fp L_Ox = 0e+0_fp ! JLOOP is the 1-D grid box index for SMVGEAR arrays JLOOP = JLOP(I,J,L) ! If this is a grid box where we have done chemistry, then ... IF ( JLOOP > 0 ) THEN !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% !%%% NOTE: If you want to restrict output to a given local !%%% time range, then uncomment these lines of code. !%%% (bmy, 7/9/15) ! ! Get the local time at box (I,J,L) !LT = GET_LOCALTIME( I, J, L ) ! ! Skip processing if we are outside of the desired ! local time range (bmy, 7/9/15) !IF ( LT < LT_START .or. LT > LT_END ) CYCLE !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% ! Increment counter of valid grid boxes COUNT(I,J,L) = COUNT(I,J,L) + 1
- (I also defined the LT variable and put in a USE statement for the GET_LOCALTIME function, not shown.)
- So if you want to restrict archiving of P(O3) and L(O3) to a given window, you can modify the LT_START and LT_END to the starting and ending hour of the local time window. Values must be in the range 0.0 to 23.99 hours. Then also uncomment the lines after the IF ( JLOOP > 0 ) block.
- In addition, I fixed a few minor issues in the DIAG20 and WRITE20 routines. For example, the bpch file was being saved with the wrong JFIRST value, so I corrected that. Also, I made the code more efficient by not trying to compute P(O3) and L(O3) where JLOOP == 0 (i.e. where chemistry wasn't done).
Bug fixes for the ND21 diagnostic
These fixes were validated with the 1-month benchmark simulation v11-01c and approved on 14 Sept 2015.
Sebastian Eastham wrote:
- I was looking at some output for a recent run with v10-01 and realized that diagnostic 21 (OD_MAP) no longer produces the expected data. I only know for certain that this happens when the UCX is enabled, as I do not run with the trop-only version of the code. It basically comes down to index mismatching between aerosol_mod, diag3_mod and gamap_mod.
- Once implemented, it should restore output of diagnostic 21 when UCX is active. Prior to this patch, selecting diagnostic 21 with “all” output seemed to result in the incorrect number of tracers and misassignments.
Bug fix for custom grid in tpcore_bc_mod.F
This fix was validated with the 1-month benchmark simulation v11-01c and approved on 14 Sept 2015.
Shannon Koplitz wrote:
- I think there may be a redundant call to allocate the custom nested grid array BC at line 2045 in tpcore_bc_mod.F.
- It looks like the call to allocate that array is given earlier in the code at line 1977, and the double call appears to result in an allocation error when attempting to save out BCs for a custom nested grid.
Melissa Sulprizio responded:
- I found a few other places where the array BC is used. From what I see, it appears that BC is used in SAVE_GLOBAL_TPCORE_BC specifically for the custom grid, but in READ_WINDOW_TPCORE_BC, GET_4x5_BC, and GET_2x25_BC it's used in a more general sense (for all nested grids). So I think the issue here is that BC is unintentionally used for two different purposes. I think if we remove the second allocation and only allocate BC when LWINDO_CU is true, we'd get errors in the last three routines that I mentioned.
- I suggest we change BC for the custom nested grid to BC_CU. In SAVE_GLOBAL_TPCORE_BC, we would modify the following code (in green text):
IF ((IC .eq. 1) .and. LWINDO_CU )THEN !---------------------------------------------------------- ! IC == 1 : Save BC's to file (custom window) !---------------------------------------------------------- ! Open file for writing, if necessary CALL OPEN_BC_FILE( am_I_Root, Input_Opt, RC, & WINDOW=IC, FOR_WRITE=.TRUE. ) ! Loop over each tracer DO N = 1, N_TOT_TRC ! Save concentrations in WINDOW REGION to disk DO L = 1, LLPAR BC_CU(1:IM_BC,1:JM_BC,L,N)= & STT(I1_BC:I2_BC,J1_BC:J2_BC,L,N) ENDDO ! Write boundary conditions to binary punch file CALL BPCH2( IU_BC, MODELNAME, LONRES, LATRES, & HALFPOLAR, CENTER180, CATEGORY, N, & UNIT, TAU, TAU, RESERVED, & IM_BC, JM_BC, LLPAR, I1_BC, & J1_BC, 1, BC_CU(1:IM_BC, 1:JM_BC, 1:LLPAR, N)) ENDDO
- and in INIT_TPCORE_BC:
! Boundary condition array for CUSTOM NESTED REGION IF ( LWINDO_CU ) THEN ALLOCATE( BC_CU( IM_BC, JM_BC, LLPAR, N_TRACERS ), STAT=AS ) IF ( AS /= 0 ) CALL ALLOC_ERR( 'BC_CU' ) BC_CU = 0e+0_fp ENDIF
- We would also have to declare BC_CU at the top of the module and deallocate it in CLEANUP_TPCORE_BC.
Outstanding issues not yet resolved in GEOS-Chem v11-01
The following issues have not yet been resolved in GEOS-Chem v11-01. Some of these issues represent questions of open research rather than bugs in the source code.
Moisture signature in dry mixing ratio
In v11-01a we separated tracer mixing ratios into dry and moist mixing ratios (Air Quantity Updates for v11-01a). This was done to correct the erroneous legacy tracer unit conversion between [v/v] and [kg] where dry air molecular weight was used with total (moist) air pressure. The output tracer concentrations of GEOS-Chem are now dry mixing ratio [v/v]. However, we discovered a problem where output tracer concentration maps show a moisture signature in the dry mixing ratio. This behavior occurs because tracer mass is distributed relative to moist air mass rather than dry air mass in transport. The Carbon and Transport Working Groups are currently investigating possible fixes.
Cause of negative tracer concentrations at the poles is unknown
Tracer concentrations at the poles may become negative within advection. A fix is included in v11-01 that resets negative values to positive at the end of TPCORE_FVDAS but the underlying cause of the negative values is not addressed. All negative values are small enough to not significantly impact advection mass balance.
Mercury simulation inconsistently failing optimization unit tests in v11-01c
There are inconsistent differences in the diagnostics and ocean restart files when running unit tests on GEOS-5 4x5 and 2x2.5 mercury simulations in v11-01c. When running a test of ten GEOS-5 4x5 Hg unit tests, the trac_avg file failed with differences seven times and the ocean restart file failed with differences eight times. We have isolated the issue to a single call to depo_mercury_mod.F routine ADD_HG2_SNOWPACK that occurs in convection_mod.F routine NFCLDMX. Commenting out the call and conducting ten GEOS-5 4x5 Hg unit tests results in all tests passing. The issue appears to be an optimization problem but is currently unresolved.