GEOS-Chem v8-01-04

From Geos-chem
Revision as of 20:21, 4 November 2011 by Bmy (Talk | contribs) (Overview)

Jump to: navigation, search

Overview

EMISSIONS RELEASE date March 10, 2009

What's new in this version

Contains everything in v8-01-03 plus the following updates:

  1. Annual scale factors for NOx-CO-SOx from, applied to ALL inventory if needed (Aaron)
  2. Daily NOx variability from EDGAR applied to all others anthro NOx inventories
  3. Several emissions updates
    • Streets 2006, with monthly variability
    • EMEP 1990-2005 w/ ship emissions separated
    • Canadian CAC,
    • EPA fixes = CA transport + Rynda scaling,
    • VISTAS/ARP for NOx in the USA
    • MEGAN2
  4. new EPA mask (Overlap Bravo/Epa and CAC/Epa)
  5. ship NOx emitted as HNO3 + 10*O3
  6. pre-Arctas SO2 ship emissions from Streets
  7. GFED2 biomass options:
    • Monthly mean (as was in the code previously)
    • 8-day mean
    • 3-hourly mean (selected months only)
    • 3-hourly synoptic (selected months only)

Philippe Le Sager has prepared a document (PDF format) which describes the anthropogenic emissions sources in some detail.

1-year benchmarks

The following benchmark simulations have been done with a few different versions of GEOS-5 meteorology:

  • GEOS-5.0.1
  • GEOS-5.1.0
  • GEOS-5.2.0

At present, all of the GEOS-5.0.1 met data files for GEOS-Chem are being replaced with the newer GEOS-5.1.0 data product. GEOS-5.1.0 will have data available from late 2003 thru September 2008. Data after September 2008 will be the newest GEOS-5.2.0 data product. For more information about these GEOS-5 data versions, please see the GEOS-5 documentation from GMAO.

Run0

Three GEOS-Chem model versions were compared to each other:

Color Quantity Plotted Met Field Type A.T.E. algorithm Anthro Emissions Biogenic Emissions Advection Annual Mean OH
[105 molec/cm3]
Red v8-01-01 Run0 GEOS-5 4x5
version 5.0.1

spinup: 2005
run: 2005
ISORROPIA GEIA/Piccot (scale year 1998)
EPA/NEI99 w/o ICARTT fix
EDGAR ship emissions
"old" MEGAN AEF's for isoprene "old" tpcore 12.006
Green v8-01-01 Run1 " " RPMARES " " " " " " 12.040
Blue v8-01-04-Run0 GEOS-5 4x5
version 5.0.1
with "quick fix"
for optical depth


spinup: 2005
run: 2005
" " " " "new" MEGAN AEF's for isoprene "new" tpcore 10.689
Black Observations            

SUMMARY:

  1. v8-01-01 Run0 vs. v8-01-01 Run1 is a clean comparison for the switch from ISORROPIA to RPMARES.
  2. v8-01-04 Run1 used the same emissions as v8-01-01 Run1, but also implemented the the "quick fix" for the GEOS-5 optical depth. Dylan Millet reported that the the "quick fix" was responsible for lowering the mean OH concentration from ~12 x 105 to ~11 x 105 molecules/cm3 in simulations that he had performed. Therefore the "quick fix" is probably the major factor driving the drop in mean OH in the v8-01-04 runs.
  3. All runs listed here used the same:
    • biomass emissions (GFED2 monthly)
    • biofuel emissions
    • lightning NOx emissions
    • aircraft emissions

The output plots for Run0 (both PostScript and PDF formats) may be downloaded from:

ftp ftp.as.harvard.edu
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run0/output/ps
mget *
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run0/output/pdf
mget *

You may also view the PDF files online by pointing your browser to:

ftp://ftp.as.harvard.edu/pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run0/output/pdf

Run1

Three GEOS-Chem model versions were compared to each other:

Color Quantity Plotted Met Field Type Photolysis Anthro Emissions Biogenic Emissions Advection Annual Mean OH
[105 molec/cm3]
Red v8-01-01 Run1 GEOS-5 4x5
version 5.0.1

spinup: 2005
run: 2005
linear
cloud overlap
GEIA/Piccot (scale year 1998)
EPA/NEI99 w/o ICARTT fix
EDGAR ship emissions
"old" MEGAN AEF's for isoprene "old" tpcore 12.040
Green v8-01-04 Run0 GEOS-5 4x5
version 5.0.1
with "quick fix"
for optical depth


spinup: 2005
run: 2005
" " " " "new" MEGAN AEF's for isoprene "new" tpcore 10.689
Blue v8-01-04 Run1 " " approximate
random
cloud overlap
GEIA/Piccot
EDGAR emissions
EMEP European emissions
BRAVO Mexican emissions
David Streets 2006 emissions
CAC Canadian emissions
EPA/NEI99 with ICARTT fix
EDGAR ship emissions
ARCTAS ship SO2 emissions
Anthro scale year 2005
"new" MEGAN AEF's for isoprene " " 10.294
Black Observations            

SUMMARY:

  1. v8-01-04 Run1 used the same emissions as v8-01-01 Run1, but also implemented the the "quick fix" for the GEOS-5 optical depth. Dylan Millet reported that the the "quick fix" was responsible for lowering the mean OH concentration from ~12 x 105 to ~11 x 105 molecules/cm3 in simulations that he had performed. Therefore the "quick fix" is probably the major factor driving the drop in mean OH in the v8-01-04 runs.
  2. v8-01-04 Run1 vs. v8-01-04 Run0 is a clean comparison for the new anthropogenic emissions options that were implemented into v8-01-04.
  3. All runs used the same:
    • A.T.E. Algorithm = RPMARES
    • GFED2 biomass emissions
    • Aircraft NOx emissions
    • Lightning NOx emissions

The output plots for Run1 (both PostScript and PDF formats) may be downloaded from:

ftp ftp.as.harvard.edu
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run1/output/ps
mget *
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run1/output/pdf
mget *

You may also view the PDF files online by pointing your browser to:

ftp://ftp.as.harvard.edu/pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run1/output/pdf

Run2

Three GEOS-Chem model versions were compared to each other:

Color Quantity Plotted Met Field Type Anthro Emissions Lightning NOx
[Tg N/yr]
Annual Mean OH
[105 molec/cm3]
Red v8-01-04 Run0 GEOS-5 4x5 version 5.0.1
with "quick fix" for optical depth

spinup: 2005
run: 2005
GEIA/Piccot (scale year 1998)
EPA/NEI99 w/o ICARTT fix
EDGAR ship emissions
5.979 10.689
Green v8-01-04 Run1 " " GEIA/Piccot
EDGAR emissions
EMEP European emissions
BRAVO Mexican emissions
David Streets 2006 emissions
CAC Canadian emissions
EPA/NEI99 with ICARTT fix
EDGAR ship emissions
ARCTAS ship SO2 emissions
Anthro scale year 2005
5.979 10.294
Green v8-01-04 Run2 GEOS-5 4x5 version 5.1.0,
"reprocessed" met fields

spinup: 2004
run: 2005
" " 5.936 11.099
Black Observations        

SUMMARY:

  1. v8-01-04 Run1 vs. v8-01-04 Run0 is a clean comparison for the new anthropogenic emissions options that were implemented into v8-01-04.
  2. v8-01-04 Run2 vs. v8-01-04 Run1 is a clean comparison between the GEOS-5.0.1 and GEOS-5.1.0 met products.
  3. All runs used the same:

The output plots for Run1 (both PostScript and PDF formats) may be downloaded from:

ftp ftp.as.harvard.edu
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run2/output/ps
mget *
cd pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run2/output/pdf
mget *

You may also view the PDF files online by pointing your browser to:

ftp://ftp.as.harvard.edu/pub/geos-chem/1yr_benchmarks/v8-01-04/geos5/2005/Run2/output/pdf

Outstanding issues in GEOS-Chem v8-01-04

Unallocated Diagnostic Arrays in TPCORE

We found that passing unallocated ND24/25/26 (mass flux) arrays into TPCORE can cause a run to crash. It happened with a modified version of tagged CO simulation in GC v8-01-04 that uses Flambe emissions and was running at 2x2.5 and on multiple processors. Since the mass flux arrays are not optional but intent(inout) for TPCORE, they should be allocated even if they are not used, i.e. even if the diagnostic are turned off.

A general fix that allocates these arrays but lets you turn off the output diagnostic will be in the next GC version (scheduled to be v8-02-01). Meanwhile we strongly advise users to **always** switch on the ND24/25/26 mass flux in their input.geos.

--phs 10:40, 2 April 2009 (EDT)

Previous issues now resolved in v8-01-04

The following user-reported issues have now been corrected in this version:

Seg fault in GEOS-5 China nested grid simulation

Xiaoguang Gu wrote:

I ran into a problem while running GEOS-5, version v8-01-02 for 0.5x0.66 China nested simulation. I followed the instructions on GEOS-Chem Manual to make the simulation. And, I successfully run the model with 4x5 resolution to save out the boundary conditions. However, the China nested simulation stops after saving out ctm.bpch file and restart file with the error message as following. I run the model on a LINUX cluster with LINUX/IFORT (64-bit) compiler. The processor number I set is 32 for 4 threads.
The error message was:
   ---> DATE: 2006/07/03  GMT: 00:00  X-HRS:      48.000
        - DIAG3: Diagnostics written to bpch!
        - MAKE_RESTART_FILE: Writing restart.2006070300.05x0667
        - INITIALIZE: Diag arrays zeroed!
        - INITIALIZE: Diag counters zeroed!

   ===============================================================================
   ND23: Mass-Weighted OH Concentration
   Mean OH =    20.8207222069296       [1e5 molec/cm3]
   ===============================================================================
        - CLEANUP: deallocating arrays now...
   forrtl: severe (174): SIGSEGV, segmentation fault occurred 
   Image   PC                Routine            Line        Source
     ...
   geos    0000000000AE3447  tpcore_geos5_wind         318  tpcore_geos5_window_mod.f90
   geos    0000000000AFF834  transport_mod_mp_        1256  transport_mod.f
   geos    0000000000AF02B7  transport_mod_mp_         140  transport_mod.f
   geos    0000000000C47BE5  MAIN__                    812  main.f 

Bob Yantosca replied:

I think the problem is an out-of-bounds error somewhere. If you are going outside of array bounds in an allocatable array then that could cause the error you saw at the end of the run.
Try adding -traceback -CB to the FFLAGS line in the makefile. Traceback will give you more detailed error output and -CB will turn on checking for "array out of bounds" errors.

Xiaoguang Gu wrote:

Thanks for your response! By adding -traceback -CB to the FFLAGS line in the makefile following your suggestion, I have found the problem and solved it.
It was showed in the log file that the problem is originated from an allocatable array 'COSE'. The error messages in the log file are the following:
   - UPBDFLX_NOY: Reading /home/jwang7/xxu/
                          GEOS-Chem/data/GEOS_0.5x0.666_CH/pnoy_200106/pnoy_nox_hno3.geos5.05x0666
   forrtl: severe (408): fort: (2): Subscript #1 of the array COSE has value 134 
   which is greater than the  upper bound of 133
   
So, I checked the array COSE in the code tpcore_geos5_window_mod.f90, and found those lines:
          .......
   263 !----------------
   264 ! Allocate arrays
   265 !----------------
   266
   267  allocate ( cosp(jm) )
   268  allocate ( cose(jm) )
          ........
   308  do j=1,jm+1                    !(dan)
   309     elat(j) = 0.5*(clat(j-1) + clat(j))
   310     sine(j) = sin(elat(j))
   311     !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   312     !%%% MODIFICATION by Harvard Atmospheric Chemistry Modeling Group
   313     !%%%
   314     !%%% Initialize SINE_25 array (bmy, bdf, 10/29/04)
   315     !%%%
   316     SINE_25(J) = SIN( CLAT(J) )
   317     !%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
   318     cose(j) = cos(elat(j))
   319  enddo
           ..........
So, I modified
   allocate ( cose(jm) )
to be
   allocate ( cose(jm+1) )
in the line 268. And, I got a successful run after this modification.

Dan Chen wrote:

I made a test on Xiaoguang's modification and I found that it's indeed a bug that may cause memory allocating problem. It would not change the simulation result, but may cause the machines running out of memory. I think this modification is necessary and correct!

--Bob Y. 09:31, 22 January 2009 (EST)

Error message in partition.f

Tzung-May Fu wrote:

I ran into a bug in v8-01-03 in partition.f. In line 285~291:
              ! Get NOx concentration from STT
              CONCNOX = STT(I,J,L,IDTNOX)

              ! Stop w/ error
              IF ( CONCNOX - SUM1 < 0.d0 ) THEN
                 CALL ERROR_STOP( 'STOP 30000', 'partition.f' )
              ENDIF
However, it is possible to run into a situation where CONCNOX=0d0, but SUM1=NO2+NO3 = 1d-99 + 1d-99, due to the underflow check in line 258~262. If this happens, CONCNOX-SUM1=-2d-99, and will stop the simulation.
I think the stop w/ error statement should be revised to something more robust, or at least changed to:
              ! Stop w/ error
              IF ( CONCNOX - SUM1 < -3.d-99 ) THEN
                 CALL ERROR_STOP( 'STOP 30000', 'partition.f' )
              ENDIF
In case CONCNOX-SUM1 is a very small negative value, it can still be caught in the underflow check in line 317.
Do you agree? I am a little worried that this will propagate unwanted error into the code.

Bob Yantosca replied:

Yes, we found that out recently too, particulary when you use the new TPCORE. I ran into the same problem in the NRT. As you say, it is an underflow error. I might not have posted this on the wiki yet, if so I'll do it today.
The cheap solution is to just print a warning message, and then the user can decide if it's a terrible error or not. If the CONCNOX - SUM1 is of order 1e-99 or 2e-99 then it's OK. This will go into the next version (v8-01-04). In the meantime, try this:
             ! Error test
             IF ( CONCNOX - SUM1 < 0.d0 ) THEN
                !------------------------------------------------------
                ! Prior to 1/7/09
                ! Don't stop w/ error, but just print warning msg.
                ! Sometimes the new TPCORE can cause this error to
                ! trap if there CONCNOX = 0, but that can be purely
                ! a numerical condition and not really an error.
                ! (phs, ccc, bmy, 1/7/09)
                !CALL ERROR_STOP( 'STOP 30000', 'partition.f' )
                !------------------------------------------------------
!$OMP CRITICAL
                PRINT*, '### In partition.f: CONCNOX - SUM1 < 0'
                PRINT*, '### If CONCNOX = 0 and SUM1 ~ 1e-99 it is OK'
                PRINT*, '### I, J, L : ', I, J, L
                PRINT*, '### CONCNOX : ', CONCNOX
                PRINT*, '### SUM1    : ', SUM1
!$OMP END CRITICAL
             ENDIF

--Ccarouge 15:31, 11 March 2009 (EDT)

Instance of this error when compiling with IFORT 10

Bob Yantosca wrote:

Prasad Kasibhatla noted that this error in partition.f occurred in a GEOS-Chem v9-01-01 simulation that was compiled with the Intel Fortran Compiler version 10 (Build 20080312 Package ID: l_fc_p_10.1.015). I was able to replicate the error with IFORT 10 but not IFORT 11. Furthermore, the error seems to occur only if the default -O2 optimization is used.
The quick fix is to compile the entire code with -O1 optimization. To do this change this line in Code.v9-01-01/Makefile_header.mk from:
   FFLAGS   = -cpp -w -O2 -auto -noalign -convert big_endian -vec-report0
to:
   FFLAGS   = -cpp -w -O2 -auto -noalign -convert big_endian -vec-report0 –O1
When IFORT (and all compilers) find that there are multiple settings for the same option on the command line, it will take the last setting. Thus in the FFLAGS line above, IFORT will ignore the –O2 and take the –O1.
We are currently investigating if we can get away with only compiling certain files with –O1 while leaving the majority compiled with –O2. Stay tuned for results.

--Bob Y. 09:05, 29 June 2011 (EDT)

Erroneous O3 diagnostic (ND45)

Lee Murray and Claire Carouge reported a typo in the way that v8-01-02 and v8-01-03 outputs the O3 ND45 diagnostic. Ox is used in place of O3 line 2601 of diag3.f. You need to replace:

                    ARRAY(:,:,1:LD45) = AD45(:,:,1:LD45,N) /
    $                    FLOAT( CTO3 )

with

                    ARRAY(:,:,1:LD45) = AD45(:,:,1:LD45,N_TRACERS+1) /
    $                    FLOAT( CTO3 ) 

--phs 08:52, 3 February 2009 (EST)

Post-Release Patches

A couple of issues were reported after the release of v8-01-04. These will be fully corrected in GEOS-Chem v8-02-01, but in the meantime we created a patch with the fixes that you can use in v8-01-04 (see below).

Fabien Paulot (fabienpaulot@gmail.com) wrote:

The tracer id is hardcoded in the sulfate module (sulfate_mod.f which causes error when calling the ship emission module (GET_EMEP_ANTHRO) if the user wants to modify the tracer list. I just replaced 26 by IDTSO2 and 30 by IDTNH3 to fix the problem. See line 6594.

This issue was due to a typo in sulfate_mod.f. The error only occurred during aerosol-only offline simulations.


Prasad Kasibhatla (psk9@duke.edu) wrote:

I also noticed in the standard distribution, that the CPU time for the one month full chem simulation at Harvard went from 56 minutes for v8-01-02 to 75 minutes for v8-01-03 to 121 minutes for v8-01-04, an increase of a factor of 2.2. I was wondering if you have looked at this slowdown?

This issue was traced to the way the !$OMP PARALLEL DO loops were implemented in the TPCORE advection module tpcore_fvdas_mod.f. Claire Carouge has fixed this by changing the order of some of the parallel loops in the code. The updated TPCORE now runs much faster than before!


Hongyu Liu (hyl@nianet.org) reported:

The following references need to be deleted from the OBJS section in the Makefile:
   arctas_ship_emiss_mod.o       \
   cac_anthro_mod.o              \
   scale_anthro_mod.o            \
   tpcore_fvdas_mod.o            \
   vistas_anthro_mod.o           \
since these are already listed in the MODS section. Having these listed twice will cause a link-time error.


Colette Heald (heald@atmos.colostate.edu) wrote:

I wanted to let you know that to get v8-01-04 to compile [with the SunStudio compiler], I needed to modify the code in Makefile.sparc on L354 (i.e. under the line for gamap_mod.o) from:
   f90 -fpp -stackvar -O0 -xfilebyteorder=big16:%all
to the following:
   f90 -fpp -O0 -stackvar -xfilebyteorder=big16:%all $(FARCH) -c $*.f
otherwise it fails in input_mod.f compilation with messages that it did not recognize gamap_mod.f.
I also wanted to point out that there is a syntax error on L1040 of rpmares_mod.f (a comma after the write(6,*) but before the statement), which can cause compile to fail depending on the compiler options.


Fabien Paulot (fabienpaulot@gmail.com) also reported...

..an overestimate in the North-South mass flux computed in the new TPCORE. A geometric scale factor was applied twice when computing the flux. The updated TPCORE now correctly computes the NS mass flux.


Claire Carouge (ccarouge@seas.harvard.edu) wrote:

A GEOS-Chem user reported a slowdown also in the mass conservation calculation in transport_mod.f. I've changed the loops to parallelize over the tracers (it previously parallelized over levels) and it runs faster now.

All users of GEOS-Chem v8-01-04 should replace the existing versions of sulfate_mod.f, tpcore_fvdas_mod.f, and transport_mod.f with the fixed files. If you are using the PGI compiler, then also get the new Makefile.pgi file. If you are using the SunStudio compiler, then get the new Makefile.sparc file.

The updated files are available for download from:

ftp ftp.as.harvard.edu
cd pub/geos-chem/patches/v8-01-04
get sulfate_mod.f
get tpcore_fvdas_mod.f90
get transport_mod.f
get Makefile.sparc
get Makefile.pgi