GEOS-5 met field reprocessing

NASA/GMAO ceased production of GEOS-5.2.0 data as of June 2013. See GMAO's GEOS Near-Real Time Data Products page for more details. We recommend that you use either GEOS-FP, which is the current operational GMAO met data product, or MERRA-2, which is the current GMAO reanalysis product. GEOS-5 was de-supported in GEOS-Chem v11-02.

NOTE: the reprocessing of GMAO GEOS-5 data described on this wiki page was completed in 2009. We reprocessed all of the GEOS-5.0.1 data to GEOS-5.1.0 data.

On this page, we describe a recent reprocessing of the GEOS-5 met fields that was done to correct an error in the optical depths.

For the latest information about GEOS-5, please consult the following GEOS-5 resources:

Overview

NOTE: The reprocessing of the GEOS-5 met field reprocessing to correct the problem in the optical depth regridding was completed as of 10 July 2009. The necessary source code changes have been incorporated into GEOS-Chem v8-02-03 and later versions. We shall keep this information here for future reference.

Data versions

GEOS-Chem uses the GEOS-5 operational data product, which is the product that starts approximately in 2004 and goes forward in time. Over the last couple of years there have been 3 different versions of GEOS-5 met data:

GEOS-5.0.1 (produced prior to approx 1/1/2007)
GEOS-5.1.0 (produced between 1/1/2007 to approx. 9/2008)
GEOS-5.2.0 (produced after approx. 9/2008)

When we started preparing GEOS-Chem to ingest the GEOS-5 met data, the only available GEOS-5 data was the very oldest data product, GEOS-5.0.1. Therefore when we downloaded and regridded our met data years 2005-2007, these data were the GEOS-5.0.1 product.

Around the beginning of 2008, GEOS-5.0.1 was turned off and replaced by a newer version of the dataset, GEOS-5.1.0. Therefore, the GEOS-5 met data that we downloaded and regridded for the calendar year 2008 was the GEOS-5.1.0 product. It also turned out that GEOS-5.1.0 had several improvements in the convective fields (e.g. CMFMC, DTRAIN) used by GEOS-Chem with respect to GEOS-5.0.1. Therefore we were recommended to replace all of our GEOS-5.0.1 data (i.e. years 2005-2007) with the newer GEOS-5.1.0 data.

In September 2008, the GEOS-5.1.0 data set was switched off and replaced by a newer data set, GEOS-5.2.0. The GEOS-5.2.0 is now the stable operational dataset of GEOS-5. This is also the version that will be used to create the MERRA reanalysis that will extend for many years into the past. Thus, all GEOS-5 met data that we downloaded and regridded for GEOS-Chem on or after 9/1/2008 is the GEOS-5.2.0 data. For more information about the switchover to GEOS-5.2.0, please visit this link.

Optical depth error

After several years of GEOS-5 met data had been downloaded & regridded for GEOS-Chem, it was discovered that there was an error in the optical depth field. For detailed information please see this link.

Long story short: the optical depth fields provided to us by GMAO were mislabled. We had thought that these were grid-box optical depths but in reality they were in-cloud optical depths. We were able to make a quick fix in order to let GEOS-Chem users continue to use the existing GEOS-5 met data. However, the more robust solution was to reprocess the GEOS-5 met data so as to handle the optical depth regridding properly. (Hongyu Liu gave us a regridding algorithm for this).

Data regridding

NOTE: Replacing GEOS-5.0.1 with GEOS-5.1.0 data was was completed on July 10, 2009. We shall leave this information here for future reference.

As described above, we have already commenced to reprocess the entire GEOS-5 met data set to:

To fix the optical depth error.
To replace our 2005-2007 GEOS-5 data files from the older GEOS-5.0.1 data product with the more recent GEOS-5.1.0 data product (and thus take advantage of the better convection fields).
To extract the GEOS-5.1.0 data product for the year 2004 (which had not been previously downloaded).

The GEOS-5 met data reprocessing is totally automated, but the limiting factor is the file transfer speeds. We therefore ask for your patience in this matter. At present we can process about a month of GEOS-5 met field files per business day.

Status

As of July 10, 2009, we have reprocessed the following GEOS-5 met data:

2003: December
2004: entire year
2005: entire year
2006: entire year
2007: entire year
2008: Jan 1 - Aug 31 (GEOS-5.2.0 data begins on Sep 1, 2008)

--Bob Y. 09:52, 4 February 2010 (EST)

Required source code modifications

NOTE: These source code modifications are now obsolete, as the GEOS-5 met field reprocessing effort was completed in July 2009. If you are using a version of GEOS-Chem prior to v8-02-03 you may have to make these modifications. However, We strongly recommend migrating to the latest version of GEOS-Chem.

The reprocessing of the GEOS-5 met field reprocessing to correct the problem in the optical depth regridding was completed as of 10 July 2009. The necessary source code changes have been incorporated into GEOS-Chem v8-02-03 and later versions.

In order to use the new GEOS-5 met data fields, we recommend that you upgrade to GEOS-Chem v8-01-04 as soon as it is released. Version v8-01-04 will have all of the necessary source code modifications that you need to use the newly reprocessed GEOS-5 met data.

However, if you are already using GEOS-Chem v8-01-02 or GEOS-Chem v8-01-03, these have some (but maybe not all) of the required changes. You may have to add some of the changes into these versions manually.

If you are using a version of GEOS-Chem older than v8-01-02, then you will have to upgrade.

File list

In include file define.h make sure that you have the IN_CLOUD_OD switch uncommented:

!----- FOR GEOS-5 MET FIELDS ONLY -----
! NOTE: If you are using GEOS-5 met fields that were reprocessed to 
! correctly regrid the in-cloud optical depth and cloud fraction fields, 
! then be sure to uncomment the following line of code.  This will cause 
! FAST-J to interpret the optical depth correctly.  Leaving this option
! commented will cause a "quick fix" (i.e. multiplying the optical depth
! by the cloud fracton) to be applied, which should be a good enough fix
! in the meantime. (bmy, hyl, 10/24/08)
#define IN_CLOUD_OD  'IN_CLOUD_OD'

The source-code routines that depend on the IN_CLOUD_OD switch are:

a3_read_mod.f
a6_read_mod.f
i6_read_mod.f
fast_j.f
lightning_nox_mod.f

In particular, if you are using v8-01-02 or GEOS-Chem v8-01-03, please double-check the following files and make sure you have these changes implemented. (These were based on bug reports from users after v8-01-02 and v8-01-03 had already been released:

Fix in a6_read_mod.f

We need to manually set the number of expected A6 fields in the file to be 21. Otherwise GEOS-Chem will not read in all of the fields properly. Please place the following #if defined block after the READ statement in subroutine OPEN_A6_FIELDS (in source code file a6_read_mod.f):

         ! The last 2 digits of the ident string
         ! is the # of fields contained in the file
         READ( IDENT(7:8), '(i2.2)' ) N_A6_FIELDS

#if   defined( GEOS_5 ) && defined( IN_CLOUD_OD ) 
         !%%% KLUDGE: set N_A6_FIELDS=21 when using the reprocessed
         !%%% GEOS-5 met.   This accounts for CMFMC (which doesn't seem
         !%%% to get counted) as well as for MOISTQ, which is an extra
         !%%% derived field.  (jmao, bmy, 2/12/09)
         N_A6_FIELDS = 21
#endif

Fix in lightning_nox_mod.f

You will need to obtain a modified OTD-LIS local redistribution file for the 4 x 5 and 2 x 2.5 global grids. These must be placed into your lightning_NOx_200709 subdirectories of your 4 x 5 and 2 x 2.5 global data directory path. You can obtain these files from

ftp ftp.as.harvard.edu
cd pub/geos-chem/data/GEOS_4x5/lightning_NOx_200709
get OTD-LIS-Local-Redist.CTH.v3.geos5.4x5.reprocessed_geos5_fix
cd pub/geos-chem/data/GEOS_2x2.5/lightning_NOx_200709
get OTD-LIS-Local-Redist.CTH.v3.geos5.2x25.reprocessed_geos5_fix

and then you must make sure that you have the following fixes in source code file: lightning_nox_mod.f:

(1) In routine READ_LOCAL_REDIST, make sure you add the "TEMPORARY FIX" section just before the WRITE( 6, 100 ) statement where the filename is printed out:

!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
!%%% TEMPORARY FIX FOR REPROCESSED GEOS-5 MET FIELDS (ltm, bmy, 2/18/09)
!%%%  
!%%% We need to read from a new OTD-LIS redistribution file that has taken
!%%% account of the new GEOS-5 convection.  Assume that this file is located
!%%% in the lightning_NOx data directory. (ltm, bmy, 2/19/09)
!%%%
!%%% NOTE: This change will be rendered obsolete when the entire data set of 
!%%% GEOS-5 is reprocessed and the OTD-LIS redistribution files are recomputed.
!%%%
#if   defined( IN_CLOUD_OD ) 
      FILENAME = TRIM( FILENAME ) // '.reprocessed_geos5_fix'
#endif
!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

      ! Echo info
      WRITE( 6, 100 ) TRIM( FILENAME )
 100  FORMAT( '     - READ_LOCAL_REDIST: Reading ', a )

(2) In routine GET_OTD_LIS_SCALE add the "TEMPORARY FIX" section after the existing IF block for the 4 x 5 grid:

#elif defined( GEOS_5 ) && defined( GRID4x5 )

      !-------------------------------------
      ! GEOS-5: 4 x 5 global simulation
      !-------------------------------------
      IF ( LCTH ) THEN
         IF ( LOTDLOC ) THEN
            SCALE = ANN_AVG_FLASHRATE / 17.5892d0
         ELSE
            SCALE = ANN_AVG_FLASHRATE / 13.9779d0
         ENDIF
      ELSE
         WRITE( 6, '(a)' ) 'Warning: OTD-LIS GEOS5 scaling only for CTH'   
         SCALE = 1.0d0
      ENDIF

!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
!%%% TEMPORARY FIX FOR REPROCESSED GEOS-5 MET FIELDS (ltm, bmy, 2/18/09)
!%%%
!%%% Deal with the change in convection used by GMAO by resetting the 4 x 5
!%%% resolution factor to match that for the new convection (ltm, bmy, 2/18/09)
!%%% 
!%%% NOTE: This change will be rendered obsolete when the entire data set of 
!%%% GEOS-5 is reprocessed and the OTD-LIS redistribution files recomputed.
!%%%
#if   defined( IN_CLOUD_OD )
      IF ( LCTH ) THEN
         IF ( LOTDLOC ) THEN
            SCALE = ANN_AVG_FLASHRATE / 22.4383d0
         ELSE
            SCALE = ANN_AVG_FLASHRATE / 18.3875d0
         ENDIF
      ENDIF
#endif
!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

(3) In routine GET_OTD_LIS_SCALE add the "TEMPORARY FIX" section after the existing IF block for the 2 x 2.5 grid:

#elif defined( GEOS_5 ) && defined( GRID2x25 )

      !-------------------------------------
      ! GEOS-5: 2 x 2.5 global simulation
      !-------------------------------------
      IF ( LCTH ) THEN
         IF ( LOTDLOC ) THEN
            SCALE = ANN_AVG_FLASHRATE / 47.7117d0
         ELSE
            SCALE = ANN_AVG_FLASHRATE / 36.2878d0
         ENDIF
      ELSE
         WRITE( 6, '(a)' ) 'Warning: OTD-LIS GEOS5 scaling only for CTH'
         SCALE = 1.0d0
      ENDIF

!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
!%%% TEMPORARY FIX FOR REPROCESSED GEOS-5 MET FIELDS (ltm, bmy, 2/18/09)
!%%%
!%%% Deal with the change in convection used by GMAO by resetting the 2 x 2.5
!%%% resolution factor to match that for the new convection (ltm, bmy, 2/18/09)
!%%%
!%%% NOTE: This change will be rendered obsolete when the entire data set of 
!%%% GEOS-5 is reprocessed and the OTD-LIS redistribution files recomputed.
!%%%
#if   defined( IN_CLOUD_OD )
      IF ( LCTH ) THEN
         IF ( LOTDLOC ) THEN
            SCALE = ANN_AVG_FLASHRATE / 67.6701d0
         ELSE
            SCALE = ANN_AVG_FLASHRATE / 52.0135d0
         ENDIF
      ENDIF
#endif
!%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

That's it!!!

--Bob Y. 12:39, 23 March 2009 (EDT)

How can I determine if I have the reprocessed GEOS-5 data?

Rebecca Buchholz wrote:

I have also been reading on the G-C site that GEOS-5 met data have been reprocessed. Is there a definitive way to determine whether we already have the reprocessed fields?

Bob Yantosca wrote:

The reprocessed fields have these file sizes:

   2 x 2.5:
   -rw-r--r-- 1 bmy geos  13846288 2010-06-04 10:29 20100601.a3.2x25
   -rw-r--r-- 1 bmy geos 317224336 2010-06-04 10:31 20100601.a6.2x25
   -rw-r--r-- 1 bmy geos   1048976 2010-06-04 10:29 20100601.i6.2x25

   4 x 5:
   -rw-r--r-- 1 bmy geos  3505936 2010-06-04 10:29 20100601.a3.4x5
   -rw-r--r-- 1 bmy geos 80179600 2010-06-04 10:31 20100601.a6.4x5
   -rw-r--r-- 1 bmy geos   265616 2010-06-04 10:29 20100601.i6.4x5

You can also use the GAMAP program test_met.pro to view a list of the met fields. Call it with:

   test_met, file='20100601.a3.2x25'

etc. A-3 files should have these 33 fields:

   ALBEDO  CLDTOT EFLUX    EVAP    FRLAKE  FRLAND  FRLANDIC
   FROCEAN GRN    GWETROOT GWETTOP HFLUX   LAI     LWGNET
   LWTUP   PARDF  PARDR    PBLH    PRECANV PRECCON PRECTOT
   PRECSNO QV2M   SNOMAS   SNODP   SWGNET  T2M     TROPP
   TSKIN   U10M   USTAR    V10M    Z0M

A-6 files should have these 21 fields:

   DQRCON DQRLSC DTRAIN   QL       QI       OPTDEPTH TAUCLI
   TAUCLW CLOUD  PV       OMEGA    QV       RH       T
   U      V      DQIDTMST DQLDTMST DQVDTMST MOISTQ   CMFMC

I-6 files should have these 5 fields:

   PS  LWI SLP TO3 TTO3

So you should be able to readily tell if you have the older or newer met fields.

--Bob Y. 13:15, 6 July 2010 (EDT)

GEOS-5 met field reprocessing

Contents

Overview

Data versions

Optical depth error

Data regridding

Status

Required source code modifications

File list

Fix in a6_read_mod.f

Fix in lightning_nox_mod.f

How can I determine if I have the reprocessed GEOS-5 data?

Navigation menu

Personal tools

Namespaces

Variants

Views

More

Search

Navigation

Tools