Difference between revisions of "GEOS-FP implementation details"

Latest revision as of 14:15, 15 May 2017

NOTE: GEOS-FP is the current NASA/GMAO operational met data product. It is produced with a native resolution of 0.25° (lon) x 0.3125° (lat) x 72 hybrid vertical layers. GEOS-FP replaced the older GEOS-5.2.0 operational data product in June 2013.

GEOS-FP was originally given the designation GEOS-5.7.1, which was later updated to GEOS-5.7.2. Currently, GEOS-FP is being produced with version 5.11.0 of the GEOS Data Assimilation System (aka GEOS-DAS). This means that the "5.7.x" nomenclature is no longer correct. For this reason, we shall refer to this product as GEOS-FP from now on.

On this page we list the necessary modifications to GEOS-Chem for compatibility with the GEOS-FP meteorological data.

Overview

The GEOS-FP data product (aka GEOS5-FP) is the newest version of the operational meteorological data produced by NASA/GMAO. Overall, the structure of GEOS-FP is very similar to MERRA and MERRA-2 met data products.. Therefore, within GEOS-Chem we can (for the most part) treat GEOS-FP in the same way as we do for the MERRA and MERRA-2.

The table below compares many features of GEOS-FP with other met field products.

--Bob Y. (talk) 17:48, 17 August 2015 (UTC)

Required coding changes in GEOS-Chem

As noted in the section above, the similarities between MERRA and GEOS-FP (especially of the vertical grids) means that in most cases we can just treat MERRA in the same way as for GEOS-5. In many areas of the code, the coding changes are straightforward and simply involve an extension of C-preprocessor statements such as:

#if defined( GEOS_5 ) || defined( MERRA )

to

#if defined( GEOS_5 ) || defined( MERRA ) || defined( GEOS_FP )

In other areas of GEOS-Chem the changes are more involved. Below we provide a description of the areas of GEOS-Chem that were modified for MERRA:

Top-level directory

--Bob Y. 13:39, 7 November 2013 (EST)

Headers directory

The following modifications for GEOS-FP were made in the Headers/ directory:

--Bob Y. 13:52, 7 November 2013 (EST)

GeosUtil directory

The following modifications for GEOS-FP were made in the GeosUtil/ directory:

--Bob Y. 13:55, 7 November 2013 (EST)

GeosCore directory

The following modifications for GEOS-FP were made in the GeosCore/ directory:

--Bob Y. 14:23, 7 November 2013 (EST)

GeosApm directory

The following modifications for GEOS-FP were made in the GeosTomas/ directory:

--Bob Y. 13:49, 9 February 2012 (EST)

NcdfUtil directory

We have added the NcdfUtil/ directory, which contains routines for netCDF file I/O. These routines are based on the NcdfUtilities package by Bob Yantosca. These are built when you compile GEOS-Chem with the NETCDF=yes Makefile option.

--Bob Y. 10:34, 3 February 2012 (EST)

NcdfUtil/perl subdirectory

In this directory, we have added scripts for automatically generating Fortran code with the proper calls from the NcdfUtilities package. We anticipate that this will be a convenience for GEOS-Chem users, as it will relieve some of the drudgery of writing these calls by hand.

--Bob Y. 15:49, 12 May 2014 (EDT)

Resolved issues

Fractional sea ice

In order to preserve some information about sea ice leads during the regridding process, we have binned the fractional sea ice coverage into deciles (0-10%, 10-20% ... 90-100% coverage) and saved those as separate fields in the GEOS-FP data archive for GEOS-Chem. We have used the same algorithm we developed for MERRA met fields to do the binning.

--Bob Y. 16:12, 24 January 2012 (EST)

SNOMAS

The SNOMAS (snow depth) fields in both GEOS-5.7.2 and MERRA differ from that in GEOS-5.2.0. It appears that in GEOS-5.2.0 a default snow mass value was applied to grid boxes that had > 90% land ice coverage. This was not done in MERRA or GEOS-5.7.2. Bob Yantosca had implemented an algorithm to adjust the MERRA SNOMAS field to be backwards-compatible with GEOS-5.2.0. We have used this algorithm to adjust the GEOS-5.7.2 SNOMAS field accordingly.

--Bob Y. 16:14, 24 January 2012 (EST)

Dust emissions

The dust emissions in GEOS-Chem rely on "tunable parameters" to scale the dust emissions to a specified total. This is because the dust emissions are very highly dependent on the surface winds, and a small difference in winds (e.g. from GEOS-5 to GEOS-FP) can cause a huge difference in dust emission flux.

In routine DST_MBL (in dust_dead_mod.f), we have added the following code:

     !----------------------------------------------------------------
     ! Based on results from GEOS-FP 4x5 for 04/2012-04/2013
     !
     !   (GEOS-5 - GEOS-FP)/GEOS-5 * 100 is 23.56% in each size bin.
     !
     ! We need to scale to the parameter FLX_MSS_FDG_FCT to make the
     ! dust emissions consistent.  Consequently, to bring GEOS-FP 4x5
     ! dust emissions up to GEOS-5 levels, we need to MULTIPLY the
     ! FLX_MSS_FDG_FCT used for GEOS-5 by:
     !
     !             1 / (1. - 0.2356) = 1 / 0.7644 = 1.3082
     !
     !    -- R. Yantosca, M. Sulprizio, 07 Nov 2013
     !----------------------------------------------------------------
     REAL*8,  PARAMETER     :: FLX_MSS_FDG_FCT = 7.0d-4 * 1.3082d0

--Melissa Sulprizio 11:42, 7 November 2013 (EST)

Lightning

We now use OTD-LIS local redistribution files (provided by Lee Murray, dated 2013-11-07) from the lightning_NOx_201311/ subdirectory.

--Bob Y. 14:45, 7 November 2013 (EST)

Wet deposition and scavenging

Helen Amos, Bess Corbitt, and Qiaoqiao Wang had implemented an updated algorithm for wet scavenging that takes advantage of the new precipitation fields from the MERRA data archive. This was included in GEOS-Chem v9-01-01 and higher versions.

Because GEOS-5.7.2 contains the same precipitation data fields as MERRA (i.e. DQRCU, DQRLSAN, PFICU, PFILSAN, PFLCU, PFLLSAN, REEVAPCN, REEVAPLSAN), we can use the new wet deposition algorithm with the GEOS-5.7.2 met data.

--Bob Y. 16:16, 24 January 2012 (EST)

Implementation history

Initial source code modifications for GEOS-FP

This update was tested in the 1-month benchmark simulation v9-01-03g and approved on 27 Feb 2012.

1. Modify the Headers, GeosUtil, and GeosCore directories for GEOS-FP
2. Update the GeosApm and GeosTomas directories, primarily to synchronize files with GeosCore
3. Add NcdfUtil directory
4. Resolve several issues, including: Fractional sea ice, SNOMAS, Wet deposition and scavenging

--Melissa Payer 17:29, 24 February 2012 (EST)

Bug fix for declaration of GEOS-5.7.2 PFICU, PFLCU, PFILSAN, PFLLSAN fields

This update was tested in the 1-month benchmark simulation v9-02g and approved on 24 Mar 2013.

In module file GeosCore/gigc_state_met_mod.F90, it was discovered that the PFICU, PFLCU, PFILSAN, and PFLLSAN met fields for GEOS-FP were being allocated with the wrong vertical dimension. In GEOS-FP, these fields are placed on level edges and need to be allocated with LLPAR+1 levels. But in MERRA, these fields are placed on level centers and need to be allocated with LLPAR levels.

We have introduced the following fix in routine Init_GIGC_State_Met of Headers/gigc_state_met_mod.F90:

    INTEGER :: LX

    ... 

#elif defined( GEOS_FP ) || defined( MERRA )

    !=======================================================================
    ! GEOS_FP / MERRA met fields
    !=======================================================================
#if defined( GEOS_57 )
    LX = LM + 1
#elif defined( MERRA )
    LX = LM
#endif

    ALLOCATE( State_Met%PFICU     ( IM, JM, LX   ), STAT=RC )
    IF ( RC /= GIGC_SUCCESS ) RETURN
    State_Met%PFICU    = 0d0

    ALLOCATE( State_Met%PFILSAN   ( IM, JM, LX   ), STAT=RC )
    IF ( RC /= GIGC_SUCCESS ) RETURN
    State_Met%PFILSAN  = 0d0

    ALLOCATE( State_Met%PFLCU     ( IM, JM, LX   ), STAT=RC )
    IF ( RC /= GIGC_SUCCESS ) RETURN
    State_Met%PFLCU    = 0d0

    ALLOCATE( State_Met%PFLLSAN   ( IM, JM, LX   ), STAT=RC )
    IF ( RC /= GIGC_SUCCESS ) RETURN
    State_Met%PFLLSAN  = 0d0

This will make sure that the fields will be declared properly for either GEOS-FP or MERRA. (NOTE: LM is set to LLPAR, and thus LM+1 = LLPAR+1.

--Bob Y. 14:37, 7 November 2013 (EST)

Make sure State_Met%T is initialized properly for GEOS-FP simulations

For simulations with the GEOS-FP met fields, temperature is an I3 (i.e. 3-hr instantaneous) met field. We need to read temperature at times t and t+3hrs, and then interpolate linearly to the proper time.

At time t=0 of a GEOS-FP simulation, the initial temperature is read from netCDF files into State_Met%TMPU1. However, we must also save this into State_Met%T. Otherwise, temperature will be not initialized properly until the first call to INTERP. This can cause a floating-point error further upstream.

We have added the following code to routine Geos57_Read_I3_1 in GeosCore/geosfp_read_mod.F90:

     ! For now, copy State_Met%TMPU1 to State_Met%T.  At the next met field 
     ! read, we will State_Met%T from the values of State_Met%TMPU1 and
     ! State_Met%TMPU2. (bmy, 9/6/13)
     State_Met%T = State_Met%TMPU1

This error condition was caught by the GEOS-Chem unit tester when compiling with option FPE=yes.

--Bob Y. 13:13, 12 September 2013 (EDT)

Modifications to reduce the number of times netCDF files are opened

This update was validated in the 1-month benchmark simulation v11-01e (approved 04 Jan 2016).

NOTE: The following only applies to simulations done with GEOS-Chem "Classic". The HPC-enabled version of GEOS-Chem (aka "GEOS-Chem HP" uses the ESMF and MAPL libraries for file I/O.

In GEOS-Chem v11-01e, we have taken steps to reduce the number of times each GEOS-FP or MERRA-2 netCDF file is opened. This should reduce

1. We added several new global module variables for the netCDF file ID's to the top of GeosCore/geosfp_read_mod.F90 and GeosCore/merra2_read_mod.F90:

     ! netCDF file ID's
     INTEGER :: fA1     = -1
     INTEGER :: fA3cld  = -1
     INTEGER :: fA3dyn  = -1
     INTEGER :: fA3mstC = -1
     INTEGER :: fA3mstE = -1
     INTEGER :: fI3_1   = -1
     INTEGER :: fI3_2   = -1

2 In both GeosCore/geosfp_read_mod.F90 and GeosCore/merra2_read_mod.F90, reorganized routines that read netCDF data files from disk according to the following template:

     SUBROUTINE GeosFp_Read_A1( YYYYMMDD, HHMMSS, Input_Opt, State_Met )
       . . . 

       NOTE: added this line:
       LOGICAL, SAVE      :: first    = .TRUE.  ! First time reading data?
                
       . . .
 
       NOTE: Moved this code up front
       ! Find the proper time-slice to read from disk
       time_index = ( HHMMSS / 10000 ) + 1

       ! Stop w/ error if the time index is invalid
       IF ( time_index < 1 .or. time_index > 24 ) THEN
          WRITE( 6, 100 ) time_index
    100   FORMAT( 'Time_index value ', i5, ' must be in the range 1 to 24!' )
          CALL Error_Stop( errMsg, caller )
       ENDIF
 
       . . .        

       NOTE: Put this code in an IF block:
       IF ( time_index == 1 .or. first ) THEN 
          . . . 
   
          ! Open netCDF file
          CALL NcOp_Rd( fA1, TRIM( nc_file ) )
       
          ! Read the dimensions from the netCDF file
          CALL NcGet_DimLen( fA1, 'lon',   X )
          CALL NcGet_DimLen( fA1, 'lat',   Y )
          CALL NcGet_DimLen( fA1, 'time',  T )

          ! Make sure the dimensions of the file are valid
          CALL Check_Dimensions( lon=X,            lat=Y,        time=T,  &
                                 time_expected=24, caller=caller         )

          NOTE: added this line:
          ! Reset first-time flag
          first = .FALSE.
       ENDIF

       . . .

       NOTE: Use one of the module variables for the netCDF file ID.
       ! Read ALBEDO
       v_name = "ALBEDO"
       CALL NcRd( Q, fA1, TRIM(v_name), st3d, ct3d )
       State_Met%ALBD = Q

       . . . read other met fields from disk . . .

       NOTE: Close the file on the last timestep
       ! If it's the last time slice, then close the netCDF file
       ! and set the file ID to -1 to indicate that it's closed.
       IF ( time_index == 24 ) THEN
          CALL NcCl( fA1 )
          fA1 = -1
       ENDIF

and we do the same for all of the other file-reading routines in the both modules.

3. We added subroutine Cleanup_GeosFp_Read to the end of GeosCore/geosfp_read_mod.F90. This subroutine will be called from the CLEANUP routine at the end of a simulation to close any netCDF files that may still be open. This can happen if a simulation ends in the middle of a day.

     SUBROUTINE Cleanup_GeosFp_Read()
       . . .
       !=======================================================================
       ! Close any netCDF file ID's that are still open
       !=======================================================================
       IF ( fA1 > 0 ) THEN
          CALL NcCl( fA1 )
          WRITE( 6, 100 ) 'A1    '
       ENDIF

       IF ( fA3cld > 0 ) THEN
          CALL NcCl( fA3cld  )
          WRITE( 6, 100 ) 'A3cld '
       ENDIF

       IF ( fA3dyn > 0 ) THEN
          CALL NcCl( fA3dyn  )
          WRITE( 6, 100 ) 'A3dyn '
       ENDIF
 
       IF ( fA3mstC > 0 ) THEN
          CALL NcCl( fA3mstC )
          WRITE( 6, 100 ) 'A3mstC'
       ENDIF
   
       IF ( fA3mstE > 0 ) THEN
          CALL NcCl( fA3mstE )
          WRITE( 6, 100 ) 'A3mstE'
       ENDIF

       IF ( fI3_1 > 0 ) THEN
          CALL NcCl( fI3_1   )
          WRITE( 6, 100 ) 'I3_1  '
       ENDIF

       IF ( fI3_2 > 0 ) THEN
          CALL NcCl( fI3_2   )
          WRITE( 6, 100 ) 'I3_2  '
       ENDIF

       ! Format strings
   100 FORMAT( '     - Closing the GEOS-FP ', a6, ' file at end of simulation' )

     END SUBROUTINE Cleanup_GeosFp_Read

We also added the corresponding subroutine Cleanup_Merra2_Read to GeosCore/merra2_read_mod.F90.

4. We modified GeosCore/cleanup.F to call the Cleanup_GeosFp_Read and Cleanup_Merra2_Read routines:

     USE GEOSFP_READ_MOD,         ONLY : CLEANUP_GEOSFP_READ
     USE MERRA2_READ_MOD,         ONLY : CLEANUP_MERRA2_READ
     . . .
          ! Close GEOS-FP or MERRA2 met field files that may still be open
   #if defined( GEOS_FP )
         CALL Cleanup_GeosFp_Read()
   #elif defined( MERRA2 )
         CALL Cleanup_Merra2_Read()
   #endif

5. We added geosfp_read_mod.o and merra2_read_mod.o to the dependencies listing in GeosCore/Makefile:

   cleanup.o                   : cleanup.F                                      \
                                 aerosol_mod.o                                  \
                                 . . .
                                 geosfp_read_mod.o       merra2_read_mod.o      \
                                 . . .

--Bob Yantosca (talk) 22:41, 4 December 2015 (UTC)