GEOS-FP implementation details: Difference between revisions
| Line 890: | Line 890: | ||
=== Modifications to reduce the number of times netCDF files are opened === | === Modifications to reduce the number of times netCDF files are opened === | ||
<span style="color:green">'''''This update was validated in the 1-month benchmark simulation [[GEOS-Chem v11-01#v11-01e|v11-01e]] (approved 04 Jan 2016).'''''</span> | <span style="color:green">'''''This update was validated in the 1-month benchmark simulation [[GEOS-Chem v11-01 benchmark history#v11-01e|v11-01e]] (approved 04 Jan 2016).'''''</span> | ||
<span style="color:green">'''''NOTE: The following only applies to simulations done with GEOS-Chem "Classic". The HPC-enabled version of GEOS-Chem (akak "GEOS-Chem HP" uses the ESMF and MAPL libraries for file I/O.'''''</span> | <span style="color:green">'''''NOTE: The following only applies to simulations done with GEOS-Chem "Classic". The HPC-enabled version of GEOS-Chem (akak "GEOS-Chem HP" uses the ESMF and MAPL libraries for file I/O.'''''</span> | ||
Revision as of 18:57, 4 January 2016
NOTE: GEOS-FP is the current NASA/GMAO operational met data product. It is produced with a native resolution of 0.25° (lon) x 0.3125° (lat) x 72 hybrid vertical layers. GEOS-FP replaced the older GEOS-5.2.0 operational data product in June 2013.
GEOS-FP was originally given the designation GEOS-5.7.1, which was later updated to GEOS-5.7.2. Currently, GEOS-FP is being produced with version 5.11.0 of the GEOS Data Assimilation System (aka GEOS-DAS). This means that the "5.7.x" nomenclature is no longer correct. For this reason, we shall refer to this product as GEOS-FP from now on.
On this page we list the necessary modifications to GEOS-Chem for compatibility with the GEOS-FP meteorological data.
Overview
The GEOS-FP data product (aka GEOS5-FP) is the newest version of the operational meteorological data produced by NASA/GMAO. Overall, the structure of GEOS-FP is very similar to MERRA and MERRA2 met data products.. Therefore, within GEOS-Chem we can (for the most part) treat GEOS-FP in the same way as we do for the MERRA and MERRA2.
The table below compares many features of GEOS-FP with other met field products.
| Feature | MERRA2 | GEOS-FP | MERRA | GEOS-5 |
|---|---|---|---|---|
| Native vertical grid |
|
|
|
|
| Native horizontal grid |
|
|
|
|
| Time-invariant fields |
|
|
|
|
| Time archiving: surface fields |
|
|
|
|
| Time archiving: 3D fields |
|
|
|
|
| Timestamps | ||||
| Precipitation fields |
|
|
|
|
| Temperature |
|
|
|
|
| Relative humidity |
|
|
|
|
| Specific humidity |
|
|
|
|
| Vertical pressure velocity |
|
|
|
|
| Ertel potential vorticity |
|
|
|
|
| Snow mass |
|
|
|
|
| Sea level pressure |
|
|
|
|
| Surface pressure |
|
|
|
|
| Tropopause pressure |
|
|
|
|
| GEOS-Chem convection routine |
|
|
|
|
| GEOS-Chem wet deposition routine |
|
|
|
|
| GEOS-Chem transport routine |
|
|
|
|
| File format read by GEOS-Chem |
|
|
|
|
--Bob Y. (talk) 17:48, 17 August 2015 (UTC)
Required coding changes in GEOS-Chem
As noted in the section above, the similarities between MERRA and GEOS-FP (especially of the vertical grids) means that in most cases we can just treat MERRA in the same way as for GEOS-5. In many areas of the code, the coding changes are straightforward and simply involve an extension of C-preprocessor statements such as:
#if defined( GEOS_5 ) || defined( MERRA )
to
#if defined( GEOS_5 ) || defined( MERRA ) || defined( GEOS_FP )
In other areas of GEOS-Chem the changes are more involved. Below we provide a description of the areas of GEOS-Chem that were modified for MERRA:
Top-level directory
| File | Modifications made |
|---|---|
| Makefile_header.mk |
|
--Bob Y. 13:39, 7 November 2013 (EST)
Headers directory
The following modifications for GEOS-FP were made in the Headers/ directory:
| File | Modifications made |
|---|---|
| CMN_SIZE_mod.F |
|
| gigc_input_opt_mod.F90 |
|
| gigc_state_met_mod.F90 |
|
--Bob Y. 13:52, 7 November 2013 (EST)
GeosUtil directory
The following modifications for GEOS-FP were made in the GeosUtil/ directory:
| File | Modifications made |
|---|---|
| bpch2_mod.F |
In routine GET_MODELNAME:
In routine GET_NAME_EXT:
In routine GET_RES_EXT:
|
| grid_mod.F |
In routine COMPUTE_GRID:
In routine INIT_GRID:
|
| pressure_mod.F |
In routines GET_PEDGE and GET_PEDGE_FULLGRID:
In routine INIT_PRESSURE:
|
| time_mod.F |
In routine GET_A1_TIME:
|
| transfer_mod.F |
In several routines:
|
--Bob Y. 13:55, 7 November 2013 (EST)
GeosCore directory
The following modifications for GEOS-FP were made in the GeosCore/ directory:
| File | Modifications made |
|---|---|
| calcrate.F | Multiple instances:
|
| carbon_mod.F | Multiple instances:
|
| cldice_HbrHOBr_mod.F | Multiple instances:
|
| convection_mod.F |
In routine DO_CONVECTION:
|
| dao_mod.F |
In routine COPY_I6_FIELDS:
In routines IS_ICE, IS_LAND, IS_NEAR, IS_WATER:
In routine INTERP:
|
| depo_mercury_mod.F |
In routine ADD_Hg2_SNOWPACK
|
| diag3.F |
|
| dust_mod.F |
In routine SRC_DUST_DEAD:
|
| dust_dead_mod.F |
In routine DST_MBL:
|
| fast_j.F |
|
| gamap_mod.F |
In routine INIT_TRACERINFO:
|
| geosfp_read_mod.F90 |
|
| global_ch4_mod.F |
In routines WETLAND_EMIS and READ_COPROD:
|
| global_hno3_mod.F |
In routine GET_GLOBAL_HNO3:
|
| global_o3_mod.F |
In routine GET_GLOBAL_O3
|
| initialize.F |
|
| input_mod.F |
In routine READ_SIMULATION_MENU:
In routine READ_CONVECTION_MENU:
In routine READ_EMISSIONS_MENU:
In routine VALIDATE_DIRECTORIES:
|
| land_mercury_mod.F |
In routines LAND_MERCURY_FLUX:
In routines SOILEMIS:
|
| lightning_nox_mod.F |
In routine LIGHTNING:
In routine GET_OTD_LIS_SCALE:
|
| Makefile |
|
| main.F |
|
| megan_mod.F |
At top of module
In routine INIT_MEGAN:
|
| mercury_mod.F |
In routine CHEM_Hg0_Hg2:
|
| nei2005_anthro_mod.F |
In routines EMISS_NEI2005_ANTHRO and EMISS_NEI_ANTHRO_05x0666:
|
| ocean_mercury_mod.F |
Multiple instances:
|
| planeflight_mod.F |
In routine PLANEFLIGHT
|
| strat_chem_mod.F90 |
Multiple instances:
|
| sulfate_mod.F |
Multiple instances:
|
| tagged_ox_mod.F |
Multiple instances:
|
| tpcore_bc_mod.F: |
Multiple instances:
|
| tropopause_mod.F: |
Multiple instances:
|
| vdiff_mod.F90 |
In routine VDIFFDR:
|
| wetscav_mod.F |
In routines DO_WETDEP, DO_WASHOUT_ONLY
In routine MAKE_QQ:
In routine RAINOUT:
|
--Bob Y. 14:23, 7 November 2013 (EST)
GeosApm directory
The following modifications for GEOS-FP were made in the GeosTomas/ directory:
| File | Modifications made |
|---|---|
| Makefile |
|
| carbon_mod.F |
|
| dao_mod.F |
|
| dust_mod.F |
|
| dust_dead_mod.F |
|
| main.F |
|
| sulfate_mod.F |
|
| wetscav_mod.F |
|
--Bob Y. 13:49, 9 February 2012 (EST)
NcdfUtil directory
We have added the NcdfUtil/ directory, which contains routines for netCDF file I/O. These routines are based on the NcdfUtilities package by Bob Yantosca. These are built when you compile GEOS-Chem with the NETCDF=yes Makefile option.
| File | Description |
|---|---|
| Makefile | GNU Makefile used to build the NcdfUtil code. |
| TestNcdfUtil.F90 | Test program to verify that the netCDF libraries were built properly. (This is executed only when you issue a make ncdfcheck command.) |
| m_do_err_out.F90 | Routine to print out netCDF error messages. |
| m_netcdf_io_checks.F90 | Routines to check whether a netCDF file contains a specified variable or dimension. |
| m_netcdf_io_close.F90 | Routines to close a netCDF file. |
| m_netcdf_io_create.F90 | Routines used to open a netCDF file for output and to synchronize a netCDF file. |
| m_netcdf_io_define.F90 | Routines to define netCDF variables and attributes. Use this when writing data to a netCDF file. |
| m_netcdf_io_get_dimlen.F90 | Routines to return dimension information from a netCDF file. |
| m_netcdf_io_handle_err.F90 | Routines for netCDF I/O error handling. |
| m_netcdf_io_open.F90 | Routines for opening a netCDF file for reading or for writing. |
| m_netcdf_io_read.F90 | Routines for reading variables (of INTEGER, REAL*4, REAL*8, and CHARACTER types) from a netCDF file. |
| m_netcdf_io_readattr.F90 | Routines for reading global and variable attributes (of INTEGER, REAL*4, REAL*8, and CHARACTER types) from a netCDF file. |
| m_netcdf_io_write.F90 | Routines for writing data to a netCDF file |
--Bob Y. 10:34, 3 February 2012 (EST)
NcdfUtil/perl subdirectory
In this directory, we have added scripts for automatically generating Fortran code with the proper calls from the NcdfUtilities package. We anticipate that this will be a convenience for GEOS-Chem users, as it will relieve some of the drudgery of writing these calls by hand.
| File | Description |
|---|---|
| StrTrim.pm | Perl module used by the ncCode* scripts. Contains routines for stripping withe space from strings. |
| ncCodeDef | Perl script that generates Fortran code to open a netCDF file and define the various dimensions, variables, and attributes. |
| ncCodeWrite | Perl script that will generate Fortran code to write data to a netCDF file. |
| ncCodeRead | Perl script that will generate Fortran code to read data from a netCDF file. |
| nc_definitions.rc | Generic input file used by the ncCode* scripts. You can edit this. |
| definitions_cn.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP CN data. |
| definitions_a1.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP A1 data. |
| definitions_a3cld.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP A3cld data. |
| definitions_a3dyn.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP A3dyn data. |
| definitions_a3mstc.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP A3mstC data. |
| definitions_a3mste.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP A3mstE data. |
| definitions_i3.rc | Customized version of nc_definitions.rc that will generate Fortran code for reading GEOS-FP I3 data. |
--Bob Y. 15:49, 12 May 2014 (EDT)
Resolved issues
Fractional sea ice
In order to preserve some information about sea ice leads during the regridding process, we have binned the fractional sea ice coverage into deciles (0-10%, 10-20% ... 90-100% coverage) and saved those as separate fields in the GEOS-FP data archive for GEOS-Chem. We have used the same algorithm we developed for MERRA met fields to do the binning.
--Bob Y. 16:12, 24 January 2012 (EST)
SNOMAS
The SNOMAS (snow depth) fields in both GEOS-5.7.2 and MERRA differ from that in GEOS-5.2.0. It appears that in GEOS-5.2.0 a default snow mass value was applied to grid boxes that had > 90% land ice coverage. This was not done in MERRA or GEOS-5.7.2. Bob Yantosca had implemented an algorithm to adjust the MERRA SNOMAS field to be backwards-compatible with GEOS-5.2.0. We have used this algorithm to adjust the GEOS-5.7.2 SNOMAS field accordingly.
--Bob Y. 16:14, 24 January 2012 (EST)
Dust emissions
The dust emissions in GEOS-Chem rely on "tunable parameters" to scale the dust emissions to a specified total. This is because the dust emissions are very highly dependent on the surface winds, and a small difference in winds (e.g. from GEOS-5 to GEOS-FP) can cause a huge difference in dust emission flux.
In routine DST_MBL (in dust_dead_mod.f), we have added the following code:
!----------------------------------------------------------------
! Based on results from GEOS-FP 4x5 for 04/2012-04/2013
!
! (GEOS-5 - GEOS-FP)/GEOS-5 * 100 is 23.56% in each size bin.
!
! We need to scale to the parameter FLX_MSS_FDG_FCT to make the
! dust emissions consistent. Consequently, to bring GEOS-FP 4x5
! dust emissions up to GEOS-5 levels, we need to MULTIPLY the
! FLX_MSS_FDG_FCT used for GEOS-5 by:
!
! 1 / (1. - 0.2356) = 1 / 0.7644 = 1.3082
!
! -- R. Yantosca, M. Sulprizio, 07 Nov 2013
!----------------------------------------------------------------
REAL*8, PARAMETER :: FLX_MSS_FDG_FCT = 7.0d-4 * 1.3082d0
--Melissa Sulprizio 11:42, 7 November 2013 (EST)
Lightning
We now use OTD-LIS local redistribution files (provided by Lee Murray, dated 2013-11-07) from the lightning_NOx_201311/ subdirectory.
--Bob Y. 14:45, 7 November 2013 (EST)
Wet deposition and scavenging
Helen Amos, Bess Corbitt, and Qiaoqiao Wang had implemented an updated algorithm for wet scavenging that takes advantage of the new precipitation fields from the MERRA data archive. This was included in GEOS-Chem v9-01-01 and higher versions.
Because GEOS-5.7.2 contains the same precipitation data fields as MERRA (i.e. DQRCU, DQRLSAN, PFICU, PFILSAN, PFLCU, PFLLSAN, REEVAPCN, REEVAPLSAN), we can use the new wet deposition algorithm with the GEOS-5.7.2 met data.
--Bob Y. 16:16, 24 January 2012 (EST)
Implementation history
Initial source code modifications for GEOS-FP
This update was tested in the 1-month benchmark simulation v9-01-03g and approved on 27 Feb 2012.
- Modify the Headers, GeosUtil, and GeosCore directories for GEOS-FP
- Update the GeosApm and GeosTomas directories, primarily to synchronize files with GeosCore
- Add NcdfUtil directory
- Resolve several issues, including: Fractional sea ice, SNOMAS, Wet deposition and scavenging
--Melissa Payer 17:29, 24 February 2012 (EST)
Bug fix for declaration of GEOS-5.7.2 PFICU, PFLCU, PFILSAN, PFLLSAN fields
This update was tested in the 1-month benchmark simulation v9-02g and approved on 24 Mar 2013.
In module file GeosCore/gigc_state_met_mod.F90, it was discovered that the PFICU, PFLCU, PFILSAN, and PFLLSAN met fields for GEOS-FP were being allocated with the wrong vertical dimension. In GEOS-FP, these fields are placed on level edges and need to be allocated with LLPAR+1 levels. But in MERRA, these fields are placed on level centers and need to be allocated with LLPAR levels.
We have introduced the following fix in routine Init_GIGC_State_Met of Headers/gigc_state_met_mod.F90:
INTEGER :: LX
...
#elif defined( GEOS_FP ) || defined( MERRA )
!=======================================================================
! GEOS_FP / MERRA met fields
!=======================================================================
#if defined( GEOS_57 )
LX = LM + 1
#elif defined( MERRA )
LX = LM
#endif
ALLOCATE( State_Met%PFICU ( IM, JM, LX ), STAT=RC )
IF ( RC /= GIGC_SUCCESS ) RETURN
State_Met%PFICU = 0d0
ALLOCATE( State_Met%PFILSAN ( IM, JM, LX ), STAT=RC )
IF ( RC /= GIGC_SUCCESS ) RETURN
State_Met%PFILSAN = 0d0
ALLOCATE( State_Met%PFLCU ( IM, JM, LX ), STAT=RC )
IF ( RC /= GIGC_SUCCESS ) RETURN
State_Met%PFLCU = 0d0
ALLOCATE( State_Met%PFLLSAN ( IM, JM, LX ), STAT=RC )
IF ( RC /= GIGC_SUCCESS ) RETURN
State_Met%PFLLSAN = 0d0
This will make sure that the fields will be declared properly for either GEOS-FP or MERRA. (NOTE: LM is set to LLPAR, and thus LM+1 = LLPAR+1.
--Bob Y. 14:37, 7 November 2013 (EST)
Make sure State_Met%T is initialized properly for GEOS-FP simulations
For simulations with the GEOS-FP met fields, temperature is an I3 (i.e. 3-hr instantaneous) met field. We need to read temperature at times t and t+3hrs, and then interpolate linearly to the proper time.
At time t=0 of a GEOS-FP simulation, the initial temperature is read from netCDF files into State_Met%TMPU1. However, we must also save this into State_Met%T. Otherwise, temperature will be not initialized properly until the first call to INTERP. This can cause a floating-point error further upstream.
We have added the following code to routine Geos57_Read_I3_1 in GeosCore/geos57_read_mod.F90:
! For now, copy State_Met%TMPU1 to State_Met%T. At the next met field
! read, we will State_Met%T from the values of State_Met%TMPU1 and
! State_Met%TMPU2. (bmy, 9/6/13)
State_Met%T = State_Met%TMPU1
This error condition was caught by the GEOS-Chem unit tester when compiling with option FPE=yes.
--Bob Y. 13:13, 12 September 2013 (EDT)
Modifications to reduce the number of times netCDF files are opened
This update was validated in the 1-month benchmark simulation v11-01e (approved 04 Jan 2016).
NOTE: The following only applies to simulations done with GEOS-Chem "Classic". The HPC-enabled version of GEOS-Chem (akak "GEOS-Chem HP" uses the ESMF and MAPL libraries for file I/O.
In GEOS-Chem v11-01e, we have taken steps to reduce the number of times each GEOS-FP or MERRA-2 netCDF file is opened. This should reduce
- 1. We added several new global module variables for the netCDF file ID's to the top of GeosCore/geosfp_read_mod.F90 and GeosCore/merra2_read_mod.F90:
! netCDF file ID's
INTEGER :: fA1 = -1
INTEGER :: fA3cld = -1
INTEGER :: fA3dyn = -1
INTEGER :: fA3mstC = -1
INTEGER :: fA3mstE = -1
INTEGER :: fI3_1 = -1
INTEGER :: fI3_2 = -1
- 2 In both GeosCore/geosfp_read_mod.F90 and GeosCore/merra2_read_mod.F90, reorganized routines that read netCDF data files from disk according to the following template:
SUBROUTINE GeosFp_Read_A1( YYYYMMDD, HHMMSS, Input_Opt, State_Met )
. . .
NOTE: added this line:
LOGICAL, SAVE :: first = .TRUE. ! First time reading data?
. . .
NOTE: Moved this code up front
! Find the proper time-slice to read from disk
time_index = ( HHMMSS / 10000 ) + 1
! Stop w/ error if the time index is invalid
IF ( time_index < 1 .or. time_index > 24 ) THEN
WRITE( 6, 100 ) time_index
100 FORMAT( 'Time_index value ', i5, ' must be in the range 1 to 24!' )
CALL Error_Stop( errMsg, caller )
ENDIF
. . .
NOTE: Put this code in an IF block:
IF ( time_index == 1 .or. first ) THEN
. . .
! Open netCDF file
CALL NcOp_Rd( fA1, TRIM( nc_file ) )
! Read the dimensions from the netCDF file
CALL NcGet_DimLen( fA1, 'lon', X )
CALL NcGet_DimLen( fA1, 'lat', Y )
CALL NcGet_DimLen( fA1, 'time', T )
! Make sure the dimensions of the file are valid
CALL Check_Dimensions( lon=X, lat=Y, time=T, &
time_expected=24, caller=caller )
NOTE: added this line:
! Reset first-time flag
first = .FALSE.
ENDIF
. . .
NOTE: Use one of the module variables for the netCDF file ID.
! Read ALBEDO
v_name = "ALBEDO"
CALL NcRd( Q, fA1, TRIM(v_name), st3d, ct3d )
State_Met%ALBD = Q
. . . read other met fields from disk . . .
NOTE: Close the file on the last timestep
! If it's the last time slice, then close the netCDF file
! and set the file ID to -1 to indicate that it's closed.
IF ( time_index == 24 ) THEN
CALL NcCl( fA1 )
fA1 = -1
ENDIF
- and we do the same for all of the other file-reading routines in the both modules.
- 3. We added subroutine Cleanup_GeosFp_Read to the end of GeosCore/geosfp_read_mod.F90. This subroutine will be called from the CLEANUP routine at the end of a simulation to close any netCDF files that may still be open. This can happen if a simulation ends in the middle of a day.
SUBROUTINE Cleanup_GeosFp_Read()
. . .
!=======================================================================
! Close any netCDF file ID's that are still open
!=======================================================================
IF ( fA1 > 0 ) THEN
CALL NcCl( fA1 )
WRITE( 6, 100 ) 'A1 '
ENDIF
IF ( fA3cld > 0 ) THEN
CALL NcCl( fA3cld )
WRITE( 6, 100 ) 'A3cld '
ENDIF
IF ( fA3dyn > 0 ) THEN
CALL NcCl( fA3dyn )
WRITE( 6, 100 ) 'A3dyn '
ENDIF
IF ( fA3mstC > 0 ) THEN
CALL NcCl( fA3mstC )
WRITE( 6, 100 ) 'A3mstC'
ENDIF
IF ( fA3mstE > 0 ) THEN
CALL NcCl( fA3mstE )
WRITE( 6, 100 ) 'A3mstE'
ENDIF
IF ( fI3_1 > 0 ) THEN
CALL NcCl( fI3_1 )
WRITE( 6, 100 ) 'I3_1 '
ENDIF
IF ( fI3_2 > 0 ) THEN
CALL NcCl( fI3_2 )
WRITE( 6, 100 ) 'I3_2 '
ENDIF
! Format strings
100 FORMAT( ' - Closing the GEOS-FP ', a6, ' file at end of simulation' )
END SUBROUTINE Cleanup_GeosFp_Read
- We also added the corresponding subroutine Cleanup_Merra2_Read to GeosCore/merra2_read_mod.F90.
- 4. We modified GeosCore/cleanup.F to call the Cleanup_GeosFp_Read and Cleanup_Merra2_Read routines:
USE GEOSFP_READ_MOD, ONLY : CLEANUP_GEOSFP_READ
USE MERRA2_READ_MOD, ONLY : CLEANUP_MERRA2_READ
. . .
! Close GEOS-FP or MERRA2 met field files that may still be open
#if defined( GEOS_FP )
CALL Cleanup_GeosFp_Read()
#elif defined( MERRA2 )
CALL Cleanup_Merra2_Read()
#endif
- 5. We added geosfp_read_mod.o and merra2_read_mod.o to the dependencies listing in GeosCore/Makefile:
cleanup.o : cleanup.F \
aerosol_mod.o \
. . .
geosfp_read_mod.o merra2_read_mod.o \
. . .
--Bob Yantosca (talk) 22:41, 4 December 2015 (UTC)