GEOS-Chem Newsletter Spring 2012
GEOS-Chem Steering Committee
Minutes from the last GCSC telecon which was held on Wednesday 23 May 2012 @ 10:00 AM ET (14:00 GMT) are available HERE.
The next scheduled GEOS-Chem Steering Committee telecon will take place in the Sept-Oct time frame. We will keep you posted. As always, if you have any items for the Steering Committee please be sure to identify them to your Working Group chairpersons.
Since the last GEOS-Chem Steering Committee telecon (15 Feb 2012), the following 1-month benchmarks have been performed for GEOS-Chem v9-01-03. For the complete table of v9-01-03 1-month benchmarks, click HERE.
- PARANOX ship plume emissions (G.C.M. Vinken, Eindhoven)
- Bug fixes for David Streets emissions for nested model (J. Fisher, Harvard)
- Several minor bug fixes not affecting the full-chemistry simulation (GEOS-Chem Support Team, aka GCST)
- Addition of lightning scale factors for MERRA 2 x 2.5 met (GCST)
- Initial source code modifications for GEOS-5.7.2 (GCST)
- Support added for netCDF file I/O (GCST)
- Hg(II) gas-aerosol partitioning (H. Amos, Harvard)
- Nested grid CO2 simulation (Y.X. Wang, Tsinghua)
- Historical emission inventories of SO2, NOx, BC, POA (E. Leibensperger, MIT)
- CH4 simulation updates (K. Wecht, Harvard)
- Removed support for GEOS-3 (GCST)
- Bug fix in David Streets emissions for offline simulation (F. Paulot, Harvard)
- Updates to facilitate inclusion of Grid-Independent model code (GCST)
- Replace RETRO C2H6 with emissions from offline C2H6 simulation (GCST)
- Updated LAI code, to facilitate Grid-Independent model (GCST)
- Preliminary interface to connect G-C to an external GCM (GCST)
- Replace existing regridding routines with MAP_A2A package (M. Cooper, GCST)
- Daily and hourly GFED3 biomass emissions (P. Kasibhatla, Duke)
The following 1-month benchmarks are currently planned before the final 1-year benchmarking and public release of v9-01-03:
- Stratospheric monthly climatological P and K (Lee Murray, Harvard)
We will perform a 1-year benchmark for v9-01-03m in addition to the customary 1-month benchmarks. The bromine simulation needs to be in place before the Stratopsheric P/K update can be validated into the standard code.
One of our primary focuses in this past quarter was to ingest the grid-Independent modifications into the standard GEOS-Chem repository. This was necessary to ensure that the Grid-Independent development code would not lag behind the standard GEOS-Chem. Frequent synchronization of these two streams of model development will be necessary going forward.
Our goal has been to introduce the Grid-Independent functionality without destroying any of the existing GEOS-Chem functionality. We have done this in a manner akin to building a new highway bridge next to the old bridge. For a time, both bridges are situated side-by-side, but eventually the old bridge gets taken down and replaced by the new.
We have added new sections of Grid-Independent code into GEOS-Chem routines while keeping the existing code as well. We used a C-preprocessor switch named DEVEL to block off the new sections of code from the existing sections of code. For the most part, these modifications occur at the entry points of subroutines, so that the arguments that need to be passed into the subroutine may be passed by Fortran derived type objects rather than via arrays or by USE statements. (This will make it easier to interface with the ESMF framework). Once the new sections of code have been fully tested and validated with benchmark simulations, they will replace the older sections of GEOS-Chem code. We plan on updating many sections of code prior to the public release of v9-01-03, in order to avoid the confusion.
We have also started to add modifications necessary for driving GEOS-Chem with the GEOS-5.7.2 met data to the standard code.
We have an ongoing collaboration with NASA where we seek to embed GEOS-Chem within the NASA GEOS-5/GCM. In order to do this, the GEOS-Chem Support Team has been working towards making GEOS-Chem Grid-Independent, or able to accept an arbitrary horizontal grid definition. This will finally enable GEOS-Chem to utilize MPI parallelization (via the ESMF framework) and to run on many more CPUs than is possible with the current configuration.
The first phase of this project has involved preparing GEOS-Chem for the GCM interface. To date, Mike Long and Bob Yantosca have made significant progress towards this goal. Much of this work involved replacing legacy code based on Fortran-77 (i.e. common blocks, etc) with modern Fortran-90 code. We have already added many of these source code modifications to the standard GEOS-Chem repository for GEOS-Chem v9-01-03.
A secondary partner in our Grid-Independent GEOS-Chem project is the Beijing Climate Center (BCC), who wishes to interface the Grid-Independent GEOS-Chem into its GCM (which is a version of CAM). In February 2012, Mike Long visited the BCC in order to test the interface that links GEOS-Chem to the BCC/GCM. This exercise proved to be very fruitful, and Mike identified a number of technical issues that needed to be resolved immediately. These issues would have also prevented the Grid-Independent GEOS-Chem from interfacing effectively with the NASA GEOS-5/GCM. We therefore made it a priority to correct each of these issues as soon as possible.
Here is a list of the issues that Mike found, and the solutions that we implemented:
|Problem: The BCC/GCM cannot read data from ASCII input files. Several of GEOS-Chem's legacy codes (dry deposition, leaf area index, etc) read from ASCII files. Because the BCC/GCM runs with MPI parallelization, each CPU would have to read the ASCII input files independently, which would adversely impact scalability.
|Solution: We converted several of GEOS-Chem's data files (e.g. drydep.table, drydep.coef, vegtype.global, lai*.global) from ASCII to netCDF. We introduced the capability to read netCDF data (based on Bob Yantosca's NcdfUtilities package) into GEOS-Chem v9-01-03. The input.geos file that is used to specify GEOS-Chem options was left untouched for the moment.
|Problem: The GEOS-Chem dry deposition and soil NOx emissions modules rely on Olson land map (1992) data contained in an ASCII file named vegtype.global. A different version of vegtype.global is currently required for each different horizontal grid resolution (4° x 5°, 2° x 2.5°, nested-grids). However, for grid-independent operation, we cannot assume a-priori what the grid definition shall be.
|Solution: Bob Yantosca wrote a new land type module which reads the Olson land map (1992) data from its native resolution (0.5° x 0.5° ) in netCDF format and regrids it to the current GEOS-Chem resolution. The existing arrays used by the GEOS-Chem dry deposition and soil NOx emissions code are populated in the exact same manner as before. Also, the user now has the option to switch to the Olson land map (2001) data at 0.25° x 0.25° resolution. This modification has already been added to GEOS-Chem v9-01-03.
|Problem: The MODIS leaf area index (LAI) data were being read from several different sources. The Dry deposition and Soil NOx emissions routines were reading MODIS LAI data from ASCII files named lai*.global. The data in lai*.global were indexed by land type in the same order as the land types were listed in vegtype.global. Different versions of lai*.global had to be created for each horizontal grid. On the other hand, the MEGAN biogenic emissions module was reading MODIS LAI data from binary punch files at 1° x 1° native resolution, which were then regridded to the current horizontal resolution. Finally, the nested-grid simulations were reading from a separate set of LAI data files that were created for each at 0.5° x 0.666° nested region.
|Solution: Bob Yantosca wrote a new MODIS LAI module that reads LAI data at the native resolution of 0.5° x 0.5° (in netCDF format) and regrids to the current horizontal grid. The LAI arrays used by GEOS-Chem are populated in exactly the same manner as before. Thus, all LAI data now used by GEOS-Chem now comes from the same source at the same resolution. Also, the user has the option of switching to the MODIS LAI data at 0.25° x 0.25° resolution (corresponding to the Olson 2001 land map). This modification has already been added to v9-01-03.
|Problem: The FAST-J photolysis module uses a default overhead ozone climatology that is overwritten with TOMS/SBUV ozone values (for the months and latitudes where TOMS data exists). Monthly TOMS/SBUV data is read from disk directly in the driver routine fast_j.F and the interpolation to the current day and overwriting of the default O3 columns was done in the lower-level routine set_prof.F. However, when GEOS-Chem is interfaced with a GCM, it will need to use the overhead O3 columns that is generated by the GCM. Also, when we interface GEOS-Chem to the GCM, file I/O operations will not be allowed to be performed at that level in the code.
|Solution: We now pass the overhead O3 columns as an input to the FAST-J photolysis module. A new routine in main.F is used to determine if the default overhead O3 climatology needs to be overwritten with TOMS/SBUV O3. The resultant overhead O3 is then passed as an argument directly to the FAST-J module. FAST-J will also be able to directly accept overhead O3 columns from a GCM. This modification has already been added to v9-01-03.
|Problem: GEOS-Chem currently assumes a pure Cartesian grid. Query functions GET_XMID(I), GET_YMID(J), GET_XEDGE(I), GET_YEDGE(J) etc. from grid_mod.F are used to return longitude and latitude information at grid box centers and edges. However, in the future we may not be able to assume that GEOS-Chem will always be running with a regular Cartesian grid.
|Solution: We have introduced a new module grid_mod.F90 and have extended the query functions to accept lon, lat, and level indices, i.e., GET_XMID(I,J,L), GET_YMID(I,J,L), GET_XEDGE(I,J,L), GET_YEDGE(I,J,L). This will leave room for future expansion in case GEOS-Chem ever should use a non-regular horizontal grid.
It should be noted that several of these modifications, in particular new Olson land map and MODIS leaf area index modules, also facilitated the effort to make GEOS-Chem compatible with the GEOS-5.7.2 meteorology.
We have currently reached the end of Year 1 of the Grid-independent GEOS-Chem project. At the end of May 2012, we shall deliver the first GEOS-Chem module to NASA for inclusion into the GEOS-5/GCM. This will be our Chemistry Component (i.e. GEOS-Chem's chemical mechanism solver, photolysis, and dry deposition routines). Postdoc Christoph Keller is working on a new Grid-Independent Emissions Component, which will occupy most of Year 2.
GEOS-5.7.2 met field update
NASA/GMAO has updated the operational GEOS system from GEOS-5.2.0 to GEOS-5.7.2. Here is a quick summary of the new GEOS-5.7.2 data product:
- Horizontal resolution: 0.25° lat x 0.3125° lon (i.e. 1/4° x 5/16°)
- Vertical resolution: 72 hybrid layers (identical to the vertical grids used by the GEOS-5.2.0 and MERRA met data products)
- Time resolution: Hourly and 3-hourly data (timestamps very similar to MERRA)
- Input data format: netCDF-4 (as opposed to HDF4-EOS, used for past GMAO data products)
- Output data format: netCDF-3 (instead of flat binary). This will allow the files to be read by IDL routines such as GAMAP.
We plan to store the GEOS-5.7.2 following data at Harvard (which will be downloadable by the community). Data will be processed for Aug 2011-onward. Priority will be given to these grids:
- Global 2° x 2.5°
- Global 4° x 5°
- SEA4CRS Nested grid at 0.25° x 0.3125° resolution
The large size of the nested data files makes disk storage an issue. At present, we plan on giving priority to the SEA4CRS nested grid.
We have added several structural modifications for GEOS-5.7.2 met into GEOS-Chem v9-01-03. Because the structure of the GEOS-5.7.2 met data is very similar to MERRA, many of the modifications simply involved simply extending C-preprocessor statements such as:
#if defined( MERRA )
#if defined( MERRA ) || defined( GEOS_57 )
which tells the code to treat GEOS-5.7 in the same way as we already do for MERRA.
Patrick Kim has done some initial benchmarking of GEOS-Chem driven with the GEOS-5.7.2 met product and have compared the results to GEOS-5.2 and MERRA. His findings are included in the separate document entitled SEAC4RS_Update.pdf which I have distributed along with this report. The GEOS-5.7.2 moist fields seem to be significantly different as compared to MERRA. We are still investigating.
Yuxuan Wang and her team at Tsinghua have been working on the nested-grid transport at 0.25° x 0.3125° resolution.
Transition to netCDF file format
One of the necessary steps that we must undertake as we prepare to interface GEOS-Chem with an external GCM (such as the NASA GEOS-5/GCM) is to adopt netCDF file I/O. The current binary punch format used by GEOS-Chem is incompatible with MPI parallelization in the sense that small chunks of a binary punch file cannot be read simultaneously across CPUs in a distributed memory environment. Rather, the entire file must be read on each CPU that is executing, thus adding much more file I/O overhead to the system, which in turn negatively impacts model performance and scalability.
Starting with v9-01-03g, we have added netCDF file I/O capability to GEOS-Chem. Going forward, all GEOS-Chem users will have to have access to a netCDF library installation on their systems in order to be able to run GEOS-Chem. Where there are several GEOS-Chem users at a single institution, the libraries only have to be built once (by the IT guru) and then placed in a common directory where all users may access them. Bob Yantosca and Lee Murray have created a user-friendly netCDF installer which should simplify the library build process and also help to standardize the GEOS-Chem compilation sequence.
We realize the transition to full netCDF I/O cannot be achieved overnight. The first step shall be to modify GEOS-Chem to be able to read certain netCDF data files from disk. This has already been done with the following input files:
- GEOS-5.7.2 met data files
- Olson land map (1992) and (2001)
- MODIS LAI data
- Dry deposition input files
As GEOS-Chem development progresses, the list of input files in netCDF format shall grow until eventually all of GEOS-Chem's input files will have been migrated to netCDF format.
At a later time, in conjunction with the Grid-independent GEOS-Chem project, we shall also begin to convert GEOS-Chem's diagnostic outputs to netCDF format. Fortran code for writing netCDF files is necessarily much more wordy than code for reading from netCDF files because one must supply information (aka metadata) about the variables that will constitute each netCDF file. To simplify the coding process for the GEOS-Chem community, Bob Yantosca has included into GEOS-Chem some Perl scripts which will automatically generate the proper Fortran for both writing to and reading from netCDF files.
De-supporting of GEOS-3 meteorology
All support for GEOS-3 met data has now been officially removed from GEOS-Chem. GEOS-3 is now several versions behind the current GMAO met data product, and is obsolete.
We shall still keep the archive of GEOS-3 met data at both Harvard and Dalhousie in case there are users who still wish to access this data.
GEOS-Chem Adjoint news
Please see the new GEOS-Chem Adjoint v33 wiki page for a complete list of updates pertaining to the current GEOS-Chem adjoint version. This wiki page is updated often, so please check back frequently for the latest news!
Also, any technical questions that you may have pertaining to the adjoint source code may be directed to Nicolas Bousserez, who recently joined the GEOS-Chem Support Team as GEOS-Chem Adjoint liaison. You may contact Nicolas (and all other Support Team) members by sending an email to email@example.com.
- Support for reading/plotting GEOS-5.7.2 met data
- Improved support for netCDF file I/O
- Updated code for plotting results from GEOS-Chem 1-month benchmarks
- Flexible colorbar scaling with the /CBSCALE option (submitted by Chris Holmes)
- Now can read binary files larger than 2.4 GB in size (submitted by Prasad Kasibhatla)
This release shall standardize several fixes that had been made to the GAMAP source code over the past year. We shall also update the GAMAP manual to remove outdated information.
We would like to each GEOS-Chem user group to take a moment to review the research blurb and list of group members on the GEOS-Chem people and projects page. Also, we apologize if you sent us updates that were not posted on the website. As you can see from all of the news above, it has been a busy few months for us on the GEOS-Chem Support Team!
(with input from Mike Long, Melissa Payer, and Matthew Cooper)
21 May 2012
--Bob Y. 16:34, 21 May 2012 (EDT)