GEOS-Chem Newsletter Winter 2012
Dear GEOS-Chem users,
We are happy to bring you the Winter 2012 edition of the GEOS-Chem Newsletter! Please read below to find out the latest goings-on in the GEOS-Chem community.
The GEOS-Chem Support Team
- 1 GEOS-Chem Steering Committee
- 2 GEOS-Chem Support Team
- 3 Public Release of v9-01-02
- 4 GEOS-Chem v9-01-03
- 5 Grid-Independent GEOS-Chem
- 6 GEOS-5.7.2 met field update
- 7 MERRA met field update
- 8 Transition to netCDF file format
- 9 De-supporting of GEOS-3 meteorology
- 10 Other development tasks
- 11 Downtime at Harvard: Feb 19-23, 2012
GEOS-Chem Steering Committee
The next GEOS-Chem Steering Committee meeting shall take place on Wednesday, February 15th, 2012 at 10AM ET. If you have any issues you would like to bring to the attention of the Steering Committee, please bring it to the attention of your Working Group Chairpersons.
We invite you to read meeting minutes.
The next telecon is scheduled for May-June 2012, date TBD.
GEOS-Chem Support Team
Here is a summary of GEOS-Chem Support Team activities:
- Bob Yantosca has been working on the code to process GEOS-5.7.2 data (see below). He also oversees general GC development and the development of the Grid-Independent model.
- Michael Long is primarily working on the Grid-Independent GEOS-Chem project (see below).
- Melissa Payer has been working on general GC development tasks (i.e. code merging, generating 1-month benchmark simulations, user support, documentation, etc.).
- Matt Cooper (Dalhousie) has been working on regridding the MERRA met data to 2° x 2.5° degree resolution (see below). He has also begun replacing the existing regridding routines in GEOS-Chem with the MAP_A2A regridding package (which is what is currently use for regridding the GMAO met fields).
Public Release of v9-01-02
Since the previous GCSC telecon in November, the GEOS-Chem Support Team has issued a public release (on 28 Nov 2011) for GEOS-Chem v9-01-02. Most of the updates in this version consisted of bug fixes and structural code changes.
In addition to the bug fixes and structural changes, several several scientific updates were added to v9-01-02, namely:
- Imposed seasonality on David Streets NH3 emissions (Jenny Fisher, Harvard)
- Updated dry deposition velocities for aerosols over ice and snow (Jenny Fisher, Harvard)
- APM Aerosol Microphysics Package (SUNY/Albany group)
- GFED3 biomass burning emissions (Prasad Kasibhatla, Duke)
- Implementation of RETRO Anthropogenic Emissions (Wes Reinhart & Dylan Millet, UMN)
- GEIA 2005 Hg emissions + updates for the Hg simulation (Team Hg)
For detailed information about GEOS-5.7.2, please see the GEOS-Chem v9-01-02 wiki page.
Here is a progress report on the development of GEOS-Chem v9-01-03:
Updates that have been benchmarked
As of this writing, the GEOS-Chem Support Team has added the following features to GEOS-Chem v9-01-03, and has validated each feature with the customary 1-month benchmark simulation.
- Fix for GLCO3/GLPAN bug in standard chemistry mechanism (by Fabien Paulot). This fixes a typo in the chemical mechanism input files which results in an artificial loss of NOx.
- OVOC dry deposition (by Jingqiu Mao). This update changes the reactivity factor for OVOC’s from 0 to 1 in the dry deposition code (cf. Karl et al, 2010)
- Improvements to sea salt emission (by Lyatt Jaegle). Sea salt emissions now include a SST and wind speed dependency (cf. Jaegle et al 2011)
- Improvements to dry deposition (by Lyatt Jaegle):
- Use Zhang et al (2001) scheme for sea salt drydep over land
- Use Slinn & Slinn (1980) scheme for sea salt drydep over oceans
- Now use actual surface pressure in routine DEPVEL instead of a constant value (NOTE: this affects all species, not just sea salt)
- Dust submicron size distribution for optics (by David Ridley, Colo State). This update uses a more realistic partitioning of dust mass for photolysis.
- Add scavenging by snow (by Qiaoqiao Wang). This update only lets dust, hydrophobic BC, and HNO3 be “rained out” by snow.
- Updates for aerosol scavenging efficiency (by Qiaoqiao Wang). This follows the scheme of Feng (2007, 2009) to avoid over-scavenging of aerosols.
- NOTE: A 1-year Rn-Pb-Be benchmark simulation was performed in addition to the customary 1-month fullchem benchmark.
Updates slated for inclusion
The following updates are currently slated for inclusion into v9-01-03 (order subject to change):
- PARANOX ship plume emissions model (Geert Vinken, Eindhoven)
- Updates from the Grid-independent_GEOS-Chem (Mike Long, Harvard)
- SOA simulation with semi-volatile POA (Havala Pye, EPA + GCST)
- Stratopsheric monthly climatological P and k (Lee Murray, Harvard)
- Various emissions updates, including:
- EPA/NEI05 North American Hg emissions
- Streets future Hg emissions
- Satellite-based NOx emission trends
- Historical emission inventories of SO2, NOx, BC, and POA
- Daily and hourly GFED3 biomass emissions
- Add scaling of lightning NOx for 2x2.5 MERRA simulations
- Nested grid simulation updates:
- Offine simulation updates:
- Various minor bug fixes, including:
- Bug fixes for David Streets emissions for years 2005 and earlier (Jenny Fisher, Harvard)
- Bug fix in emep_mod.F for offline simulations (GCST)
- Bug fix to prevent crash in wetdep when encountering bad met input (GCST)
- Minor fix in Rn-Pb-Be simulation to prevent missing drydep diagnostics (GSCT)
- Bug fixes in diag3.F (Mat Evans, York)
- Bug fixes for nested-grid simulation (Dylan Millet, UMN)
- Bug fix for reading GWETTOP fields in offline CH4 simulation (GCST)
- Correct out-of-bounds error in offline aerosol simulation (GCST)
- Updating photolysis from FAST-J to FAST-JX
The GEOS-Chem Support Team shall lump together updates that do not affect the full chemistry simulation for expediency in benchmarking.
Also, the GEOS-Chem Support Team recommends that updates from the Grid-independent GEOS-Chem be merged into the standard repository as soon as possible (ideally before the SOA + POA updates). This will prevent the Grid-independent GEOS-Chem code from significantly diverging from the standard mainline GEOS-Chem code. In the future, further synchronization between the Grid-Independent GEOS-Chem and the standard GEOS-Chem will be necessary at frequent intervals.
Since the previous GEOS-Chem Steering Committee telecon in November 2011, work on the Grid-independent GEOS-Chem (GIGC) has progressed swiftly. Among the milestones:
- Old Fortran-77 style legacy code has been replaced with modern Fortran-90 code.
- Common blocks have been replaced by Fortran modules.
- The IIPAR and JJPAR parameters are now defined at runtime rather than being left as fixed parameters. This will allow a larger model to be able to tell the GIGC code the size of the grid rather than having to use a hardwired grid.
- Fortran-90 derived-type arrays have been created for both emissions tendencies and met fields. At present, these point to the existing emissions and met field variables in the code. In the future, these new derived type arrays will replace the existing arrays.
Mike Long summarizes:
After a few false-starts, the system is now capable of operating on a flexible grid, primarily due to flexing F90 strengths in place of F77. I am right now in the process of validating that the model results are still good & I haven't broken anything. But, it has not yet been tested with ESMF. That will be next.
The data streams have been largely reconfigured in preparation for ESMF coupling through simple-bundle sockets, and that will be the largest next-step prior to running the 'sandbox' as Bob Yantosca and Arlindo da Silva had set up previously.
Also, it is noteworthy that this development has been in stride with GEOS-Chem science development and updates; and new features and fixes are being brought into this system as they are brought into the standard code.”
As of this writing, Mike Long is visiting the Beijing Climate Data Center. He is working with the staff there to insert the Grid-Independent GEOS-Chem code into the Center’s GCM. This will help to further refine the GIGC source code.
Mike writes from Beijing:
“We're making progress. We're effectively reading some of the emissions files onto the GCM.
DO_CHEMISTRY (i.e. GEOS-Chem’s chemistry driver routine) is working in [the Bejing Climate Data Center’s MPI-parallel version of the CAM GCM] on an externally initialized decomposed grid w/o blowing up. Note, that there aren't any 'true' data links between CAM & GC other than the geophysical grid and the the model’s time & time-stepping states. [T]he system isn't dying.
What’s left to do:
- Getting CAM to read GC's land-type, dry depositon, and other ASCII input files regardless of the grid. We may settle for a blunt-force regridding operation for the time being.
- Complete a proper emissions read & build-up. My approach is simply going to have CAM read in all the data from file, and then modify the DO_EMISSIONS routine (i.e. GEOS-Chem’s emissions driver routine) to [accept the data input].”
Upon Mike’s return from Beijing, we shall ingest the GIGC source code into the standard GEOS-Chem repository. This step is critical to ensure that the GIGC code does not diverge significantly from the mainline GEOS-Chem code.
Work remaining to be done:
- Further debugging
- Wrapping the GIGC code with calls to the Earth System Model Framework library.
- Separating the emissions routines from the rest of the code
- Converting file I/O from binary to netCDF
- Integrating GIGC code within NASA/GMAO’s GEOS-5 GCM
GEOS-5.7.2 met field update
This past summer, NASA/GMAO updated the operational GEOS system from GEOS-5.2.0 to GEOS-5.7.2. Here is a quick summary of the new GEOS-5.7.2 data product:
- Horizontal resolution: 0.25° lat x 0.3125° lon (i.e. 1/4° x 5/16°)
- Vertical resolution: 72 hybrid layers (identical to the vertical grids used by the GEOS-5.2.0 and MERRA met data products)
- Time resolution: Hourly and 3-hourly data (timestamps very similar to MERRA)
- Input data format: netCDF-4 (as opposed to HDF4-EOS, used for past GMAO data products)
Bob Yantosca has been preparing GEOS-Chem to run with the new GEOS-5.7.2 met data. He has completed the following tasks:
- Added documentation to the GC wiki with information about the GEOS-5.7.2 data.
- Activated a “data subscription” for the GEOS-5.7.2 data.
- Written scripts to transfer GEOS-5.7.2 data from NASA to Harvard and to automate the data download process.
- Finalized the GEOS-5.7.2 data processing/regridding software.
- Created a new Git repository for GEOS-5.7.2 data processing code and scripts.
- Completed initial modifications in GEOS-Chem to read the GEOS-5.7.2 data.
For more information about the implementation of GEOS-5.7.2 into GEOS-Chem (including a detailed list of all modifications made to individual GEOS-Chem routines), please see the following wiki pages:
We plan to store the GEOS-5.7.2 following data at Harvard (which will be downloadable by the community). Data will be processed for Aug 2011-onward. Priority will be given to these grids:
- Global 2° x 2.5°
- Global 4° x 5°
- SEA4CRS Nested grid 0.25° x 0.3125°: 20S-50N; 60E-150E
Consistent with our established nested grid protocol, we will distribute the GEOS-5.7.2 data processing code & scripts to any user groups who wish to pull the nested data themselves.
One important new feature is that the GEOS-Chem Support Team shall be archiving the processed GEOS-5.7.2 data in netCDF format instead of the traditional binary format (see below for more information).
At present, GEOS-Chem can read the GEOS-5.7.2 met data from within the timestepping loop in the main program for 2° x 2.5°, 4° x 5° global grids and the 0.25° x 0.3125° SEA4CRS nested China grid. Further testing is required to make sure that the individual operations (i.e. convection, deposition, chemistry) will work properly with the GEOS-5.7.2 data.
Bob Yantosca and the SEA4CRS team (Jenny Fisher, Patrick Kim, Lin Zhang, Yuxuan Wang) are coordinating several of the remaining tasks (e.g. regridding emissions data files, debugging, etc.) among themselves.
MERRA met field update
Matt Cooper has been regridding the GMAO MERRA data to 2° x 2.5° resolution at Dalhousie. He will continue the data processing until the full record is completed. He writes:
As for MERRA 2x2.5, I have finished processing the whole data record, and we have 1979-2010 available on our server here at Dal. I've already updated the wiki to indicate that, and have been contacted by a few people already. Feel free to spread the word that we have these met fields available.
Transition to netCDF file format
Several GEOS-Chem users have identified a clear need to migrate GEOS-Chem from the traditional sequential binary file format (aka “binary punch” or “bpch”) to netCDF format. NetCDF is a self-describing file format that allows one to save the data along with any descriptions of the data (field names, units, descriptions. Several software packages for data visualization and analysis support netCDF format.
The GEOS-Chem Support Team recommends migration to netCDF format in order to facilitate the Grid-Independent GEOS-Chem (GIGC) project. As described above, the GIGC seeks to incorporate GEOS-Chem within larger Earth System Models, Climate Models, GCMs, and data assimilation systems. These larger models employ Message Passing Interface (MPI) parallelization in order distribute or “decompose” data among several hundreds or thousands of CPUs.
The current “binary punch” format used by GEOS-Chem is incompatible with MPI parallelization in the sense that small chunks of a binary punch file cannot be read simultaneously across CPUs in a distributed memory environment. Rather, the entire file must be read on each CPU that is executing, thus adding much more file I/O overhead to the system, which in turn negatively impacts model performance and scalability. On the other hand, it is possible to read small chunks from a netCDF file independently on several CPUs simultaneously, thus preserving scalability.
The transition to netCDF file format will involve several steps. We realize that the full transition cannot be achieved overnight. The first step shall be to modify GEOS-Chem to be able to read netCDF data files from disk, starting with the new GEOS-5.7.2 met data product. To this end, Bob Yantosca has installed his NcdfUtilities package into a development version of GEOS-Chem v9-01-03. Bob has also written a Fortran module (“geos57_read_mod.F90”) which calls NcdfUtilities routines to read the GEOS-5.7.2 met data from disk. The netCDF functionality can be activated simply by compiling GEOS-Chem code with an additional switch (NETCDF=yes).
At a later time, in conjunction with the GIGC project, we shall also begin to convert GEOS-Chem’s diagnostic outputs to netCDF format. Fortran code for writing netCDF files is necessarily much more “wordy“ than code for reading from netCDF files because one must supply information (aka “metadata”) about the variables that will constitute each netCDF file. To simplify the coding process for the GEOS-Chem community, Bob Yantosca has included into GEOS-Chem some Perl scripts which will automatically generate the proper Fortran for both writing to and reading from netCDF files.
The Earth System Model Framework environment contains many file I/O and regridding utilities. Once the GIGC code becomes fully compatible with ESMF, we will take advantage of ESMF’s regridding capabilities. During construction of the GIGC, we will need to continue to provide a Fortran-only package for netCDF I/O (cf. Bob Yantosca’s NcdfUtilities, or similar programs).
The migration to netCDF will require that GEOS-Chem end users install the netCDF libraries on their system. We shall provide instructions for installing these libraries on the GEOS-Chem wiki.
The next release of the GAMAP visualization software (v2-16) will feature improved support for reading and plotting data in netCDF files. The GAMAP routines BPCH2COARDS and MAKE_MULTI_NC may be used to convert existing binary punch files to netCDF format.
De-supporting of GEOS-3 meteorology
Bob Yantosca queried the GEOS-Chem user community to see if there were any users who still relied on the GEOS-3 met data for their research. He found that none of the GEOS-Chem users still rely on GEOS-3 meteorology.
Therefore, the GEOS-Chem Support Team recommends that we remove support for GEOS-3 met data from within GEOS-Chem. This will help to clean up the GEOS-Chem source code and remove unnecessary routines and coding constructs which may impede development of the Grid-independent GEOS-Chem.
Other development tasks
Matt Cooper has been working on replacing the traditional GEOS-Chem regridding routines (which sometimes involve two separate regrdidding processes) with the MAP_A2A regridding package. He writes:
I'm making progress on getting MAP_A2A into GEOS-Chem. I have made the switch in some places and now am just working my way through the code finding all the places where it needs to be switched. Hopefully it doesn't take me too much longer.
Downtime at Harvard: Feb 19-23, 2012
The Harvard Atmospheric Chemistry Modeling Group will be moving its computational resources from its existing location into a more modern server room. The transition will take place starting on Sunday Feb 19, 2012 and will take at 3-4 days to complete.
Our sysadmin Jack Yatteau writes:
Downtime: 1) Sunday, February 19, 16:00. Most services will go down. Running jobs on the computational cluster will be rescheduled. Minimal services will remain up overnight and go down early Monday morning. Some services might be available Tuesday. Interactive and batch systems should be available by EOB Wednesday 2/22. 2) Thursday, February 23. Backup power for our equipment will be unavailable for a few hours due to UPS maintenance. No downtime is anticipated, but we will not be protected against power failure while the work is being performed.
The following services will be unavailable during the transition:
- ftp.as.harvard.edu: main FTP site for data downloads
- git.as.harvard.edu: Git server for GEOS-Chem source code & run directory downloads
- GEOS-Chem web site
- GAMAP web site
- Jacob Group web site
The GEOS-Chem wiki is hosted on a different server and should not be affected. However, some links to documents on the GEOS-Chem web server will not function during the downtime.
Please plan accordingly. Met data may also be obtained from the Dalhousie server (rain.ucis.dal.ca)
--Bob Y. 11:21, 14 February 2012 (EST)