Python code for GEOS-Chem

From Geos-chem
Revision as of 19:30, 16 May 2016 by Barronh (Talk | contribs) (→‎PseudoNetCDF)

Jump to: navigation, search

On this page, we list information about Python software packages that are being created for use with GEOS-Chem.

Overview

Traditionally, GEOS-chem users have relied on IDL-based software, such as GAMAP, for data analysis and visualization. But to many GEOS-Chem user groups, the IDL software has become prohibitively expensive. Several members of the GEOS-Chem community have begun to develop file software programs in the Python language for reading, visualizing, and processing GEOS-Chem data. Python is a free and open-source computer language that comes with several pre-packaged libraries for numerical computation and visualization.

On this page we provide a list of Python software that is being developed for use with GEOS-Chem. For more information, please contact the individual authors listed below.

--Bob Y. 13:51, 28 May 2013 (EDT)

Development Teams

Institution Contact Persons Packages Status
City University of Hong Kong Gerrit Kuhlmann Gchem: a reader/writer for bpch files
U. Edinburgh (UK) Liang Feng
  1. bpchdump.py: to read/write/convert/regrid/visualize bpch2 data
  2. Python tools for EnKF data assimilation
  3. We are also developing python tools for setting up and managing GEOS-Chem nested simulations.
U. Florida (USA) Barron Henderson
  1. bpch: a bpch to netCDF conversion tool
  2. #pseudonetcdf: a NetCDF like system including visualization (maps, profiles, timeseries, etc)
  3. Process analysis diagnostics: A tool for examining the change in each species due to each process and reaction.
U. Liege (Belgium) Emanuel Mahieu
Benoit Bovy

PyGChem : a Python interface to GEOS-Chem (currently allows dataset handling using different backends - IO bpch/netCDF4, and will soon provide an interface to HEMCO).

U. Wisconsin-Madison / UW (USA) Andre Perkins Python Ensemble Manager for ensembles of GEOS-Chem adjoint simulations using MPI available on github

--Bob Y. 11:40, 28 May 2013 (EDT)

Information about existing Python packages

bpchdump

Author: Liang Feng (U. Edinburgh, UK)

Here are some screenshots of our bpchdump.py package, which allow you to browse the contents of a bpch file and plot them:

bpchdump.py GUI interface:

Bpchdump gui.png

Sample plot generated with bpchdump.py:

Bpchdump plot.png

--Bob Y. 11:40, 28 May 2013 (EDT)

bpch to netCDF converter

Author: Barron Henderson (U. Florida)

Barron Henderson wrote:

I wanted to clarify a little bit about my own project, which I have named simply bpch. bpch is registered with pypi (and installable with the command `pip install bpch`).
bpch is netcdf-like reader for bpch files returns a netcdf-like object with CF-1.6 compliant meta-data. The interface for bpch mimics the netcdf4-python interface (the successor to SciPy.io.netcdf.netcdffile).
If the bpch module is called from the command line (see below), it creates a netcdf file. If the module is used in a python program, then no netcdf file is created. At the command line, it also supports nco-like dimension and variable subsetting. Optionally, it can output figures and animations as well.
Example command line:
   $ python -m bpch ctm.bpch -o ctm.bpch.nc
would create ctm.bpch.nc that is readable by Panoply. The output is not currently readable by IDV because of some dimension naming assumptions for vertical dimensions.
In script form, the bpch module is simple a reader. The script below would simply print out some statistics twice. The first time demonstrates the netcdf3-classic style and the second demonstrates the netcdf4 style. The module uses numpy array structures, so it is fully compatible with matplotlib plotting functions.
   #!/usr/bin/env python
   #--Begin Script--
   from bpch import bpch
   import numpy as np

   f = bpch('ctm.bpch')
   key = 'IJ-AVG-$_Ox'
   v = f.variables[key]
   print key
   print v.units, v.dimensions
   print 'Min, Median, Max, Mean, Std:', v[:].min(), np.median(v[:]), v[:].max(), v[:].mean(), v[:].std()

   group = 'IJ-AVG-$'
   key = 'Ox'
   v = f.groups[group].variables[key]
   print key
   print v.units, v.dimensions
   print 'Min, Median, Max, Mean, Std:', v[:].min(), np.median(v[:]), v[:].max(), v[:].mean(), v[:].std()

   #--End Script--

--Bob Y. 11:40, 28 May 2013 (EDT)

PseudoNetCDF

PseudoNetCDF supersedes the python bpch library. It has all the same functionality and much much more. There is a core library that can be used for all kinds of programming, and a series of utility scripts to make easy things easy. Evaluating GEOS-Chem against AQS observations, for example, takes three commands.

  • Converts bpch to CF compliant NetCDF pncgen -f bpch inpath outpath
  • View a lon/lat point pncdump -f bpch --extract=-74,25 inpath
  • Make maps - pncmap.py -f bpch -r time,mean -s layer47,0 inpath outpath
  • Make profiles - pncvertprofile.py -f bpch -r time,mean -r latitude,mean -r longitude,mean inpath output
  • Evaluate against AQS (pncaqsraw4pnceval.py makes a netcdf file of AQS for comparison; pncgen -f bpch --extract-file=aqs.nc

The examples below use the benchmark run that was run for the GEOS-Chem timing exercise.

pncmap.py makes maps simply. Note that you can add states and counties or custom shapefiles. 

   $ pncmap.py -f "bpch,vertgrid='GEOS-5-NATIVE'" --norm="Normalize()" -v IJ-AVG-\$_O3 -r time,mean -s layer72,0 trac_avg.geosfp_4x5_benchmark.201307010000 pnc_trac_avg.geosfp_4x5_benchmark.20130701000_


pnc_trac_avg.geosfp_4x5_benchmark.20130701000_IJ-AVG_O3.png


pncvertprofile.py makes vertical profile plots and, optionally, can plot tes and/or omi (not shown) for comparison. tes and/or omi will be spatially subset automatically (good for ND51). 

   $ pncvertprofile.py -f "bpch,vertgrid='GEOS-5-NATIVE'" -v IJ-AVG-\$_O3 -r time,mean trac_avg.geosfp_4x5_benchmark.201307010000 --tes-paths=/path/to/tes/*.nc pnc_trac_avg.geosfp_4x5_benchmark.20130701000_pncprofile

pnc_trac_avg.geosfp_4x5_benchmark.20130701000_pncprofile_IJ-AVG_O3.png

Comparing GEOS-Chem to AQS observations (or other networks) can be done using three simple commands, you can evaluate GEOS-Chem against AQS observations 

   $ pncaqsraw4pnceval.py --param=44201 -s 2013-07-01 -e 2013-07-31 -r "1985-01-01 00:00:00" --wktpolygon="POLYGON((-182.5 -90, 177.5 -90, 177.5 90, -182.5 90, -182.5 -90))"
   $ pncgen -O --extract-file="AQS_DATA_20130701-20130731.nc" -f "bpch,vertgrid='GEOS-5-NATIVE'" -s layer72,0 trac_avg.geosfp_4x5_benchmark.201307010000 GC_AT_AQS.nc
   $ pnceval.py --funcs=MB,ME,NMB,NME -v Ozone -- -r time,mean AQS_DATA_20130701-20130731.nc --sep --expr="Ozone=O3/1000." --rename=v,IJ-AVG-\$_O3,O3 GC_AT_AQS.nc

Note that pncaqsraw4pnceval.py will either use a pre-downloaded file or download it for you. You can substitute any other network that is in a text file using the `pncgen -f csv,... ` command in place of pncaqsraw4pnceval.py. There are also many more statistical functions that can be used.

Latest code at github.com/barronh/pseudonetcdf Details for basic operation: https://github.com/barronh/pseudonetcdf/wiki/GC-Tutorials

-- Barron H. Henderson 11:49pm, April 20 2016

Process analysis diagnostics

Author: Barron Henderson (U. Florida)

Barron Henderson wrote:

Process-based Analysis examines the change in each species due to each process and reaction. Models predict atmospheric state, which in a time-series can be used to create net-change of each species. What this cannot tell us, is which processes led to that change. To supplement state (or concentration, GEOS-Chem has long archived emissions and employed advanced diagnostics to predict gross chemical production or loss. Process Analysis goes a step further archiving grid-cell budgets for each species, and decomposing gross production/loss into individual reaction contributions. Process Analysis extensions are currently available in CAMx, WRF-Chem, CMAQ, and now GEOS-Chem. This allows for direct comparisons of models at a fundamental, process level.

For more information, or to download the code, please see our Process Analysis Diagnostics page on the GEOS-Chem wiki.

Gchem

Author: Gerrit Kuhlmann (City University of Hong Kong)

My package Gchem is mainly a reader/writer for bpch files. To visualize data I use available libraries (Matplotlib and basemap).

PyGChem

Authors: Benoit Bovy and Emanuel Mahieu (U. Liege, Belgium)

Benoit Bovy wrote:

The PyGChem Python package aims to connect GEOS-Chem to the Python Scientific (SciPy) Stack, which consists of several core packages and many specialized packages for scientific computing. PyGChem is under active development. Currently, it allows reading GEOS-Chem datasets from both bpch and netCDF formats and handling it as Iris's cubes. Other dataset handling backends such as xray will be added soon. An API for HEMCO is also under development ; it will allow users to dynamically read/write, create, edit, test, and visualize HEMCO configurations.

Preliminary documentation is available as IPython notebooks.

--Benoit B. 10:45, 7 May 2015 (EDT)


Python Ensemble Manager

Author: Andre Perkins (U. Wisconsin-Madison, USA)


This code is useful for running ensembles of GEOS-Chem adjoint model simulations within an MPI environment. For more information about our Python Ensemble Manager, please see the source code on github.

--Daven 14:46, 8 May 2015 (EDT)

Python resources

Retrieved from "https://wiki.seas.harvard.edu/geos-chem/index.php?title=Python_code_for_GEOS-Chem&oldid=26840"