Difference between revisions of "Python tools for use with GEOS-Chem"

From Geos-chem
Jump to: navigation, search
(Created page with "== GCPy == The GEOS-Chem Support Team is currently developing [https://github.com/geoschem/gcpy GCPy], which is used to create the evaluation plots from GEOS-Chem benchmark s...")
 
(29 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 +
__FORCETOC__
 +
'''''[[Guide to visualization and analysis tools for GEOS-Chem|Previous]] | [[Other tools for visualizing and analyzing GEOS-Chem output|Next]] | [[Getting Started with GEOS-Chem]]'''''
 +
#[[Minimum system requirements for GEOS-Chem|Minimum system requirements]]
 +
#[[Configuring your computational environment]]
 +
#[[Downloading GEOS-Chem source code|Downloading source code]]
 +
#[[Downloading GEOS-Chem data directories|Downloading data directories]]
 +
#[[Creating GEOS-Chem run directories|Creating run directories]]
 +
#[[GEOS-Chem input files|Configuring runs]]
 +
#[[Compiling GEOS-Chem|Compiling]]
 +
#[[Running GEOS-Chem|Running]]
 +
#[[GEOS-Chem output files|Output files]]
 +
#[[Guide to visualization and analysis tools for GEOS-Chem|Visualizing and processing output]]
 +
#*<span style="color:blue">'''Python tools'''</span>
 +
#*[[Other tools for visualizing and analyzing GEOS-Chem output|Other tools]]
 +
#*[[Visualization and analysis examples|Examples]]
 +
#[[GEOS-Chem_coding_and_debugging|Coding and debugging]]
 +
#[[GEOS-Chem_overview#Further_reading|Further reading]]
 +
 +
 +
On this page, we provide information about free and open-source Python-language tools for visualizing or analyzing GEOS-Chem data.
 +
 
== GCPy ==
 
== GCPy ==
  
The GEOS-Chem Support Team is currently developing [https://github.com/geoschem/gcpy GCPy], which is used to create the evaluation plots from GEOS-Chem benchmark simulationsWhile GCPy is still in development, we encourage interested users to start using GCPy and to report any issues.
+
The GEOS-Chem Support Team is currently developing [https://github.com/geoschem/gcpy '''GCPy'''], which is a Python-based toolkit containing useful functions and routines for working with GEOS-Chem.  GCPy is meant to update and replace the IDL-based GAMAP utility.  While not a complete re-write of the GAMAP IDL pacakage, GCPy aims to build on the well-established scientific python technical stack, leveraging tools like cartopy and xarray to simplify the task of working with model output and performing atmospheric chemistry analyses.
  
Also, if you have your own Python plotting or analysis code, and would like to make it a part of GCPy, please contact the [[GEOS-Chem Support Team]].
+
The GEOS-Chem Support Team is presently working to update the GCPy documentation on readthedocs.io.  At present, the documentation is not complete, but will be updated as time allows.  GCPy also includes a few example scripts and notebooks that you can work through.
  
== Third-party packages ==
+
As of this writing (June 2019), most of the functions in GCPy are geared towards creating the evaluation plots for the [[GEOS-Chem Benchmarks|GEOS-Chem benchmark simulations]].  The [[GEOS-Chem Support Team]] will be adding new functionality to GCPy in the near future.  If you have your own Python plotting or analysis code, and would like to make it a part of GCPy, please contact the Support Team.
 +
 
 +
== Third-party Python tools ==
  
 
Several third-party Python packages have been developed for GEOS-Chem.  If you are interested in using any of these, we recommend that you follow up with the developers directly.  The [[GEOS-Chem Support Team]] is not responsible for maintaining these tools.
 
Several third-party Python packages have been developed for GEOS-Chem.  If you are interested in using any of these, we recommend that you follow up with the developers directly.  The [[GEOS-Chem Support Team]] is not responsible for maintaining these tools.
Line 18: Line 41:
 
|Jiawei Zhuang
 
|Jiawei Zhuang
 
|[https://xesmf.readthedocs.io/en/latest/why.html xESMF]: A universal regridder.  Leverages ESMF's regridding capabilities to easily regrid data between lon-lat and cubed-sphere grids.  Other grids are supported as well.
 
|[https://xesmf.readthedocs.io/en/latest/why.html xESMF]: A universal regridder.  Leverages ESMF's regridding capabilities to easily regrid data between lon-lat and cubed-sphere grids.  Other grids are supported as well.
|[https://github.com/JiaweiZhuang/xESMF Available on github]
+
|[https://github.com/JiaweiZhuang/xESMF Available on Github]
  
 
|-valign="top"
 
|-valign="top"
 
|[[User:darothen|Daniel Rothenberg]]
 
|[[User:darothen|Daniel Rothenberg]]
 
|[[#xbpch|xbpch]]: Backend for reading bpch output into [http://xarray.pydata.org xarray]/[http://dask.pydata.org dask]
 
|[[#xbpch|xbpch]]: Backend for reading bpch output into [http://xarray.pydata.org xarray]/[http://dask.pydata.org dask]
|[https://github.com/darothen/xbpch Available on github]
+
|[https://github.com/darothen/xbpch Available on Github]
  
 
|-valign="top"
 
|-valign="top"
Line 30: Line 53:
 
#[[#PseudoNetCDF|PseudoNetCDF]]: a NetCDF like system including visualization (maps, profiles, timeseries, etc)
 
#[[#PseudoNetCDF|PseudoNetCDF]]: a NetCDF like system including visualization (maps, profiles, timeseries, etc)
 
#[[#Process analysis diagnostics|Process analysis diagnostics]]: A tool for examining the change in each species due to each process and reaction.  
 
#[[#Process analysis diagnostics|Process analysis diagnostics]]: A tool for examining the change in each species due to each process and reaction.  
|[https://github.com/barronh/pseudonetcdf Available on github]
+
|[https://github.com/barronh/pseudonetcdf Available on Github]
 +
 
 +
|-valign="top"
 +
|[https://github.com/frodre Andre Perkins]
 +
|[https://github.com/frodre/pyEnsemble pyEnsemble]: This code is useful for running ensembles of GEOS-Chem adjoint model simulations within an MPI environment.
 +
|[https://github.com/frodre/pyEnsemble Available on Github]
  
 
|}
 
|}
  
== Tutorials ==
+
== Core Python packages for Earth Science data ==
  
If you are new to Python, we strongly recommend that you take our GEOS-Chem Python tutorialIt will walk you though the steps of creating sample plots from GEOS-Chem output:
+
For the most part, you don't need to use a lot of Python packages to read and plot GEOS-Chem dataFor example, the [[#GCPy|GCPy]] package mostly relies on the following packages, which can be downloaded with either PyPI and/or Conda.
  
*[https://github.com/geoschem/GEOSChem-python-tutorial]
+
*[http://xarray.pydata.org/en/stable/ xarray]: For reading netCDF, HDF5, GRIB, etc. data into a common data structure for later manipulation
 +
*[https://matplotlib.org/ matplotlib]: For general plotting operations
 +
*[https://scitools.org.uk/cartopy/docs/latest/ cartopy]: For plotting data on world maps
 +
*[http://www.numpy.org/ numpy]: For math operations
 +
*[https://xesmf.readthedocs.io/en/latest/ xESMF]: For regrididdng between cubed-sphere to lat-lon grids
 +
*[https://pandas.pydata.org/ pandas]: For spreadsheet-like data analysis
  
You may also find these tutorials useful:
+
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 16:15, 14 June 2019 (UTC)
 +
 
 +
== Python tutorials and resources ==
 +
 
 +
If you are new to Python, we strongly recommend that you take our [https://github.com/geoschem/GEOSChem-python-tutorial '''GEOS-Chem Python Tutorial'''].  It will walk you though the steps of creating sample plots from GEOS-Chem output:
 +
 
 +
You may also find these resources useful:
  
 +
*[http://www.python.org Python.org]: Main site for all things Python
 
*[https://developers.google.com/edu/python/ Google's Python course]
 
*[https://developers.google.com/edu/python/ Google's Python course]
 
*[http://docs.python.org/3/tutorial/ Python 3 tutorial]
 
*[http://docs.python.org/3/tutorial/ Python 3 tutorial]
*[http://learnpythonthehardway.org/book/ ''Learn Python the hard way'' book]
+
*[https://www.geeksforgeeks.org/python-programming-language/ GeeksForGeeks Python page]
 +
*[http://wiki.python.org/moin/BeginnersGuide/Programmers PythonWiki.org: List of tutorials and resources]
 +
*[https://developers.google.com/edu/python/ Google's Python tutorial]
  
For the most part, you don't need to use a lot of Python packages to read and plot GEOS-Chem data.  For example, GCPy package mostly relies on [http://xarray.pydata.org/en/stable/ xarray] (to read netCDF data), [https://matplotlib.org/ matplotlib] and [https://scitools.org.uk/cartopy/docs/latest/ cartopy] (to plot data), [http://www.numpy.org/ numpy] (for math operations) and [https://xesmf.readthedocs.io/en/latest/ xESMF] (to regrid between cubed-sphere to lat-lon grids).
+
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 16:21, 14 June 2019 (UTC)
  
--[[User:Bmy|Bob Yantosca]] ([[User talk:Bmy|talk]]) 16:15, 14 June 2019 (UTC)
+
 
 +
----
 +
'''''[[Guide to visualization and analysis tools for GEOS-Chem|Previous]] | [[Other tools for visualizing and analyzing GEOS-Chem output|Next]] | [[Getting Started with GEOS-Chem]]'''''

Revision as of 21:55, 19 December 2019

Previous | Next | Getting Started with GEOS-Chem

  1. Minimum system requirements
  2. Configuring your computational environment
  3. Downloading source code
  4. Downloading data directories
  5. Creating run directories
  6. Configuring runs
  7. Compiling
  8. Running
  9. Output files
  10. Visualizing and processing output
  11. Coding and debugging
  12. Further reading


On this page, we provide information about free and open-source Python-language tools for visualizing or analyzing GEOS-Chem data.

GCPy

The GEOS-Chem Support Team is currently developing GCPy, which is a Python-based toolkit containing useful functions and routines for working with GEOS-Chem. GCPy is meant to update and replace the IDL-based GAMAP utility. While not a complete re-write of the GAMAP IDL pacakage, GCPy aims to build on the well-established scientific python technical stack, leveraging tools like cartopy and xarray to simplify the task of working with model output and performing atmospheric chemistry analyses.

The GEOS-Chem Support Team is presently working to update the GCPy documentation on readthedocs.io. At present, the documentation is not complete, but will be updated as time allows. GCPy also includes a few example scripts and notebooks that you can work through.

As of this writing (June 2019), most of the functions in GCPy are geared towards creating the evaluation plots for the GEOS-Chem benchmark simulations. The GEOS-Chem Support Team will be adding new functionality to GCPy in the near future. If you have your own Python plotting or analysis code, and would like to make it a part of GCPy, please contact the Support Team.

Third-party Python tools

Several third-party Python packages have been developed for GEOS-Chem. If you are interested in using any of these, we recommend that you follow up with the developers directly. The GEOS-Chem Support Team is not responsible for maintaining these tools.

Developer Packages/Description Status
Jiawei Zhuang xESMF: A universal regridder. Leverages ESMF's regridding capabilities to easily regrid data between lon-lat and cubed-sphere grids. Other grids are supported as well. Available on Github
Daniel Rothenberg xbpch: Backend for reading bpch output into xarray/dask Available on Github
Barron Henderson Several software packages, including:
  1. PseudoNetCDF: a NetCDF like system including visualization (maps, profiles, timeseries, etc)
  2. Process analysis diagnostics: A tool for examining the change in each species due to each process and reaction.
Available on Github
Andre Perkins pyEnsemble: This code is useful for running ensembles of GEOS-Chem adjoint model simulations within an MPI environment. Available on Github

Core Python packages for Earth Science data

For the most part, you don't need to use a lot of Python packages to read and plot GEOS-Chem data. For example, the GCPy package mostly relies on the following packages, which can be downloaded with either PyPI and/or Conda.

  • xarray: For reading netCDF, HDF5, GRIB, etc. data into a common data structure for later manipulation
  • matplotlib: For general plotting operations
  • cartopy: For plotting data on world maps
  • numpy: For math operations
  • xESMF: For regrididdng between cubed-sphere to lat-lon grids
  • pandas: For spreadsheet-like data analysis

--Bob Yantosca (talk) 16:15, 14 June 2019 (UTC)

Python tutorials and resources

If you are new to Python, we strongly recommend that you take our GEOS-Chem Python Tutorial. It will walk you though the steps of creating sample plots from GEOS-Chem output:

You may also find these resources useful:

--Bob Yantosca (talk) 16:21, 14 June 2019 (UTC)



Previous | Next | Getting Started with GEOS-Chem