GEOS-Chem Adjoint v33: Difference between revisions

From Geos-chem
Jump to navigation Jump to search
 
(65 intermediate revisions by 4 users not shown)
Line 14: Line 14:
!Type
!Type
!Submitted by
!Submitted by
!Committed by
!Version
!Version
!Status
!Status
Line 19: Line 20:
|[[#Add_TES_O3_IRK_.28v33a.29|Add TES O3 IRK observation operator]]
|[[#Add_TES_O3_IRK_.28v33a.29|Add TES O3 IRK observation operator]]
|Adjoint Update
|Adjoint Update
|Daven Henze (CU Boulder), Kevin Bowman (JPL)
|Daven Henze (CU Boulder)
|Daven Henze (CU Boulder)
|v33a
|v33a
Line 26: Line 28:
|[[#Fix_wet_deposition_adjoint_.28v33b.29|Fix wetdep adjoint]]
|[[#Fix_wet_deposition_adjoint_.28v33b.29|Fix wetdep adjoint]]
|Adjoint bug fix
|Adjoint bug fix
|Fabien Paulot (Harvard University), Daven Henze (CU Boulder)
|Daven Henze (CU Boulder)
|Daven Henze (CU Boulder)
|v33b
|v33b
|Approved 20 Apr 2012
|Approved 20 Apr 2012
|-bgcolor="#CCFFFF"
|-
|[[#Fix_checkpointing_of_aircraft_SO2_emissions_.28v33c.29|Fix checkpointing of aircraft SO2 emissions]]
|Adjoint bug fix
|Daven Henze, Kateryna Lapina (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33c
|Approved 20 Apr 2012
|-bgcolor="#CCFFFF"
|-
|[[#Fix_linoz_adjoint_.28v33d.29|Fix linoz adjoint]]
|Adjoint bug fix
|Hyung-Min Lee (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33d
|Approved 07 May 2012
|-bgcolor="#CCFFFF"
|-
|[[#Adjoint_stability_update_.28v33e.29| Adjoint stability update]]
|Adjoint bug fix
|Jamin Koo (MIT), Daven Henze (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33e
|Approved 09 May 2012
|-bgcolor="#CCFFFF"
|-
|[[#Fix_SE_Asia_NH3_seasonality_.28v33e.29| Fix SE Asia NH3 seasonality]]
|Forward bug fix
|Shailesh Kharol (Dalhousie), Steven Vogel (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33e
|Approved 09 May 2012
|-bgcolor="#CCFFFF"
|-
|[[#Fix_problem_with_LICS_flag_in_using_SCHEM_.28v33f.29| Fix problem with LICS flag in using SCHEM]]
|Adjoint bug fix
|Hyung-Min Lee (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33f
|Approved 13 July 2012
|-bgcolor="#CCFFFF"
|-
|[[#Inverse_Hessian_and_pseudo-obs_cost_function_formulation_updates_.28v33g.29| L-BFGS inverse Hessian and pseudo-obs cost function formulation updates]]
|Adjoint update
|Nicolas Bousserez (CU Boulder)
|Nicolas Bousserez (CU Boulder)
|v33g
|Approved 10 August 2012
|-bgcolor="#CCFFFF"
|-
|[[#BC_fix_to_support_offline_simulation_and_AOD_.28v33h.29| BC fix to support offline simulation and AOD]]
|Adjoint update
|Li Zhang (CU Boulder)
|Yanko Davila (CU Boulder)
|v33h
|Approved 5 September 2012
|-bgcolor="#CCFFFF"
|-
|[[#Two_Minor_Bug_Fixes_.28v33h.29| Two minor bug fixes]]
|Adjoint bug fix
|Daven Henze (CU Boulder)
|Yanko Davila (CU Boulder)
|v33h
|Approved 5 September 2012
|-bgcolor="#CCFFFF"
|-
|[[#Fix_Monoterpene_SOA_zero_bug | Biomass Emissions bug fix]]
|Forward bug fix
| Jeffrey Pierce (CSU)
|Yanko Davila (CU Boulder)
|v33h
|Approved 5 September 2012
|-bgcolor="#CCFFFF"
|-
|[[#L-BFGS_inverse_Hessian_Update_.28v33i.29 | L-BFGS inverse Hessian update]]
|Adjoint Update
|Nicolas Bousserez (CU Boulder)
|Yanko Davila (CU Boulder)
|v33i
|Approved 21 September 2012
|-bgcolor="#CCFFFF"
|-
|[[#Pseudo-Obs_Test_Bug_Fix_.28v33i.29 | Pseudo-obs test bug fix]]
|Adjoint bug fix
|Nicolas Bousserez (CU Boulder)
|Yanko Davila (CU Boulder)
|v33i
|Approved 21 September 2012
|-bgcolor="#CCFFFF"
|-
|[[#Version_33i_patches_.28v33i-patch1.29 | Version 33i patch1]]
|Adjoint bug fix
|Nicolas Bousserez (CU Boulder)
|Yanko Davila (CU Boulder)
|v33i-patch1
|Approved 9 October 2012
|-bgcolor="#CCFFFF"
|-
|[[#Version_33i_patches_.28v33i-patch2.29| Version 33i patch2]]
|Adjoint bug fix
|Nicolas Bousserez (CU Boulder)
|Yanko Davila (CU Boulder)
|v33i-patch2
|Approved 22 October 2012
|-bgcolor="#CCFFFF"
|-bgcolor="#CCFFFF"
|}
|}
Line 35: Line 142:


==== Bug fixes in forward model ====
==== Bug fixes in forward model ====
===== Fix SE Asia NH3 seasonality (v33e) =====
The bug fix from the forward model [http://wiki.seas.harvard.edu/geos-chem/index.php/Anthropogenic_emissions#Streets_2000 fixing the seasonality of the NH3 emissions over SE Asia] has also been applied.
1 week benchmark is approved [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0  33e_003]
--[[User:Daven|Daven]] 02:40, 10 May 2012 (EDT)
===== Fix Monoterpene SOA zero bug =====
The bug fix from the forward model [http://wiki.seas.harvard.edu/geos-chem/index.php/Setting_up_GEOS-Chem_nested_grid_simulations#Monoterpene_SOA_zero_bug_fix Monoterpene SOA zero bug fix] has been applied.
--[[User:Yanko|Yanko]] 16:29, 6 September 2012 (EDT)


==== Updates to adjoint model ====
==== Updates to adjoint model ====
Line 42: Line 165:
Implemented observation operator for assimilation of TES O3 IRKs.   
Implemented observation operator for assimilation of TES O3 IRKs.   


See benchmark results 33a_001 [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0|33a_001]
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0 33a_001]


--[[User:Daven|Daven]] 19:03, 13 April 2012 (EDT)
--[[User:Daven|Daven]] 19:03, 13 April 2012 (EDT)
===== BC fix to support offline simulation and AOD (v33h) =====
- Fix to support offline BC simulation and AOD calculation
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33h]
--[[User:Yanko|Yanko]] 16:43, 6 September 2012 (EDT)
===== L-BFGS inverse Hessian Update (v33i) =====
Now use the background error instead of the initial L-BFGS inverse Hessian diagonal value where the gradient of the cost function is very small (in this case the inverse Hessian value is not updated by the algorithm).
Calculate and output the Degree of Freedom for Signal.
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33i]
--[[User:Yanko|Yanko]] 20:34, 21 September 2012 (EDT)


==== Bug fixes in adjoint model ====
==== Bug fixes in adjoint model ====
Line 52: Line 191:




See benchmark results 33a_002 vs 33b_002 [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0|33a_001]
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0 33a_002 vs 33b_002]


--[[User:Daven|Daven]] 12:51, 20 April 2012 (EDT)
--[[User:Daven|Daven]] 12:51, 20 April 2012 (EDT)
===== Fix checkpointing of aircraft SO2 emissions (v33c) =====
The SO2 aircraft emissions are distributed in the model based on a mapping from the 20 level inventory to the GEOS-Chem levels.  This mapping depends upon the air masses in the models.  The emissions are read in during the first hour of the month in the forward model, but on the last hour of the month in the adjoint.  Thus they were being calculated inconsistently in the adjoint compared to the forward model.  We now checkpoint these emissions rather than recalculate them
See benchmark results  [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0  33c_001 - 33c_004]
--[[User:Daven|Daven]] 23:08, 7 May 2012 (EDT)
===== Fix linoz adjoint (v33d) =====
While the linoz routine is linear, and should be self-adjoint, it isn't exactly self-adjoint. For example, given initial forcings of STT_ADJ = 0d0 in every grid cell, it can generate non-zero values of STT_ADJ as output, which is incorrect. 
In this update we use the discrete adjoint based upon reformulating the code to be consistent with the strat chem routine for the non-Ox species.
See benchmark results  [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0  33d_001 - 33c_007]
--[[User:Daven|Daven]] 23:14, 7 May 2012 (EDT)
'''Important: If you check out version 33d (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake.
--[[User:Nicolas Bousserez|Nicolas Bousserez]] 22:12, 7 July 2012 (EDT)
===== Adjoint stability update (v33e) =====
Several fixes were made to increase the numerical stability of the code
*Change the filter to 1d-10 in partition_adj.f
*Better error trapping in gckpp_adj_Integrator.f90 to prevent NaNs following a failed integration step during the adjoint
A few other small changes to improve performance were included in this update.  Overall, the benchmark performance is not impacted by these update, see [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=0  33e_001]
--[[User:Daven|Daven]] 02:26, 10 May 2012 (EDT)
'''Important: If you check out version 33e (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake.'''
--[[User:Nicolas Bousserez|Nicolas Bousserez]] 22:12, 7 July 2012 (EDT)
===== Fix problem with LICS flag in using SCHEM (v33f) =====
New stratospheric chemistry module (strat_chem_mod.f), was set to loop over the number of tracers (NSTPL) that has read from input.gcadj. It caused a problem when LADJ_STRAT is turned off, LADJ_STRAT is not appeared in input.gcadj (NSTPL = 0), or LADJ = F for spin-up.
--[[User:Hyung-Min Lee|Hyung-Min Lee]] 19:26, 14 Jul 2012 (EDT)
===== Inverse Hessian and pseudo-obs cost function formulation updates (v33g) =====
- Possibility to compute the L-BFGS inverse Hessian, which gives the posterior error covariance matrix of the inversion. Here only the diagonal elements (the error variances) of the matrix are computed. Unlike the DFP method, this method allows to compute a posterior errors for large dimensional problems (e.g. high-resolution inversions).
- The equations for the initial estimate of the inverse Hessian in the DFP algorithm were also updated to allow for quicker convergence. 
- The cost function for the pseudo-observations inversion test has been modified to properly account for the error term specified in the input.gcadj input file (ERROR).
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE#gid=1 v33g]
--[[User:Nicolas Bousserez|Nicolas Bousserez]] 05:34, 11 August 2012 (EDT)
===== Two Minor Bug Fixes (v33h) =====
- Fix for OpenMP in subroutine EMITHIGH_ADJ.  This can correct some errors for the BC emissions adjoints.  Without this fix, users may notice some error in FD vs ADJ calculations for shorter simulations.
- Update subroutine CHEM_SO2_ADJ to avoid underflow.
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33h]
--[[User:Yanko|Yanko]] 16:43, 6 September 2012 (EDT)
===== Pseudo-Obs Test Bug Fix (v33i) =====
- Bug fix for the definition of the observation error in the pseudo-obs inversion test and filter to avoid NaN adjoint value.
See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33i]
--[[User:Yanko|Yanko]] 20:34, 21 September 2012 (EDT)
=====Version 33i patches (v33i-patch1) =====
- Bug fix for too small observation errors when doing pseudo-assimilation experiments
- Bug Fix for the DFP.
--[[User:Yanko|Yanko]] 17:42, 1 November 2012 (EDT)
=====Version 33i patches (v33i-patch2) =====
- Bug Fix for memory problems caused by the amount of iteration steps kept in memory while doing a Hessian approximation.
--[[User:Yanko|Yanko]] 17:42, 1 November 2012 (EDT)


== Outstanding issues not yet resolved in v33 ==
== Outstanding issues not yet resolved in v33 ==


TBD
=== Updates in the pipeline ===
*Reaction rate sensitivities (Paulot, Harvard; Walker, UT)
*Update inverse Hessian approximation (Bousserez, CUB)
*Off-diagonal error covariance matrices (Singh, VT; Liu, JPL)
*Deposition-based cost function and updated chemistry (Paulot, Harvard)
*ETOH simulations (Millet, UMN)
*NH3 updates (Zhu, Colorado)
*NO2/SO2 obs operators (Bousserez, Dalhousie)
*ANISO (Capps, Georgia Tech)

Latest revision as of 23:06, 3 July 2013

Overview

BETA RELEASE

What's new in this version

GEOS-Chem Adjoint v33 contains the following major updates and improvements.

Feature Type Submitted by Committed by Version Status
Add TES O3 IRK observation operator Adjoint Update Daven Henze (CU Boulder), Kevin Bowman (JPL) Daven Henze (CU Boulder) v33a Approved 13 Apr 2012
Fix wetdep adjoint Adjoint bug fix Fabien Paulot (Harvard University), Daven Henze (CU Boulder) Daven Henze (CU Boulder) v33b Approved 20 Apr 2012
Fix checkpointing of aircraft SO2 emissions Adjoint bug fix Daven Henze, Kateryna Lapina (CU Boulder) Nicolas Bousserez (CU Boulder) v33c Approved 20 Apr 2012
Fix linoz adjoint Adjoint bug fix Hyung-Min Lee (CU Boulder) Nicolas Bousserez (CU Boulder) v33d Approved 07 May 2012
Adjoint stability update Adjoint bug fix Jamin Koo (MIT), Daven Henze (CU Boulder) Nicolas Bousserez (CU Boulder) v33e Approved 09 May 2012
Fix SE Asia NH3 seasonality Forward bug fix Shailesh Kharol (Dalhousie), Steven Vogel (CU Boulder) Nicolas Bousserez (CU Boulder) v33e Approved 09 May 2012
Fix problem with LICS flag in using SCHEM Adjoint bug fix Hyung-Min Lee (CU Boulder) Nicolas Bousserez (CU Boulder) v33f Approved 13 July 2012
L-BFGS inverse Hessian and pseudo-obs cost function formulation updates Adjoint update Nicolas Bousserez (CU Boulder) Nicolas Bousserez (CU Boulder) v33g Approved 10 August 2012
BC fix to support offline simulation and AOD Adjoint update Li Zhang (CU Boulder) Yanko Davila (CU Boulder) v33h Approved 5 September 2012
Two minor bug fixes Adjoint bug fix Daven Henze (CU Boulder) Yanko Davila (CU Boulder) v33h Approved 5 September 2012
Biomass Emissions bug fix Forward bug fix Jeffrey Pierce (CSU) Yanko Davila (CU Boulder) v33h Approved 5 September 2012
L-BFGS inverse Hessian update Adjoint Update Nicolas Bousserez (CU Boulder) Yanko Davila (CU Boulder) v33i Approved 21 September 2012
Pseudo-obs test bug fix Adjoint bug fix Nicolas Bousserez (CU Boulder) Yanko Davila (CU Boulder) v33i Approved 21 September 2012
Version 33i patch1 Adjoint bug fix Nicolas Bousserez (CU Boulder) Yanko Davila (CU Boulder) v33i-patch1 Approved 9 October 2012
Version 33i patch2 Adjoint bug fix Nicolas Bousserez (CU Boulder) Yanko Davila (CU Boulder) v33i-patch2 Approved 22 October 2012

Updates to forward model

Bug fixes in forward model

Fix SE Asia NH3 seasonality (v33e)

The bug fix from the forward model fixing the seasonality of the NH3 emissions over SE Asia has also been applied.

1 week benchmark is approved 33e_003


--Daven 02:40, 10 May 2012 (EDT)

Fix Monoterpene SOA zero bug

The bug fix from the forward model Monoterpene SOA zero bug fix has been applied.

--Yanko 16:29, 6 September 2012 (EDT)


Updates to adjoint model

Add TES O3 IRK (v33a)

Implemented observation operator for assimilation of TES O3 IRKs.

See benchmark results 33a_001

--Daven 19:03, 13 April 2012 (EDT)

BC fix to support offline simulation and AOD (v33h)

- Fix to support offline BC simulation and AOD calculation

See benchmark results v33h

--Yanko 16:43, 6 September 2012 (EDT)

L-BFGS inverse Hessian Update (v33i)

Now use the background error instead of the initial L-BFGS inverse Hessian diagonal value where the gradient of the cost function is very small (in this case the inverse Hessian value is not updated by the algorithm). Calculate and output the Degree of Freedom for Signal.

See benchmark results v33i

--Yanko 20:34, 21 September 2012 (EDT)

Bug fixes in adjoint model

Fix wet deposition adjoint (v33b)

The adjoint was not properly handling evaporation. This fix helps make the adjoint of species such as NH3 better in a few locations where adjoints were diverging from forward model sensitivities by up to 50%.


See benchmark results 33a_002 vs 33b_002

--Daven 12:51, 20 April 2012 (EDT)

Fix checkpointing of aircraft SO2 emissions (v33c)

The SO2 aircraft emissions are distributed in the model based on a mapping from the 20 level inventory to the GEOS-Chem levels. This mapping depends upon the air masses in the models. The emissions are read in during the first hour of the month in the forward model, but on the last hour of the month in the adjoint. Thus they were being calculated inconsistently in the adjoint compared to the forward model. We now checkpoint these emissions rather than recalculate them

See benchmark results 33c_001 - 33c_004

--Daven 23:08, 7 May 2012 (EDT)

Fix linoz adjoint (v33d)

While the linoz routine is linear, and should be self-adjoint, it isn't exactly self-adjoint. For example, given initial forcings of STT_ADJ = 0d0 in every grid cell, it can generate non-zero values of STT_ADJ as output, which is incorrect.

In this update we use the discrete adjoint based upon reformulating the code to be consistent with the strat chem routine for the non-Ox species.

See benchmark results 33d_001 - 33c_007

--Daven 23:14, 7 May 2012 (EDT)

Important: If you check out version 33d (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake. --Nicolas Bousserez 22:12, 7 July 2012 (EDT)

Adjoint stability update (v33e)

Several fixes were made to increase the numerical stability of the code

  • Change the filter to 1d-10 in partition_adj.f
  • Better error trapping in gckpp_adj_Integrator.f90 to prevent NaNs following a failed integration step during the adjoint


A few other small changes to improve performance were included in this update. Overall, the benchmark performance is not impacted by these update, see 33e_001


--Daven 02:26, 10 May 2012 (EDT)

Important: If you check out version 33e (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake. --Nicolas Bousserez 22:12, 7 July 2012 (EDT)


Fix problem with LICS flag in using SCHEM (v33f)

New stratospheric chemistry module (strat_chem_mod.f), was set to loop over the number of tracers (NSTPL) that has read from input.gcadj. It caused a problem when LADJ_STRAT is turned off, LADJ_STRAT is not appeared in input.gcadj (NSTPL = 0), or LADJ = F for spin-up.

--Hyung-Min Lee 19:26, 14 Jul 2012 (EDT)

Inverse Hessian and pseudo-obs cost function formulation updates (v33g)

- Possibility to compute the L-BFGS inverse Hessian, which gives the posterior error covariance matrix of the inversion. Here only the diagonal elements (the error variances) of the matrix are computed. Unlike the DFP method, this method allows to compute a posterior errors for large dimensional problems (e.g. high-resolution inversions).

- The equations for the initial estimate of the inverse Hessian in the DFP algorithm were also updated to allow for quicker convergence.

- The cost function for the pseudo-observations inversion test has been modified to properly account for the error term specified in the input.gcadj input file (ERROR).

See benchmark results v33g

--Nicolas Bousserez 05:34, 11 August 2012 (EDT)

Two Minor Bug Fixes (v33h)

- Fix for OpenMP in subroutine EMITHIGH_ADJ. This can correct some errors for the BC emissions adjoints. Without this fix, users may notice some error in FD vs ADJ calculations for shorter simulations.

- Update subroutine CHEM_SO2_ADJ to avoid underflow.

See benchmark results v33h

--Yanko 16:43, 6 September 2012 (EDT)

Pseudo-Obs Test Bug Fix (v33i)

- Bug fix for the definition of the observation error in the pseudo-obs inversion test and filter to avoid NaN adjoint value.

See benchmark results v33i --Yanko 20:34, 21 September 2012 (EDT)

Version 33i patches (v33i-patch1)

- Bug fix for too small observation errors when doing pseudo-assimilation experiments

- Bug Fix for the DFP.

--Yanko 17:42, 1 November 2012 (EDT)

Version 33i patches (v33i-patch2)

- Bug Fix for memory problems caused by the amount of iteration steps kept in memory while doing a Hessian approximation.

--Yanko 17:42, 1 November 2012 (EDT)

Outstanding issues not yet resolved in v33

Updates in the pipeline

  • Reaction rate sensitivities (Paulot, Harvard; Walker, UT)
  • Update inverse Hessian approximation (Bousserez, CUB)
  • Off-diagonal error covariance matrices (Singh, VT; Liu, JPL)
  • Deposition-based cost function and updated chemistry (Paulot, Harvard)
  • ETOH simulations (Millet, UMN)
  • NH3 updates (Zhu, Colorado)
  • NO2/SO2 obs operators (Bousserez, Dalhousie)
  • ANISO (Capps, Georgia Tech)
Retrieved from "https://wiki.seas.harvard.edu/geos-chem/index.php?title=GEOS-Chem_Adjoint_v33&oldid=13624"