# Difference between revisions of "GEOS-Chem Adjoint v33"

(→L-BFGS inverse Hessian and pseudo-obs cost function formulation updates (v33g)) |
(→What's new in this version) |
||

(19 intermediate revisions by 4 users not shown) | |||

Line 44: | Line 44: | ||

|[[#Fix_linoz_adjoint_.28v33d.29|Fix linoz adjoint]] | |[[#Fix_linoz_adjoint_.28v33d.29|Fix linoz adjoint]] | ||

|Adjoint bug fix | |Adjoint bug fix | ||

− | | | + | |Hyung-Min Lee (CU Boulder) |

|Nicolas Bousserez (CU Boulder) | |Nicolas Bousserez (CU Boulder) | ||

|v33d | |v33d | ||

Line 68: | Line 68: | ||

|[[#Fix_problem_with_LICS_flag_in_using_SCHEM_.28v33f.29| Fix problem with LICS flag in using SCHEM]] | |[[#Fix_problem_with_LICS_flag_in_using_SCHEM_.28v33f.29| Fix problem with LICS flag in using SCHEM]] | ||

|Adjoint bug fix | |Adjoint bug fix | ||

− | | | + | |Hyung-Min Lee (CU Boulder) |

|Nicolas Bousserez (CU Boulder) | |Nicolas Bousserez (CU Boulder) | ||

|v33f | |v33f | ||

Line 74: | Line 74: | ||

|-bgcolor="#CCFFFF" | |-bgcolor="#CCFFFF" | ||

|- | |- | ||

− | |[[# | + | |[[#Inverse_Hessian_and_pseudo-obs_cost_function_formulation_updates_.28v33g.29| L-BFGS inverse Hessian and pseudo-obs cost function formulation updates]] |

|Adjoint update | |Adjoint update | ||

|Nicolas Bousserez (CU Boulder) | |Nicolas Bousserez (CU Boulder) | ||

Line 80: | Line 80: | ||

|v33g | |v33g | ||

|Approved 10 August 2012 | |Approved 10 August 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#BC_fix_to_support_offline_simulation_and_AOD_.28v33h.29| BC fix to support offline simulation and AOD]] | ||

+ | |Adjoint update | ||

+ | |Li Zhang (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33h | ||

+ | |Approved 5 September 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#Two_Minor_Bug_Fixes_.28v33h.29| Two minor bug fixes]] | ||

+ | |Adjoint bug fix | ||

+ | |Daven Henze (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33h | ||

+ | |Approved 5 September 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#Fix_Monoterpene_SOA_zero_bug | Biomass Emissions bug fix]] | ||

+ | |Forward bug fix | ||

+ | | Jeffrey Pierce (CSU) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33h | ||

+ | |Approved 5 September 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#L-BFGS_inverse_Hessian_Update_.28v33i.29 | L-BFGS inverse Hessian update]] | ||

+ | |Adjoint Update | ||

+ | |Nicolas Bousserez (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33i | ||

+ | |Approved 21 September 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#Pseudo-Obs_Test_Bug_Fix_.28v33i.29 | Pseudo-obs test bug fix]] | ||

+ | |Adjoint bug fix | ||

+ | |Nicolas Bousserez (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33i | ||

+ | |Approved 21 September 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#Version_33i_patches_.28v33i-patch1.29 | Version 33i patch1]] | ||

+ | |Adjoint bug fix | ||

+ | |Nicolas Bousserez (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33i-patch1 | ||

+ | |Approved 9 October 2012 | ||

+ | |-bgcolor="#CCFFFF" | ||

+ | |- | ||

+ | |[[#Version_33i_patches_.28v33i-patch2.29| Version 33i patch2]] | ||

+ | |Adjoint bug fix | ||

+ | |Nicolas Bousserez (CU Boulder) | ||

+ | |Yanko Davila (CU Boulder) | ||

+ | |v33i-patch2 | ||

+ | |Approved 22 October 2012 | ||

|-bgcolor="#CCFFFF" | |-bgcolor="#CCFFFF" | ||

|} | |} | ||

Line 95: | Line 151: | ||

--[[User:Daven|Daven]] 02:40, 10 May 2012 (EDT) | --[[User:Daven|Daven]] 02:40, 10 May 2012 (EDT) | ||

+ | |||

+ | ===== Fix Monoterpene SOA zero bug ===== | ||

+ | |||

+ | The bug fix from the forward model [http://wiki.seas.harvard.edu/geos-chem/index.php/Setting_up_GEOS-Chem_nested_grid_simulations#Monoterpene_SOA_zero_bug_fix Monoterpene SOA zero bug fix] has been applied. | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 16:29, 6 September 2012 (EDT) | ||

+ | |||

==== Updates to adjoint model ==== | ==== Updates to adjoint model ==== | ||

Line 105: | Line 168: | ||

--[[User:Daven|Daven]] 19:03, 13 April 2012 (EDT) | --[[User:Daven|Daven]] 19:03, 13 April 2012 (EDT) | ||

+ | |||

+ | ===== BC fix to support offline simulation and AOD (v33h) ===== | ||

+ | - Fix to support offline BC simulation and AOD calculation | ||

+ | |||

+ | See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33h] | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 16:43, 6 September 2012 (EDT) | ||

+ | |||

+ | ===== L-BFGS inverse Hessian Update (v33i) ===== | ||

+ | |||

+ | Now use the background error instead of the initial L-BFGS inverse Hessian diagonal value where the gradient of the cost function is very small (in this case the inverse Hessian value is not updated by the algorithm). | ||

+ | Calculate and output the Degree of Freedom for Signal. | ||

+ | |||

+ | See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33i] | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 20:34, 21 September 2012 (EDT) | ||

==== Bug fixes in adjoint model ==== | ==== Bug fixes in adjoint model ==== | ||

Line 158: | Line 237: | ||

New stratospheric chemistry module (strat_chem_mod.f), was set to loop over the number of tracers (NSTPL) that has read from input.gcadj. It caused a problem when LADJ_STRAT is turned off, LADJ_STRAT is not appeared in input.gcadj (NSTPL = 0), or LADJ = F for spin-up. | New stratospheric chemistry module (strat_chem_mod.f), was set to loop over the number of tracers (NSTPL) that has read from input.gcadj. It caused a problem when LADJ_STRAT is turned off, LADJ_STRAT is not appeared in input.gcadj (NSTPL = 0), or LADJ = F for spin-up. | ||

− | --[[User:Hyung-Min Lee| | + | --[[User:Hyung-Min Lee|Hyung-Min Lee]] 19:26, 14 Jul 2012 (EDT) |

− | ===== | + | ===== Inverse Hessian and pseudo-obs cost function formulation updates (v33g) ===== |

- Possibility to compute the L-BFGS inverse Hessian, which gives the posterior error covariance matrix of the inversion. Here only the diagonal elements (the error variances) of the matrix are computed. Unlike the DFP method, this method allows to compute a posterior errors for large dimensional problems (e.g. high-resolution inversions). | - Possibility to compute the L-BFGS inverse Hessian, which gives the posterior error covariance matrix of the inversion. Here only the diagonal elements (the error variances) of the matrix are computed. Unlike the DFP method, this method allows to compute a posterior errors for large dimensional problems (e.g. high-resolution inversions). | ||

Line 170: | Line 249: | ||

--[[User:Nicolas Bousserez|Nicolas Bousserez]] 05:34, 11 August 2012 (EDT) | --[[User:Nicolas Bousserez|Nicolas Bousserez]] 05:34, 11 August 2012 (EDT) | ||

+ | |||

+ | ===== Two Minor Bug Fixes (v33h) ===== | ||

+ | - Fix for OpenMP in subroutine EMITHIGH_ADJ. This can correct some errors for the BC emissions adjoints. Without this fix, users may notice some error in FD vs ADJ calculations for shorter simulations. | ||

+ | |||

+ | - Update subroutine CHEM_SO2_ADJ to avoid underflow. | ||

+ | |||

+ | See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33h] | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 16:43, 6 September 2012 (EDT) | ||

+ | |||

+ | ===== Pseudo-Obs Test Bug Fix (v33i) ===== | ||

+ | - Bug fix for the definition of the observation error in the pseudo-obs inversion test and filter to avoid NaN adjoint value. | ||

+ | |||

+ | See benchmark results [https://docs.google.com/spreadsheet/ccc?key=0As2MewHKyWpDdDlTeFdPanU2RzlWc0luaFpEQXlyclE&pli=1#gid=0 v33i] | ||

+ | --[[User:Yanko|Yanko]] 20:34, 21 September 2012 (EDT) | ||

+ | |||

+ | =====Version 33i patches (v33i-patch1) ===== | ||

+ | - Bug fix for too small observation errors when doing pseudo-assimilation experiments | ||

+ | |||

+ | - Bug Fix for the DFP. | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 17:42, 1 November 2012 (EDT) | ||

+ | |||

+ | =====Version 33i patches (v33i-patch2) ===== | ||

+ | |||

+ | - Bug Fix for memory problems caused by the amount of iteration steps kept in memory while doing a Hessian approximation. | ||

+ | |||

+ | --[[User:Yanko|Yanko]] 17:42, 1 November 2012 (EDT) | ||

== Outstanding issues not yet resolved in v33 == | == Outstanding issues not yet resolved in v33 == |

## Latest revision as of 23:06, 3 July 2013

## Contents

- 1 Overview
- 1.1 What's new in this version
- 1.1.1 Updates to forward model
- 1.1.2 Bug fixes in forward model
- 1.1.3 Updates to adjoint model
- 1.1.4 Bug fixes in adjoint model
- 1.1.4.1 Fix wet deposition adjoint (v33b)
- 1.1.4.2 Fix checkpointing of aircraft SO2 emissions (v33c)
- 1.1.4.3 Fix linoz adjoint (v33d)
- 1.1.4.4 Adjoint stability update (v33e)
- 1.1.4.5 Fix problem with LICS flag in using SCHEM (v33f)
- 1.1.4.6 Inverse Hessian and pseudo-obs cost function formulation updates (v33g)
- 1.1.4.7 Two Minor Bug Fixes (v33h)
- 1.1.4.8 Pseudo-Obs Test Bug Fix (v33i)
- 1.1.4.9 Version 33i patches (v33i-patch1)
- 1.1.4.10 Version 33i patches (v33i-patch2)

- 1.1 What's new in this version
- 2 Outstanding issues not yet resolved in v33

## Overview

BETA RELEASE

- Previous version: GEOS-Chem_Adjoint_v32

### What's new in this version

GEOS-Chem Adjoint v33 contains the following major updates and improvements.

Feature | Type | Submitted by | Committed by | Version | Status |
---|---|---|---|---|---|

Add TES O3 IRK observation operator | Adjoint Update | Daven Henze (CU Boulder), Kevin Bowman (JPL) | Daven Henze (CU Boulder) | v33a | Approved 13 Apr 2012 |

Fix wetdep adjoint | Adjoint bug fix | Fabien Paulot (Harvard University), Daven Henze (CU Boulder) | Daven Henze (CU Boulder) | v33b | Approved 20 Apr 2012 |

Fix checkpointing of aircraft SO2 emissions | Adjoint bug fix | Daven Henze, Kateryna Lapina (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33c | Approved 20 Apr 2012 |

Fix linoz adjoint | Adjoint bug fix | Hyung-Min Lee (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33d | Approved 07 May 2012 |

Adjoint stability update | Adjoint bug fix | Jamin Koo (MIT), Daven Henze (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33e | Approved 09 May 2012 |

Fix SE Asia NH3 seasonality | Forward bug fix | Shailesh Kharol (Dalhousie), Steven Vogel (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33e | Approved 09 May 2012 |

Fix problem with LICS flag in using SCHEM | Adjoint bug fix | Hyung-Min Lee (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33f | Approved 13 July 2012 |

L-BFGS inverse Hessian and pseudo-obs cost function formulation updates | Adjoint update | Nicolas Bousserez (CU Boulder) | Nicolas Bousserez (CU Boulder) | v33g | Approved 10 August 2012 |

BC fix to support offline simulation and AOD | Adjoint update | Li Zhang (CU Boulder) | Yanko Davila (CU Boulder) | v33h | Approved 5 September 2012 |

Two minor bug fixes | Adjoint bug fix | Daven Henze (CU Boulder) | Yanko Davila (CU Boulder) | v33h | Approved 5 September 2012 |

Biomass Emissions bug fix | Forward bug fix | Jeffrey Pierce (CSU) | Yanko Davila (CU Boulder) | v33h | Approved 5 September 2012 |

L-BFGS inverse Hessian update | Adjoint Update | Nicolas Bousserez (CU Boulder) | Yanko Davila (CU Boulder) | v33i | Approved 21 September 2012 |

Pseudo-obs test bug fix | Adjoint bug fix | Nicolas Bousserez (CU Boulder) | Yanko Davila (CU Boulder) | v33i | Approved 21 September 2012 |

Version 33i patch1 | Adjoint bug fix | Nicolas Bousserez (CU Boulder) | Yanko Davila (CU Boulder) | v33i-patch1 | Approved 9 October 2012 |

Version 33i patch2 | Adjoint bug fix | Nicolas Bousserez (CU Boulder) | Yanko Davila (CU Boulder) | v33i-patch2 | Approved 22 October 2012 |

#### Updates to forward model

#### Bug fixes in forward model

##### Fix SE Asia NH3 seasonality (v33e)

The bug fix from the forward model fixing the seasonality of the NH3 emissions over SE Asia has also been applied.

1 week benchmark is approved 33e_003

--Daven 02:40, 10 May 2012 (EDT)

##### Fix Monoterpene SOA zero bug

The bug fix from the forward model Monoterpene SOA zero bug fix has been applied.

--Yanko 16:29, 6 September 2012 (EDT)

#### Updates to adjoint model

##### Add TES O3 IRK (v33a)

Implemented observation operator for assimilation of TES O3 IRKs.

See benchmark results 33a_001

--Daven 19:03, 13 April 2012 (EDT)

##### BC fix to support offline simulation and AOD (v33h)

- Fix to support offline BC simulation and AOD calculation

See benchmark results v33h

--Yanko 16:43, 6 September 2012 (EDT)

##### L-BFGS inverse Hessian Update (v33i)

Now use the background error instead of the initial L-BFGS inverse Hessian diagonal value where the gradient of the cost function is very small (in this case the inverse Hessian value is not updated by the algorithm). Calculate and output the Degree of Freedom for Signal.

See benchmark results v33i

--Yanko 20:34, 21 September 2012 (EDT)

#### Bug fixes in adjoint model

##### Fix wet deposition adjoint (v33b)

The adjoint was not properly handling evaporation. This fix helps make the adjoint of species such as NH3 better in a few locations where adjoints were diverging from forward model sensitivities by up to 50%.

See benchmark results 33a_002 vs 33b_002

--Daven 12:51, 20 April 2012 (EDT)

##### Fix checkpointing of aircraft SO2 emissions (v33c)

The SO2 aircraft emissions are distributed in the model based on a mapping from the 20 level inventory to the GEOS-Chem levels. This mapping depends upon the air masses in the models. The emissions are read in during the first hour of the month in the forward model, but on the last hour of the month in the adjoint. Thus they were being calculated inconsistently in the adjoint compared to the forward model. We now checkpoint these emissions rather than recalculate them

See benchmark results 33c_001 - 33c_004

--Daven 23:08, 7 May 2012 (EDT)

##### Fix linoz adjoint (v33d)

While the linoz routine is linear, and should be self-adjoint, it isn't exactly self-adjoint. For example, given initial forcings of STT_ADJ = 0d0 in every grid cell, it can generate non-zero values of STT_ADJ as output, which is incorrect.

In this update we use the discrete adjoint based upon reformulating the code to be consistent with the strat chem routine for the non-Ox species.

See benchmark results 33d_001 - 33c_007

--Daven 23:14, 7 May 2012 (EDT)

**Important: If you check out version 33d (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake.**
--Nicolas Bousserez 22:12, 7 July 2012 (EDT)

##### Adjoint stability update (v33e)

Several fixes were made to increase the numerical stability of the code

- Change the filter to 1d-10 in partition_adj.f
- Better error trapping in gckpp_adj_Integrator.f90 to prevent NaNs following a failed integration step during the adjoint

A few other small changes to improve performance were included in this update. Overall, the benchmark performance is not impacted by these update, see 33e_001

--Daven 02:26, 10 May 2012 (EDT)

**Important: If you check out version 33e (using the tag option), make sure to also check out the most recent version of linoz_mod.f (cvs checkout linoz_mod.f), since this file had not been updated by mistake.**
--Nicolas Bousserez 22:12, 7 July 2012 (EDT)

##### Fix problem with LICS flag in using SCHEM (v33f)

New stratospheric chemistry module (strat_chem_mod.f), was set to loop over the number of tracers (NSTPL) that has read from input.gcadj. It caused a problem when LADJ_STRAT is turned off, LADJ_STRAT is not appeared in input.gcadj (NSTPL = 0), or LADJ = F for spin-up.

--Hyung-Min Lee 19:26, 14 Jul 2012 (EDT)

##### Inverse Hessian and pseudo-obs cost function formulation updates (v33g)

- Possibility to compute the L-BFGS inverse Hessian, which gives the posterior error covariance matrix of the inversion. Here only the diagonal elements (the error variances) of the matrix are computed. Unlike the DFP method, this method allows to compute a posterior errors for large dimensional problems (e.g. high-resolution inversions).

- The equations for the initial estimate of the inverse Hessian in the DFP algorithm were also updated to allow for quicker convergence.

- The cost function for the pseudo-observations inversion test has been modified to properly account for the error term specified in the input.gcadj input file (ERROR).

See benchmark results v33g

--Nicolas Bousserez 05:34, 11 August 2012 (EDT)

##### Two Minor Bug Fixes (v33h)

- Fix for OpenMP in subroutine EMITHIGH_ADJ. This can correct some errors for the BC emissions adjoints. Without this fix, users may notice some error in FD vs ADJ calculations for shorter simulations.

- Update subroutine CHEM_SO2_ADJ to avoid underflow.

See benchmark results v33h

--Yanko 16:43, 6 September 2012 (EDT)

##### Pseudo-Obs Test Bug Fix (v33i)

- Bug fix for the definition of the observation error in the pseudo-obs inversion test and filter to avoid NaN adjoint value.

See benchmark results v33i --Yanko 20:34, 21 September 2012 (EDT)

##### Version 33i patches (v33i-patch1)

- Bug fix for too small observation errors when doing pseudo-assimilation experiments

- Bug Fix for the DFP.

--Yanko 17:42, 1 November 2012 (EDT)

##### Version 33i patches (v33i-patch2)

- Bug Fix for memory problems caused by the amount of iteration steps kept in memory while doing a Hessian approximation.

--Yanko 17:42, 1 November 2012 (EDT)

## Outstanding issues not yet resolved in v33

### Updates in the pipeline

- Reaction rate sensitivities (Paulot, Harvard; Walker, UT)
- Update inverse Hessian approximation (Bousserez, CUB)
- Off-diagonal error covariance matrices (Singh, VT; Liu, JPL)
- Deposition-based cost function and updated chemistry (Paulot, Harvard)
- ETOH simulations (Millet, UMN)
- NH3 updates (Zhu, Colorado)
- NO2/SO2 obs operators (Bousserez, Dalhousie)
- ANISO (Capps, Georgia Tech)