Data-Driven Parameter Estimation for Complex Physics Models
Dive into the world of data-driven parameter estimation for complex physics models without using ensembles or gradients. Explore the challenges and approaches to estimating unknown parameters in large-scale physics-based models. Understand the complexities of modeling high-dimensional systems and the methods used to tackle them effectively.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Data-Driven Parameter Estimation for Large-Scale Physics-Based Models without Ensembles or Gradients Ankit Goel Postdoctoral Research Fellow Workshop on machine Learning, data Mining and data Assimilation in Geospace, 2020 24thSeptember 2020 Collaborators: Dennis S. Bernstein, Aaron J. Ridley, Garima Malhotra, Brandon Ponder
What Makes a Model Complex? High dimensional ~? 106 states Nonlinear Complex architecture hybrid systems multi-rate subsystems Contain representational models subgrid models closure models Nonlinearly parameterized Inaccessible 3
Complex Model - GITM Global Ionosphere Thermosphere Model Magnetic Field EUV Inside? GITM? Electron Characteristics ??? ??= 2 2 3??? Ion Dynamics Ion Species Initialize Ionosphere and Thermosphere Vertical O2+, O+(4S), O+ (2D), O+ (2P), N+, N2+, NO+, H+, He+ 3??? ?? ?? ??? Neutral Species O, O2, N(2D), N(2P), N(4S), N2, NO, H, He Neutral Dynamics Electrons Horizontal + ?.??+ ?? ?? Electron Characteristics High Latitude Magnetic Field Ion Species Density Ion Temperature O+ Velocity Electron Temperature Electron Density Neutral Species Density Neutral Species Wind Bulk Neutral Wind Neutral Temperature Vertical Horizontal qt Nsr qNs ??= ????+ ? ?? u u r Ns 0, The unknown parameter ?? is representational inaccessible 4
Parameter Estimation Problem Physical system model ??+1= ? ??,??,? ??= ? ??,??,? Unknown parameters Estimate ? using the input data ?? the output measurements ?? Unknown Measured Standard assumptions Structurally identifiable Sufficiently exciting input 5
Estimation Model Estimation model ??+1= ? ??,??, ? ??= ? ??,??, ? In complex models such as GITM, Explicit expressions for ? and ? usually not available only executable code is available State ?? may also not be available Gradient computation Ensemble computation 6
Possible Approaches ??+1= ? ??,??,? ??= ? ??,??,? ?? ?? ?? ??,?? ??+1 ??+1 = ? ,?? 2 ? ? = ? ? ?? ??= ??? ??= ? Linear regression Gradient descent method ?? ???=?? State estimation techniques Extended Kalman Filter Requires gradient computation Ensemble filters-UKF, EnKF, Particle filters Requires at least 2??+ 1 sigma points Estimation model and state are used to compute ?? ??+1= ?? ? 1 T ? = T Numerical gradient Adjoints Simulation executed multiple times Requires measurements 7
Retrospective Cost Parameter Estimation (RCPE) Physical system ?+= ? ?,?,? ? = ? ?,?,? ? ? Atmospheric process Chemical process Composite structure Electric engine Unknown Measured Computed Estimation model + ? ?+= ? ?,?, ? ? = ? ?,?, ? Estimated Parameters Output Error ? ? Parameter Estimator Retrospective Cost Optimization ? = ? ? 8
Parameter Estimator Parameter estimate Integrator state Parameter pre-estimate Output error ?? ?? ? |?| ? ? ?p ?? Discrete-time Integrator ??= ?? 1+ ?? 1 Absolute Value Component-wise Permutation ??= ?p?? Integrator Gain ??= ???? Maps pre-estimate to the first orthant Integrator gain ?? is updated using Retrospective cost optimization ?p permutes the entries of ? 0 0 1 ?1 ?2 ?3 ?2 ?3 ?1 1 0 0 0 1 0 ?231? = = ??= ????= ??? 9 ? = ? + ?
Retrospective Cost Optimization Unknown Hyperparameters Measured Computed Retrospective performance variable ?? ? ??+ ?f? ? ? ?? = ?? ?f,? + f,? ? ?f?? ?f,?, f,? ??, ? ?f? = ?? ?=1 Quadratic Positive definite Minimize the retrospective cost function ? ?? ? ?? T ? ?? ? ? ? + ?? ?T?? ? ?=1 ??+1= argmin ?? ? Computed exactly by RLS at each ? ? ?? Update the parameter estimate as ??+1= ?p??+1 = ?p ?+1??+1
The Filter and Search Space The coefficients ?? constrain ?? T T ?? ?1 ??f ? Choose ?? of the form ?1 1 ,1 0 0 0 1 0 , 0 1 ?? 0 ?2 such that T T = ?? ?1 ??f For example, to estimate 3 parameters with 1 measurement, 1 0 0 0 1 0 0 0 1 , , 11
Toy Problem with 3 Parameters cos?1 sin?1 ? ?1 3 2) 3 ??+log(1 + ?2 1 + sin?2 ??+1= ?? ??,3 0.5 2 1.1 + ?1 ??,1 2?? ??= ?3 4?3 ??,2 15 2?? 100? + ?2 ??= 2 + sin ?=1 No model information is used ?f=?1 ?+?2 ?2+?3 ?3 ??,1 ??,2 ??,3 ??= ?p 12
A Drawback of the Method RCPE uses only an executable code, and thus it cannot map parameter estimates to parameters ? ? ?? ?? ?? Parameter Estimator Model ? ?? Permutation 2 Permutation 1 Need to determine the correct permutation There are ??! permutation matrices For ??= 3, there are 6 possible ways to inject the estimate in the model
Toy Problem with 3 Parameters Revisited cos?1 sin?1 ? ?1 3 2) 3 ??+log(1 + ?2 1 + sin?2 ??+1= ?? 0.5 2 1.1 + ?1 2?? ??= ?3 4?3 Parameter Permutations 15 2?? 100? + ?2 ??= 2 + sin ?=1 231 works ?f=?1 ?+?2 ?2+?3 ?3 ??,1 ??,2 ??,3 ??= ?p 14
Estimation Of Parameters In GITM Earth simulated with ? Measured output Computed output Magnetic Field EUV Inside? GITM? Ion Dynamics Ion Species Initialize Ionosphere and Thermosphere Vertical O2+, O+(4S), O+ (2D), O+ (2P), N+, N2+, NO+, H+, He+ Neutral Species O, O2, N(2D), N(2P), N(4S), N2, NO, H, He Neutral Dynamics Electrons Horizontal Electron Characteristics High Latitude Magnetic Field Ion Species Density Ion Temperature O+ Velocity Electron Temperature Electron Density Neutral Species Density Neutral Species Wind Bulk Neutral Wind Neutral Temperature Vertical ntinuity e quatio Horizontal qt Nsr qNs u u r Ns 0, ?? Parameter Estimator ? =Measured-Computed
Photoelectron Heating Efficiency Estimate PHE using simulated Champ Density measurements 90-minute averaged ?????????? along the CHAMP orbit Initial PHE guess 90 minute averaged ??????????? along the CHAMP orbit Modeled PHE 90-minute averaged ?????????? along the GRACE orbit Estimated PHE Modeled PHE Initial Guess Estimated 90-minute averaged ??????????? along the GRACE orbit PHE 16
Photoelectron Heating Efficiency Estimate PHE using real Champ density measurements PHE updated by RCPE PHE constant 17
Photoelectron Heating Efficiency Estimate PHE using real Champ density measurements 18
Estimation Of Thermal Conductivity Coefficients Thermal conductivity relates temperature variation with the vertical gradient of the temperature in the atmosphere ?? ?? where ? =?O2?O2+ ?O2??2+ ?O?O ?O2+ ??2+ ?O ? ????? ?? ?? ?T using measurements of Estimate ? = ?O2 Global max density Global mean density Global min density ?O Simulated measurements 19
Estimation Of Thermal Conductivity Coefficients ??,1 ??,2 ??,3 ??= ? + 0.69? 4 0.56? 4 0.36? 4 Parameter Permutations 20
Computational Cost Of RCPE The underlying minimization problem is ???? dimensional RLS propagates a ???? ???? matrix cos?1 sin?1 ? ?1 3 2) 3 ??+log(1 + ?2 1 + sin?2 ??+1= ?? 0.5 2 1.1 + ?1 2?? ??= ?3 4?3 ??= 3,??= 1 ????= ? ??= 3,??= 3 ????= ? 21
ML DA RCPE Input Input Training Input Magnetic Field EUV Magnetic Field EUV Inside? GITM? Inside? GITM? Ion Dynamics Ion Species Ion Dynamics Initialize Ionosphere and Thermosphere Ion Species Vertical O2+, O+(4S), O+ (2D), O+ (2P), N+, N2+, NO+, H+, He+ Initialize Ionosphere and Thermosphere Vertical O2+, O+(4S), O+ (2D), O+ (2P), N+, N2+, NO+, H+, He+ Neutral Species O, O2, N(2D), N(2P), N(4S), N2, NO, H, He Neutral Dynamics Neutral Species O, O2, N(2D), N(2P), N(4S), N2, NO, H, He Neutral Dynamics Electrons Horizontal Electrons Horizontal Electron Characteristics Electron Characteristics High Latitude High Latitude Magnetic Field Ion Species Density Ion Temperature O+ Velocity Electron Temperature Electron Density Neutral Species Density Neutral Species Wind Bulk Neutral Wind Neutral Temperature Vertical Magnetic Field Ion Species Density Ion Temperature O+ Velocity Electron Temperature Electron Density Neutral Species Density Neutral Species Wind Bulk Neutral Wind Neutral Temperature Vertical Horizontal qt Nsr Horizontal qt Nsr qNs u u r Ns 0, qNs u u r Ns 0, Current Input Current Input RCPE seeks to improve the model Can improve the forecasting accuracy by improving the state estimates Forecast Forecast Forecast ML seeks to approximate models of the physical process Ex: Ampere Data -> Continuous predictions Approximate model can be used for forecasting DA seeks to improve the state estimates Ex: DART Applications: smoothing, improving state estimates 22
Retrospective Cost Parameter Estimation (RCPE) Thank you ? ?+= ? ?,?,? ? = ? ?,?,? ? ankgoel@umich.edu Physical system Relevant Papers A. Goel, and D. S. Bernstein, Gradient-, Ensemble-, and Adjoint- Free Data-Driven Parameter Estimation, AIAA Journal of Guidance, Control, and Dynamics, vol. 42, no. 8, pp. 1743 1754, 2019. DOI: 10.2514/1.G004158. A. Goel, B. Ponder, A. J. Ridley, and D. S. Bernstein, Estimation of Thermal-Conductivity Coefficients in the Global Ionosphere- Thermosphere Model, in Journal of Aerospace Information Systems. DOI: 10.2514/1.I010819 A. Goeland D. S. Bernstein, Injection-Constrained Filters for Linear and Nonlinear State Estimation, submitted to AIAA Journal of Guidance, Control, and Dynamics. + ?+= ? ?,?, ? ? = ? ?,?, ? Estimation model ? ? ? Parameter Estimator ? = ? ? 23
