Unconstrained Minimization Problems and Strong Convexity
Explore unconstrained minimization problems and strong convexity in convex optimization. Learn about Taylor expansion, Cauchy-Schwarz inequality, and optimization conditions for optimal solutions.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
CSE 203B: Convex Optimization Week 7 Discuss Session Xinyuan Wang 02/20/2019 1
Unconstrained Minimization Problems Assume that the objective function is strongly convex on ?, there exists an ? > 0 such that ?2?(?) ?? (1) for all ? ?. Upper bound on ?2?(?), there exist a constant ? such that ?2?(?) ?? (2) for all ? ?. In class we are given two inequality condition for the optimal 1 2? 1 2? 2 2 2 ? ? ?? ? ?? ? 2 ? 1 2? 2 2 ? ? ? How to prove? 2 ?? ? ?? ? 2 2
Unconstrained Minimization Problems Taylor expansion with any ?,? ?. ? ? = ? ? + ?? ??? ? +1 2? ???2?(?)(? ?) for ? on the line segment between ? and ?. Consider the strong convexity ?2?(?) ?? for any ? ? ? ? ? ? + ?? ??? ? +? 2 ? ? 2 2 2 = ? ? +? ? ? +1 1 2 ??? ? 1 2? ?? ? 2 2? 2 2 2 ? ? ?? ? 2 It holds for any ? ?, let ? = ? be the optimal 1 2 ? = ? ? ? ? ?? ? 2? 2 1 2 ? ? ? ?? ? 2? 2 3
Unconstrained Minimization Problems Taylor expansion with any ?,? ?. ? ? = ? ? + ?? ??? ? +1 2? ???2?(?)(? ?) for ? on the line segment between ? and ?. Consider the upper bound ?2?(?) ?? for any ? ? ? ? ? ? + ?? ??? ? +? 2 ? ? 2 2 1 ???(?) 2 try to minimize the rhs, the minimum is achieved at ? = ? ? ? 1 1 ???(?) ? ? ?? ? 2? 2 Let ? be the optimal ? = ? ? ? ? 1 1 2 ???(?) ? ? ?? ? 2? 2 1 2 ? ? ? ?? ? 2? 2 1 1 2 2 ? ? ? 2 ?? ? ?? ? 4 2 2? 2?
Unconstrained Minimization Problems Taylor expansion with any ?,? ?. ? ? = ? ? + ?? ??? ? +1 2? ???2?(?)(? ?) for ? on the line segment between ? and ?. Consider the strong convexity ?2?(?) ?? for any ? ?, we have ? = ? ? ?(?) + ?? ??? ? +? 2 ? ? 2 2 2Cauchy-Schwarz inequality 2+? 2? ? ? ? ? ? ?? ? 2 2 Since? ?(?)for all ? ? 2+? 2 2 2 0 2? ? ? ? ?? ? 2 2 ? ? ?? ? ? 2 5
Unconstrained Minimization Problems Taylor expansion with any ?,? ?. ? ? = ? ? + ?? ??? ? +1 2? ???2?(?)(? ?) for ? on the line segment between ? and ?. Consider the first-order condition for any ? ? and ? is optimal ? = ? ? ? ? + ?? ??? ? 2? ? ? ? ?? ? 2Cauchy-Schwarz inequality We already prove that 1 2 ? = ? ? ? ? ?? ? 1 2? ? ? 2? 2 2 2? ? ?? ? 2 ?? ? 2 1 ?? ? 2 2? 2 1 2 2 2 ? ? ?? ? ?? ? 2 2? ? 6
