Handling SessionState Issues in HiveServer2 Codepath
Explore the challenges with SessionState in HiveServer2's codepath and discover solutions like refactoring OperationState, ensuring proper synchronization, and addressing related tickets to enhance performance and stability. Prepare for Hive 2.x release changes deprecating SessionState for better backward compatibility.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Let's Talk About SessionState Anthony Hsu November 16, 2015
SessionState From HiveServer2's HiveSessionImpl.java: // TODO: some SessionState internals are not thread safe. // The compile-time internals are synced via session-scope // or global compile lock. // The run-time internals work by magic! // They probably work because races are relatively // unlikely and few tools run parallel queries // from the same session. // 1) OperationState should be refactored out of // SessionState, and made thread-local. // 2) Some parts of session state, like mrStats and // vars, need proper synchronization. private SessionState sessionState;
Some SessionState-related tickets HIVE-9095: permanent functions' ClassLoader should be global instead of per-session HIVE-10876: Get rid of Driver and SessionState from HiveServer2 codepath HIVE-10893: Expose a global isHS2 check HIVE-11402: HS2 - disallow parallel query execution within a single Session HIVE-11403: Authorization policy init should be delinked from SessionState HIVE-11681: sometimes when query mr job progress, stream closed exception will happen HIVE-11878: ClassNotFoundException can possibly occur if multiple jars are registered one at a time in Hive HIVE-12409: make sure SessionState.initTxnMgr() is thread safe
Addressing SessionState Issues HIVE-10511: Replacing the implementation of Hive CLI using Beeline HIVE-10876: Get rid of Driver and SessionState from HiveServer2 codepath HIVE-11878: ClassNotFoundException can possibly occur if multiple jars are registered one at a time in Hive Hive 2.x release is an opportunity to deprecate SessionState and make backward-incompatible changes
