Status of the Framework
This content pertains to the current status of the Framework for Teng Li, as of August 301. It likely discusses updates, progress, or issues related to the framework. The information can provide insights into the development and implementation of this specific framework, possibly in a particular context or industry. Readers may expect to find detailed analysis, evaluations, or future implications regarding Teng Li's framework based on the provided information.
Uploaded on Feb 22, 2025 | 0 Views
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Status of the Framework Teng LI 2024.8.30 1
Recent development activities of the framework Database interface Parallel simulation Upgrade of the TES implementation Improvement of the user interface Installation, documentation and validation 2
Database Interface In HERD, conditions data (calibration data, alignment data) and some parameters are managed by the database The framework provides database interface for applications to access these data DBI Interface services GeometrySvc, CalibrationSvc: make queries, unpack the payload and read parameters ParamDBSvc, CondDBSvc: handle the tag/global tag, internal of validity and generate queries. DBISvc: connection to the DB backend 3 DBIAPI: concrete implementation of different DB backends
Database Interface: status and plan Implementation of the DBI interface is an urgent task Design and implementation of ParamDBSvc, CondDBSvc, DBISvc and DBIAPI are basically done Implementation of CalibrationSvc (of different detectors) is on-going (partially finished) A few reconstruction algorithms have already been using CalibrationSvc (e.g. PSDRec) Source code can be found in 38-DBISvc branch https://code.ihep.ac.cn/herdos/offline/-/tree/38-DBISvc Plans: Improvement of CalibrationSvc Code quality check before merging (coding conversion, documentation, tests, ...) 4
Parallel Simulation Long term development Inter-event and intra-event level of parallel simulation have been developed in HERDOS Inter-event level https://code.ihep.ac.cn/herdos/offline/-/tree/MT Intra-event level https://code.ihep.ac.cn/herdos/offline/-/tree/MT-track Most development work has been finished List of TODOs: Parallelize the CRMC pakcage: Native CRMC cannot be used in parallel mode. Some patches are needed. More tests and fine-tunning before the code can be merged 5
Upgrade of the TES implementation SniperKernel does not provide concrete TES implementation HERDOS TES is implemented based on podio::EventStore (integrated with SNiPER) Source code: https://code.ihep.ac.cn/herdos/offline/-/tree/master/DataManagement/PodioDataSvc https://code.ihep.ac.cn/herdos/offline/-/tree/master/DataManagement/PodioSvc https://code.ihep.ac.cn/herdos/offline/-/tree/master/DataManagement/PodioRootIO Recently, we are planning to upgrade the base implementation from using podio::EventStore to podio::Frame 6
Upgrade of the TES implementation Motivation of the upgrade (from EventStore to Frame) podio::Frame supports event-level, run-level and job-level data all together Podio has dropped the EventStore implementation From podio v00-17, EventStore is marked as deprecated From podio v01-00, EventStore code is officially removed Upgrade of podio should be made asap New features (support of schema evolution is a strongly desired feature) Impact of the upgrade: Concrete implementation of the TES is transparent to user code, so the impact to the user code is minimum Impact to the framework code is big. More evaluation is needed. The upgrade still needs to be evaluated 7
Improvement of the user interface Old user interface Naked python code A lot of properties to be configured Easy to have hard-coded stuff 8
Improvement of the user interface New user interface A simple python script to define modules to be called Options are then exported to CLI 9
Improvement of the user interface Status: Code is available in 28-UIOptimize branch: https://code.ihep.ac.cn/herdos/offline/-/tree/28-UIOptimize Python code for most published packages are developed Detector simulation, digitization and reconstruction workflow is supported More advanced features unpon request 10
Installation, documentation and validation Recently, HERDOS is enabled on AlmaLinux9 (both for amd64 and arm64 build) The auto installation scripts need to be improved to be cleverer (auto detection of OS, architecture, gcc version and apply corresponding patches) Documentation update has been made mandatory before merging of any code components Include comments needed to generate doxygen document and updates of markdown-based user mannual Documentation source code: https://code.ihep.ac.cn/herdos/docs (connected with Gitlab CD) Documentation website: https://herd.ihep.ac.cn/internal/herdos/manual/index.html 11
Installation, documentation and validation Automated software validation is being improved A python toolkit to build test cases and profile software performance https://code.ihep.ac.cn/herdos/offline/-/tree/3-CI- deploy/HerdUtil/valprod CMake rules to collect all test cases Based on CTest https://code.ihep.ac.cn/herdos/offline/-/blob/3-CI- deploy/cmake/HerdTest.cmake Connection with Gitlab CI system Docker containers are developed to run tests (now support CentOS7 and el9. Full set of test matrix to be developed) Code is available in 3-CI-deploy branch https://code.ihep.ac.cn/herdos/offline/-/tree/3-CI-deploy Need to collect more test cases from applications Providing test cases is also made mandatory before merging code 12
Data formats and data analysis Raw Data Calib. Data Rec. Data Det. Sim. hits Digi. Sim. hits Pre-selected DSTs not yet existed podio data format The podio data format can be analyzed within or without HERDOS Advanced features are only provided in HERDOS (relationship, geometry) The format of data product has not been concluded. This is something to be discussed. 13
