Sustainable Utilization of Global Hydrological Research Software: WaterGAP Development Progression
WaterGAP, a hydrological research software, is being reprogrammed in Python to meet sustainability requirements. The goal is to enhance maintainability, extensibility, and modularity while improving computational performance and accessibility. Challenges include understanding legacy code, acquiring functional user stories, and optimizing computational performance in Python. The project aims to rewrite WaterGAP with a modular structure, making it usable, testable, and easily maintainable for the community.
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
Towards a sustainable utilization of the global hydrological research software WaterGAP Emmanuel Nyenah (Goethe University Frankfurt) Prof. Dr. Petra D ll (Goethe University Frankfurt) Dr. Robert Reinecke (University of Postdam) 27th April 2023 Session ESSI3.6 Collaborative Science Through Free and Open Source Software Tools and Frameworks in Earth sciences (Geo, Weather, Climate)
Summary Motivation Motivation Sustainable research software is software that: [1] is maintainable, [2] is extensible, [3] is flexible (adapts to user requirements), [4] has a defined software architecture, [5] is testable, [6] has a comprehensive in-code and external documentation, and [7] is accessible (the software is licensed as Open Source with a digital object identifier (DOI) for proper attribution). WaterGAP has been developed and modified for about 30 years by researchers with little to no training in software development resulting in software with over thousand lines of code, little modularity and documentation. Goal Goal Applied Best Practices Reprogram WaterGAP in Python, applying best practices to meet sustainability requirements while maintaining computational performance. Challenges lesson learned Challenges & lesson learned Agile MAKE INTERNAL AND EXTERNAL DOCUMENTATION COMPREHENSIBLE APPLY PROJECT MANAGEMENT PRACTICES CONSIDER SOFTWARE ARCHITECTURE, LANGUAGE VERSION CONTROL APPLY DESIGN PATTERNS AND QUALITY CONTROL Understanding legacy code needs time and effort . Getting functional user stories is hard. Packages are available for improving the computational performance of Python. CONSIDER SOFTWARE ARCHITECTURE, LANGUAGE & VERSION CONTROL APPLY DESIGN PATTERNS AND QUALITY CONTROL MAKE INTERNAL AND EXTERNAL DOCUMENTATION COMPREHENSIBLE APPLY PROJECT MANAGEMENT PRACTICES nyenah@em.uni-frankfurt.de
Motivation WaterGAP Development Progression: Very low comment density 30 25 Lines of code (x1000) 20 WaterGAP 2.2e has 83 files ranging from 1 to 4000+ source lines of code 15 10 5 0 2.1d 2.1e 2.1f 2.1g 2.1h Model version Total Lines of code 2.2 2.2b 2.2c 2.2d 2.2e Comment Lines Home Home nyenah@em.uni-frankfurt.de
Project Goals Modular Usable & Testable Usable by WaterGAP community. Software components can be tested Rewrite WaterGAP with a modular structure. Flexible, Extensible & Maintainable Easily change resolution. New features can be added in the future and are easily repairable Modern Language Use Python as it is more readable than C++ Open source Documentation Provide comprehensive documentation Open source with a License and a DOI for proper attribution Home Home nyenah@em.uni-frankfurt.de
Agile Project Management Burndown chart (Apr 2022 Apr 2023) 6 Planned and completed user stories (Apr 2022 - Apr 2023) 20 25 Outstanding man hours (x100 hours) Remaining and completed user stories 18 5 16 20 14 1 big task remaining: Implementing human water abstraction in the model Number of user stories 12 15 4 10 8 10 6 3 4 5 2 2 0 0 0 1 2 3 4 5 6 7 8 9 10 11 12 Month User stories completed Ideal man hours Remaining man hours 1 0 1 2 3 4 5 6 7 8 9 10 11 12 Month User stories completed User stories planned Home Home nyenah@em.uni-frankfurt.de
Software Architecture, Language & Version Control Architecture Language Legacy code C++ New Code Python Version Control Model controller View (MVC) Model: Manages data and logic (hydrological processes) Controller: Controls the data flow into model View: Visualizes output data in desired format. GNU Lesser General Public License v3.0 Home Home nyenah@em.uni-frankfurt.de
Comprehensive External & Internal Documentation New Documentation Comment Density 30 Model Version Comment Density (%) 25 Lines of code (x1000) 20 ReWaterGAP 35.3 15 2.2e 27.6 10 5 0 2.2e RewaterGAP (Incomplete) Model version Comment Lines Total Lines of code Automated with GitHub workflow Home Home nyenah@em.uni-frankfurt.de
Design Principles & Quality Control Design Patterns Quality checks Design patterns are ready-to-use solutions to recurring software design problems. Apply design patterns if required. e.g. Factory Method , Fa ade, etc . https://refactoring.guru/design- patterns/catalog Internal and external review: Check code quality Linting: Testing Analyzing code for issues such as style deviations and unsafe constructs. Home Home nyenah@em.uni-frankfurt.de
Challenges & Lesson Learned Challenge Lesson learned Getting functional user stories Requires iteration Improving computational performance Packages are available (eg. Numba) Knowledge of two languages (C++ & Python) There will be some differences (new user stories and model processes) Model outputs similarity Understanding legacy or complex scientific code Needs time and effort Home Home nyenah@em.uni-frankfurt.de
User Stories A user story is an informal, general explanation of a software feature written from the perspective of the end user or customer. The purpose of a user story is to articulate what the software should and should not do. https://www.atlassian.com/agile/project-management/user-stories Example user stories Home Home nyenah@em.uni-frankfurt.de
Performance and Interface Performance Interface Model Version Runtime per year (mins) Remarks ReWaterGAP 8-9 You can only get reasonable runtime with performance improving packages. 2.2e 4-6 Written in C++ Home Home nyenah@em.uni-frankfurt.de
What is WaterGAP? WaterGAP is a global hydrological model that quantifies human use of groundwater and surface water as well as water flows and water storage and thus water resources on all land areas of the Earth. It is also used to access climate change impacts on these natural resources https://gmd.copernicus.org/articles/14/1037/2021/ Home Home nyenah@em.uni-frankfurt.de
