Bank of Italy VTL Engine & Editor: Lessons Learned and Future Perspectives
"Explore the journey of developing the VTL engine and editor at the Bank of Italy, showcasing its features, benefits, and architecture. Discover the importance of building a tool for VTL rule writing and integration into enterprise applications."
Download Presentation
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
The VTL Engine & Editor of the Bank of Italy. Lessons learned and future perspectives Attilio Mattiocco, IT Advisor, Bank of Italy 12thSDMX Experts workshop 7-11 October 2024 Amsterdam, the Netherlands
VTL E&E - overview A project aimed at helping business users write VTL rules A resource for learning VTL hands-on A flexible, scalable engine to integrate VTL in enterprise applications An open architecture for connecting VTL to different data sources and metadata systems Easy to install, very few requirements Standalone engine (pure Java), as part of a wider IT framework Seamless integration with R available out-of-the-box (R package) VTL Studio (Shiny VTL IDE) with editor and visualization of output, graph and lineage
Why developing an engine? When we started the development (ESCB SDMX Tools Task Force, 2018) the situation regarding VTL software was not clear The development of an engine from scratch forced us to develop a deep understanding of the language and a practical interpretation of the specifications All the discussions in the VTL TaskForce (some have not ended yet ) have benefited from this practical knowledge We can test new ideas and changes quickly We have a tool that can be used to show VTL in action to potential users, demystifying the fear of an eccessive complexity of this technology We can use the engine to test the documentation, using the examples as test cases, with a double benefit for the engine and for the quality of VTL
VTL E&E - architecture RVTL VTL Kernel (Jupyter notebook kernel) REST Web App (recently deprecated, waiting for standard) CLI interface (to launch VTL from scripts) Front ends (R package with VTL Studio and RVTL API) VTL Engine VTL In-Memory Executor (Rule execution, data processing) data process VTL Compiler (Code parsing, model checks) VTL Apache Spark Executor (Rule execution, data processing) Data and Metadata DATA METADATA Scratch Environment R Environment SDMX Environment CSV Environment Spark Environment Custom (in-memory) (dataframes) (sdmx-core) (flexible behaviour) (with Spark engine) (user developed) Scratch Repository SDMX Repository Json Repository Custom (in-memory) (Fusion MR) (VTL Guidelines) (user developed)
Future work Align to VTL specs v.2.1 and build a documentation environment for testing the examples Join SDMX.io initiative and enhance the integration with FMR Engine documentation revamp Work to VTL REST API VTL Task Force Maybe E&E for education (using the examples in the VTL documentation) Python front-end
Thanks! attilio.mattiocco@bancaditalia.it valentino.pinna@bancaditalia.it https://github.com/vpinna80/VTL
