Explore the goals of the RDA Data Fabric Interest Group in paving the way for reproducible science through integrated data fabrics. Learn about essential components, characteristics, and infrastructure perspectives to drive collaborative efforts within the data science community.
Please find below an Image/Link to download the presentation.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author. If you encounter any issues during the download, it is possible that the publisher has removed the file from their server.
You are allowed to download the files provided on this website for personal or commercial use, subject to the condition that they are used lawfully. All files are the property of their respective owners.
The content on the website is provided AS IS for your information and personal use only. It may not be sold, licensed, or shared on other websites without obtaining consent from the author.
E N D
Presentation Transcript
RDA Data Fabric (DF) Interest Group Peter Wittenburg & Gary Berg-Cross DF IG Goals Introduction to Group Coming to a reproducible data science is a high priority. Only highly automated self-documenting procedures respecting proper data organization principles will overcome current barriers. The Data Fabric group needs to work out directions in the belief that integrated data fabrics are a critical component of infrastructures paving the way to reproducible science. The goal is to develop alternative views, components and aspects of the DF concept and related infrastructure. Conceptualizations be discussed to come to an agreed RDA view on how the evolving DF landscape can be productively described. The idea for this IG emerged from the discussions amongst the chairs of various RDA WGs As part of this, essential DF components and their interrelation need to be identified and defined. Characteristics of a Data Fabric: We are just beginning to scout out the landscape of data fabric . In one view it is a minimalistic set of infrastructure and service requirements by which services can plug into (belong to) the defined fabric. In a data fabric we see how the separate components, developed separately, can be made to work together, this means that for different sets of components the data fabric will be different. We note, strongly, that it is meant as a descriptive/conceptual way to deal with the interrelation between many components, rather than prescriptive (like you would have with an architecture). This diagram provides a high-level view of possible actions within a Data Fabric running from raw data to increasingly documented data that has been enriched and analyzed creating referable and citable data. As shown publications are part of this Data Fabric since they are often used for data mining and other analysis. Some of the existing RDA groups including metadata WGs and IGs are working on DF components and need to be positioned in such a landscape. New working groups need to be defined to work on identified components and interfaces.
Data Fabric IG DF Organizing Chairs: Gary Berg-Cross & Peter Wittenburg
Some early statements on Data Fabric & Components Infrastructure Component View (after Reagan Moore) Data Fabric Service View (after Beth Plale) A DF should: Data Object View (after Peter Wittenburg) A data fabric is the set of software and hardware infrastructure components that are used to manage data, information, and knowledge. When an enterprise implements a data management solution, one of multiple types of DFs infrastructure is typically chosen to enable the: The data fabric covers a domain of registered digital objects (DO) that are stored in well managed repositories. Be self-documenting a service contributes to the lifecycle of data objects it handles and must keep track of the scientifically relevant actions it performs on those data objects. DOs are associated with metadata describing its creation context and history (provenance). Data management enterprise to build a data repository, manage an information catalog, & enforce management policies The resulting log files are periodically be sent to a provenance consolidator. The Data Fabric covers a domain of registered software components (workflows, services) that are in fact a special class of DOs. Data analysis enterprise to process a data collection, apply analysis tools, and automate a processing pipeline. Track data objects through its service processing using one of the well- known object identifier schemes Data preservation enterprise to build reference collections and knowledge bases that comprise the intellectual capital, while managing technology evolution Actions on DOs may be guided by abstract policies that are explicit and thus auditable. Identify itself as one type of service as drawn from an RDA- agreed upon list of service types. Data publication discovery and access of data collections. There can be multiple data fabric implementations that should be highly interoperable. Implement an interface to a publish- subscribe system which serves as the Data Fabric Control mechanism. Data sharing controlled sharing of a data collection, shared analysis workflows, and information catalogs.
People and Plans Who The DF group is a joint initiative of the first 5 working groups who understood that all what they started needs to be seen as part of a bigger plan full of dynamics. They also realized that the work which they started needs to be maintained to meet new requirements. Others interested in the goal of improving our data creation machine is welcome to participate in the DF group. Status & Planned Outcomes The suggested Data Fabric IG is planned as a forum to discuss these alternative views, components and aspects of the DF concept. The DF group will first work on an initial position paper to characterize the landscape of components and interfaces that have the potential to realize a reproducible data science. To be discussed: What is the agreed RDA view on a Data Fabric. How the outputs from the RDA working groups fit in the DF concept and how they relate to each other and to various related WGs and IGs within the RDA. Such a landscape will change over years where components will be replaced by others requiring that the position paper will need to be amended frequently. The DF group will be a continuous source for identifying new requirements and barriers that need to be addressed by new RDA groups, in particular working groups. Which further activities are required to push the data fabric concept ahead. Continuation and initialization of working group activities related to the DF. Initiating Members Rebecca Koskela, Keith Jefferey, Jane Greenberg, Reagan Moore, Rainer Stotzka, Tim Delauro, Tobias Weigel, Raphael Ritz, Gary Berg-Cross, Peter Wittenburg, Daan Broeder, Larry Lannom, Beth Plale, Juan Bicarregui, Herman Stehouwer Improving the uptake of the WG outputs by communicating them as a coherent whole within the DF concept. Updating the results will be an interesting but worthwhile challenge. Prepared for the 4th RDA Plenary in Amsterdam, Sept. 2014,
