Realising Data-Centric Scientific Workflows with Provenance-Capturing on Data Lakes

Since their introduction by James Dixon in 2010, data lakes get more and more attention, driven by thepromise of high reusability of the stored data due to the schema-on-read semantics. Building on this idea,several additional requirements were discussed in literature to improve the general usability of the concept,like a central metadata catalog including all provenance information, an overarching data governance, orthe integration with (high-performance) processing capabilities. Although the necessity for a logical and aphysical organisation of data lakes in order to meet those requirements is widely recognized, no concreteguidelines are yet provided. The most common architecture implementing this conceptual organisation is thezone architecture, where data is assigned to a certain zone depending on the degree of processing. This paperdiscusses how FAIR Digital Objects can be used in a novel approach to organize a data lake based on datatypes instead of zones, how they can be used to abstract the physical implementation, and how they empowergeneric and portable processing capabilities based on a provenance-based approach.