Welcome to our demonstrator, a preview of an architecture designed to streamline and automate the process of data storage, conversion and mapping. This demonstrator is capable of taking raw files (xslx) from this CKAN data portal and transforming them into RDF (Resource Description Framework) without the need of any code.
The transformation process is guided by draw.io diagrams, which are stored in GitLab repositories. These diagrams are not just visual representations but also functional blueprints that can be converted into RDF. They define the mapping from the raw files to a graph and also define how the mapped entities relate to each other.
Our demonstrator includes a main diagram that outlines the alloy development process. Additionally, there are sub-diagrams that detail specific processes such as the Vickers hardness test, diffusion annealing, and EDX measurements. The domain context for those graphs is given in this presentation.
The raw files are hosted on a CKAN instance, where they can be previewed and queried (the green Data API button). Those raw file are mapped to RDF based on the mappings defined in the diagrams. Because the backend for storing the diagrams is GitLab, a CI job is automatically triggered on a change in one of the diagrams, which integrates/maps/reason the raw data. Once the transformation is complete, the RDF data is then pushed into a triple store for further use and analysis. This process not only enhances the interoperability of your data but also makes it easier to manage, share, and reuse.
All tools involved have been containerized and can be deployed on any linux box. The CI job was containerized and modulized with nextflow.
More Links
The created knowledge graph can be explored and queried with the following tools:
Example Sparklis Query
- This Sparklis Query links the calphad simulation with the observed EDX measurement
