Scaling out ETL queries for Life Science data in production
There exists an abundance of Linked Data storage solutions, but only few meet the requirements of a production environment with interlinked life sciences data. In such environments, a triple store has to support complex SPARQL queries and handle large datasets with hundreds of millions of triples. The Ontoforce platform DisQover offers federated search for life sciences, relying on complex federated queries over open life science data, run in an ETL-pipeline to anticipate user actions in its exploratory search interface. Different state-of-the-art approaches for scaling out are compared, both in terms of their ability to execute the queries as in terms of ETL pipeline performance. This paper analyzes and discusses the features of the datasets and query mixes. An in-depth analysis is provided on an individual query basis revealing the strengths and weaknesses with respect to certain query types.
Published in 2016 in Proceedings of the 9th International Conference on Semantic Web Applications and Tools for Life Sciences.
Read this paper online
Cite this paper in your publications
- Use the BibTeX entry to easily refer to this paper.
- Alternatively, you can refer to this paper as: De Witte, D., De Vocht, L., Knecht, K., Pattyn, F., Constandt, H., Mannens, E. and Verborgh, R. (2016), “Scaling out ETL queries for Life Science data in production”, in Proceedings of the 9th International Conference on Semantic Web Applications and Tools for Life Sciences.