Bredala: Pushing semantic into In-Situ applications to drive data redistribution

Speaker: Matthieu Dreher (ANL)
Date: Thursday, 3 December 2015, 10:30-12:00
Session: Programming Models II
Talk type: Project talk (30 min)

Abstract: In-Situ applications are a promising solution to tackle the problem of imbalance between computational capabilities and I/O bandwidth in leadership supercomputers. Initially designed to focus on I/Os, In-Situ applications now include a wide range of domains such as visualization, machine learning, filtering or feature tracking. In-situ infrastructures have to deal with complex heterogeneous codes using different data structures. Data need to be transformed and reorganized along the analysis path to fit the analysis needs. However redistributing complex data structures is difficult. In some cases, splitting the arrays of a data structure is not enough to preserve the integrity of the data. In this article, we present Bredala, a lightweight library to build a data model with enough information to preserve the semantic integrity of the data during a redistribution. We coupled Bredala with Flowvr, an In-Situ infrastructure which doesn't have a data model, and demonstrate its usability by improving our previous implementation of the Quicksurf algorithm. We show that our data model can simplify the network of large scale applications, improve the reusability of modules and offer a more efficient alternative to redistribute data.