Skip to Main content Skip to Navigation
New interface
Conference papers

PaDaWAn: a Python Infrastructure for Loosely Coupled In Situ Workflows

Abstract : This paper presents PaDaWAn, an infrastructure written in Python to provide loosely coupled in situ capabilities to accelerate file-based simulation workflows. It provides services for in-memory data exchange between applications and a simple configuration model to switch from a file-based workflow to a loosely coupled in situ workflow. The infrastructure is currently based on CEA-DAM Hercule parallel I/O library by providing an ABI-compatible library to intercept simulation data in a transparent way and to facilitate integration into existing simulation codes and tools. PaDaWAn implements a producer-consumer pattern with buffering of data in an in-memory staging service with automatic memory management and running on dedicated resources. We describe the key design decisions and main architectural features, and share the lessons learned from the development of the infrastructure and from setting up test runs on two production-like workflow cases. We conclude on the perspectives for our infrastructure.
Complete list of metadata

Cited literature [17 references]  Display  Hide  Download
Contributor : Julien Capul Connect in order to contact the contributor
Submitted on : Tuesday, February 25, 2020 - 10:43:05 AM
Last modification on : Wednesday, February 26, 2020 - 1:41:56 AM
Long-term archiving on: : Tuesday, May 26, 2020 - 1:57:06 PM


Files produced by the author(s)




Julien Capul, Sébastien Morais, Jacques-Bernard Lekien. PaDaWAn: a Python Infrastructure for Loosely Coupled In Situ Workflows. ISAV '18: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization, Nov 2018, Dallas, United States. pp.7-12, ⟨10.1145/3281464.3281470⟩. ⟨cea-02490376⟩



Record views


Files downloads