Documentation and training material

This is typically performed in real-time on streamed data. It provides immediate feedback to steer the experiment. Online data analysis typically favors latency over completeness and accuracy.

Several tools to perform online data analysis are available, offering different degrees of programmatic flexibility and accessibility, ranging from GUIs to raw data access.

Karabo scenes allow quick and easy introspection into control system data. For perfomant image-based data analysis, EXtra-foam offers a rich GUI alongside with dedicated interfaces. On the programmatic side, EXtra-metro is a framework for rapid development of Python-based online analysis code. Finally, the Karabo bridge is a protocol to receive real-time data with a client library in C++ and Python.

This offers more flexibility and explorability, and the focus is on quality. Scientific data at the European XFEL is saved in HDF5 files in a specific format and can be analyzed on the Maxwell computing cluster. The latter provides a diverse computing environment, including a JupyterHub and various pre-installed software.
The data analysis group provides a high-level Python library to access the data, alongside with data processing services that apply common and established analysis steps automatically and immediately after data acquisition.