Faculty of Science, Technology and Medicine
Faculty of Law, Economics and Finance
Faculty of Humanities, Education and Social Sciences
Interdisciplinary Centre for Security, Reliability and Trust
Luxembourg Centre for Systems Biomedicine
Luxembourg Centre for Contemporary and Digital History
Luxembourg Centre for European Law
Luxembourg Centre for Socio-Environmental Systems
The Doctoral School in Science and Engineering is happy to invite you to Damien Vincent FRANCOIS’s defence entitled
Practical Deep Learning Applied to Biomedical Data
Supervisor: Prof Jacques KLEIN
Current Deep Learning methods have proven to be powerful tools for solving complex tasks in many domains. However, their efficacy truly manifested itself when massive amounts of data and computing power became available to train Deep models on extensive collections of curated data samples. Unlike convolutional or recurrent architectures that embed strong inductive biases tailored to specific data modalities (e.g., spatial locality in images or temporal order in sequences), transformer models are largely modality-agnostic and avoid such built-in assumptions. This design choice grants them flexibility across domains, but also increases their dependence on large-scale data and huge computing power for effective learning. In the context of Biomedical data, such access to both data and compute can be challenging. Ethical and practical concerns are examples of constraints that can make data collection very challenging. To mitigate these challenges, our work tackles the main hurdles for the practical use of current Deep Learning methods for biomedical data, in particular for data in the form of time-series, by (1) providing a tool to analyze and curate time-series data to yield high quality datasets in a practical way, (2) proposing a new efficient approach named CURtransformer to mitigate the cost of Transformer models, and finally (3) proposing a new data augmentation method leveraging a Flow Matching model to increase minority class representation in imbalanced datasets and increase classification performance.
For the expert curation of high-quality data, we introduce a tool which facilitates the processing, detection, and extraction of anomalous signals in Local Field Potential time-series data. Using a wavelet-based signal decomposition algorithm with user-adjustable frequency and time thresholding, our tool can allow for the automatic detection, extraction, and subsequent labeling of anomalies to create high-quality datasets. The tool uses an intuitive Graphical User Interface to enable users to reframe and modify automatically detected anomalies or manually flag anomalies. The detection algorithm enables the analysis of high-resolution data beyond real-time, and full recording sessions can be processed in a fraction of the full recording time.
To address the efficiency problem, we introduce CURformer. The Self-Attention mechanism underpinning the success of Transformer models has a main drawback, its quadratic complexity with regard to input length. Hence, our work on this front focused on an approximation of the Attention matrix to linearize its cost in time and memory. We created a novel Attention approximation method inspired by the CUR low rank approximation method, both for legacy hardware, and, with our hardware-aware fused kernel version, for modern GPUs. Both methods achieve task-related performance comparable to state-of-the-art approaches, while being much less memory and compute intensive.
For our work, the last challenge to overcome is the sample efficiency of Deep Learning approaches. To defer the use of expert analysis to downstream tasks, we focused not on the encoding of inductive biases from expert knowledge but on the generation of synthetic data. Through our Flow Matching method, we can create synthetic datasets from real Local Field Potential time-series data, compensate for low data regime and imbalanced datasets to stabilize training, and improve classification performance. The use of Flow Matching also opens the opportunity to analyze the dynamics of sample generation and harness this process to highlight divergence between classes. Altogether, our work focuses on the main challenges for the practical use of Deep Learning models for biomedical data, from gathering high-quality expert-labeled data, to memory and compute efficiency of state-of-the-art models, as well as bypassing the sample efficiency problem through synthetic data generation.