Devices and Methods for Optimizing Parallel Execution of a Machine Learning Model.
Lamprou Ioannis, Zhang Zhen, Filhol Etienne, and Barthou Denis.
106-539-523-079-778, Dec 2024. Pending.
[
Patent ]
Denis Barthou is director of the Distributed and Parallel Tech. Lab. at Huawei Paris Research Center and full professor in Computer Science, on leave from Bordeaux INP.
He graduated from ENS Lyon and obtained a PhD from the University of Versailles St Quentin in 1998, on dependency analysis in the polyhedral model. He took a position as assistant professor at the same university. In 2009 he was appointed University professor at Bordeaux INP and he joined Inria's Runtime team. In 2017, he created and led Inria's Storm team, and at the same time was in charge of the Computer Science studies at the ENSEIRB-MATMECA engineering school, Bordeaux INP. In 2023, he joined the Huawei Paris research center.
In the last 2 years, he has focused on parallelization applied to AI, and in particular to open-source LLMs, both for training and inference.
Over the years, his research interests have focused on optimization, parallel and distributed methods, compilation and runtime systems for AI and high-performance computing applications.
Software
MAQAO: MAQAO is a
Modular Assembly Quality Analyzer and Optimizer, that I started to develop in 2004. This is a parallel performance analyzer and optimizer for developers and experts. MAQAO is actively maintained and developed at the University of Versailles.
MAQAO is part of
VI-HPS Institute and is among the EU Innovation Radar technologies and has been described in a blog on AWS that shows the potential of MAQAO. MAQAO has GPL licence.
AFF3CT: A Fast Forward Error Correction Toolbox. AFF3CT is dedicated to the Forward Error Correction (FEC or channel coding) simulations. It was initiated during the PhD of A. Cassagne in 2017. It supports a large range of codes: from the well-spread Turbo codes to the very new Polar codes including the Low-Density Parity-Check (LDPC) codes.
AFF3CT is actively maintained and developed at Paris Sorbonne university and at Inria. The community of users and developers regularly meet during an AFF3CT day. AFF3CT has MIT licence. MIPP: MIPP is a portable and Open-source wrapper (MIT license) for vector intrinsic functions (SIMD) written in C++11. It works for SSE, AVX, AVX-512 and ARM NEON (32-bit and 64-bit) instructions. MIPP wrapper supports simple/double precision floating-point numbers and also signed integer arithmetic (64-bit, 32-bit, 16-bit and 8-bit).
With the MIPP wrapper you do not need to write a specific intrinsic code anymore. Just use provided functions and the wrapper will automatically generates the right intrisic calls for your specific architecture. MIPP started its development during A.Cassagne PhD and is still actively maintained. PARCOACH: As current scientific applications mainly rely on the Message Passing Interface (MPI) parallel programming model, new hardwares clearly advocate for an MPI+X solutions with X a thread-based model such as OpenMP. The PARallel COntrol flow Anomaly CHecker, aims at helping developers in their debugging phase. Its development started during the E. Saillard PhD. It combines static and dynamic analyses to detect misuse of collectives in parallel applications.
Research activities
Current PhD students
► V. Alba: PhD on Resource dimensioning for heterogeneous architectures.
► D. Orhan: PhD on Modeling and dynamic optimization of software radio chains on heterogeneous architectures. Co-directed with C.Jego
PhD Alumni
► B. Coye (with Ubisoft). Dynamic Task Graph Scheduling by Composition, 2023, U.Bordeaux PhD thesis, co-directed with Pr. R.Namyst. Now research engineer at Ubisoft.
► V.-M. Nguyen. Compile-time Validation and Optimization of MPI Nonblocking
Communications, 2022, U.Bordeaux PhD thesis, co-directed with P. Carribault (CEA). Now research engineer at Eviden.
► C. T. Ait Kaci. Static and dynamic analysis for memory access concurrency error detection in MPI-RMA applications, 2022, U.Bordeaux PhD thesis. Now scientific project manager at Cap Gemini.
► A. Cassagne. Optimization and parallelization methods for software-defined radio, 2020, U. Bordeaux PhD thesis, co-directed with Pr. C.Jego. Now Ass. Professor at Paris Sorbonne University
► P. Huchant. Static Analysis and Dynamic Adaptation for Parallelism, 2019, U. Bordeaux PhD thesis. Now Senior Software Engineer at Synopsis, Bordeaux
► H. Brunie. Optimization of data allocation for HPC applications on heterogeneous memory architectures, 2019, U. Bordeaux PhD thesis, co-directed with P. Carribault (CEA). Now Postdoc at Inria Grenoble.
► C. Haine. Kernel Optimization by Layout Restructuring, 2017, U. Bordeaux PhD thesis. Now research engineer at HPE, Swizerland.
► G. Vaumourin. Hybrid Memory Hierarchy and Dynamic Data Handling in Embedded Parallel Architectures., 2016, U. Bordeaux PhD thesis. Now research engineer at ATOS, Grenoble.
► E. Saillard. Static/dynamic/iterative analyses for validation and improvement of multi-models HPC applications, 2015, U. Bordeaux PhD thesis. Now Inria Researcher.
► B. Putigny. Benchmark-driven Approaches to Performance Modeling of Multi-core architectures, 2014, U. Bordeaux PhD thesis. Now HPC engineer at Eviden, Bordeaux.
► S. Henry. Programming Models and Runtime Systems for Heterogeneous Architectures, 2013, U. Bordeaux PhD thesis. Now engineer at IOHK.
► L. Duchateau. Automatic Algorithm Derivation and Exploration in Linear Algebra for Parallelism and Locality, 2013, UIUC PhD thesis, co-directed with Pr. D. Padua. Now senior software engineer at Pure Storage, Bellevue, USA.
► A. Mazouz. Une Etude Empirique des Performances des Applications OpenMP sur les Plateformes Multi-coeurs, 2012, UVSQ PhD thesis, co-directed with Pr. S.-A. Touati. Now senior software engineer at Intel, Paris.
► A. Charif-Rubial. On code performance analysis and optimization for multicore architectures, 2012, UVSQ PhD thesis, co-directed with Pr. W. Jalby. In memoriam.
► J. Jaeger. Source-to-source transformations for irregular and multithreaded code optimization, 2012, UVSQ PhD thesis. Now Research engineer at CEA.
► P. De Oliveira Castro Herrero. Expression and optimization of data reorganizations on data flow parallelism
, 2010, UVSQ PhD thesis. Now Professor, HDR, at Paris-Saclay University, Versailles St Quentin en Yvelines.
► S. Donadio. Iterative optimization of performance libraries by hierarchical division of codes, 2007, UVSQ PhD thesis, directed by and co-advised with Pr. W. Jalby. Now Architect/Product manager at Bloomberg, New York, USA and adjunct professor at Columbia Engineering.
► C. Alias. Program Optimization by Template Recognition and Replacement, 2005, UVSQ PhD thesis, directed by and co-advised with P. Feautrier. Now Inria Researcher, HDR and chief scientific advisor of XtremLogic
Postdoc
► Lilia Ziane Khodja (2014), Modeling of parallel HPC applications running on platforms composed by modern multicore nodes interconnected with high performance networks, with B.Goglin. Now Consultant at ANEO;
Engineers
► P. Virouleau, working with ATOS on Parcoach project (2022-2024). Now permanent research engineer at Inria.
► M. Makni, working on H2020 Microcard project (2022-2023). Now research engineer at Lytid.
► C. Sakka, working on ANR Exacard project (2021-2022). Now engineer at ANEO.
► K. He, working on AFF3CT (2018-2019), with A. Cassagne and O. Aumage. Now engineer at IHU Liryc.
► A. Cassagne, working on optimizing Error Correcting Codes (2015-2016), with B. Le Gal (IMS), C. Leroux (IMS) and O. Aumage. Now Ass. Professor at Paris Sorbonne University;
► J. Tombi A Mba, working on MAQAO for Arm (2014-2015), with O. Aumage. Now engineer senior software engineer at BePatient;
► T. Meunier, working on performance analysis for vectorization and data restructuring (2013), with O. Aumage;
Publications
Patent