A Universal Compression Algorithm for Deep Neural Networks
In the past decade, deep neural networks (DNNs) have shown state-of-the-art performance on a wide range of complex machine learning tasks. Many of these results have been achieved while growing the size of DNNs, creating a demand for efficient compression and transmission of them. This talk will present DeepCABAC, a universal compression algorithm for DNNs that through its adaptive, context-based rate modeling, allows an optimal quantization and coding of neural network parameters. It compresses state-of-the-art DNNs up to 1.5% of their original size with no accuracy loss and has been selected as basic compression technology for the emerging MPEG-7 part 17 standard on DNN compression.
Speakers, Panelists and Moderators
WOJCIECH SAMEKHead of the Machine Learning GroupFraunhofer Heinrich Hertz InstituteWojciech Samek is head of the Department of Artificial Intelligence at Fraunhofer Heinrich Hertz Institute (HHI), Berlin, Germany. He studied computer science at Humboldt University of Berlin, Heriot-Watt University and University of Edinburgh from 2004 to 2010 and received the Dr. rer. nat. degree with distinction (summa cum laude) from the Technical University of Berlin in 2014. During his studies he was awarded scholarships from the German Academic Scholarship Foundation and the DFG Research Training Group GRK 1589/1, and was a visiting researcher at NASA Ames Research Center, Mountain View, USA. After his PhD he founded the Machine Learning Group at Fraunhofer HHI, which he has directed until 2020. Dr. Samek is PI at the Berlin Institute for the Foundation of Learning and Data (BIFOLD), member of the European Lab for Learning and Intelligent Systems (ELLIS) and associated faculty at the DFG graduate school BIOQIC. Furthermore, he is an editorial board member of Digital Signal Processing, PLoS ONE and IEEE TNNLS and an elected member of the IEEE MLSP Technical Committee. He is recipient of multiple best paper awards, including the 2020 Pattern Recognition Best Paper Award, is part of the MPEG-7 Part 17 standardization, and was organizer of special sessions, workshops and tutorials on topics such as explainable AI and federated learning at top-tier machine learning and signal processing conferences. He has co-authored more than 100 peer-reviewed journal and conference papers, predominantly in the areas deep learning, explainable AI, neural network compression, and federated learning.