Model Compression for Deep Neural Networks: A Theoretical Review and Practical Application to OpenAI's Whisper Model

Abstract

The rapid growth in the size and complexity of deep learning models poses challenges for deploying deep neural networks on resource-constrained devices. Driven by real-world business needs — such as reducing cloud infrastructure costs and enhancing data privacy in applications like Jabra’s call center solution — this thesis provides both a theoretical overview and an empirical evaluation of model compression techniques aimed at enabling efficient on-device inference. Filling a gap in current research on compression for transformer-based Automatic Speech Recognition models, we apply pruning and quantization to OpenAI’s Whisper-small model. Through systematic experimentation on LibriSpeech datasets, we introduce a custom architecture-aware pruning strategy that leverages component-level sensitivity analysis. We also evaluate multiple quantization techniques across different frameworks, analyzing their trade-offs in terms of performance and accuracy. Finally, our sequential compression pipeline achieves up to a 75% reduction in model size and an 87% inference speedup, with minimal degradation in recognition accuracy. These findings demonstrate the viability of deploying cost-efficient, low-latency, and privacy-preserving systems at scale.