Disentangled Loss for Low-Bit Quantization-Aware Training - Archive ouverte HAL Access content directly
Conference Papers Year : 2022

Disentangled Loss for Low-Bit Quantization-Aware Training

(1) , (1) , (1) , (2)
1
2

Abstract

Quantization-Aware Training (QAT) has recently showed a lot of potential for low-bit settings in the context of image classification. Approaches based on QAT are using the Cross Entropy Loss function which is the reference loss function in this domain. We investigate quantization-aware training with disentangled loss functions. We qualify a loss to disentangle as it encourages the network output space to be easily discriminated with linear functions. We introduce a new method, Disentangled Loss Quantization Aware Training, as our tool to empirically demonstrate that the quantization procedure benefits from those loss functions. Results show that the proposed method substantially reduces the loss in top-1 accuracy for low-bit quantization on CIFAR10, CIFAR100 and ImageNet. Our best result brings the top-1 Accuracy of a Resnet-18 from 63.1\% to 64.0\% with binary weights and 2-bit activations when trained on ImageNet.
Fichier principal
Vignette du fichier
93.pdf (1.44 Mo) Télécharger le fichier
Origin : Files produced by the author(s)

Dates and versions

cea-03776535 , version 1 (13-09-2022)

Identifiers

Cite

Thibault Allenet, David Briand, Olivier Bichler, Olivier Sentieys. Disentangled Loss for Low-Bit Quantization-Aware Training. CVPR 2022 - IEEE / CVF Computer Vision and Pattern Recognition Conference, Jun 2022, New Orleans, United States. pp.2788-2792, ⟨10.1109/CVPRW56347.2022.00315⟩. ⟨cea-03776535⟩
23 View
12 Download

Altmetric

Share

Gmail Facebook Twitter LinkedIn More