Efficient data encoding for convolutional neural network application

H.-P. Trinh (1) , M. Duranton (1) , M. Paindavoine (2)

1 DACLE-LIST - Département d'Architectures, Conception et Logiciels Embarqués-LIST
2 LEAD - Laboratoire d'Etude de l'Apprentissage et du Développement [Dijon]

This article presents an approximate data encoding scheme called Significant Position Encoding (SPE). The encoding allows efficient implementation of the recall phase (forward propagation pass) of Convolutional Neural Networks (CNN)-a typical Feed-Forward Neural Network.This implementation uses only7bits data representation and achieves almost the same classification performance compared with the initial network: on MNIST handwriting recognition task, using this data encoding scheme losses only 0.03% in terms of recognition rate (99.27% vs. 99.3%). In terms of storage, we achieve a 12.5% gain compared with an 8 bits fixed-point implementation of the same CNN. Moreover, this data encoding allows efficient implementation of processing unit thanks to the simplicity of scalar product operation-the principal operation in a Feed-Forward Neural Network.

Keywords: Character recognition; Convolution; Digital storage; Feedforward neural networks; Neural networks; Canonical signed digits; Classification performance; Convolutional neural network; Data representations; Efficient implementation; Fixed-point implementation; Forward propagation; Handwriting recognition; Encoding (symbols)

Domains: Computer Science [cs]; Engineering Sciences [physics]

H.-P. Trinh, M. Duranton, M. Paindavoine. Efficient data encoding for convolutional neural network application. ACM Transactions on Architecture and Code Optimization, 2014, 11 (4), ⟨10.1145/2685394⟩. ⟨cea-01846867⟩