Improvement of power consumption efficiency for advanced AI processing using deep learning　compression technology

OKI Electric Industry Co., Ltd.

Outline

OKI regards the contribution to reducing reduction of environmental loads as a very important theme in all R & D and technology development. We are promoting activities making it in center to improve the energy saving performance of ICT. For example, the energy-saving performance of AI by deep learning lightweight technology has been adopted as a project of the New Energy and Industrial Technology Development Organization (NEDO) on energy conservation.

Description

We have been studying deep learning model compression technology to improve energy efficiency for advanced AI processing at edge. In general, it is known that higher performance models tend to have higher energy costs, including power consumption. Therefore, the model compression technology is receiving a lot of attention because it can effectively reduce the computational cost while maintaining the performance of deep learning.

As one of the model compression technologies, we developed a new method that belongs to a type of channel pruning. It can remove redundant channels (groups of neurons) from the model by appending another network model, called the attention module, which can estimate the importance of channels. With this technology, the number of Multiply-accumulate (MAC) operations in the model has been reduced by approximately 80% while maintaining the accuracy degradation of the high-accuracy recognition model to about 1%.

There are two typical use cases for deep learning deployments: the first case is that a large amount of data is transferred to the server and processed, and the second case is that data is processed on the device with advanced computing power and only small data such as results is transferred to the server. Since the deep learning model has a large computational cost, the former has a problem with the communication capacity, while the latter also has a problem in terms of power consumption and processing speed. By using our model compression method, it is possible to implement high-performance AI on general-purpose edge devices, even if the computing device does not have enough computational capacity. In other words, this technology allows for significant reductions in power consumption and communication capacity.

We are conducting research of this technology in a project on energy conservation by the New Energy and Industrial Technology Development Organization (NEDO).

[Acknowledgement] This article is based on results obtained from a project commissioned by the New Energy and Industrial Technology Development Organization (NEDO).