Fujitsu Looks to Accelerate Deep Learning Workloads
The company has developed software that will speed up machine learning tasks that are spread out over multiple GPU-powered systems.Engineers at Fujitsu Laboratories have developed new software that can speed up deep learning projects run over multiple GPUs. According to Fujitsu Labs officials, tests have found that the software used with 16 and 64 GPUs are 14.7 to 27 times faster than using a single GPU to run deep learning workloads, with increases in learning speeds 46 percent (on 16 GPUs) to 71 percent (on 64 GPUs). This is important given the increasing popularity of deep learning, a subset of machine learning, which is foundational to the development of artificial intelligence (AI). Machine learning essentially comprises two parts, training (where neural networks are taught object identification and other tasks) and inference (where they use this training to recognize and process unknown inputs). The use of deep learning techniques to train neural networks has grown over the past several years, helping to drive significant advances in such work as image and speech recognition and increasing the accuracy over other technologies, according to Fujitsu Labs officials. In addition, deep learning requires massive amounts of data for machine training, and GPUs—with their ability to process huge amounts of data in parallel—are better suited than CPUs. A challenge has been finding efficient ways to run deep learning workloads across multiple GPUs in parallel, the officials said. Right now, the primary way it's done is to use multiple computers that are powered by GPUs, networked together and running in parallel. However, such arrangements are difficult to scale—the benefits of parallelization becomes increasingly more difficult to reach when the time it takes to share data between the computers grows, particularly when more than 10 systems are used in the network at the same time.
The software developed by Fujitsu is designed to overcome those limitations, the researchers said. They took the new parallelization technologies and applied them to the open-source Caffe framework for deep learning. The software enables users to reduce the time needed for R&D, which in turn will lead to improved learning models, they said.