Samsung, more widely known for making television monitors, smartphones and other popular consumer devices, also is a world leader in producing computer memory. The North Korean IT giant announced that it has developed the industry’s first high-bandwidth memory (HBM) chip that’s integrated with artificial intelligence processing power—the HBM-PIM.
Like Intel, AMD, NVIDIA and others are baking security, networking and other functionality into processors, Samsung is doing the same, only with AI. The new processing-in-memory (PIM) architecture brings real-time AI computing capabilities inside high-performance memory so as to accelerate large-scale processing in data centers, high performance computing (HPC) systems and AI-enabled mobile applications.
The pioneering HBM-PIM is the industry’s first programmable PIM solution tailored for diverse AI-driven workloads such as HPC, training and inference, Samsung said. The company plans to build upon this by further collaborating with AI solution providers for even more advanced PIM-powered applications, the company said.
The HBM-PIM design has demonstrated “impressive performance and power gains on important classes of AI applications,” Rick Stevens of Argonne Labs said in a media advisory.
Most of today’s computing systems are based on the von Neumann architecture, which uses separate processor and memory units to carry out millions of intricate data processing tasks. This sequential processing approach requires data to constantly move back and forth, resulting in a system-slowing bottleneck especially when handling ever-increasing volumes of data.
Instead, the HBM-PIM brings processing power directly to where the data is stored by placing a DRAM-optimized AI engine inside each memory bank — a storage sub-unit — enabling parallel processing and minimizing data movement. When applied to Samsung‘s existing HBM2 Aquabolt solution, the new architecture is able to deliver more than twice the system performance while reducing energy consumption by more than 70%, the company claimed. The HBM-PIM also does not require any hardware or software changes, allowing faster integration into existing systems, Samsung said.
Samsung’s paper on the HBM-PIM was selected for presentation at the renowned International Solid-State Circuits Virtual Conference (ISSCC), which ended Feb. 22. Samsung’s HBM-PIM is now being tested inside AI accelerators by leading AI solution partners, with all validations expected to be completed within the first half of this year, the company said.