Red Hat laid out its vision for the future of AI and open source software in early November when it announced the acquisition of industry-leading machine learning optimizing model Neural Magic. The move could have long-term ramifications for generative AI and model optimization.
What is Neural Magic?
Neural Magic’s machine learning model optimization runs on current-generation commodity hardware, including central processing units (CPUs) and graphic processing units (GPUs). By using off-the-shelf hardware instead of the tensor processing units (TPUs) typically used in machine learning, the company hopes to make large language models (LLMs) more accessible to everyone. To this extent, it offers a repository of pre-built templates for immediate deployment. For advanced users, Neural Magic also supports customized workflows via the ONNX or PyTorch models.
While the concept of AI optimization software isn’t exactly new, Neural Magic’s platform is available to users at no cost. It also integrates with numerous open source tools, which makes it ideal for the growing Red Hat AI portfolio. Red Hat is perhaps best known for its Red Hat Enterprise Linux distribution.
“At Neural Magic, we’ve assembled some of the industry’s top talent in AI performance engineering with a singular mission of building open, cross-platform, ultra-efficient LLM serving capabilities,” Neural Magic’s CEO Brian Stevens said in a recent interview.
Taking the Next Step in LLM Processing
As a company, Red Hat is already known for its contributions to the open source software movement. In recent years, however, the company has been making headlines around the world for its move toward AI. Not only does the acquisition of Neural Magic represent the latest step toward the embrace of generative AI, but it specifically gives Red Hat the ability to support LLM deployments and AI workloads for enterprises of any size and scale.
Since Red Hat already uses Virtual Large Language Model (vLLM) for inference in products like Red Hat Enterprise Linux AI and OpenShift AI as a port of its hybrid cloud AI solutions, the acquisition of Neural Magic was a natural next step for the company. The Red Hat team hopes to use the recent acquisition to increase its presence in the vLLM community and help drive changes, including easily scalable and highly efficient inference, among systems that are using commodity hardware.
Red Hat has been at the forefront of technology since the 1990s. The Neural Magic acquisition solidifies its position as an industry leader in generative AI integration and implementation.