Red Hat Inc., the open-source software leader, announced its acquisition of Neural Magic Inc., a machine learning startup specializing in AI model optimization. This strategic acquisition will enhance Red Hat’s capabilities in supporting efficient, hardware-agnostic AI models optimized for the hybrid cloud.
As demand for AI-driven technology grows, organizations face challenges with managing the computational intensity of large language models (LLMs). These complex models require substantial power and resources, particularly for tasks like real-time natural language processing. Neural Magic, founded in 2018 by MIT research scientist Alex Matveev and Professor Nir Shavit, addresses these demands by providing technology to run optimized AI models on existing CPUs and GPUs—making it a cost-effective solution for many businesses. Learn more about AI’s role in optimizing business processes.
Why Neural Magic’s Technology Stands Out
Neural Magic’s groundbreaking approach allows AI models to run with comparable efficiency on standard hardware setups as they would on specialized AI chips, saving companies significant costs. With Neural Magic’s software, companies can fine-tune and optimize AI models to maximize hardware performance across diverse architectures, removing the need for exclusive AI infrastructure.
The startup is particularly known for its work on an open-source project called vLLM, designed for model deployment. This technology powers a library that accelerates AI model deployment in the cloud, regardless of whether a CPU or GPU is used. Red Hat highlighted vLLM as one of the primary reasons for its acquisition decision, as the project aligns with its vision of supporting LLMs on hybrid cloud environments. Additionally, Neural Magic’s model compressor significantly improves inference speed, reducing the time needed to deliver results.
Red Hat’s Hybrid Cloud and Open-Source AI Strategy
Red Hat’s acquisition of Neural Magic aligns with its broader strategy to enhance hybrid cloud solutions with scalable AI capabilities. The company recently introduced Red Hat Enterprise Linux AI. This offering integrates the Granite family of LLMs developed by IBM Corp. with Red Hat’s own InstructLab, creating open-source, server-deployable AI models optimized for the cloud. With Neural Magic’s technology, Red Hat will be better equipped to support organizations in deploying high-performance AI solutions on a range of hardware options. For a deeper dive into AI’s evolving role in cloud technology, check out Times of Tech.
Matt Hicks, CEO of Red Hat, emphasized the acquisition’s alignment with Red Hat’s open-source approach: “AI workloads need to run wherever customer data lives across the hybrid cloud. This makes flexible, standardized, and open platforms and tools a necessity.” Neural Magic’s innovations in vLLM, paired with its high-performance LLM capabilities, strengthen Red Hat’s open-source AI framework, which supports scalability across public, private, and hybrid cloud systems.
Neural Magic’s Contributions to Open-Source AI
Neural Magic has long supported community collaboration in AI, assembling a team of industry experts in AI performance engineering to create open, cross-platform AI tools. “Open source has proven time and again to drive innovation through the power of community collaboration,” stated Neural Magic’s CEO Brian Stevens. Neural Magic’s contributions extend to pre-built and pre-optimized open-source LLMs, which are ready for deployment through vLLM, enabling rapid, high-performance model deployments at scale.
The Future of Open-Source AI Innovation
As the AI sector expands, Red Hat’s acquisition of Neural Magic represents a significant move toward making efficient, optimized AI solutions more accessible. By combining Red Hat’s infrastructure with Neural Magic’s optimized model technology, businesses can leverage the full potential of AI, regardless of their hardware resources. This acquisition not only underscores Red Hat’s commitment to open-source innovation but also enhances its hybrid cloud portfolio, making it easier for companies to adopt scalable, high-performance AI.
To explore the latest in open-source AI innovations and their applications across industries, follow Times of Tech’s AI insights section.
