Netflix co-founder backs $312M round for optical inference appliance maker Olix
Artificial intelligence hardware startup Olix Computing Ltd. today announced that it has raised $312 million in funding.
The Series C round included contributions from Arm Holding plc, Netflix Inc. co-founder Reed Hastings and several others. Olix is now valued at $3.3 billion, about triple what it was worth after its previous raise in February.
When a large language model receives a prompt, it turns the text into a collection of mathematical values called a KV cache. It then uses the KV cache to generate a prompt response. The latter phase of the inference workflow is known as the decode stage.
London-based Olix is developing a chip called the DX-1 that is specifically optimized for decode workloads. It plans to ship the DX-1 as part of a data center appliance known as the X-1. According to Olix, the chips inside the system will be linked together using an optical interconnect that transmits data as light.
The KV cache that an LLM uses during the decode phase of inference is often larger than the model. As a result, it can’t fit in the underlying chip’s internal SRAM memory. Graphics cards address the challenge by off the KV cache to off-chip HBM memory. HBM is slower than SRAM but can hold more data.
Existing decode-optimized processors such as Nvidia Corp.’s Groq 3 LPX take a different approach. They feature a significantly larger pool of on-chip SRAM memory than standard graphics cards. The SRAM pool is large enough to store KV caches on-chip, which removes the need for off-chip HBM memory.
Olix hinted in a blog post published today that its DX-1 chip takes a similar approach. According to the company, it doesn’t contain any HBM memory or the advanced packaging used to integrate HBM modules with graphics cards. Olix stated that the DX-1 “holds a model in fast on-chip memory, SRAM, for higher energy efficiency and lower latency.”
The company plans to ship the chip as part of a data center appliance called the X-1. Olix says that the processors in the system will be linked together by a slow and wide optical interconnect.
Historically, data center operators used copper wires to link together chips in racks. Optical interconnects provide better performance because light travels through glass faster than electrons through metal. Typically, an optical interconnect comprises a relatively small number of high-speed data channels that each shuffle information-carrying light beams between two chips.
The slow and wide design used by Olix works differently. Instead of a few high-speed channels, it features a large number of slower channels, hence the name. The speed difference between the two implementations stems from the way they encode data into light beams.
A traditional optical interconnect with a handful of high-speed channels encodes data into light using a technology called PAM4. The technology’s speed comes at the expense of reliability. In many cases, optical interconnects must use chips called digital processors to remove the errors that find their way into PAM4-encoded data.
Slow and wide interconnects use a less performant but more reliable encoding method called NRZ. It’s less prone to errors than PAM4, which removes the need for digital signal processors and thereby avoids the associated costs. The technology also helps mitigate hardware failures. When an interconnect contains a large number of channels, a localized malfunction in one of them has limited impact on the host sysem.
Olix says that its DX-1 chip enables LLMs with 100 billion parameters to process more than 10,000 tokens per second. Furthermore, customers can link together multiple X-1 racks into clusters to run models with 10 trillion parameters.
Olix hopes to start shipping its chips in the first half of 2027. The company will use its latest funding round to enhance the “wider custom silicon platform behind” the DX-1 and accelerate manufacturing efforts.
Photo: Unsplash
Support our mission to keep content open and free by engaging with theCUBE community. Join theCUBE’s Alumni Trust Network, where technology leaders connect, share intelligence and create opportunities.
15M+ viewers of theCUBE videos, powering conversations across AI, cloud, cybersecurity and more** 11.4k+ theCUBE alumni**— Connect with more than 11,400 tech and business leaders shaping the future through a unique trusted-based network.
About SiliconANGLE Media
theCUBE AIand theCUBE SuperStudios — with flagship locations in Silicon Valley and the New York Stock Exchange — SiliconANGLE Media operates at the intersection of media, technology and AI.
Founded by tech visionaries John Furrier and Dave Vellante, SiliconANGLE Media has built a dynamic ecosystem of industry-leading digital media brands that reach 15+ million elite tech professionals. Our new proprietary theCUBE AI Video Cloud is breaking ground in audience interaction, leveraging theCUBEai.com neural network to help technology companies make data-driven decisions and stay at the forefront of industry conversations.