Astera Labs announces memory acceleration to remove AI / ML bottlenecks in data center

Astera Labs today announces important advances in cleaning up performance bottlenecks in enterprise data centers caused by the massive data needs of AI and ML applications.

Time to coincide with Supercomputing21, a high-performance computing conference taking place this week, launches what it claims is the industry’s first memory accelerator platform based on the Compute Express Link (CXL) standard for interconnecting general CPU processors and various other data center devices.

The news is significant because clearing bottlenecks in data centers has become a holy grail for the major processors of processors. Their customers struggle with performance, bandwidth and latency issues as they put together different types of processors like CPUs, GPUs and AI accelerators needed to run powerful applications like AI.

By combining its existing Aries product (for PCIe retimers) with the recently announced Taurus (for smart cables) and Leo SoC (for CXL memory accelerators), Astera Labs says it could become the leading cloud connectivity provider and (more than) double its revenue annually to meet the 1 billion pipeline option. dollars, as it sees it, with a total estimated, total addressable market of 8 billion. $ in 2025, which will be driven by the growth of artificial intelligence.

The goal is to create a faster connection backbone that provides low-latency interconnects, shares resources, and remains efficient with difficult technologies such as cache. Astera Labs also says that its fully cloud-based approach provides significant benefits in design productivity and quality assurance.

Supply of data to accelerators

One of the persistent challenges in computing is ensuring that CPUs and other accelerators can be supplied with data. This has become a major problem given the explosive growth of AI, with model sizes doubling in as little as every three and a half months. In recent years, DRAM scaling has not kept pace with Moore’s law, which means that memory is becoming a more limiting and expensive factor than computers. The CXL protocol, based on standard PCIe infrastructure, is an alternative to the standard DIMM slot for DRAM. It can also be used to connect accelerators to the CPU.

Intel proposed the CXL standard in 2019, and its industrial application is targeted at coinciding with PCIe 5.0 in 2022. Compared to PCIe 5.0, the CXL adds more features such as cache coherence across CPUs and accelerators and also has a much lower latency. In the future, CXL 2.0 will add rack-level memory pooling, which will make disaggregated data centers possible.

Astera Labs already has some products used by cloud service providers, such as PCIe and CXL retimers, but aims to expand this portfolio with these new announcements.

Memory accelerator for CXL 2.0

Leo, which Astera calls the industry’s first memory accelerator platform for CXL 2.0, is designed to enable CXL 2.0 to gather and share resources (memory and storage) across multiple chips in a system – including CPU, GPU, FPGA and SmartNIC – and enable separate servers. Leo also offers built-in fleet management and diagnostic features for large-scale server deployments such as in the cloud or corporate.

“CXL is a game-changer for hyperscale data centers that enables memory expansion and pooling capabilities to support a new era of data-centered and composable computing infrastructure,” said Astera Labs CEO Jitendra Mohan. “We have developed Leo SoC [system on a chip] platform locked in with leading processor vendors, system OEMs and strategic cloud customers to release the next generation of memory interconnection solutions. “

CXL consists of three protocols: CXL.io, CXL.cache and CXL.memory. However, only the implementation of CXL.io is mandatory. For the use of artificial intelligence with a cache-coherent interconnection between memory, the CPU and accelerators such as GPUs and NPUs (neural processing units), the CXL.memory protocol is relevant. Although the latency of the CXL is higher than a standard DIMM slot, it resembles current (proprietary) inter-CPU protocols such as Intel’s Ultra Path Interconnect (UPI). Because one of the goals of CXL 2.0 is to enable resource pooling on the rack scale, latency will be similar to today’s solutions for inter-node connections. CXL.memory also supports both conventional DRAM and persistent memory, especially Intel’s Optane.

The Leo SoC memory accelerator platform places Astera to play a crucial role in supporting the industry in adopting CXL-based solutions for AI and ML. Because CXL is based on PCIe 5.0, Leo supports a bandwidth of 32 GT / s per second. course, with a maximum of 16 courses. The maximum capacity is 2TB.

“Astera Labs’ Leo CXL Memory Accelerator Platform is a key facilitator of Intel’s ecosystem to implement shared memory space between hosts and connected devices,” said Jim Pappas, Director of Technology Initiatives at Intel.

“Solutions such as Astera Labs’ Leo Memory Accelerator Platform are key to enabling closer coupling and coherence between processors and accelerators, specifically for memory expansion and pooling capabilities,” said Michael Hall, AMD’s Director of Customer Compatibility.

Inside the CXL

By digging a little deeper into CXL, the Intel-proposed standard was the last for a cache-coherent pairing that was announced. For example, Arm was already promoting its CCIX standard, and various other vendors were working on a similar solution in the Gen-Z consortium. But with the absence of Intel – still the dominant provider in the data center – in these initiatives, they gained little traction. So when Intel proposed the CXL as an open interconnection standard based on the PCIe 5.0 infrastructure, the industry quickly switched to backing the CXL initiative, as Intel promised support in their upcoming Sapphire Rapids Xeon Scalable processors.

Within six months of the CXL announcement, Arm announced that it would also move away from its own CCIX in favor of CXL. Earlier this month, the Gen-Z consortium announced that it had signed a letter of intent (following a previous memorandum of understanding) to transfer the Gen-Z specifications and assets to the CXL consortium, making CXL the “only industry standard” going forward.

Other vendors have already announced support. In 2021, Samsung and Micron each announced that they would bring DRAM based CXL connectivity to market. In November, AMD announced that they would start supporting CXL 1.1 in 2022 with its Epyc Genoa processors.

Outside CXL

Astera also announced the Taurus SCM, which relates to smart cable modules (SCM) for Ethernet. These “smart cables” serve to maintain signal integrity, as the bandwidth doubles in 200G, 400G and 800G Ethernet (which is starting to replace 100GbE) in 3m or longer copper cables, and they support latencies up to 6x lower than the specification. Other smart features include security, cable breakdown monitoring and self-testing. The cables support up to 100G-per-lane serializer-deserializer (SerDes).

Astera Labs is an Intel Capital portfolio company. The startup is collaborating with chip providers such as AMD, Arm, Nvidia and Intel’s Habana Labs, which have also supported the CXL standard. In September, the company announced a Series C $ 50 million investment for a $ 950 million valuation.

VentureBeat

VentureBeat’s mission is to be a digital marketplace for tech makers to learn about transformative technology and trade. Our site provides essential information on data technologies and strategies to guide you as you lead your organizations. We invite you to join our community to access:

  • updated information on topics of interest to you
  • our newsletters
  • gated thoughtful content and reduced access to our valued events, such as Transformation 2021: Learn more
  • networking features and more

sign up

Leave a Comment