Untether AI nabs $125M for AI acceleration chips

Untether AI, a startup developing custom-built chips for AI inferencing workloads, today announced that it raised $125 million from Tracker Capital Management and Intel Capital. The round, which was oversubscribed and had participation from Canada Pension Plan Investment Board and Radical Ventures, will be used to support the expansion of Untether’s customer engagements, CEO Arun Iyengar says.

The increase in the use of AI — along with AI’s hardware requirements — poses a challenge for traditional datacenter compute architectures. Untether is among the companies proposing at-memory or near-memory computation as a solution. Essentially, this type of hardware builds memory and logic into an integrated circuit package. In a “2.5D” near-memory compute architecture, processor dies are stacked atop an interposer that links the components and the board, incorporating high-speed memory to bolster chip bandwidth.

Founded in 2018 by CTO Martin Snelgrove, Darrick Wiebe, and Raymond Chik, Untether says it continues to make progress mass-producing a chip — RunA1200 — that combines efficiency with robustness. Snelgrove and Wiebe claim that data in their architecture moves up to 1,000 times faster than is typical. That would be a boon for machine learning computation, where data sets are frequently dozens or hundreds of gigabytes in size. Untether says its RunA1200 chips can be used in a range of industries and applications including banking and financial services, natural language processing, autonomous vehicles, smart city and retail, and other applications that require high-throughput and low-latency AI acceleration.

High-speed architecture

Each RunA1200 chip contains a RISC processor and 511 memory banks, with the banks comprising 385KB of SRAM and a 2D array of 512 processing elements (PE). There are 261,632 PEs in total per RunA1200 chip with 200MB of memory, and the chip runs at 502 trillion operations per second (TOPS).

Untether’s first product is the TsunAImi, a PCIe card containing four runA1200s. App-specific AI processors are spread throughout a memory array in the RunA1200s, enabling the TsunAImi to delivers over 80,000 frames per second on the popular ResNet-50 computer vision benchmark. That’s three times the throughput of its nearest competitor and outperforms a single Nvidia A100 GPU at about the same power rating (400W).

Untether is shipping TsunAImi card samples and aims for general availability this summer.

“Untether AI has a scalable architecture that provides a revolutionary approach to AI inference acceleration. Its industry-leading power efficiency can deliver the compute density and flexibility required for current and future AI workloads in the cloud, for edge computing, and embedded devices,” Tracker Capital senior advisor Shaygan Kheradpir said in a statement.

There’s no shortage of adjacent startup rivals in a chip segment market that’s anticipated to reach $91.18 billion by 2025. California-based Mythic has raised $85.2 million to develop custom in-memory compute architecture. Hailo, a startup developing hardware to speed up AI inferencing at the edge, in March 2020 nabbed $60 million in venture capital. Graphcore, a Bristol, U.K.-based startup creating chips and systems to accelerate AI workloads, has a war chest in the hundreds of millions of dollars. SambaNova has raised over $1 billion to commercialize its AI acceleration hardware. And Baidu’s growing AI chip unit was recently valued at $2 billion after funding.

With the latest round of funding, Toronto-based Untether’s total raised stands at $152 million.