Even though GPUs in data centers theoretically only need 700 watts to run large language models, in practice, they may require 1700 watts due to inefficient power delivery. Peng and his team at the startup PowerLattice say they solved this problem by miniaturizing and repackaging a high-voltage regulator.

The company claims its new chip reduces power consumption by up to 50% and doubles performance per watt by shrinking the voltage conversion process and moving it significantly closer to the processor.

The decline and shift of power transmission

Traditional systems power AI chips by converting alternating current (AC) from the grid to direct current (DC), and then converting the DC back to low-voltage (around 1 volt) DC usable by the GPU. Because of the voltage drop, the current must be increased to conserve energy.

This energy exchange occurs near the processor, but even at low voltage, the current needs to travel a considerable distance. High current traveling any distance is detrimental because the system loses energy as heat, the amount of which is proportional to the square of the current. "The closer to the processor, the shorter the distance the high current needs to travel, so we can reduce power loss," says Hanh-Phuc Le, who studies power electronics at UC San Diego and has no affiliation with PowerLattice.

Combied with AI

"Given the ever-increasing energy consumption of AI data centers, 'this has almost become a major obstacle to development today,'" said Mr. Zou of PowerLattice.

Mr. Zou believes he and his colleagues have found a way to avoid this enormous power loss. Instead of reducing the voltage a few centimeters away from the processor as was done previously, they found a way to reduce the voltage a few millimeters inside the processor package. PowerLattice designed miniature power supply chips that integrate inductors, voltage control circuitry, and software-programmable logic into an integrated circuit only twice the size of a pencil eraser. These chips are located beneath and connected to the processor package substrate.

One challenge faced by PowerLattice's researchers was how to shrink the size of inductors without compromising their performance. Inductors temporarily store energy and then release it smoothly, helping regulators maintain a stable output. The physical size of an inductor directly affects its energy storage capacity, so shrinking its size diminishes its effectiveness.

The startup solved this problem by using a special magnetic alloy to manufacture the inductors. Mr. Zou stated that this alloy "allows us to operate inductors efficiently at high frequencies." "Our operating frequency is a hundred times higher than traditional solutions." At higher operating frequencies, circuits can be designed to use inductors with lower inductance values, meaning the components themselves can use less material. The unique feature of this alloy is that it maintains better magnetism than comparable materials, even at such high frequencies.

Professor Zou stated that the resulting microchip is less than one-twentieth the size of current voltage regulators and only 100 micrometers thick, roughly the thickness of a human hair. This small size allows the microchip to be mounted as close to the processor as possible, freeing up valuable space for other components.

Professor Zou stated that even with its small size, this proprietary technology is "highly configurable and scalable." Customers can use multiple chipsets to achieve a more comprehensive solution, or fewer chipsets if their architecture doesn't require them. Professor Zou believes this is "one of the key advantages" of the PowerLattice voltage regulation solution.

The company claims that using chipset technology can reduce operators' power requirements by 50%, thereby effectively improving performance. However, Le believes this figure is overly optimistic. He stated that a 50% energy saving "might be achievable, but that would mean PowerLattice would have to be able to directly control the load, including the processor." He believes that this goal can only be truly achieved when the company can manage power in real time based on the processor's load—a technology known as dynamic voltage and frequency regulation—which PowerLattice currently lacks.

Facing competition

Currently, PowerLattice is undergoing reliability and verification testing and expects to release its first product to customers in two years. However, bringing these chips to market will not be easy, as PowerLattice faces some formidable competitors. For example, Intel is developing a fully integrated voltage regulator, partly aimed at solving the same problem.

Mr. Zou believes Intel is not a competitor because, aside from the difference in power supply methods, he doesn't believe Intel would provide technology to its competitors. "From a market positioning perspective, we are very different from Intel," Mr. Zou said.

Le stated that ten years ago, PowerLattice had no room to grow because processor manufacturers only guaranteed chip reliability if customers also purchased their power supply products. "For example, Qualcomm could sell its processor chips, but most customers also had to buy its proprietary Qualcomm power management chips, otherwise Qualcomm would say, 'We don't guarantee the reliable operation of the entire system.'"

However, there may be hope now. "There's a trend we call chipset implementation, which is heterogeneous integration," Le said. He indicated that customers are mixing and matching components from different companies to achieve better system optimization.

While established vendors like Intel and Qualcomm may still hold an advantage thanks to their large customer base, smaller companies—primarily startups—also require significant power to build processors and AI infrastructure. Le says these companies need to find power sources, and companies like PowerLattice can meet that need. "That's just how the market is," he says. “We see startups partnering with other startups to develop products that are actually comparable to, or even competing with, some of the larger companies."

Source: Compiled from IEEE

View more at EASELINK