EaPU Could Cut AI Energy Use With Memristors

Large language models, image generators and smart assistants – AI is currently consuming vast amounts of energy.

A Chinese research team has now presented a surprisingly straightforward way forward.

The current AI boom has a downside: data centres are overheating, and electricity meters are racing too. Training large models in particular sends energy use and costs soaring. Researchers in China have introduced a method that could dramatically cut the energy requirements of neural networks without sacrificing accuracy.

Why modern AI consumes so much electricity

Whether it is ChatGPT, an image generator or an in-car assistance system, enormous computing clusters packed with specialised chips operate behind the scenes. Training deep neural networks especially requires millions or billions of matrix operations. Operators mainly rely on GPUs, or graphics processing units, for this work.

This architecture has two major sources of energy consumption:

• the computational work carried out by processors and GPUs
• the continual transfer of data back and forth between memory and computing units

The new approach focuses precisely on the latter issue: performing calculations directly in memory, using an entirely different hardware foundation.

Memristors: components that retain a memory

The key term is the memristor. Put simply, it is a resistor with memory. Its electrical resistance depends on how the component was previously “programmed”, and it retains that state.

This makes memristors well suited to physically representing the weights of a neural network. Rather than storing numbers in conventional memory and constantly sending them to processors, calculations take place exactly where the weights are stored. In theory, this can save huge amounts of energy.

In real-world use, however, there is a problem: memristors are imperfect. They produce noise, vary in behaviour and do not respond with complete precision. Even minor errors can disrupt AI models.

The central challenge is to live with the hardware’s imprecision rather than fighting it with extremely costly corrections.

EaPU training method: allowing for errors instead of fighting them

This is where the technique presented by the team at China’s Zhejiang Lab comes in. It is called “error-aware probabilistic update”, or EaPU for short. Its underlying idea is surprisingly human: small errors are acceptable, provided they remain within a tolerance range.

How EaPU works in principle

With conventional training, a neural network adjusts a large proportion of its weights after every step. On memristor hardware, that would consume a great deal of energy, because rewriting the components requires substantial power.

EaPU reverses that process:

• The system accepts minor deviations as long as they stay below a defined threshold.
• A corresponding weight in the memristor is rewritten only when a change is genuinely significant.
• In each training round, the network updates less than 0.1 per cent of all parameters.

As a result, the largest energy expense in training is almost entirely eliminated: the constant rewriting of countless memory cells.

Energy demand falls sharply and hardware lasts far longer

Compared with previous memristor methods, the researchers report several striking effects:

• Up to 50 times lower energy consumption when training on memristor hardware.
• Up to a 1,000-fold increase in lifespan, as the components are written far less often.
• Around 60 per cent higher accuracy than older memristor-based training methods.

The comparison with conventional GPU hardware is even more notable. According to the team, EaPU requires around six orders of magnitude less energy for the same task – in simpler terms, roughly one million times less electricity than a comparable GPU setup.

Energy demand that falls by a factor of 106 compared with GPU systems would fundamentally change the operation of today’s AI models.

Initial tests: removing image noise and improving resolution

The researchers have already tested EaPU in practice on a memristor array with a feature size of 180 nanometres. They trained neural networks for two typical image-processing tasks:

• removing noise from images
• artificially upscaling images (super-resolution)

The results were at a similar level to established methods running on conventional hardware, but with much lower energy use. This suggests that the concept is not merely effective on paper in a laboratory, but also makes sense for real applications.

Can large language models benefit too?

For now, the available hardware limits the experiments. The memristor chips used so far are relatively small. However, the team believes the approach should also be suitable in principle for large language models (LLMs).

If that proves true, it would directly affect systems such as chatbots, code assistants and analytical tools, which currently require huge computing resources. The researchers expect to test EaPU with substantially larger networks in future.

Importantly, the method is not tied to memristors. Those involved hope it could also be transferred to other emerging memory technologies, including:

• ferroelectric transistors
• magnetoresistive RAM (MRAM)

This opens up an entire toolkit of hardware options for energy-efficient AI.

What this could mean for data centres and everyday life

If the approach can be scaled up, AI-focused data centres could look very different from those of today. Instead of halls full of GPUs, compact analogue chips that combine computing and memory could become possible. That would not only lower electricity bills, but also reduce cooling requirements and the carbon footprint.

Potential practical effects include:

• considerably cheaper training cycles for businesses
• smaller specialised AI chips for smartphones, cars and industrial facilities
• more computing power directly at the network edge, without constantly sending data to the cloud

In everyday life, smart functions could then reach areas where power consumption is currently a deal-breaker, including battery-powered sensors, wearables and autonomous robots.

What the technical terms mean

Anyone without a deep background in hardware can easily stumble over some of the terminology. Two concepts are particularly important:

• Analogue in-memory computing: computational operations take place directly within the memory component, using analogue currents and voltages rather than purely digital ones and zeroes.
• Noise tolerance: an AI model is trained to cope with imprecise or noisy signals without its predictions becoming unstable.

Neural networks are already inherently somewhat robust to disruption. EaPU makes consistent use of this latitude instead of insisting on maximum mathematical precision, which is often unnecessary in practice.

Risks, open questions and the road ahead

The new technology still has unresolved issues. For example, it remains unclear how the approach will perform with extremely large models containing billions of parameters. The real cost of manufacturing dependable memristor chips at industrial scale is also not yet foreseeable.

There are further risks in critical applications such as medicine or autonomous driving. Those responsible must examine very carefully how much hardware inaccuracy can be permitted alongside EaPU without exceeding safety limits.

Despite these question marks, the work sends a clear message: the trend towards ever-larger AI models does not necessarily have to end in an energy dead end. When hardware and training methods are developed together, the power appetite of new generations of AI systems can be noticeably reduced – perhaps one day even to the point where a powerful model fits on a single energy-efficient chip.