Invented by TURASOV; Ivan, KAMPHUIS; Johannes Marinus Hendrikus

Machine learning is everywhere. Phones, home gadgets, cameras, and even watches are filled with smart models. All these devices need to run on batteries, and batteries do not last forever. So, if you can guess how much energy a machine learning model will use before you run it, you can make much better choices. This article explains a new way to predict energy use for machine learning models—especially on small, low-power devices. We will look at why this matters, what science led to this, and what makes this new method special.

Background and Market Context

Almost everyone uses something powered by artificial intelligence (AI) every day. AI models are behind apps that recognize faces in photos, help with voice commands, or spot objects in security footage. These models are built using machine learning, a way for computers to “learn” from data without being told step-by-step what to do.

Big companies have big servers to run these models. But many smart devices—like watches, sensors, and little cameras—are not big or powerful. They are “edge devices.” Edge devices get their name because they sit at the “edge” of the network, close to where data is made. They collect data and run models right there, instead of sending everything up to a server or the cloud.

Edge devices are everywhere: in smart homes, in cars, in factories, and even in farms. Their jobs range from simple (like checking temperature) to complex (like spotting a person in a picture). The problem is, these devices have small batteries and limited memory. If a model uses too much energy, it drains the battery, and the device stops working too soon.

Imagine a smart doorbell that runs out of power in a day because the face detection model is too “hungry.” Or a farm sensor that cannot monitor crops because it needs to recharge all the time. No one wants that. So, people who build these devices want to know, before they start, how much energy a machine learning model will use. That way, they can pick the right model, or tune it, to make sure the device lasts as long as possible.

Until now, there has been no quick way to predict energy use for a model running on a specific device. The normal way is to put the model on the device, run it, and measure how much energy is used. This takes a lot of time and needs special tools. It is not practical for most people, especially when you want to test many models on many devices.

The ability to predict energy use without actually running the model would be a huge step. It would help engineers pick better models, make devices last longer, and save money. It also helps the environment by reducing waste. All in all, predicting energy use is a big deal for the next wave of smart, battery-powered gadgets.

Scientific Rationale and Prior Art

Let’s talk about how people have tried to solve this problem before, and why it is hard.

Machine learning models, especially neural networks like convolutional neural networks (CNNs), are made up of many tiny math steps. Each step uses the device’s processor and memory. In big computers, there are ways to track how many times memory is used, how many times certain instructions run—these are called “performance counters.” Some older ways to predict energy use tried to use these performance counters from the device itself.

One old method is to run the model on the device and use built-in counters to watch how often the processor does certain things, like reading from memory or doing math. After collecting this data, you can do math (like regression) to see how those actions match up with real energy use. But there are two big problems:

First, not all edge devices have these counters or make it easy to get the data out. Second, you still need to run the model on the device, collect the data, and measure the energy. This is slow and not possible if you do not have the device.

Some people tried to use simulations. This means you make a computer program that acts like the device, then run the model in the simulator. But these are slow and often do not match the real device exactly. Even small mistakes can lead to wrong guesses about energy use.

Others tried to “guess” using simple numbers, like counting the number of multiply-accumulate (MAC) operations in the model. MACs are basic math steps that make up most of the work in a neural network. But counting MACs alone is not enough. Two models with the same MAC count can use different amounts of memory, which changes how much energy they use.

Another problem: edge devices use special tricks to save energy, like special instructions (SIMD) that handle many numbers at once, or hardware for digital signal processing (DSP). The way these tricks work changes between devices and models. It is very hard to guess how they affect energy use just by looking at the model’s size or type.

So, most old methods were either too slow, too simple, or too hard to use. They did not help someone who wants a quick answer without running tests on every device.

The new method described in this patent application aims to solve these problems by connecting the dots between model properties (like MACs and number of parameters) and energy use, without running the model on the device. It uses smart math (regression) and insights about how memory and processor use in a model relates to energy on real hardware. This is a step forward from just counting MACs or running slow simulations.

Invention Description and Key Innovations

Now, let’s dig into what’s new and special about this invention.

The main goal is to predict, before running anything on a device, how much energy a machine learning model will use during inference (when it is making predictions, not being trained). This is aimed at “edge” devices, with small processors and low power.

Here’s how it works, step by step:

First, the system looks at the machine learning model itself. It checks basic properties:

• The number of multiply-accumulate (MAC) operations in the model (this tells you how much math needs to be done).
• The total number of output parameters of all the model’s layers (this tells you how much memory is used).
• The number of output parameters for any layer that is not a 1×1 convolution (this helps account for special memory use cases).

These three values—let’s call them P1 (MACs), P2 (parameters), and P3 (parameters not from 1×1 convolutions)—are collected just by looking at the model’s “blueprint.” No need to run it.

Next, the system uses these values to estimate how many times the device will need to access memory during inference. This is important because memory access (especially reading from or writing to memory) uses a lot of energy—sometimes more than the math itself.

The system breaks memory access into two types:

• Simple bus accesses (like loading or storing a single value at a time). This is called D1.
• SIMD bus accesses (special instructions that handle two or more values at once). This is called D2.

The patent describes a series of small math models (regressions) that use P1, P2, and P3 to estimate D1 and D2. For example, using the number of MACs and total parameters helps guess how many simple memory actions will happen. Adding in the number of parameters from non-1×1 convolutions helps guess SIMD actions.

After getting D1 and D2, another regression model uses these to predict the total energy used by the model when running inference on the target device. This way, the system connects what it knows about the model to what it knows about the device’s hardware, all without running any code on the device.

The process is smart because it does not need to run the model on the device or collect performance counters for every new model. Instead, once the regression models are trained (using a set of real data collected once for a device), they can be used again and again for new models.

This invention is also flexible. It works for different types of neural networks, especially convolutional neural networks (CNNs), which are popular in edge devices. It can also be extended to cover digital signal processing (DSP) steps that often come before or after the model runs, like cleaning up audio or sensor data.

What makes this especially useful is that it lets developers and engineers make quick choices. They can test different models, change model size, or pick new layers, and see right away how much energy will be used. This saves time, makes devices last longer, and helps bring smarter, more efficient products to market.

On a technical level, the invention is built to be clear and repeatable. The steps are:

1. Analyze the model to get P1, P2, and P3.
2. Use trained regression models to estimate D1 and D2.
3. Use another regression model to combine D1 and D2 into a final energy estimate.

This chain of steps is simple, fast, and does not need the actual device in hand. Once the regression models are built for a device, they can be reused for any model. This is a big leap over the old ways.

The patent also covers implementation as software, as part of a tool, or even embedded in a device. It works for models running on separate edge devices or on the same device doing the analysis. It also covers ways to store the models and run the analysis on different types of processors and memory setups.

Conclusion

Predicting energy use for machine learning models on edge devices is a big challenge, and it is only getting more important as AI moves into more places. The invention described here gives a practical, fast, and repeatable way to get good estimates of energy use, just by looking at the model’s blueprint. It connects what we know about model math and memory to real energy use, using smart regression models.

This means engineers and companies can make better devices, save time, and help batteries last longer. It also helps the environment and makes it easier to bring smart products to people everywhere. If you are building or choosing machine learning models for edge devices, this invention gives you the tools you need to make the best choices, right from the start.

Click here https://ppubs.uspto.gov/pubwebapp/ and search 20250363417.