Invented by Kocamaz; Mehmet K., Xu; Ke, Oh; Sangmin, Kwon; Junghyun

Welcome! Today, we’re going to talk about a new invention for machines that can drive or move on their own, like self-driving cars or smart robots. This invention helps these machines see and understand the world better, so they can move safely and smartly. We’ll break down the big ideas, explain how this invention is different from what came before, and show what makes it special.

Background and Market Context

Let’s begin with why this invention matters. More and more, cars and robots are learning to do things by themselves. You see cars that can park, drive on highways, or stop if someone walks in front of them. In factories, robots move boxes without help. Even delivery drones are starting to bring packages to our doors. All these machines need to know what is around them—cars, people, animals, signs, and many other things.

To “see” the world, these machines use sensors. A sensor is like a camera or a scanner. Some sensors take pictures, some use lasers to measure distance, and some use radio waves. These sensors help machines “look” in all directions and find out what’s happening nearby. But just seeing is not enough. The machine also needs to understand what it’s looking at and remember where everything is as it moves. If a car sees a bike in one moment, it needs to make sure it’s looking at the same bike in the next moment, even if the bike moves or something gets in the way.

This is where object tracking comes in. Object tracking means following things as they move. For cars and robots, tracking is very important. It helps them plan where to go, when to stop, and how to avoid hitting things. If a car can track people and other cars, it can make better choices and keep everyone safe.

The world is also changing fast. There are more cars on the roads, new delivery robots, self-driving trucks, and even drones flying above us. Companies want their machines to be smarter, safer, and able to work in many places: on city streets, in warehouses, on farms, and even in the air or under water.

But there are big problems to solve. Sometimes, sensors get confused. Maybe the weather is bad, or something blocks the view. Sometimes, when two people cross paths, the machine might mix them up. And with so many things around, the computer inside the machine has to work really hard to keep up.

So, companies and inventors are looking for new ways to help machines “see” and “think” better. They want their machines to keep track of many things at once, even if things move fast or get hidden for a moment. That’s why this invention is important. It gives machines a smarter way to track things, using both clever computer tricks and fast hardware.

Scientific Rationale and Prior Art

Now, let’s talk about how machines used to handle seeing and tracking before this new invention. For a long time, machines used simple rules and models. They tried to spot things using special “key points” in pictures—tiny dots or shapes that the computer could find again and again. Some common tricks were called SIFT and KLT. These methods looked for things like corners or edges and tried to connect the dots from one moment to the next.

With these old methods, the computer would draw a box around what it saw—like drawing a rectangle around a person or a car. Then it would try to follow that box as things moved. It used math to guess where the box should go next, based on where it was before. If the guess was close enough to what the sensor saw, the computer said, “Yes, that’s the same object.”

But these tricks don’t always work well. If the picture is blurry or something gets in the way, the computer can lose track. If two things look alike or cross paths, the computer can get confused and mix them up. Also, these old tricks often needed a person to set “rules” for each new place—like how many dots to look for, or how close is close enough. That means lots of work and not much flexibility.

Later, scientists started using deep neural networks (DNNs). These are big computer programs that can learn from thousands or millions of pictures. DNNs are good at spotting things in pictures. They can find cars, people, and pets, even if they look different or are in new places.

But even with DNNs, there was a problem. Most DNNs were trained to find things, not to follow them as they moved. After the DNN found an object, another program had to try to match the object from one moment to the next. This caused mistakes, especially if two things looked similar or crossed each other. Also, saving all the details for every object took a lot of computer power, so it was hard to track many things at once.

Some inventions tried to fix these problems by making better detectors or adding smarter matching tricks. But usually, the object detector and the tracker were separate. The features (the details about each object) were not picked to help with tracking, just with detection. This led to errors, like mixing up two people when they crossed paths. And when many things appeared at once, the computer slowed down.

Other inventors tried to use more sensors at once—like using a camera and a laser scanner together. But matching what one sensor sees to what another sees is tricky. The sensors might look at the same thing from different angles, or at slightly different times.

In short, the old ways had these main problems:

• They needed lots of hand-tuning and rules for each place or sensor.
• They got mixed up when things crossed, got hidden, or looked similar.
• They could not keep up when there were many things to track.
• They wasted computer power by making the detector and the tracker work separately.

This invention is different. It trains the computer to use the details it finds (features) in a way that helps with tracking, not just detection. It uses a smart math trick called “triplet loss” to teach the computer which details should be the same for the same object, and different for different objects. It also uses faster hardware and works with many kinds of sensors.

Invention Description and Key Innovations

Let’s dive into what makes this invention special. The heart of the invention is a smart way for machines to keep track of objects, using learned details (feature vectors) that help tell things apart and follow them over time or across different sensors.

The machine (like a car or robot) in this invention has several parts:

• One or more CPUs—these are the brains that do most of the thinking.
• One or more GPUs—these are fast at handling lots of pictures and heavy math.
• One or more hardware accelerators—these help with special jobs, like running neural networks even faster.
• Multiple sensors—these look at the world in different ways: cameras for pictures, LIDAR for distance, RADAR for finding objects in the dark or fog, and more.

Here’s how the invention works:

1. The sensors gather data from outside. This could be a picture, a 3D scan, or any kind of map of what’s around the machine.

2. The data is sent to a neural network. But this isn’t just any neural network—it’s trained in a special way. The network learns to make a set of numbers (a feature vector) for each object it sees. These numbers are like a “fingerprint” for the object.

3. To train the network, the inventors use a clever method called “triplet loss.” The idea is to show the computer three examples at a time:

• An “anchor” (a piece of data about an object at one time or from one sensor),
• A “positive” (the same object, but at a different time or from a different sensor),
• A “negative” (a different object).

The network learns to make the anchor and positive look very similar (their numbers are close together), but to keep the negative far apart. This way, the computer learns what makes the same object look similar, even if the view changes, and what makes different objects look different.

4. Once trained, the network can look at new data and quickly decide if two things are the same object, even if they are seen at different times or by different sensors. It does this by checking if their feature vectors are close (for example, if the distance between the numbers is less than a certain threshold).

5. The computer then uses this information to plan what to do next: steer, stop, speed up, or avoid something. It can also follow objects as they move, even if they get hidden for a moment or cross paths with others.

What’s new and special here?

– The invention lets the neural network learn features that are made for tracking, not just for finding objects. This means it’s better at following things that move, change shape, or get partly hidden.

– It works with many types of sensors, so it can handle tough situations like bad weather, crowds, or busy streets. It can even match objects seen by different sensors at the same time.

– The feature vectors are simple, often just a row of numbers, so the computer can work faster and use less memory. This is important because self-driving cars and robots can’t waste time—they need to make decisions quickly.

– The invention can be used in many different machines, not just cars. It works for drones, delivery robots, boats, construction machines, and more.

– It’s ready for real-world use. The invention is designed to fit into existing systems with little extra work. It can run in the car, in a robot, in a data center, or even in the cloud.

Key technical details:

– The network can use many kinds of math models, including deep neural networks, convolutional networks, and more.

– The feature vectors are often computed at the pixel level (tiny parts of the sensor data), but for tracking, they can be averaged over a whole object (like the pixels inside a box around a car).

– The system uses bounding shapes (like rectangles or curves) to help group the features for each object.

– During training, the system can use real data, synthetic data, or a mix. This means it can learn from real-world driving or from virtual tests.

– The system can use different types of triplets in training: easy, hard, and semi-hard. This helps the network get better at telling apart tricky cases, like when two people cross paths.

– The tracking method works for both “single sensor, many times” (following an object as it moves) and “many sensors, one time” (matching what one sensor sees with what another sees).

– The invention is ready for big systems, like fleets of cars or robots connected to the cloud. It can share data, learn from new experiences, and get updates from a data center.

Why does all this matter? Because machines need to see the world as clearly as possible. If they can track objects better, they can make smarter, safer choices. This helps make self-driving cars safer, delivery robots more reliable, and drones better at avoiding danger.

Actionable Takeaways for Industry

If you build self-driving cars, delivery robots, or any machine that moves by itself, this invention can help you:

– Track more objects at once, even in tough conditions.

– Reduce mistakes when things get hidden, cross paths, or look similar.

– Use less computer power, so your machines can run longer and react faster.

– Adapt to new places, sensors, or jobs with less time spent on setup.

– Fit the system into your current machines, whether you use cameras, LIDAR, RADAR, or other sensors.

To use this invention, you’ll need to set up the right hardware (CPUs, GPUs, accelerators) and collect training data from your sensors. You’ll train the neural network using the triplet method, then deploy it inside your machine. Once running, your machine will be able to spot and track things in real time, planning its moves with confidence.

Conclusion

The future of smart machines depends on seeing and understanding the world as humans do. This invention is a big step forward, teaching computers not just to spot objects, but to follow them, even in busy, changing places. By using clever training methods, fast hardware, and flexible design, this system makes autonomous machines safer and more reliable. Whether you build cars, drones, or factory robots, this invention can help your products see better, think faster, and act smarter—bringing us all closer to a world where machines work alongside us safely every day.

Click here https://ppubs.uspto.gov/pubwebapp/ and search 20250218195.