Invented by Lortie; Chase B., Meyer; David J., Dandu; Bharat C.

This article will help you understand a new patent application focused on how electronic devices interpret user touch in 3D extended reality (XR) spaces. We’ll look at why this technology matters, how it builds on what came before, and what makes this invention unique. Whether you are a developer, investor, or just curious about XR, this is for you.

Background and Market Context

Touch is an essential part of how we use phones, tablets, and computers. We tap, swipe, and drag on screens every day. But in XR — which includes both augmented and virtual reality — things are different. There is no physical screen to touch. Instead, people interact with floating menus, virtual buttons, and 3D objects that exist only in their headset or glasses.

As XR devices become more common, from headsets for gaming to smart glasses for work, the way people interact with these digital worlds needs to feel natural and easy. If you reach for a button in XR, it should respond just like a button in the real world. But this goal is hard to reach. The device must track where your hand is, how fast it moves, and decide if you really meant to “touch” that virtual object or not.

The market is hungry for XR systems that are fast, accurate, and reliable. Big tech companies are investing billions into XR, and startups are racing to invent better ways for users to interact with virtual content. The stakes are high. If you can make XR feel as smooth and intuitive as using a smartphone, you can win over millions of users.

Current XR devices often struggle with delays, missed touches, or false triggers. Sometimes, the system is too slow, and the user feels frustrated when their actions do not match what they see. Other times, the device may misinterpret a simple hand movement as a command, leading to mistakes. This gap between intention and response breaks the magic of XR and keeps many people from using it for work or play.

This patent steps into this market at the perfect time. It promises to make touch in XR much more like touch in the real world — fast, predictable, and natural. By solving this challenge, the invention could push XR into the mainstream and make devices more useful in homes, schools, and offices everywhere.

Scientific Rationale and Prior Art

To understand why this patent matters, we need to look at how XR touch has worked so far, and what problems have remained unsolved.

In early XR systems, user interaction often relied on hand-held controllers with buttons and triggers. These worked for games, but didn’t feel natural or direct. As cameras and sensors improved, some XR devices began to track hand and finger movements. The idea was simple: let people point, tap, and grab virtual objects just like they do in the real world.

But hand tracking is tricky. The device must know where your hand is in space, how it moves, and when it crosses a virtual surface — like a button or menu. Many systems tried to solve this by measuring the position of the user’s fingertip and comparing it to the location of the virtual object. If the fingertip was close enough, it was considered a “touch.”

This worked in theory, but in practice, it wasn’t always reliable. There are many reasons for this:

First, cameras and sensors are not perfect. Sometimes, the system thinks your finger is somewhere it is not. This leads to missed touches or false triggers.

Second, XR devices have some delay (latency) between what the user does and what the system detects. If the system waits until a finger is already at the surface, it may react too late, making the interface feel laggy.

Third, people move their hands in different ways. Some may tap quickly, others slowly. They may swipe, drag, or make gestures that are hard to classify. Simple rules based on distance alone can’t always figure out what the user intends.

Prior art attempted to fix these issues with better sensors, faster processors, or clever algorithms. Some systems used “fuzzy” boundaries — regions around a virtual surface where a touch might be detected even if the finger isn’t exactly on the surface. Others tried to predict hand movements or use machine learning to recognize gestures.

But these approaches still had problems. Too much prediction could lead to accidental touches. Too little could make the system feel slow. Most systems didn’t combine information about both position (where is the finger?) and velocity (how fast is it moving?) in a way that matched how people really move their hands.

Moreover, previous inventions often treated all gestures the same, without adjusting for whether the user was tapping, dragging, or swiping. They did not account for user intention in a fine-grained way. This made it hard to distinguish between, for example, a quick tap and a slow drag.

The scientific challenge, then, was to create a system that feels immediate and accurate. The system must know not just where the finger is, but where it is going, how fast, and what the user probably wants to do. It must predict the start and end of touches in a way that keeps up with the user, even with imperfect sensors and system delays.

This patent answers that challenge by mixing position, velocity, prediction, and gesture type into a smart decision-making process. It does not simply wait for a finger to hit a surface. Instead, it considers how the finger is moving, predicts what will happen next, and triggers touch events at just the right time — making XR feel much more like the real world.

Invention Description and Key Innovations

This invention describes a new way for XR devices to detect and predict touch interactions between a user and virtual elements in a 3D environment. Let’s explore how it works, step by step, and what makes it stand out from earlier solutions.

At the heart of the invention is the idea that touch in XR should be fast, accurate, and natural. To accomplish this, the system follows a series of steps:

First, the device — for example, a headset or smart glasses — creates a virtual environment that matches the real world, including the user’s hands and virtual surfaces like menus or buttons. Using cameras and sensors, the system tracks the position of the user’s hand, finger, or other body parts in 3D space.

Next, the system calculates two important things: where the user’s finger is, and how fast it is moving, especially in the direction perpendicular to the virtual surface (imagine moving your finger toward or away from a floating virtual button). These two pieces of information — position and velocity — are the key ingredients for predicting touch.

Instead of waiting until the finger actually reaches the surface, the system predicts when the finger will hit or leave the surface based on its current speed and direction. If the finger is moving toward the button quickly, the system can “guess” that a touch will happen in the next split-second. This prediction lets the system trigger a touch event early enough to feel immediate to the user, even if the sensors or software have a bit of delay.

Similarly, when the finger moves away from the surface, the system predicts when the touch should end. This helps avoid situations where the user thinks they have stopped touching a button, but the system is slow to react.

The system uses the idea of a “fuzzy” region or boundary — a zone near the surface where touch can be detected or predicted. This accounts for small errors in hand tracking and human movement. If the finger is inside this zone and moving toward or away from the surface at the right speed, the system can confidently predict a touch or release, even if the finger isn’t exactly on the virtual surface.

A key innovation is that the system adjusts its sensitivity based on the type of gesture. For example, if the user is tapping quickly, the system can be more sensitive and predict touches further in advance, making the interface feel snappy. If the user is dragging or swiping, it can be less sensitive, helping prevent accidental breaks in the interaction.

This dynamic adjustment uses a state machine that looks at the movement pattern — is the finger tapping, swiping, or dragging? — and sets the prediction threshold and sensitivity accordingly. This approach allows the system to handle fast taps, slow drags, and everything in between with the right balance of speed and accuracy.

Another clever part of the invention is that it can decide if a touch is intentional or not. For example, if the user’s hand is just passing by a button and not really aiming for it, the system can avoid triggering a touch by looking at the path and speed of the hand. Or if the user releases a pinch gesture near a virtual surface, the system can tell that this was not meant as a touch and ignore it.

The invention can work with different parts of the hand — not just fingertips, but also the palm or even other objects the user is holding. It is flexible enough to support many kinds of gestures and user actions, making it suitable for a wide range of XR applications.

The patent also describes practical ways to implement the system. It can run on head-mounted devices, mobile devices, or servers. It uses standard sensors like cameras, depth sensors, and motion sensors, making it accessible to many hardware platforms.

Let’s put this into a simple example. Imagine you are wearing XR glasses and see a floating menu in front of you. You reach out with your finger to tap a button. As your finger moves toward the button, the system tracks both its position and how fast it is moving. When your finger enters the fuzzy zone in front of the button and is moving fast enough toward it, the system predicts you will touch the button in the next fraction of a second. It triggers the button’s response right as your finger arrives, making the experience feel instant. If you pull your finger back, the system predicts when your finger will leave the button and ends the touch at just the right moment.

Because the system can adjust for different gestures and user speeds, it works well for all kinds of users — fast or slow, careful or clumsy. It also helps reduce mistakes, like accidental touches when your hand just brushes past a button.

Finally, the patent includes ways to keep user data private and secure. It lets users choose what information to share and how. The system can work with anonymous data and uses strong protections to keep personal information safe.

In summary, the key innovations of this patent are:

– Predicting touch events based on both position and speed, not just position alone.
– Using a fuzzy boundary to make touch detection more forgiving and natural.
– Dynamically adjusting sensitivity based on gesture type.
– Distinguishing between intentional and unintentional touches.
– Working with standard sensors and supporting user privacy.

These advances make XR interaction smoother, faster, and closer to real-life touch, opening new possibilities for apps, games, and productivity tools in 3D spaces.

Conclusion

This patent marks a big step forward for how we interact with virtual worlds. By focusing on the way people really move and act, and by predicting their intentions, it makes XR touch feel as quick and natural as touching a real object. The invention tackles long-standing problems with delay, missed touches, and false triggers, making it possible for more people to enjoy XR in their everyday lives.

For developers and device makers, this technology offers a path to more responsive and reliable XR products. For users, it means less frustration and more fun. As XR keeps growing, inventions like this will shape the future of how we live, work, and play in digital spaces. This patent could be the foundation for the next generation of XR experiences, where the line between virtual and real touch finally disappears.

Click here https://ppubs.uspto.gov/pubwebapp/ and search 20250216951.