Time-of-flight: what you need to know about these new means of computer vision

You may not realize this but you’ve probably come across technologies that operate on the time-of-flight (ToF) principle. They are used in medicine (for example for MRA scanning), automotive industry (for advanced driver-assistance systems), and even in some of the newest smartphone cameras! Is this your first time that you’re hearing about time-of-flight? Read on to learn what it is, how it works, and how you can benefit from it in your business.

What is time-of-flight (ToF)

The time-of-flight principle is based on measuring the time it takes for a wave to travel from a source (a time-of-flight sensor) to an object and back. Based on that data – as well as some knowledge of maths and physics (such as wave propagation) – you can establish the distance of that object from the source. Depending on the technology, different types of waves may be used with different results, though mostly light signal is used, either laser or LED.

Time-of-flight is one of several methods of capturing 3D images, alongside for example stereoscopic cameras (ones that have two separate lenses to imitate human vision and recreate depth perception) or structured light imaging (where a structured image, such as a grid, is projected onto an object and the shape and distance of that object is calculated based on the distortion to the grid).

As with anything, ToF has its benefits and drawbacks, compared to other imaging techniques. Here are some things to consider.

Working conditions

You will often hear that the advantage of the time-of-flight technology is that it works even when there is little to no light. And that is true – ToF is considered an “active” imaging technique, i.e. one that relies on projecting its own light. Because of that, it doesn’t need any external source of light to scan its surroundings like passive imaging techniques (such as stereo cameras) do.

However, there is another side to that coin. Because technologies based on time-of-flight rely on emitting and receiving back reflections of their own light, they may perform poorly in natural surroundings (e.g. when exposed to the sun). This is because waves emitted by other sources of light interfere with the light from the device.

Similarly, there are other factors that make it harder for ToF sensors to work properly, such as “shiny” surfaces (which reflect light in different directions) or corners (which create multiple reflections), etc.

Processing speed

The time of flight principle is based on pretty straightforward math and relatively simple algorithms, as opposed to stereoscopic cameras that require complex calibration and processing to generate the image. This is why devices based on ToF technologies have lower system requirements and, simply speaking, work faster on smaller processors. This factor comes into play especially in applications where processing speeds are of crucial importance, such as in autonomous vehicles or factory robots and production lines.Range

One of the most pronounced benefits of the time-of-flight technology is that it operates on a variety of ranges, anywhere from half a meter to five meters. Some ToF sensors even have a range of up to 10 meters! It needs to be remembered, though, that other technologies – such as structured light imaging – may only have the range of up to two or three meters, but they also tend to be much more accurate. Thus, while ToF may work on short ranges, it may not be the best for use-cases where not the range, but accuracy is the ultimate consideration.

Precision

So how accurate exactly is time-of-flight? Well, their accuracy is largely dependent on the distance from the object as it is generally estimated at 1% of that value (so if an object is 5 meters away, a ToF camera can achieve an accuracy of about 5 cm). This places them somewhere in between stereo cameras (with precision of about 5-10% of the distance) and structured light sensors – the most accurate technology that can have an accuracy of as little as 1 mm.

Where do you apply time-of-flight principle?

As you can see, there are many factors that differentiate particular 3D imaging techniques and make them more suitable for certain kinds of deployments. Moreover, even within one deployment it is possible that more than one technology will be required – think autonomous vehicles that need long range imaging for solutions such as pedestrian detection, as well as precision for parking assistance, on top of being effective in all lighting conditions. But where are technologies based on time-of-flight most frequently used?

People counting

Time-of-flight technologies really came to the fore in 2020 when social distancing and occupancy management became a necessity for many businesses, for example in retail. It is then when devices based on time-of-flight technologies started to be used in people counting software to provide more precise and reliable data than simplified sensors.

The biggest benefit of using time-of-flight technologies for people counting is that they ensure high count accuracy (up to 98%), which is particularly important when occupancy is limited by legal regulations. At the same time, the sensors provide a non-participating, yet also non-intrusive means of people counting. This means that visitors don’t have to do anything (such as scanning a QR code) except enter the venue to be counted. At the same time, they maintain their privacy, as their real image isn’t captured, which cannot be said for commonly used CCTV-based software.

Robotics

Time-of-flight sensors can also be used in logistics and assembly line automation. Autonomous robots used there require computer vision and thanks to 3D imaging and high processing speeds, ToF appears to be a perfect solution for that. With ToF sensors, for example, robotic arms can sort objects down an assembly line and autonomous forklifts can pick up and carry cargo.

But not just the smart factory can benefit from time-of-flight technologies. Individual consumers can, too. 3D ToF cameras are used in autonomous drones and some smart appliances such as robotic vacuum cleaners, where they ensure your device can safely avoid all obstacles.

Smartphones

Finally, it is quite possible that your smartphone already has a ToF camera. Whatever for? Mostly, it is used to enhance the quality of photos you take. Since it can sense depth, the phone can use it to better understand what constitutes a background in the photo. Because it works well in dim light, it can also be used to take better photos where there is no optimal lighting. Technologies based on time-of-flight are also used for motion detection and gesture recognition, making it possible to use your phone without actually touching it. There are even apps that use ToF sensors in your phone for distance measuring! As the technology will be applied to more and more smartphone models, you can be sure to see even more clever applications pop up.

So do I need it?

As you can see, time-of-flight technologies have a range of applications for business, industry and individual users. They can prove to be a breakthrough for certain developments in areas such as robotics, but the recent worldwide events show that they carry huge potential and are extremely versatile whatever their use. It is well worth it to learn more about this technology and keep an eye on how it develops. Who knows, maybe it’s just the solution your business needs.