When you buy through affiliate links in our content, we may earn a commission at no extra cost to you. Learn 
Rudimentary infrared products and applications, recognizable by their characteristic red lights that respond to gestures, are a familiar sight to many.
Touchless household garbage cans that open on their own in response to hand gestures and autonomous robotic household vacuum cleaners that dodge walls and other barriers, or even recharge on their own, are examples of other familiar sights in many homes these days.
In the context of coronavirus and other debilitating viruses, touchless hand washing for infection control is noticeable in intensive care units in hospitals.
3D Sensing Matures
Humans see the world with visible light that diffuses. Machines see the world around them with structured invisible light, like infrared and lasers, that is linear and bounces from surfaces in ways measurable by analytical engines that use the measurements of the dimensions to reconstruct the 3D reality in their environment and the objects that populate it.
Inertial sensors, accelerometers (motion sensors), gyroscopes (direction), and magnetometers (height), aid the touchless movement of machines in the 3D world.
3D sensing technology is maturing, as is evident from the anticipation building around the expected launch of Apple's iPhone 11 Pro and iPhone 11 Pro Max later this year — a culmination of the process that began with the acquisition of Prime Sense in 2013.
Apple has a bigger plan for 3D imaging, augmented reality, and virtual reality, for its wearable devices that will emerge by 2023. Depth sensing with Time-of-Flight is a cornerstone of Apple's plan for 3D sensing — extending its range, and achieves a higher level of precision.
Infrared is also able to see the world unaffected by the lighting conditions in the external environment.
Perceptions of Machines in Motion
3D sensing has a far-reaching impact on moving objects like robots, drones, and connected cars that can navigate with greater ease in fluid environments making frequent adjustments autonomously as they encounter obstacles.
Their ability to maneuver comes from an array of sensors assembled in miniaturized packs.
Occipital's structure sensor, for example, has 3D depth-sensing capabilities — two infrared cameras measure dimensions. It gets its broad field of view from a very wide-angle lens. IMU sensors determine objects' position in a mapped space around them and aid navigation indoors.
Robotics
Misty Robotics mimics the human eye's wide angles and depths sensing with a fisheye camera. It is a prominent user of Occipital's structure sensor that enables the robot to navigate deftly inside offices and homes.
It has plans to graduate to navigation in the wild or out in the fields where it can expect much higher traffic from many more moving objects.
"There is a lot that goes into making a robot robust for outside 'wild' environments, which could increase the price beyond an affordable threshold in the near-term future. We would instead have companies and developers begin working with a robot quickly," said Jeremy Dillingham, head of Customer Success and Product Management.
To be sure, small companies have used Misty Robotics for data collection, such as measurements of temperature, and sounds, to feed them to developers who can use them to write applications.
Misty Robotics has strategic partnerships with Microsoft, Google, Twilio, and the developer community to build applications with its robots. "Our partnership with Microsoft's OS is the key to enable all of the real-time functions of the robot," said Dillingham.
"We also use their vast array of APIs to add functionality like emotion detection for the care of the elderly. It helps to keep them mentally healthy by responding to their mood swings," Dillingham explained.
"We have added layers which make it easier for developers to access the hardware functionality of the robot simply with help from REST APIs native to Javascript and C# without prior knowledge of hardware," said Diingham.
The partnership with Google has yielded a concierge service using its dialog flow. The robot acts as a concierge, either independently or as an aide, who responds to questions by guests and entertains them with cute gestures and speech.
The partnership with Twilio creates fun communication services to engage audiences.
"We've taken the Misty robot to conferences where attendees can use a phone number to drive Misty. The robot, for example, can take pictures and text back to the senders' phones or read the messages sent to it," said Diingham.
Drones
Drone companies have also used the Occipital structural sensor for developing applications for remote inspection of infrastructure assets or industrial facilities without manual effort.
3D sensing and inertial sensors help them to navigate to designated corners of large bridges where GPS signals are inaccessible or are blocked.
Automodality is one such company in the domain — Aaron Singer, its CEO, spoke about how it leverages its 3D vision for bridge inspection.
"We use 3D structural sensors in combination with Lidar and other sensors aided by our patented Perceptive Navigation technology to navigate without the use of GPS or beacons of any kind. Perceptive navigation is a combination of artificial intelligence algorithms and machine perceptions to enable our drones to recognize features of targeted assets."
"We train the drone to recognize standard features and separate abnormal features that indicate deficiencies and anomalies," Singer explained.
Connected Cars
3D sensing is executed not only with infrared alone but in conjunction with related technologies like lasers, radar, and ultrasound, depending on the use cases.
Infrared is preferred for its efficiency in identifying objects, especially at night time, such as pedestrians crossing a street based on thermal characteristics.
Additionally, lasers come in the form of LIDAR (Light Detection and Ranging) or VCSEL (Vertical Cavity Surface Emitting Laser). LIDAR is for scanning distant views beyond cars' headlights. VCSEL has a narrower wavelength that filters out ambient light, and is immune to temperature levels, to focus on an object.
Andre Wong, Vice President, Product Line Management, Lumentum, spoke about the use of 3D sensing in connected cars.
"We use VCSEL for in-cabin applications such as observing driver behavior to determine whether they are paying attention or are distracted and LIDAR for long-distance monitoring of objects on the road. The narrow illumination bandwidth of the laser provides a view unaffected by the ambient conditions in the car or the external environment," said Wong.
Final Thoughts
Machine vision, whether for robots, drones, or connected cars, brings a human quality that has opened a whole new vista for application development.
The hardware devices with machine vision, together with emerging X-Reality applications — augmented, virtual, and mixed reality — will create new possibilities for application development.
They will interact with the environment, aided by a range of sensors and IoT devices, for autonomous execution of industrial and other activities in myriad ways that mimic a combination of human senses in as yet largely unforeseen ways.
