The Eyes of the IoT

Worldwide, hours of digital video are generated every second, and even more data from IoT sensors will be added with the intention to improve situational awareness. In less than four years, about 50% of the streams feeding into video management systems will not come from fixed cameras by themselves, but will be streams from any type of sensor, fixed and mobile, visual or non-visual.

The question is, how we will manage this data and information in the most intelligent, smart and secure way? I believe that intelligent video will become a vital tool for analyzing and categorizing IoT-generated data. Humans can’t do it alone; the amount of data is simply becoming too overwhelming. On the other hand, it’s not a task that we can rely on intelligent learning technologies to handle singlehandedly either. The solution is to combine machine intelligence with human judgement.

Intelligent learning technologies are now bringing video content analysis far beyond the capabilities of legacy rule-based video analytics systems. Traditionally, video analytics have been rule-based with a human programmer setting fixed parameters for each situation that the system must recognize, evaluating a few pre-defined situations. In comparison, AI-based intelligent video content analysis technology can learn directly from the video about objects, their relations to each other and normal behavior. This makes it possible to intelligently identify objects and classify situations. The system will be able to understand what’s normal behavior and alert the operator to unusual activities, leading to predictive systems in the future.

No doubt that the use of intelligent learning technologies will help us analyze the data generated from IoT network devices and bring opportunities that create unprecedented advantages for us all. Some people may worry whether this wealth of IoT devices, and their data, can be trusted. I believe they can, but we need visual confirmation of the data generated from IoT devices, so video will be a crucial tool to provide this. Having both device data and video is the foundation for combining machine intelligence with human judgement.

When the machine is uncertain, it can flag the situation for human intervention. With video to guide their reasoning, humans can either confirm the machine’s decision, or guide the process to a different outcome. For instance, when securing a safe landing of an airplane, humans must understand a vast amount of data generated from the IoT sensors. However, not all the data will be relevant, and it is here video can come to play a vital role – by using video as a visual confirmation of the data, humans can quickly separate relevant data from irrelevant data, enabling them to make the right decisions faster.

Here’s a good example: if a network sensor alarm goes off in an offshore windmill park, usually a maintenance crew would be sent out to fix the problem. However, this is costly and time-consuming. But with video providing visual confirmation of the problem, it is possible to identify what led the sensor to trigger the alarm, making it easier for humans to decide whether a crew needs to be sent out or whether the problem can be solved remotely.

As more and more data are generated from IoT sensors, more and more processes will be monitored and controlled by IoT network devices. But we need humans to make the final judgement of the data provided, and video will no doubt be a crucial tool in this process, ensuring real-time visual access, converting data into visual insight.

Combining machine intelligence with human judgement is a prerequisite for making the most of IoT-generated data. And I have no doubt that video will be the foundation for humans to trust this data in the future.

By Bjørn Skou Eilertsen, Chief Technology Officer, Milestone Systems.

Bjørn took part last month in the following sessions at the IoT Tech Expo Global in Olympia London:

Data analytics for AI & IoT – 18 April Keynote Panel: IoT and AI data analytics for intelligent decision making
Connected Transportation- 18 April Can you trust IoT network devices with access to your enterprise network?