Video Analytics: Specific by Design
Whether you are trying to secure a border, monitor a precarious piece of road, or track the number of people entering and leaving a department store, you are usually looking for something specific. Video content analysis (VCA) assists in these endeavors and is coming to play an increasingly prominent role in video surveillance worldwide.
Systems using VCA or video analytics, as it is also frequently referred to, are often highly specialized with above-average demands for accuracy and availability. Since they are generally used to detect particular items, VCA applications require highly customized installations and configurations. This white paper explores the technology behind VCA and explains on a case by case basis why these advanced software programs are specific by design.
Understanding the ins and outs of analytics
While it is amazing what technology can do these days, the nitty-gritty of it does not work like the latest Sci-Fi movie. VCA works in a highly organized, structured, and logical manner. It is therefore important to have a basic understanding of how VCA works to better grasp how it is applied in various situations.
One common misconception today is that video motion detection (VMD) is a form of video analytics. This is not the case. VMD compares image frames to detect motion in the camera scene. While this can be useful in weeding out static video material to optimize available storage space, VMD is not capable of detecting objects and as such, the false alarm rate is high.
Contrastingly, video analytics actually examine the video material. VCA uses intelligent algorithms to continually analyze camera images for unusual occurrences and alert operators within seconds of the type of incident and its location. From a centralized control center, the notified operator can visually verify and assess the situation and react appropriately.
In sum, VCA applications are intelligent software algorithms capable of three things:
- Differentiating between the general camera scene and objects passing through the camera’s field of view (FOV)
- Using specified parameters known as “metadata” to detect objects; metadata includes things like the speed and relative position of the object within the camera scene
- Classifying and evaluating the detected objects based on predefined rule sets, including applications and characteristics such as tripwires, counting, minimum object lifespan, and aspect ratio; a rule set can roughly categorize objects, for example, as humans, animals, or vehicles, or determine the density of objects in a specific area. In more advanced analytics systems, rules might include tracking objects, accumulating (behavioral) statistics, and using optical character recognition (OCR) to read license plates or container labels.
The defining nature of metadata and rules means that there is very little margin for imprecision or inaccuracies in VCA applications. This brings up an important point about VCA in general. For whatever application video analytics are intended, they always look for something specific in a particular situation based on exact factors. VCA deployments can therefore only be successful if they are designed for a certain application.
Tailoring VCA to a specific situation entails determining how to allocate resources across the network, for example, in the central control room, in the cameras, or in a distributed setup. Implementing VCA in just one place can burden that area of the system, though using multiple intelligent devices requires higher levels of compatibility. As a result, a large part of creating an efficient and effective system is deciding where and how the analytics algorithms will reside on the network.
Getting the best of both worlds
Video analytics programs are relatively flexible in where they can be deployed; they can run on standard off-the-shelf computers, recording devices, specialized video processing units, or be embedded in IP cameras or encoders. This essentially boils down to three options.
VCA programs can be installed:
- In stand-alone devices at the edge of the network
- On centralized computers and servers in a control room
- Across the network in a distributed setup
There are certainly pros and cons to every configuration. A centralized setup makes it possible to use different types of devices (e.g., IP or analog) and to combine information from the various devices. However, it requires a lot of processing power (CPU) and the analytics algorithms work on compressed video rather than on the original images, which can potentially inhibit the overall effectiveness of the system.
Embedding VCA in edge devices (e.g., cameras and codecs) enables algorithms to scan raw video signals. It also makes it possible for a relatively simple but intelligent system to be set up virtually anywhere. Yet stand-alone edge devices are not capable of tracking objects or analyzing recorded material. Moreover, they have relatively limited processing power.
Therefore, many analytics deployments today opt for a distributed solution to optimize the allocation of resources. In distributed networks, edge devices take on some of the number crunching while the central server provides ample amounts of CPU and memory power for demanding tasks. Ultimately, a hybrid setup reaps the best of both worlds.
Implementing a distributed analytics network also facilitates the customization of the VCA solution. Although the devices need to be compatible with the VCA application, they are relatively independent of the rules employed by the central server. Therefore, the best devices for a particular situation can be selected and appropriately positioned at the edge of the network while the VCA rules and actions can be honed and perfected at the control center end of the operation. As a result, metadata and rules can be precisely defined on the right equipment for a particular application in the most efficient way possible.
The following case studies will give a better idea of how VCA applications are designed and deployed in the real world.
Setting up a new system in a tunnel
Due to new EU regulations, a European transportation authority needs to implement an AID-equipped video surveillance system in a 600-meter long tunnel on a two-lane rugged mountain highway.
The highway authorities have a lot to think about as they work with the systems integrator to design the system. One of the first things they do is set up cameras to document what the recurring scene fluctuations are. Then, the authorities start to design a distributed network and to build a central control room located just outside the north end of the tunnel that will employ an operator twenty-four hours a day.
The control room will house the servers, PCs, and monitors upon which the traffic management system (TMS) will run. The TMS software platform will include both a configuration client and server. It will enable tunnel personnel to draw up incidents that the cameras need to monitor. Incidents might include anything from pedestrians, stopped vehicles, wrong-way drivers, and lost cargo to queuing, speed drops, and even the presence of smoke.
Tunnel personnel can also specify various scenarios in the TMS, such as maintenance projects or day and night lighting parameters. Armed with these scenarios, the TMS can discern between actual events and anodyne occurrences. For example, the cameras might be configured to notify the TMS if pedestrians are detected in the tunnel. If the TMS is using a normal, everyday scenario, it will trigger an alarm. However, if part of the tunnel is under maintenance, the TMS will use a different scenario. This maintenance scenario will allow the TMS to interpret the information from the cameras as irrelevant if the pedestrian happens to be a road worker.
The TMS will use configured AID events to trigger a network video recorder (NVR) to save any relevant video clips to the system for future use. These recordings can be reviewed later to help improve the traffic monitoring system and hence the safety of the tunnel.
In the tunnel itself, intelligent cameras will be installed every fifty meters since they have a detection range of sixty meters and a blind spot just under the camera of ten meters. Given that the intelligence of the cameras in this installation is of critical importance, each camera will use a dedicated digital signal processor (DSP) to implement the incident detection algorithms, so all the number crunching will be done locally. This means that only video material of relevant events will be transmitted to the central control room. Consequently, the system will use less bandwidth while ensuring higher video quality of actual incidents.
The dedicated AID DSP makes the cameras capable of readily discovering and exposing incidents as well as carrying out traffic monitoring tasks, such as measuring lane speed and occupancy. The cameras will optimize detection performance by storing and transmitting the metadata alongside the video material. Each individual stream can be merged with incident detection information, giving operators a complete overview of the traffic data and events and allowing them to easily access, maintain, and enhance traffic conditions. Consequently, the AID backbone of this traffic management solution will ensure that travelers arrive at their destinations safe and sound.
Managing a metro
With millions of commuters every day, delayed trains are not an option for a metro system in a world renowned mega metropolis, even when operational issues arise. That’s why this metro system installed a centralized IP video surveillance system equipped with VCA capability.
The system allows authorities to define a number of incidents that could hinder metro schedules. The analytics algorithms continually monitor the camera scenes and trigger an alarm when certain parameters are met. Such incidents might include an object on the tracks or a stopped train. When the analytics system triggers an alarm, operators can promptly take the appropriate actions to ensure that the metro continues to run safely and on time.
One very dangerous incident that metro authorities want to be sure to avoid is having a person in a metro tunnel when a train goes by. This could potentially cause a fatal accident. Therefore, the analytics need to promptly and accurately detect the presence of pedestrians in the tunnels and immediately notify operators.
When initially setting up the system, the video analytics sent a number of false alarms to the control center. After surveying the situation, it became apparent that the VCA algorithms were being confused by the extreme difference in the camera scene between when a train was and when a train was not passing through the tunnel. Very little usually happens in the camera scene unless a train is going past. Then, there is a lot of motion, light, and vibration. As a result, the VCA application was not working properly.
The metro authorities and systems integrator met to discuss how they might remedy this problem. The systems integrator noted that for an analytics system to work correctly, it was necessary to pinpoint exactly what the system needed to detect. The answer was clear: People in the tunnels.
Since this was only possible when a train is not going through a particular camera scene, the systems integrator worked with metro authorities to come up with a simple but efficient solution. They equipped the tunnels with sensors to shut down the analytics system while trains passed. When the camera’s FOV returned to the preconfigured “normal” scene, the VCA application would start up again.
This setup did the trick and the analytics system now proficiently ensures the safety and security of the metro system and its commuters.
Retrofitting surveillance systems for intelligent solutions
A factory has secured its grounds with an analog CCTV video surveillance system for over twenty years. The security guards on the premises sit in a local control room and monitor an established camera tour on a video matrix. When they notice something strange, someone goes to investigate the area.
While this system has worked alright for a number of years, the factory has expanded its operations and facilities over the years and it is no longer feasible for just a few security guards to manually secure the factory. Therefore, management opted for a VCA application to assist its security personnel. The new system includes detection, classification, and tracking capabilities, so operators in the control room can more swiftly and effectively respond to incidents.
To the satisfaction of the factory’s management, the systems integrator suggested they make use the legacy analog system for the analytics. However, it will take a lot of work to make sure the old cameras work appropriately in the new system.
When the legacy analog system was set up, the cameras were mounted to have overlapping but broad FOVs along the factory’s perimeter. This will not work for the analytics algorithms. For example, the detection and classification rules cannot distinguish a leaf from a person with such a broad camera scene. Therefore, all the cameras have to be realigned. While some just need to be pointed in a different direction, others will have to be moved to new locations and mounted anew.
In addition to repositioning, the cameras also have to be reconfigured for an analytics situation. Since the cameras are outside and VCA applications are very sensitive to noise levels in camera images, varied lighting conditions must be accounted for.
By modifying the legacy equipment in this way during the retrofit, the systems integrator was able to help the factory upgrade its surveillance installation to an advanced security management system at a relatively low cost.
Undermining Universal Applicability
Video analytics are changing the face of video surveillance applications. They offer intelligent solutions to detect, classify, and track almost anything. This ultimately enhances the overall effectiveness of surveillance installations on the whole as well as their supported operative infrastructures. This makes it possible for operators and security guards to focus more on dealing with incidents rather than discovering them.
However, for video analytics to work correctly, applying accurate parameters is essential. You need to know exactly what you want the system to identify and the conditions in which the cameras are mounted must be well understood. As a result, there are vast possibilities available with VCA systems, but every installation must be designed and fine‑tuned for the specific task at hand.
