The future of embedded video analytics

Please, note that the article is automatically translated from Russian into English, so the translation may not be accurate.

Nikolai  Ptitsyn

What is embedded video analytics? What are its advantages and disadvantages compared to traditional server-based video analysis? Why now embedded algorithms are particularly important? What is the structure of the market? What should I look for when choosing intelligence and how it can be tested?

Architecture of video analytical systems

Embedded video analytics (video analysis system) is called the software that recognizes streaming video directly into the camera or encoder in the system of surveillance. Compared with the server implementation (Fig. 1) embedded analytics analyzes the signal without distortion to its compression and transmission over digital communication channels (Fig. 2). This increases the recognition accuracy by processing video at a higher resolution and higher frame rate.

Fig. 1. The system architecture based on server-side analytics. Significant flow of data transmitted from the camera to the server, where the decompression and video analysis. Finite computational resources limit the scalability of the server system.

Fig. 2. The system architecture based on embedded video analytics. Video analysis is implemented "on board" camera. The camera simultaneously transmits media data (video and audio) and metadata.

The result of video analysis algorithms is the flow of metadata, ie structured description of what is happening in the surveillance zone. The metadata includes information about moving objects, their trajectory and speed signs for the automatic classification on the server, information about the video quality and damage the camera (the status of service detector).

Thus, the smart camera simultaneously transmits real-time media data (compressed video and audio) and metadata (the result of video analysis). Videoanaliticheskoe device (IP-camera or encoder) can work independently and to write to local memory or buffer transfer.

The disadvantages of embedded intelligence in comparison with the server are higher design complexity and less flexibility for a stand-alone device. However, despite the substantial capital costs, total cost of ownership system based on embedded intelligence is often lower than the cost of systems based on server-side analysts.

The reasons for the increase in demand

Consider the major industry trends, leading to increased interest in end-users the embedded video analytics.

Distribution of IP-Surveillance. It is obvious that the transition from analog cameras to the network requires a radical revision of the architecture of the entire security system. Centralized video analysis becomes practically impossible because of the significant increase in computational load on the server where you want to make the decompression of compressed IP-video on all channels at high resolution. More vexing problem lies in the fact that the accuracy and range of analytics server is substantially reduced from the artifacts of compression H.264 or MJPEG. Unacceptable decrease in sensitivity Server Analysis occurs during meterologicheskih precipitation, because codec, loss of video stream to a fixed bit rate, casts a significant part of the "useful video information."

Open standards ONVIF and PSIA . One of the main obstacles to widespread internal videoalitiki is the problem of poor compatibility of IP-devices such as cameras and encoders on the one hand, and video management systems and digital recorders on the other side. The active work of producers to adapt their products to the open international standards, today gives hope for overcoming this obstacle in the near future. The new standards, particularly ONVIF 2.0, covering various aspects of the interaction of network devices: the detection of the available video analytics services, configuration parameters, the transfer of metadata, sign up to receive event-messages. This will make it possible to connect videoanaliticheskogo with the device plug and play.

Megapixel resolution. Increasing resolution of video surveillance systems to high-definition (HD 720p, 1080p) followed by repeated (4-8 times) increase in the computational load on the compression subsystem, and video analysis. Thus, to avoid overloading the channels of communication and server compression and video analysis to be implemented directly in the HD-camera or HD video server.

Channels with low bandwidth. Network CCTV is often necessary to deploy over the telecommunications infrastructure with a relatively low bandwidth, for example, based on wireless networks, 3G/4G (WI-MAX), a cable network of ATMs or telemetry in the industry. In these conditions it is difficult to ensure adequate quality of broadcast video in real time from a few or even one chamber. On the other hand, the embedded algorithms are used to analyze the video in the highest quality on-site, generate alarms and to the buffered broadcast on the events or on demand, even at extremely slow communication channels. In other words, video analytics provides autonomous operation of cameras and sets priorities for the selective transmission of video over a channel with low bandwidth.

Managed Video as a Service (MVaS) is one of the hottest trends in the industry in the west, but has not yet come to Russia. According to the model MVaS, infrastructure management and storage of video is located on Internet servers, and video service is provided as a service with a monthly payment. The volume of transmitted video from the subscriber's server is limited by bandwidth of the outgoing link, which is usually significantly less than the input. Just limited disk resources on a central server that serves a large number of subscribers. As discussed above, integrated analytics plays a key role in systems with limited resources.

The structure and size of the market

Market analysts have built in the early stages of development. A considerable number of vendors offering videoanaliticheskoe software, which is badly differentiated and poorly integrated with third-party development.Often the technical capabilities of video analytics and technical requirements of the customer poorly formalized and does not allow us to make a formal comparative analysis.

In the area of the perimeter of the leading developers of embedded video analytics are companies IOImage (Israel), AgentVi (Israel), ObjectVideo (USA), Bosch (Germany / USA) and Synesis (RF / RB). The company Axis, a Swedish provider of network cameras, recently announced a basic videoanaliku in the chambers of his senior product line. In Russia, the equipment with embedded intelligence make the perimeter of Byterg and Aggregator device (MagicBox).

A pioneer in the field of integrated recognition of license plates of vehicles the company CRS (UK). Company Texas Instruments (USA) offers a detection algorithm and face recognition, embedded processor family DaVinci. For more complex applications such as anti-terrorist security in the transport or marketing of retail sales, adequate implementations of embedded analytics does not exist yet.

By the manufacturers of video surveillance systems market leader iTV / Axxon, Genetec and Milestone is currently already support equipment with embedded analytics.

According to research agency IMS Research, about 50 000 channel embedded intelligence have been deployed worldwide in 2009, but annual sales have grown significantly to 400 000 channels by 2013. For comparison, the sales of server-side analysts to grow at a slower pace with 38,000 in 2009 to 100 000 channels in 2013. At the beginning of the reporting period, approximately 50% of sales accounted for by government and transport sectors.

Price analytics varies considerably ranging from $ 10 to $ 10,000 per channel, with most of the added value created during hardware-software integration of all surveillance systems. At this price, difficult to identify the most cost "of mathematics." Thus, the agency IMS Research in 2008 suggested that the market for video analytics in 2010 amounted to 3.4 billion dollars is obvious that we are talking primarily about revenue producers of cameras and control systems with embedded intelligence, and not about royalties videoanaliticheskih algorithms.

How to choose a embedded analytics?

According to a survey of analytical portal IPVideoMarket, specializing in network video, the most urgent areas of improvement of the video analysis are: increasing recognition accuracy (55%), simplification of installation / configuration (26%), other improvements (5%). Only 2% of respondents felt that videoanaliticheskaya technology needs no further improvement. Consider a more detailed discussion of these and other criteria.

Recognition accuracy. The main criterion for selecting video analytics - indicators of accuracy - is the most challenging in terms of testing and comparative analysis. Need a large base of content online, which covers a variety of states of the environment and behavior of the enemy, as well as automatic test equipment. In some cases, non-trivial to ensure the supply of test videos directly videoanaliticheskoe device where you want the video input interface is usually not provided.

The test base is marked by security experts who determine the true types of events and times of their occurrence. Calculated measures of accuracy: sensitivity and specificity Where a - number istinnopolozhitelnyh positives (no error), b - the number of false positives [alarms (error type I) and c - the number of false-positives (Type II error). As an integral metric, you can use a weighted average Where α - a weight parameter that determines the ratio of the significances of error types I and II. Repeated tripping the motion detector on the same targets were considered type I error. For example, for the scenario "security perimeter" in assessing the accuracy of the formation of operational troubles α = 0.65, and for assessing the accuracy of recording events for indexing videos α = 75.00. A good level of accuracy is F ₁ = 0.99.To test the recommended set of TV spots, with not less than 500 violations of the true and the same situations with the manifestations of interference, and potential sources of error (moving shadows, shaking the camera, the appearance of animals, birds, insects).

Support for open standards. As noted above, one of the major obstacles distribution integrated analytics is the problem of compatibility with hardware platforms and video management systems. According to our research, the standard ONVIF provides the most ample opportunities for setting up a centralized intelligence and transfer of metadata (see article). Support for this standard from the ip-cameras and encoders, control systems, video recorders provides maximum investment protection for end users, which comes freedom of choice of each component of the overall solution.

Remote software update. Updating the embedded intelligence, by analogy with antiviral drugs is a prerequisite for maintaining security at a high level of efficiency. When you select a embedded video analytics provider must pay attention to the reliability function remote firmware updates over the network and frequency of software updates, including "mathematics".

Other factors. Additional features embedded video analytics and server are the ability to work in a changing background, resistance to camera shake, the presence of daylight and night modes, time target recognition (the typical range of 1-10 seconds), range (10-50 meters), the permissible angles camera settings, the types of detected human movement (eg walking, running, somersault).

The role of the server software

Building features embedded intelligence must be accompanied by a corresponding development of "intelligence" video management systems on the server side or on the side of the digital recorder. Note the most important functions of the server software.

Hashing metadata for fast search. Full search all the metadata generated by multiple cameras, is resource-intensive operation. Implementing Instant Search the archive requires the construction of special indices, similar to those used in Internet search engines or on the workstation.

Statistical analysis and improvement of recognition accuracy. Server software can analyze the metadata to a higher level and detect unusual situation with more precision than the firmware, due to statistics accumulated in the database server and the application of more sophisticated algorithms. Server-based statistical algorithms are used to identify patterns in the behavior of the observed objects and to make quantitative estimates.

Multi-tracking and control of PTZ-camera. Accompanying the object from camera to camera requires a centralized data processing on the server. Absolute three-dimensional coordinates of the object to automatically target the PTZ camera, or automatically switch channels on the display.

Dynamic control of bit rate. An important task of the server is an adaptive flow control video and audio data from the camera. Event occurs, congestion of communication channels, the priority of the observation, the presence of the operator are non-exhaustive list of factors affecting the optimum transfer mode media.

In some cases, much cheaper to differentiate on the server than the camera or encoder. Development process on the x86 is much faster and more flexible. This allows us to efficiently build customized solutions around unified embedded analytics.

Conclusion

We have considered the issues of designing security systems based on integrated video analytics. Compared with the server implementation of embedded algorithms provide better scalability due to the decentralized processing of media data. Real benefit of embedded video analysis for the end user increases as the spread of IP-video, megapixel cameras, standards ONVIF and PSIA. At the same time expanding the space of possible differentiation of video management system by intelligently processing videoanaliticheskih metadata on the server side. When choosing to use analytics should pay attention to formal accuracy parameters, test conditions, compatibility with the standards and the availability of Remote firmware upgrade.