In the security industry, large-scale data-related technology is constantly infiltrated, but also the emergence of a number of large data-related applications and products, which indicates that the security industry, large data era will gradually kicked off. Overall, the current large-scale data security industry is still in its infancy, faced with including technology, business, system, standardization and other aspects of the many problems in the future development, the need to gradually resolve these issues in order to make the advantages of large data reflected More and more obvious, in order to make greater use of data to greater value, so that the security industry's competitiveness has been improved.

What is Big Data?

On the definition of large data, the current different industries have different voices. Some people say that large data is a particularly large amount of data, previously a TB level, it is PB level; also said that large data is represented by Hadoop new technology, it can handle massive amounts of data. These arguments seem to be somewhat one-sided, an emphasis on data, an emphasis on technology. McKinsey on the definition of large data "4V" feature, while the definition of large data: "Large data refers to the size of the data set than the typical database software and tools processing capacity, at the same time, timely capture, "The new technologies and capabilities to aggregate, manage, and analyze data in depth are growing rapidly, just as Moore's Law predicts the growth rate of chips." McKinsey's definition covers data and technology. Data development, such a definition can not fully interpret the content of large data. We say that large data not only data, technology, more importantly, it can provide better service. Large data can carry on the deep relevance analysis to the massive data, then forecast the trend of the development of the thing, this also is the core of the big data, big data can apply mathematics algorithm to the magnanimous data to predict the possibility of happening.

"The Big Data Age," a book that: the essence of large data is that we analyze the data of the three changes, these changes will help us in-depth understanding of large data.

First, in the era of large data, we can analyze the data collection, rather than data sampling. The quantitative change of data can produce qualitative change, and can make up the deficiency of the algorithm at the same time. And the following cases, the Word program syntax check, there is a simple algorithm, when the amount of data is only 5 million, the performance of the algorithm is poor, but when the amount of data reached 1 billion level, the performance of the algorithm the most outstanding; On the contrary, there is a complex algorithm, the best performance in the 5 million data, but in the 1 billion level of data when the effect is not as simple as the algorithm.

Second, the large data under the data so much, so we are no longer keen to pursue accuracy. In the era of lack of information, we often pursue the accuracy of the data; in the era of large data, the amount of data so large, so many types of data, we can not guarantee that each data is accurate, but as long as most of the data is accurate , It will not affect the reliability of the analysis results.

Third, in the era of large data, we are more concerned about the relationship, rather than causal relationship. Wal-Mart's analysis found that beer and diaper sales there is a certain correlation. According to the results of the analysis, Wal-Mart diapers and beer, these two irrelevant goods together, the results are magically found diapers and beer sales have increased. It turned out that women in the United States are usually at home to take care of their children, so they often asked her husband on the way home from work to buy diapers for children, while her husband bought diapers at the same time they will easily buy their favorite beer. In this case, we find its reasons, but more often, we can not find the cause, but in fact we do not need to care about its causes, because the correlation from the analysis of the results, we can benefit from .

Big Data Development

Big data is now the hottest technology in 2012, the Ministry of Science and Technology, "China cloud technology development," Twelve Five "special plan" and the Ministry of Industry and "Internet of Things" second five "development plan" will be large data technology as A priority to be supported.

In the IT field, large data development has been quite mature. Such as Google's use of more than 3 billion users of the instruction to successfully predict the spread of influenza, the use of trillions of corpus to provide users with a relatively accurate translation; Amazon predicts based on past information users interested in books; Taobao based on the user's shopping behavior accurate To push advertising; and so on.

However, in the field of security, large data is still in the embryonic and exploratory stage.

First of all, the security industry is gradually entering the era of large data. With the continuous development of urban processes, the continuous deepening of information technology, the data is to the rapid growth of geometric speed, the traditional system or tool has been unable to effectively deal with such a mass of data. For example, the traffic bayonet data, the previous 10 million level, the current situation is: a district one year of bayonet data can reach one billion level, a prefecture-level city bayonet data can even reach 10 billion level , A province of data is even greater, the face of such a huge data, the traditional system seems helpless, even a simple query command, the response time will become very slow, not to mention analysis, statistics and other functions. At the same time, more and more users of large data put forward higher requirements, such as public security users, they have a lot of data, types, large amount of data, they require massive data analysis, to predict the role of early warning , And thus be able to public security business from post-analysis to forecast changes in advance.

Second, some security companies are in contact with large data, and have a preliminary exploration and application. As early as 2012, Hikvision as to get involved in large data, based on Hadoop development and optimization of large data solutions to meet the massive data efficient processing requirements. At present, Hikvision products based on large data technology: video cloud storage, to meet the 100PB data storage; video image information database, able to quickly retrieve large amounts of event data; traffic bay large data platform, able to target Mass of the bayonet data for rapid retrieval, intelligent research and judgment, statistical analysis, some of the research and judgment functions can be used for criminal cases of reconnaissance and early warning. In addition, such as Pok Hong, Yu and other security companies, is also catching up with the pace of development of large data.

Analysis of Security Data Core Technology

IT field of large data development has been quite mature, many of which can learn from the application of technology to the field of security. However, the security industry and the IT industry is not the same place, mainly the type of data. In the IT industry, large data analysis is often log, user behavior information, web index and other data, the computer can identify the structured data; and security industry, large data needs to be analyzed mainly video, pictures, audio And other unstructured data, the computer can not directly analyze the data, but need to extract the structure of which information, and then analysis.

The basic technology of large data can be borrowed from the IT field to the field of security, including the following technologies: First, the distributed file system, responsible for mass data storage, the data stored in separate multiple devices, the system uses scalable system Structure, the use of multiple storage servers share the storage load, the use of metadata server positioning storage information, it not only improves the reliability of the system, availability and access efficiency, but also easy to expand; Second, distributed database, real-time distributed Database, suitable for constructing high concurrent low-latency online data service system for storing coarse-grained structured data; 3, distributed computing, is responsible for a very large computing power to solve the problem is divided into many small parts, And then assigned to many computers for processing, and finally the results of these calculations together to get the final results; Fourth, full-text search engine, is responsible for massive data for stable, reliable, fast real-time retrieval; Fifth, memory computing, distributed memory computing , To the massive data more quickly analysis and processing; 6, flow calculation, responsible for the analysis and processing of streaming media data. Based on these technologies, the structure of the data can be processed quickly to address the massive data processing efficiency problems.

However, as mentioned above, the security industry's largest data is not structured data, but unstructured data, how to extract structured information from these unstructured data, is the first key to be addressed. The structure information which can be extracted in the video image includes the following contents: 1. The characteristic information of human, vehicle, and thing, the characteristic information of the person includes sex, age, height, body type, skin color, whether wearing glasses, hair, The character information of the vehicle includes the color, shape, size, texture features, etc. of the object, such as the license plate number, the license plate color, the license plate type, the vehicle type, the body color, the vehicle standard, the vehicle personnel information, , Behavioral information such as crossed alert surfaces, entry / exit zones, regional invasions, people wandering, and people gathering. When these data are extracted, further analysis can be done further, such as trajectory analysis of vehicles, analysis of abnormal behavior of people. Therefore, the intelligent analysis technology in the security data is particularly important, is to achieve the security of large data base.

Integration of a large amount of data, you need to dig the depth value of the data. The true value of the data is like an iceberg in the ocean, with the first glimpse at the tip of the iceberg, and the vast majority hidden beneath the surface. Prediction is the core value of large data, depth correlation analysis algorithm is the necessary means to achieve the value of large data. The data analysis algorithm is like a drill, and you need to dig out the real diamonds from this amazing diamond mine.

Security big data problems

With the development of large data, many problems gradually exposed, mainly in the following points:

(1) intelligent analysis technology is not mature enough. The structure of video image data is to realize the data base of security. At present, the vehicle information extraction technology of traffic bayonet is more mature, but the technologies such as human body information extraction and face matching are not mature enough.

(2) the application of data is not deep enough. When the integration of sufficient data, how to use the data for predictive analysis, trend analysis, almost the current application mode on the blank. Of course, there are also some examples can learn from, such as the bayonet large data system, it can be the depth of the car data intelligent judgments: regional collision, trajectory analysis, with car research and so on, which contribute to criminal detection efficiency of a substantial increase .

(3) data sharing is not extensive enough. Especially in government, public security, transportation and other departments, the information island widespread, this is mainly due to institutional problems, not a technical problem, it is difficult to change the status quo by the relevant departments, and can only be carried out by the relevant departments change.

(4) standardization is not comprehensive enough. This is mainly due to large data is still in its infancy, but also need more in-depth exploration and try. In the standardization of construction, such as data standards, interoperability standards, data application model standards, etc., need to continue to summarize and gradually standardized.

Security big data development trend

In the future development of the security industry, large data is bound to occupy an increasingly important position. The face of development problems, the current primary task is to be able to gradually solve these problems, and continue to improve the security big data program.

(1) technological innovation. First of all, the structure of video data, through intelligent technology, from the video image to extract people, cars, objects and other characteristics of information, through the extraction and integration of these information, can easily retrieve the video data, Depth association analysis. When these technologies can be realized, the application efficiency of video data will be greatly improved, and can lay the foundation for the in-depth application of video data. Second, large data processing technology. After the video data is structured, it becomes data which can be recognized by the computer. When more and more data are collected, the traditional technology or system can not be effectively processed. At this time, large data technology must be used to carry on the massive data deal with. Large data technologies include distributed file system, distributed database, full-text search engine, distributed computing, memory computing, flow computing, etc., with excellent reliability, scalability and processing performance, for massive data for rapid analysis, To provide users with better service.

(2) business innovation. With the massive video data after the structure, through the large data technology, these massive data can be deep excavation, prediction and trend analysis can be done, but the related business model also need to constantly explore and innovate. If the public security departments, video surveillance can only be an auxiliary means at present, if the use of large data technology to predict and early warning, then video detection will become a very important means, through the detection technology to reduce the number of cases Incidence, improve detection rate.

(3) Institutional improvements. More data can produce greater value, in order to be able to integrate more data, we must eliminate the information silos, which in the government sector is an objective problem. Of course, in the wisdom of the city under the impetus of this situation has changed, more and more government departments aware of the importance of data sharing. However, to really achieve the centralized and sharing of large data, there is still a long way to go.

(4) standard perfect. The integration of large amounts of data is inseparable from the standardization process, in the standardization process, the need to focus on the following points: First, the data structure of the standard specifications, including what data needs structured, structured data, how to design dictionary specifications, How to design database tables and so on, through the standard structured data, all systems are able to identify and deal with; Second, data interoperability standards, including how the platform and front-end interoperability platform and platform interoperability. The front end can be structured on the video data, the background can also be structured on the video data, front-end and background need to cooperate with each other, then the front end to inform the background which data has been structured, and what data needs further structured to standard Norms; Third, the application of data standards, including the data service model, type, rules and so on. Such as large data platform for cleaning and classification of large amounts of data, the depth of mining, the need to provide services to the upper business applications, this service will need to provide a standardized interface out.

Big data boost the security industry

Big data in the security industry will have a start, to the development and maturity of the process. In the initial stage, there are some intelligent analysis techniques, and through large data technology to solve the massive data processing efficiency problem; in the development stage, intelligent analysis technology will continue to mature, and will continue to appear innovative data applications; in the mature stage, Analysis technology is quite mature, and a systematic application of the depth of the data. Big data help security industry, mainly in the following areas:

First, the data application efficiency is rising. Through intelligent analysis technology, large data technology, can make the application of video data efficiency rising to solve the problem of low efficiency in the past. Application of the efficiency of the video data can produce greater value.

Second, the depth of data applications. Depth application of data to reflect the real value of large data, which also enhance the overall security system strength, the edge of the video data to the core position closer to the security industry's competitiveness has been improved.

Third, improve the system and standards. Standards and institutional improvements can further promote the development of large data, and master the standard security companies will have a more powerful voice.

Large data can build a smarter system, can change our country security system construction "reconstruction, light application" status, help to further enhance the public security department's core combat effectiveness, further consolidate the foundation of social stability.

Conclusion

Big data can build a smarter system for users, providing more valuable services. In the security industry, the rapid growth of data, emerging user needs, indicates that the demands of large data more and more intense, while more and more security companies involved in large data, with the initial exploration and application. Security field of large data is different from the IT field of large data, it has a higher demand for intelligent analysis technology, intelligent analysis technology is the basis for the realization of large security data, in addition to its large data-based technology, data depth analysis algorithms are The same high demand. Of course, large data security industry is still in its infancy, not mature enough big data also faces many problems, including intelligent analysis technology is not mature enough, the data application is not enough depth, data sharing is not wide enough, standardization is not comprehensive and so on. In the future development, first of all to solve these problems, and constantly improve the security of large data programs, including technological innovation, business innovation, system improvement, standard improvement. Only a more complete security large data, in order to reflect the more obvious advantages, to play a greater value. With the continuous development of large data is mature, it will bring to the security industry to enhance quality. Big data is the future trend of development, it will lead the next security era, let us wait and see.