Speaker
Ms
Hao Hu
(Institute of High Energy Physics)
Description
Huge amount of experimental data are produced by large scientific facilities of IHEP, such as Daya Bay, JUNO, LHASSO and CSNS. The performance and efficiency of the data exchange are playing an important role in these scientific research activities.
The quality of data exchange among the members relies heavily on the stability and reliability of the network. The statistics and analysis for the network traffic is a way to know the network status and also very useful for the network performance optimization and the strategy plan for network architecture.
The paper describes the design and development a network traffic statistics and analysis platform based on PMACCT and KAFKA. The functional modules of this platform include data acquisition, data storage, data analysis and visualization. The data acquisition module is developed based on an open source software ‘PMACCT’, which collects network traffic data from the routers. For data storage module, KAFKA is used as a message queue to subscribe traffic records from PMACCT and save these records to MongoDB with high efficiency. Data analysis module is responsible for the classification and statistics of traffic data according to the requirement parameters pre-configured. Visualization module serves for network administrator which provides customized graphic reports via HTTP.
Partial functions of the platform have been finished and deployed to analysis IHEP network traffic, and more features will be released in the future.
Summary
The paper describes the design and development a network traffic statistics and analysis platform based on PMACCT and KAFKA. The functional modules of this platform include data acquisition, data storage, data analysis and visualization.Part functions of the platform has been finished and deployed to analysis IHEP network traffic, and more features will be released in the future.
Primary author
Ms
Hao Hu
(Institute of High Energy Physics)
