Pages: 1-13

Abstract

The big data security and its management are significantly important given the increasing volume of data in computer networks. In this regard, data mining algorithms have been introduced as the applicable data analysis tools. Classification algorithms are important techniques in data mining. Decision tree algorithms have many applications in intrusion detection problems due to producing classified results in a meaningful and tree structure. Given the importance of intrusion detection velocity in networks, the parallel processing of classification algorithms on a big data context has been a challenging issue. In this research implemented a distributed and scalable method based on C5.0 decision tree for the intrusion detection problem using a new and advanced Spark framework in the field of data processing and used the most important feature selection to improve the system efficiency. Due to the use this framework and its high ability in in-memory processing. Finally, the proposed algorithm was evaluated using the standard KDDCUP99 dataset. The evaluation results indicated the scalability and high speed of the proposed algorithm.