Environment for Threat Intelligence Analysis and Generation using Open Sources

José Valdy Campelo Júnior; João José Costa Gondim

doi:10.17648/jisc.v6i1.75

Environment for Threat Intelligence Analysis and Generation using Open Sources

José Valdy Campelo Júnior, João José Costa Gondim

Abstract

Analyzing attacks on computer networks is complex given the volume of data considered and the large number of machines, even in small networks. The volume of data is large and the time to process and analyze it is short. The goal is to extract and analyze information about network attacks that has been obtained from open sources. Using a robust, elastic and scalable architecture that makes use of processing techniques with the use of Hadoop so that the information is available in a timely manner. With the proposed architecture implemented all the desired characteristics were obtained allowing the processing of the data in near real time. The system provides intelligence information about large-scale attacks with agility and efficiency.

Keywords

Attack detection; K-means Clustering; Threat Intelligence; Big data

Full Text:

PDF

References

Arbor Networks, Inc. Digital Attack Map. Available in: Acess Date: 12/09/2019

Bachupally, Y. R., Yuan, X., & Roy, K. Network security analysis using Big Data technology. In SoutheastCon 2016 (pp. 1–4). IEEE. https://doi.org/10.1109/SECON.2016.7506686

HADOOP, Apache. HDFS architecture guide. URl: https://hadoop. apache. org/docs/r1, v. 2, 2013.

CheckPoint, S. ThreatMap CheckPoint. Available in: Acess Date: 12/09/2019

Dean, J., & Ghemawat, S. MapReduce: Simplified Data Processing on Large Clusters. Commun. ACM, 51(1), 107–113. https://doi.org/10.1145/1327452.1327492

Kovacs, Eduard. Easily Exploitable Vulnerability Found in Netis Routers. Available in: Acess Date: 12/09/2019

Zikoupoulos, P., & Eaton, C. Understanding big data: Analytics for Enterprise Class Hadoop and Streaming (1st ed., Vol. 11). https://doi.org/10987654321

Janeja, V. P., Azari, A., Namayanja, J. M., & Heilig, B. B-dids: Mining anomalies in a Big-distributed Intrusion Detection System. 2014 IEEE International Conference on Big Data (Big Data), 32–34. https://doi.org/10.1109/BigData.2014.7004484

Jia, W. Study on network information security based on big data. Proceedings - 9th International Conference on Measuring Technology and Mechatronics Automation, ICMTMA 2017, 408–409. https://doi.org/10.1109/ICMTMA.2017.0104

Scharr, Jill. Possible Backdoor Found in Chinese-Made Routers. Available in: Acess Date: 12/09/2019

Vavilapalli, V. K., Murthy, A. C., Douglas, C., Agarwal, S., Konar, M., Evans, R., Baldeschwieler, E. Apache Hadoop YARN: Yet Another Resource Negotiator. Proceedings of the 4th Annual Symposium on Cloud Computing, 5:1--5:16. https://doi.org/10.1145/2523616.2523633

Wu, Y., Zheng, L., Heilig, B., & Gao, G. R. HAMR: A dataflow-based real-time in-memory cluster computing engine. International Journal of High Performance Computing Applications, 31(5), 361–374. https://doi.org/10.1177/1094342016672080

DOI: https://doi.org/10.17648/jisc.v6i1.75