Reinforcement learning-based detection method for malware behavior in industrial control systems

GAO Yang; WANG Li-wei; REN Wang; XIE Feng; MO Xiao-feng; LUO Xiong; WANG Wei-ping; YANG Xi

doi:10.13374/j.issn2095-9389.2019.09.16.005

Volume 42 Issue 4

Apr. 2020

Turn off MathJax

Article Contents

Article Navigation > Chinese Journal of Engineering > 2020 > 42(4): 455-462

GAO Yang, WANG Li-wei, REN Wang, XIE Feng, MO Xiao-feng, LUO Xiong, WANG Wei-ping, YANG Xi. Reinforcement learning-based detection method for malware behavior in industrial control systems[J]. Chinese Journal of Engineering, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005

Citation:

GAO Yang, WANG Li-wei, REN Wang, XIE Feng, MO Xiao-feng, LUO Xiong, WANG Wei-ping, YANG Xi. Reinforcement learning-based detection method for malware behavior in industrial control systems[J]. Chinese Journal of Engineering, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005

Citation:

GAO Yang, WANG Li-wei, REN Wang, XIE Feng, MO Xiao-feng, LUO Xiong, WANG Wei-ping, YANG Xi. Reinforcement learning-based detection method for malware behavior in industrial control systems[J]. Chinese Journal of Engineering, 2020, 42(4): 455-462. doi: 10.13374/j.issn2095-9389.2019.09.16.005

PDF( 723 KB)

Reinforcement learning-based detection method for malware behavior in industrial control systems

doi: 10.13374/j.issn2095-9389.2019.09.16.005

GAO Yang¹,
WANG Li-wei^{2, 3, 4},
REN Wang¹,
XIE Feng¹,
MO Xiao-feng^{2, 3, 4},
LUO Xiong^{2, 3, 4
,
,},
WANG Wei-ping^{2, 3, 4},
YANG Xi⁵

1.
China Information Technology Security Evaluation Center, Beijing 100085, China
2.
School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China
3.
Institute of Artificial Intelligence, University of Science and Technology Beijing, Beijing 100083, China
4.
Beijing Key Laboratory of Knowledge Engineering for Materials Science, Beijing 100083, China
5.
Beijing Intelligent Logistics System Collaborative Innovation Center, Beijing 101149, China

More Information

Corresponding author: E-mail: xluo@ustb.edu.cn
Received Date: 2019-09-15
Publish Date: 2020-04-01

Abstract

Abstract

Due to the popularity of intelligent mobile devices, malwares in the internet have seriously threatened the security of industrial control systems. Increasing number of malware attacks has become a major concern in the information security community. Currently, with the increase of malware variants in a wide range of application fields, some technical challenges must be addressed to detect malwares and achieve security protection in industrial control systems. Although many traditional solutions have been developed to provide effective ways of detecting malwares, some current approaches have their limitations in intelligently detecting and recognizing malwares, as more complex malwares exist. Given the success of machine learning methods and techniques in data analysis applications, some advanced algorithms can also be applied in the detection and analysis of complex malwares. To detect malwares and consider the advantages of machine learning algorithms, we developed a detection framework for malwares that threatens the network security of industrial control systems through the combination of an advanced machine learning algorithm, i.e., reinforcement learning. During the implementation process, according to the actual needs of malware behavior detection, key modules including feature extraction, policy, and classification networks were designed on the basis of the intelligent features of reinforcement learning algorithms in relation to sequence decision and dynamic feedback learning. Moreover, the training algorithms for the above key modules were presented while providing the detailed functional analysis and implementation framework. In the application experiments, after preprocessing the actual dataset of malwares, the developed method was tested and the satisfactory classification performance for malware was achieved that verified the efficiency and effectiveness of the reinforcement learning-based method. This method can provide an intelligent decision aid for general malware behavior detection.
- malware,
- detection method,
- reinforcement learning,
- feature extraction,
- policy network

FullText(HTML)

References(17)

References

[1]	時憶杰. 移動互聯環境下工業控制系統安全問題研究[學位論文]. 北京: 北京郵電大學, 2016 Shi Y J. Research on the Key Security Issues of Mobile and Open Industrial Control System[Dissertation]. Beijing: Beijing University of Posts and Telecommunications, 2016
[2]	Demontis A, Melis M, Biggio B, et al. Yes, machine learning can be more secure! A case study on android malware detection. IEEE Trans Dependable Secure Comput, 2019, 16(4): 711 doi: 10.1109/TDSC.2017.2700270
[3]	Sharif M, Lanzi A, Giffin J, et al. Impeding malware analysis using conditional code obfuscation // Proceedings of the Network and Distributed System Security Symposium. San Diego, 2008: 1939
[4]	Xiao X, Wang Z, Li Q, et al. Back-propagation neural network on Markov chains from system call sequences: a new approach for detecting Android malware with system call sequences. IET Inf Secur, 2016, 11(1): 8
[5]	Su X, Zhang D F, Li W J, et al. A deep learning approach to android malware feature learning and detection // 2016 IEEE Trustcom/BigDataSE/ISPA. Tianjin, 2016: 244
[6]	Li G L, Gomez R, Nakamura K, et al. Human-centered reinforcement learning: a survey. IEEE Trans Human Mach Syst, 2019, 49(4): 337 doi: 10.1109/THMS.2019.2912447
[7]	Wu C S, Shi J Y, Yang Y X, et al. Enhancing machine learning based malware detection model by reinforcement learning // Proceedings of the 8th International Conference on Communication and Network Security. Qingdao, 2018: 74
[8]	Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518(7540): 529 doi: 10.1038/nature14236
[9]	Schultz M, Eskin E, Zadok F, et al. Data mining methods for detection of new malicious executables // Proceedings of the IEEE Symposium on Security and Privacy. Oakland, 2001: 38
[10]	Santos I, Brezo F, Ugarte-Pedrero X, et al. Opcode sequences as representation of executables for data-mining-based unknown malware detection. Inf Sci, 2013, 231: 64 doi: 10.1016/j.ins.2011.08.020
[11]	Zhang J X, Qin Z, Yin H, et al. IRMD: Malware variant detection using opcode image recognition // Proceedings of the IEEE 22nd International Conference on Parallel and Distributed Systems. Wuhan, 2016: 1175
[12]	Tandon G, Chan P. Learning rules from system call arguments and sequences for anomaly detection // Proceedings of the International Workshop on Data Mining for Computer Security. Melbourne, 2003: 20
[13]	Willems C, Holz T, Freiling F. Toward automated dynamic malware analysis using CWSandbox. IEEE Secur Privacy, 2007, 5(2): 32 doi: 10.1109/MSP.2007.45
[14]	Rieck K, Trinius P, Willems C, et al. Automatic analysis of malware behavior using machine learning. J Comput Secur, 2011, 19(4): 639 doi: 10.3233/JCS-2010-0410
[15]	Ki Y, Kim E, Kim H K. A novel approach to detect malware based on API call sequence analysis. Int J Distrib Sens Netw, 2015, 11(6): 659101 doi: 10.1155/2015/659101
[16]	Busoniu L, Babu?ka R, De Schutter B. A comprehensive survey of multiagent reinforcement learning. IEEE Trans Syst Man Cybern Part C Appl Rev, 2008, 38(2): 156 doi: 10.1109/TSMCC.2007.913919
[17]	Zhang T Y, Huang M L, Zhao L, et al. Learning structured representation for text classification via reinforcement learning // Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence. New Orleans, 2018: 6053