Abstract

Machine learning-based NIDS tend to perform well on the test sets included with their software, but don’t work well in production environments. In this paper, the cross-dataset generalization in malware traffic classification is analyzed for the datasets of UNSW-NB15 and CTU-13. Five flow features were selected and used to train and test the classifiers namely Random Forest, Gradient Boosting and XGBoost classifiers as the features have semantic similarity to each other. The Kolmogorov-Smirnov (KS) test was employed to measure the domain shift between datasets and SHAP was used to measure feature importance and the stability of explanations between domains. The experimental results illustrate that both models have similar performance in the in-domain evaluation while there is a significant drop in performance for the out-of-domain evaluation on CTU-13. Gradient Boosting is the best performing algorithm over domain and KS analysis revealed that the distributions of all the features aligned were significantly different. SHAP analysis showed that the influence of features is very sensitive among different datasets indicating instability of learned representations, highlighting the difficulty of deploying a machine learning intrusion detection system in an environment different from the one it was trained in.