Abstract

Hackers do not stay still. They test new tools, adjust their timing and blend attack traffic with normal traffic until a trained model stops catching them. This slow shift is called concept drift, and it can cause a network classifier with a 99% test score to fall apart within a few weeks of deployment. We built AdaptiveIDS to evaluate two reinforcement learning agents, Deep Q-Network (DQN) and Proximal Policy Optimization (PPO), on the CSE-CIC-IDS2018 dataset when attacks keep changing. We run the first DQN vs. PPO comparison for 14-class detection on this dataset, cut the 10-day data into five time windows, train on the first one, and test on the other four to see how scores drop as attacks evolve. A new measure called Post-Drift Detection Rate (PDDR) tracks that drop. The agent gets a bigger penalty for missing an attack (−1.5) than for a false alarm (−1.0), and we used SMOTE with reward bonuses to push the agent to learn rare attack classes. PPO finished at 97.8% accuracy and 97.4% weighted F1, with a false alarm rate of 1.4%—44% lower than the best prior result on the same dataset. Under the toughest drift window it still kept 86.1% F1, and PPO also trained about 27% faster than DQN.