Component Profiling and Prediction Models for QoS-Aware Self-Adapting DSMS Framework

Quality of Service (QoS) has been identified as an important attribute of system performance of Data Stream Management Systems (DSMS). A DSMS should have the ability to allocate physical computing resources between different submitted queries and fulfil QoS specifications in a fair and square manner. System scheduling strategies need to be adjusted dynamically to utilise available physical resources to guarantee the end-to-end quality of service levels. In this paper, we present a proactive method that utilises a multi-level component profiling approach to build prediction models that anticipate several QoS violations and performance degradations. The models are constructed using several incremental machine learning algorithms that are enhanced with ensemble learning and abnormal detection techniques. The approach performs accurate predictions in near real-time with accuracy up to 85% and with abnormal detection techniques, the accuracy reaches 100%. This is a major component within a proposed QoS-Aware Self-Adapting Data Stream Management Framework.