A Statistical Method for Middleware System Architecture Evaluation.
Abstract
The architecture of complex software systems is a collection of decisions that are very expensive to change. This makes effective software architecture evaluation methods essential in today's system development for mission critical systems. We have previously developed MEMS for evaluating middleware architectures, which provides an effective assessment of important quality attributes and their characterizations. To provide additional quantitative assessments on the overall system performance using actual runtime data, we employed a set of statistical procedures in this work. Our proposed assessment procedures comprises a standard sensitivity analysis procedure that utilizes leverage statistics to identify and remove influential data points, and an estimator for evaluating system stability and a metric for evaluating system load capacity. Experiments were conducted using real runtime datasets. Results show that our procedures effectively identified and isolated abnormal data points, and provided valuable statistics to show system stability. Our approach thus provides a sound statistical basis to support software architecture evaluation. 1. Introduction In the domain of software engineering, mission critical systems incorporate component-based and distributed computing systems and these systems are becoming more powerful and complex. This growth in system complexity results in a corresponding increase in the complexity of the software driving these systems. As a result, software intensive acquisition projects often incur schedule delays, cost overruns and reduced functionality at the end. To remedy this situation, a software architecture evaluation framework can provide a means for identifying the technical risks of a proposed architecture. 1530-0803/10 $25.00 © 2010 IEEE DOI 10.1109/ASWEC.2010.39 2. Background and Related Work This study is linked to a previous project on evaluating middleware architecture for Airborne Mission Systems that developed a Method for Evaluating Middleware architectureS (MEMS) [1], by rating multiple software quality attributes. 183 Authorized licensed use limited to: CITY UNIV OF HONG KONG. Downloaded on September 17,2026 at 02:45