Quality Assessment of Mission Critical Middleware System Using MEMS.
Abstract
poorly designed, the projects built on such middleware often incur schedule delays, cost overruns and reduced functionality. The earlier risks are identified, the greater the probability that the system will be cost effective, delivered to schedule and perform as required. Therefore, the acquisition process should have the capability of evaluating architectures built on middleware. To remedy this situation, a number of architecture evaluation methods provide general guidelines to assess quality attributes of systems. A comprehensive review can be found in [5]. These methods are applied at the application level, where middleware is considered as features or constraints of implementing the applications. The strong relationship between a middleware technology and the quality of the overall system are not explicitly addressed. While these methods can be extended to incorporate middleware in theory, there are a number of problems with trying to implement them in practice. One problem is that quality attributes (such as performance, liveness and scalability) are abstract entities to the stakeholders, making it difficult for stakeholders to quantify the expected quality of the mission critical systems built on complicated middleware. The ability of a middleware technology to support given quality attributes depends on the mechanisms and services provided by the infrastructure. This indicates that evaluation methods require detailed technical inputs regarding the middleware infrastructure, including its programming model, APIs, configuration and deployment. This kind of knowledge helps to identify the effect of different middleware architectures on quality attributes. Given the increasing complexity of middleware technologies, improved evaluation techniques are required. To this end, we have developed a Method for Evaluating Middleware architectureS (MEMS) [10], which measures middleware architectures by rating multiple quality attributes. MEMS was initially designed for comparing several architectures of Grid applications. The output of Architecture evaluation methods provide general guidelines to assess quality attributes of systems, which are not necessarily straightfo