Research>Metrics
Software Engineering Metrics Research at University of Maryland
General Information on Research:
This research
concentrates on the establishment of the relationship between the software
metrics and software reliability. Software engineering metrics relate to
multiple aspects of the software development process and product. Software
development organizations typically select a small number of such software
engineering measures to manage their development processes and products.
The basic idea behind our research is to help
identify the software engineering metrics that are best predictors of
software reliability. A structural classification scheme that helps in
more objectively assessing the intrinsic and extrinsic characteristics of
the measure is introduced. A graphical method is described for the purpose
of structural representation. The creation of this graphical
representation is shown to be another valuable tool in the analysis of a
software engineering measure. The concept of reliability prediction system
is introduced to classify software engineering measures into sets. Each
such set exclusively contains measures from which software reliability can
be predicted. A ranking methodology is also presented to be used to select
the best reliability prediction system with respect to software
reliability prediction.
Publications :
-
Smidts C., Li M., “Software Engineering Measures for Predicting Software Reliability in Safety Critical Digital Systems”, prepared for the U.S. Nuclear Regulatory Commission, NUREG/GR-0019, UMD-RE-2000-23, U.S. Nuclear Regulatory Commission, Office of Nuclear Regulatory Research (2000).
This report
presents the University of Maryland (UMD) research to identify measures
and families for the prediction and assessment of the reliability of
software-based digital systems.
A set of software engineering measures from
which the potential reliability of a digital I&C system can be
predicted is developed from a set of 30 pre-selected software engineering
measures. These measures are derived from a pool of 78 software
engineering measures identified by Lawrence Livermore National Laboratory
(LLNL). The concepts of structural classification, software development
life-cycle classification, and family are presented. These 30 measures are
categorized using these concepts. The concept of RPS and an extended
structural representation are introduced to bridge the gap between
software engineering measures and reliability. Expert opinion is elicited
as the input in ranking the pre-selected 30 measures in terms of software
reliability prediction. 10 missing measures are identified and ranked. The
potential impact of these 10 missing measures on the ranking of the
pre-selected 30 measures is analyzed. The top-ranked measures and families
are presented in this report. Use of the families of measures in each
software development phase can lead to a quantitative prediction of
software reliability.
This study is the first step towards a
systematic approach predicting the reliability of a real-time I&C
software using RPSs established from the top-ranked measures and families.
However, current knowledge prevents the quantitative estimation of the
accuracy of such prediction. Further experiments are required to
investigate the quantitative reliability as a function of the RPS
measures.
This report was prepared as an account of
work sponsored by under a Cooperative Agreement, NRC-RES 98-056, with the
US Nuclear Regulatory Commission an agency of the United States
Government. Neither the United States Government or any agency thereof,
nor any of its employees, makes any warranty, expressed or implied, or
assumes any legal liability or responsibility for any third party's use,
or the results of such use, of any information, apparatus, product, or
process disclosed in this report, or represents that its use by such a
third party would not infringe privately owned rights. The views expressed
in this paper are those of the authors and should not be construed to
reflect the U. S. Nuclear Regulatory Commission position.
The opinions and viewpoints expressed herein are the authors' personal ones and do not necessarily reflect the criteria, requirements, and guidelines of the Nuclear Regulatory Commission.
-
Li M. Smidts C., Ranking Software Engineering Measures Related to Reliability Using Expert Opinion, Proceedings of ISSRE, p. 246 – 258, San Jose, California, 8-11October, 2000.
The
field of software engineering measurement appears to the unfamiliar eye as
a chaotic environment lacking unifying principles and rigor. The number of
software engineering measures developed over the years is stupefying and
keeps increasing.
Software engineering measures
relate to multiple aspects of the software development process and
product. Software development organizations typically select a small
number of such software engineering measures to manage their development
processes and products.
The research presented in
this paper is an attempt to help software development organizations
identify the software engineering measures that are best predictors of
software reliability. The current research is based on the top thirty
measures identified in an earlier study carried out by Lawrence Livermore
National Laboratory. The set of ranking criteria was modified to fit the
needs of the study. The score of each measure for each ranking criterion
was elicited through expert opinion and then aggregated into a single
score using multi-attribute utility theory. The basic aggregation scheme
selected was a linear additive scheme. A comprehensive sensitivity
analysis was carried out. The sensitivity analysis included: variation of
levels, variation of weights, variation of aggregation schemes.
-
Smidts C., Li M., From Measures to Reliability, Proceedings of NPIC & HMIT, Washington D.C., November 13-17, 2000.
Software-based
digital systems are progressively replacing analog systems in
safety-critical applications like nuclear power plants. Although they must
satisfy more stringent reliability requirements than the old analog
systems, their behavior is still not well understood and should be studied
thoroughly. In particular it is essential to quantify their failure
behavior through software reliability prediction. Prediction can be
performed using diverse sources of information. One possible source of
information is “software engineering measures”, which address multiple
aspects of the software development process and of the product itself.
Software development organizations typically
elect to select a small number of software engineering measures by which
to manage, predict, and assess the quality of their development processes
and products. The question we attempt to address is whether these measures
are suitable for the prediction of software reliability, and if so, to
what extent. This question necessitates assessing both the intrinsic
characteristics of the measure (how good the measure is by itself, the
measure’s purpose, how much it costs) as well as its extrinsic
characteristics (how it relates to reliability).
This paper introduces a structural
classification scheme that helps in more objectively assessing the
intrinsic and extrinsic characteristics of the measure. A graphical method
is described for the purpose of structural representation. The creation of
this graphical representation is shown to be another valuable tool in the
analysis of a software engineering measure.
-
Li M., Smidts C., Ranking Software Engineering Measures Related to Reliability Using Expert Opinion, Proceedings of PSAM5, vol. 4, p. 2675 – 2680, Osaka, Japan, Nov. 27 – Dec. 1, 2000.
The field of
software engineering measurement appears to the unfamiliar eye as a
chaotic environment lacking unifying principles and rigor. The number of
software engineering measures developed over the years is stupefying and
keeps increasing.
Software engineering measures relate to
multiple aspects of the software development process and product. For
instance, one finds measures associated to estimation and/or prediction of
cost and schedule, measures characterizing organizations, staff, number of
lines in a software, logical complexity of a module, application domain,
etc. Software development organizations typically elect to select a small
number of such software engineering measures to manage their development
processes and products.The research presented in this paper is an attempt
to help software development organizations identify the software
engineering measures, which are best predictors of software reliability.
Last Updated Wednesday, December 19, 2001
©2001 University of Maryland (UMD). All rights reserved
Best viewed by IE 5,5 and 800*600 resolution.