Software Metrics 3a4i4n

SOFTWARE METRICS BY MAZWIN ABDUL WAHAB NUUROL FARHANA MOHD ZAKI NURAIN AMRAN

Software Metrics Topics            

Objectives Definitions The Scope of Software Metrics Measurement Metric Classification Types of Measures Metrics and Software Quality Measures of Sofware Quality 12 Steps to Useful Sofware Metrics Tools Improvements/Motivation of Metrics Issues

Objectives

To describe the current state-of-the-art in the measurement of software products and process in order to provide a quality product.

Definitions Measure - Quantitative indication of extent, amount, dimension, capacity, or size of some attribute of a product or process. E.g., Number of errors

Metrics - Quantifiable measures that could be used to measure characteristics of a software system or the software development process. E.g., Number of errors found per person hours expended

The Scope of Software Metrics  Software metrics is a term that embraces many

activities, all of which involve some degree of software measurement:      

cost and effort estimation productivity measures and models data collection quality models and measures reliability models performance evaluation and model

The Scope of Software Metrics

 Structural and complexity metrics  Capability-maturity assessment  Management by metrics

 Evaluation of methods and tools

Examples of Entities and Attributes Measurement is the process by which numbers or symbols are assigned to attributes of entities in the real world in such a way as to describe them according to clearly defined unambiguous rules

Entity

Attribute

Software Design

Defects discovered in design reviews

Software Design Specification Number of pages

Software Code

Number of lines of code, number of operations

Software Development Team

Team size, average team experience

Measurement 1. WHY MEASURES? "When you can measure what you are speaking about and express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: it may be the beginnings of knowledge but you have scarcely in your thoughts advanced to the stage of Science." Lord Kelvin (Physicist) "You cannot control what you cannot measure." Tom DeMarco (Software Engineer)

Measurement Measures is really important because...     

Quality of deliverables Timely delivery Cost Need quantitative data Continuous improvement

3W Measures 2. WHAT TO MEASURES ?   

Project management metrics Quality metrics Cost metrics

Types of Measures 

Direct Measures (internal attributes)     

Cost Effort LOC Speed memory

Types of Measures 

Indirect Measures (external attributes)      

Functionality Quality Complexity Efficiency Reliability Maintainability

Metric Classifications 

Products (Output)   



Processes  



Explicit results of software development activities Deliverables, documentation, by products Quantify characteristics of the product being developed Activities related to production of software Quantify characteristics of the process being used to develop the software

Resources (Input)  

Inputs into the software development activities hardware, knowledge, people

Types of Metric  Nominal

eg: no ordering, simply attachment of labels (language: 3GL, 4GL)  Ordinal

eg: ordering, but no quantitative comparison (programmer capability: low, average, high)  Interval

eg :between certain values (programmer capability: between 55th and 75th percentile of the population ability)

Types of Metric  Ratio

eg: (the proposed software is twice as big as the software that has just been completed)  Absolute

eg: the software is 350,000 lines of code long

Metrics and Software Quality FURPS     

Functionality - features of system Usability – aesthesis, documentation Reliability – frequency of failure, security Performance – speed, throughput ability – maintainability

Measures of Software Quality 







Correctness – degree to which a program operates according to specification  Defects/KLOC  Defect is a verified lack of conformance to requirements  Failures/hours of operation Maintainability – degree to which a program is open to change  Mean time to change  Change request to new version (Analyze, design etc)  Cost to correct Integrity - degree to which a program is resistant to outside attack  Fault tolerance, security & threats

Usability – easiness to use  Training time, skill level necessary to use, Increase in productivity, subjective questionnaire or controlled experiment

12 Steps to Software Metrics Step 1 - Identify Metrics Customers Step 2 - Target Goals Step 3 - Ask Questions Step 4 - Select Metrics Step 5 - Standardize Definitions Step 6 - Choose a Model Step 7 - Establish Counting Criteria Step 8 - Decide On Decision Criteria Step 9 - Define Reporting Mechanisms Step 10 - Determine Additional Qualifiers Step 11 - Collect Data Step 12 - Consider Human Factors

Step 1:Identify Metrics Customer

Who needs the informations?  Who's going to use the metrics? 

SStep 2:Target Goals  Organizational goals

-Be the low cost provider -Meet projected revenue targets  Project goals

-Deliver the product by specified date

-Finish the project within budget  Task goals (entry & exit criteria)

-Effectively inspect software module ABC

-Obtain 100% statement coverage during testing

Step 3: Ask Questions “Apply GQM (Goal-Question-Metric)” 1. Set Goal Goal: Maintain a high level of customer satisfaction

2. Derive QUESTIONS from the GOAL -What is our current level of customer satisfaction? -What attributes of our products and services are most important to our customers? -How do we compare with our competition?

GQM • A reminder on generating measurements: –

In coming up with metrics think of GQM • • •

– – –

What’s the Goal What’s the Question What’s the Metric

Goal: is Improved Quality Question: What is the Post Release Defect Rate? Metric: Number of problems found per months

Step 4: Select Metrics 





Identify problems from the basic metrics -High fault rates during coding phase Introduce strategy to correct the problems -Select metrics that provide information to help answer the questions -Be practical, realistic, pragmatic -Consider current engineering environment -Start with the possible To monitor success, collect more detailed metrics -Fault rates of individual programmers

Step 5:Standardize Definition 







Select definitions from the literature that match your organizational goals Use them as a basis for creating your own definitions Apply them consistently

Include them in an appendix to each metrics report

Step 6:Choose Model Models for code inspection metrics Metric Primitive Models: Lines of Code Inspected = loc Hours Spent Preparing = prep_hrs Hours Spent Inspecting = in_hrs Discovered Defects = defects

Other Metric Models: Preparation Rate = loc / prep_hrs Inspection Rate = loc / in_hrs Defect Detection Rate = defects / (prep_hrs + in_hrs)

Step 7:Establish Counting Criteria

Lines of Code  Variations in counting  No industry accepted standard  SEI guideline - check sheets for criteria  Advice: use a tool

Counting Criteria-Effort

Step 8:Decide on Decision Criteria Establish Baselines  Current value -Problem report backlog -Defect prone modules  Statistical analysis (mean & distribution) -Defect density -Fix response time -Cycle time -Variance from budget (e.g., cost, schedule)

Step 9:Define Reporting Mechanism 100

O pen Fixed R esolved Jan-97 23 13 3 Feb-97 27 24 11 M ar-97 18 26 15 Apr-97 12 18 27

80 60 40 20 0 1st Qtr

100

160

2nd Qtr

3rd Qtr

4th Qtr

80 120

60 40

80

20 40

0 Jan

0 1st Qtr

2nd Qtr

3rd Qtr

Mar

May

July

4th Qtr

120

80

40

0 0

20

40

60

80

100

120 1

2

3

4

5

6

7

8

9

10

11

12

Step 10:Determine Additional Qualifiers A good metric is a generic metric  Additional qualifiers:

-Provide demographic information -Allow detailed analysis at multiple levels -Define additional data requirements

Additional Qualifier Example Metric: software defect arrival rate • Release / product / product line • Module / program / subsystem • Reporting customer / customer group • Root cause • Phase found / phase introduced • Severity

Step 11: Collect Data What data to collect?  Metric primitives  Additional qualifiers Who should collect the data?  The data owner  Direct access to source of data  Responsible for generating data  Owners more likely to detect anomalies  Eliminates double data entry

Step 12:Consider Human Factors The People Side of the Metrics Equation -How measures affect people ? -How people affect measures ? “Don’t underestimate the intelligence of your engineers. For any one metric you can come up with, they will find at least two ways to beat it.” [unknown]

Motivation of Metrics 

Estimate the cost & schedule of future projects

 Evaluate the productivity impacts of new tools and

techniques

 Establish productivity trends over time  Improve software quality  Forecast future staffing needs  Anticipate and reduce future maintenance needs

Do and Don’t

Don’t…  Measure individuals  Use metrics as a “stick”  Use only one metric  Ignore the data

Do and Don’t

Do…  Select metrics based on goals  Provide  Focus on processes, products & services  Obtain “buy-in”

Tools CCCC • A metric analyser C, C++, Java, Ada-83, and Ada-95 (by Tim

Littlefair of Edith Cowan University, Australia)

• Metrics calculated

Lines Of Code (LOC) – McCabe’s cyclomatic complexity – C&K suite (WMC, NOC, DIT, CBO) • Generates HTML and XML reports • freely available –

•

http://cccc.sourceforge.net/

Tools Jmetric • OO metric calculation tool for Java code (by Cain and Vasa for a

project at COTAR, Australia) • Requires Java 1.2 (or JDK 1.1.6 with special extensions) • Metrics – Lines Of Code per class (LOC) – Cyclomatic complexity – LCOM (by Henderson-Seller) • Availability – is distributed under GPL •

http://www.it.swin.edu.au/projects/jmetric

Tools GEN++ •

GEN++ is an application-generator for creating code analyzers for C++ programs – – –

• •

simplifies the task of creating analysis tools for the C++ several tools have been created with GEN++, and come with the package these can both be used directly, and as a springboard for other applications

Freely available http://www.cs.ucdavis.edu/~devanbu/genp/down-red.html

Tools A Source of Information for Mission Critical Software Systems, Management Processes, and Strategies http://www.niwotridge.com/Resources/PM-SWEResources/SWTools.htm •

Defense Software Collaborators (by DACS) http://www.thedacs.com/databases/url/key.hts?keycode=3 http://www.qucis.queensu.ca/Software-Engineering/toolcat.html#label208 •

Object-oriented metrics http://me.in-berlin.de/~socrates/oo_metrics.html • Software Metrics Sites on the Web (Thomas Fetcke) •

Metrics tools for C/C++ (Christofer Lott) http://www.chris-lott.org/resources/cmetrics/ •