The Good and Bad of Software Metrics

Jason Pugh

Devspace Conference 2016

”When you can measure what you are speaking about, and can express it in numbers, you know something about it; but when you cannot measure it, when you cannot express it in numbers, your knowledge is of a meager and unsatisfactory kind: It may be the beginning of knowledge, but you have scarcely in your thoughts advanced to the stage of science.” -Lord Kelvin

DevSpace would like to thank our sponsors!

Agenda

Introduction
Code Scanned
Definition
Why Software Design Metrics
The Bad
The Good
Process Integration
Products
Conclusion

Introduction

Jason Pugh
- COLSA Corporation
- Zenoware
- UAH

Code Scanned

MongoDB .NET Driver
- GitHub
- C#

What are Metrics?

Type of objective scoring that is used to quantitatively measure aspects of software
More detailed results === increased efficiency && productivity

Software Design Metrics

Why Software Design Metrics

Lower Maintenance
Faster Throughput
Easier Estimation

Useful Metrics

There is NOT one metric to rule them all!!
- Combine useful metrics your TEAM finds useful
- COLSA’s Approach:
  - Complexity
  - Cohesion
  - Coupling

Complexity - Weighted Method per Class

where ci is the complexity of a method
Viewpoints:
- Time and Effort required to develop and maintain the class
- greater potential impact on children
- limits the possibility of reuse

Chidamber, S.R.; Kemerer, C.F.; , "A metrics suite for object oriented design," Software Engineering, IEEE Transactions on , vol.20, no.6, pp.476-493, Jun 1994

L. H. Etzkorn et al., “A comparison of cohesion metrics for object-oriented systems,” Information and Software Technology, vol. 46, no. 10, pp. 677-687, Aug. 2004.

\sum_{i=1}^n c_{i}

\sum_{i=1}^n c_{i}

Complexity - Weighted Method per Class

wmc = 0
for ctor in c.Constructors:
    if(calculateComplexity == True):
        wmc += ctor.DecisionsCount + 1
    else:
        wmc += 1
for m in c.Methods:
    if(calculateComplexity == True):
        wmc += m.DecisionsCount + 1
    else:
        wmc += 1

Complexity - Weighted Method per Class

/// <summary>
/// Checks whether a given collection name is valid in this database.
/// </summary>
/// <param name="collectionName">The collection name.</param>
/// <param name="message">An error message if the collection name is not valid.</param>
/// <returns>True if the collection name is valid; otherwise, false.</returns>
public virtual bool IsCollectionNameValid(string collectionName, out string message)
{
    if (collectionName == null)
    {
        throw new ArgumentNullException("collectionName");
    }

    if (collectionName == "")
    {
        message = "Collection name cannot be empty.";
        return false;
    }

    if (collectionName.IndexOf('\0') != -1)
    {
        message = "Collection name cannot contain null characters.";
        return false;
    }

    if (Encoding.UTF8.GetBytes(collectionName).Length > 121)
    {
        message = "Collection name cannot exceed 121 bytes (after encoding to UTF-8).";
        return false;
    }

    message = null;
    return true;
}

Complexity - Weighted Method per Class

/// <summary>
/// Checks whether a given collection name is valid in this database.
/// </summary>
/// <param name="collectionName">The collection name.</param>
/// <param name="message">An error message if the collection name is not valid.</param>
/// <returns>True if the collection name is valid; otherwise, false.</returns>
public virtual bool IsCollectionNameValid(string collectionName, out string message)
{
    if (collectionName == null)
    {
        throw new ArgumentNullException("collectionName");
    }

    if (collectionName == "")
    {
        message = "Collection name cannot be empty.";
        return false;
    }

    if (collectionName.IndexOf('\0') != -1)
    {
        message = "Collection name cannot contain null characters.";
        return false;
    }

    if (Encoding.UTF8.GetBytes(collectionName).Length > 121)
    {
        message = "Collection name cannot exceed 121 bytes (after encoding to UTF-8).";
        return false;
    }

    message = null;
    return true;
}

Total WMC = 5

Coupling - Direct Class Coupling

Where cpi is unique custom parameters for a method, cfi is unique custom fields in a class, and mcri is unique custom method return types
Viewpoints:
- Higher coupling reduces reuse
- High dependency on other objects

J. Bansiya and C. Davis, “A hierarchical model for object-oriented design quality assessment,” Software Engineering, IEEE Transactions on, vol. 28, no. 1, pp. 4-17, 2002.

\sum_{i=1}^n cp_{i}+cf_{i}+cmr_{i}

\sum_{i=1}^n cp_{i}+cf_{i}+cmr_{i}

Coupling - Direct Class Coupling

  dcc = 0
  types = []
  for a in c.Fields:
    if((a.Type.Name in types) == False and (a.Type.IsUserDefined == True)):
       types.append(a.Type.Name)
  
  for m in c.Methods:
    for p in m.Parameters:
      if((p.Type.Name in types) == False and (p.Type.IsUserDefined == True)):
        types.append(p.Type.Name)
  
    if((m.ReturnType.Name in types) == False and m.ReturnType.IsUserDefined == True):
        types.append(m.ReturnType.Name)
  
  dcc = len(types)

Coupling - Direct Class Coupling

/// <summary>
/// Runs an aggregate command with explain set and returns the explain result.
/// </summary>
/// <param name="args">The args.</param>
/// <returns>The explain result.</returns>
public virtual CommandResult AggregateExplain(AggregateArgs args)
{
    var messageEncoderSettings = GetMessageEncoderSettings();
    var operation = new AggregateExplainOperation(_collectionNamespace, 
                        args.Pipeline, 
                        messageEncoderSettings)
    {
        AllowDiskUse = args.AllowDiskUse,
        Collation = args.Collation,
        MaxTime = args.MaxTime
    };
    var response = ExecuteReadOperation(operation);
    return new CommandResult(response);
}

Coupling - Direct Class Coupling

/// <summary>
/// Runs an aggregate command with explain set and returns the explain result.
/// </summary>
/// <param name="args">The args.</param>
/// <returns>The explain result.</returns>
public virtual CommandResult AggregateExplain(AggregateArgs args)
{
    var messageEncoderSettings = GetMessageEncoderSettings();
    var operation = new AggregateExplainOperation(_collectionNamespace, 
                        args.Pipeline, 
                        messageEncoderSettings)
    {
        AllowDiskUse = args.AllowDiskUse,
        Collation = args.Collation,
        MaxTime = args.MaxTime
    };
    var response = ExecuteReadOperation(operation);
    return new CommandResult(response);
}

Both are custom types, therefore, DCC is 2

Cohesion - Lack of Cohesion in Methods

instance variables used in a method

\big\{ I_{j} \big\}

\big\{ I_{j} \big\}

P = \big\{ \big( I_{i}, I_{j} \big) \mid I_{i} \cap I_{j} = \oslash \big\}

P = \big\{ \big( I_{i}, I_{j} \big) \mid I_{i} \cap I_{j} = \oslash \big\}

Q = \big\{ \big( I_{i}, I_{j} \big) \mid I_{i} \cap I_{j} \neq \oslash \big\}

Q = \big\{ \big( I_{i}, I_{j} \big) \mid I_{i} \cap I_{j} \neq \oslash \big\}

LCOM =\begin{cases}\mid P \mid - \mid Q \mid && \mid P \mid > \mid Q \mid \\ \oslash && otherwise\end{cases}

LCOM =\begin{cases}\mid P \mid - \mid Q \mid && \mid P \mid > \mid Q \mid \\ \oslash && otherwise\end{cases}

Chidamber, S.R.; Kemerer, C.F.; , "A metrics suite for object oriented design," Software Engineering, IEEE Transactions on , vol.20, no.6, pp.476-493, Jun 1994

L. H. Etzkorn et al., “A comparison of cohesion metrics for object-oriented systems,” Information and Software Technology, vol. 46, no. 10, pp. 677-687, Aug. 2004.

Cohesion - Lack of Cohesion in Methods

  lcom = 0
  I = [] # An empty list of sets 
  i = 0 # A counter
  
  p = 0  # The cardinality of set P
  q = 0  # The cardinality of set Q
  N = 0  # The number of methods in class C
  
  N = c.Methods.Count

  for m in c.Methods:
    methodSet = set() # empty set, initially
    
    # Determine which of the accessed variables 
    # are instance variables of the current class.  
    for v in m.AccessedVariables:
      tempSet = set()
      tempSet.add(v)
      if(c.IsInstanceVariable(v) == True):
        methodSet = methodSet | tempSet

    # Keep the sets of accessed instance variables around. We will
    # need them for the next step of the calculation.
    I.append(methodSet)  

  for i in range(0, N):
    for j in range(0, N):
      intersection = I[i] & I[j]
      if(len(intersection) == 0):  # i.e. is the empty set
        p += 1
      else:
        q += 1

  # 
  # Now p is the number of pairs of methods which don't have  
  # have any instance variable accesses in common.
  # 
  # q is the number of pairs of methods which DO have an instance
  # variable access in common
  # 
  if(p > q):
    lcom = p - q
  else:
    lcom = 0

Cohesion - Lack of Cohesion in Methods

private readonly CollectionNamespace _collectionNamespace;

/// <summary>
/// Runs an aggregate command with explain set and returns the explain result.
/// </summary>
/// <param name="args">The args.</param>
/// <returns>The explain result.</returns>
public virtual CommandResult AggregateExplain(AggregateArgs args)
{
    var messageEncoderSettings = GetMessageEncoderSettings();
    var operation = new AggregateExplainOperation(_collectionNamespace, 
                        args.Pipeline, 
                        messageEncoderSettings)
    {
        AllowDiskUse = args.AllowDiskUse,
        Collation = args.Collation,
        MaxTime = args.MaxTime
    };
    var response = ExecuteReadOperation(operation);
    return new CommandResult(response);
}

Cohesion - Lack of Cohesion in Methods

private readonly CollectionNamespace _collectionNamespace;

/// <summary>
/// Runs an aggregate command with explain set and returns the explain result.
/// </summary>
/// <param name="args">The args.</param>
/// <returns>The explain result.</returns>
public virtual CommandResult AggregateExplain(AggregateArgs args)
{
    var messageEncoderSettings = GetMessageEncoderSettings();
    var operation = new AggregateExplainOperation(_collectionNamespace, 
                        args.Pipeline, 
                        messageEncoderSettings)
    {
        AllowDiskUse = args.AllowDiskUse,
        Collation = args.Collation,
        MaxTime = args.MaxTime
    };
    var response = ExecuteReadOperation(operation);
    return new CommandResult(response);
}

Instance Variable _collectionNamespace

- therefore p > q : LCOM is 1

The Good

"You can't control what you can't measure"

Tom DeMarco

We need some way to measure and know when things need to be reconsidered/refactored

Define Useful Metrics

What Language are you using?
What are important aspects of the language?

Complexity: 0 - 49 is Good, 50-99 is Potential Issues, and >= 100 is Concerning
Cohesion: 0 - 499 is Good, 500-899 is Potential Issues, and >= 900 is Concerning
Coupling: 0 - 11 is Good, 12-39 is Potential Issues, and >= 40 is Concerning

MongoDB Scanned Results - The Good

The Bad

Why design metrics are considered bad:
- Incorrect usage of metrics
  - Using just one metric for a decision.
  - Using too many metrics for a decision.
  - Using a single threshold for separate projects and not adapting.

The Bad - Simple Example

Sandy, Joe and Susan run in a race. Sandy comes in first, Joe second, and Susan third.
- We assign Sandy the number 1 for first place and give her $10,000
- We assign Joe the number 2 and give him $1,000
- We assign Susan the number 3 and give her $100

We assigned the numbers according to a rule.

Questions
- Is Sandy twice as fast as Joe and three times as fast as Susan?
- Is Sandy 10 times as fast as Joe and 100 times as fast as Susan?
- Isn’t the assignment of the numbers based on their speed?
- Did we measure their speed or not?
  
  Kaner, C. “Yes, But What Are We Measuring?,” 1999 PNSQC
  
  Douglas Hoffman

The Bad - Using just One Metric

How to Interpret Metrics

Measurements are… MEASUERMENTS!!
Software is inherently difficult
Example: If a car is large, it has poor gas mileage. Is that true?
- It is a good observation, but not ALWAYS true...
- You have to LOOK at the specifications and inspect/research first

Metrics in Software Design Process

Architect:
- Scan Code and Documentation Monthly*
- Retrieve Metrics and Review
Project Manager:
- Review Architect Findings
- Assign to Team Lead
Team:
- Schedule Peer Reviews
- Schedule Refactor Task (IF NECESSARY)

Available Products

Understand
- https://scitools.com/
Cantata++
- http://www.qa-systems.com/tools/cantata
McCabe IQ
- http://www.mccabe.com/iq.htm
Coverity
- https://coverity.com/

Zenoware CSMS

Currently revamping CSMS (Custom Software Metric Suite)

SaaS and Atlassian Support – December Timeframe

Actively looking for Closed Beta Users

Special Thanks

DevSpace Conference
UAH and Dr. Letha Etzkorn
Zenoware Team Members
Kurt Lawson

The Good and Bad of Software Metrics

DevSpace would like to thank our sponsors!

Agenda

Introduction

Code Scanned

What are Metrics?

Software Design Metrics

Software Design Metrics

Why Software Design Metrics

Useful Metrics

Complexity - Weighted Method per Class

Complexity - Weighted Method per Class

Complexity - Weighted Method per Class

Complexity - Weighted Method per Class

Coupling - Direct Class Coupling

Coupling - Direct Class Coupling

Coupling - Direct Class Coupling

Coupling - Direct Class Coupling

Cohesion - Lack of Cohesion in Methods

Cohesion - Lack of Cohesion in Methods

Cohesion - Lack of Cohesion in Methods

Cohesion - Lack of Cohesion in Methods

The Good

Define Useful Metrics

MongoDB Scanned Results - The Good

The Bad

The Bad - Simple Example

The Bad - Using just One Metric

How to Interpret Metrics

Metrics in Software Design Process

Available Products

Zenoware CSMS

Special Thanks

Conclusion