ABOUT US

About your professor...

• Working for USJ part-time for several years
• Works for Telefónica R&D
• Previously Head of Engineering for FirefoxOS
• Currently CTO of the Data Transparency Lab (http://datatransparencylab.org/) 
• ... and Lead Innovation Team in Telefónica AURA
• Love Software Development, in particular, anything related to how different aspects affect Software Quality
• Web Technologies are key to me: JavaScript, HTML, CSS...
• Privacy and Transparency advocate: Data + AI is powerful and requires responsible use
• On a personal part... 3 kids, love chess and music

About you?

• Programming Languages?
• Methodologies
• Tools: Trello, Bugzilla, Redmine, SVN, Git, etc..
• Anything else?

and...

WHY DO YOU THINK SOFTWARE QUALITY IS IMPORTANT?

(WE ARE GOING TO SPEND 2 HOURS PER WEEK DURING FOUR 4 MONTHS TALKING ABOUT IT)

LECTURER NOTES

(GUÍA DOCENTE)

CONTENT

Unit 1

Introduction to Software Quality

Unit 2

Software Quality Metrics

Unit 3

Software Configuration Managment

Unit 4

Testing

Unit 5

QA Activities beyond Testing

Methodology?

Online slides to guide the class (like these ones)

TEACHING NOTES

http://dcoloma.github.io/software-quality/all.html#introduction-to-software-quality

WITH MORE DETAILS SO YOU CAN REVIEW IT OFFLINE

Open Discussions

PLEASE DO INTERRUPT ME FREQUENTLY

I'D PREFER TO TALK ABOUT ANYTHING THAT WORRIES YOU THAT ABOUT WHAT WORRIES ME

Practice based as much as possible

BUT

BEAR IN MIND THAT THERE IS SOME THEORETICAL AND EVEN PHILOSOPHICAL PARTS WE CAN'T AVOID

NEED TO FOLLOW THE USJ ATTENDANCE RULES

NEED TO WORK AT HOME!

NOT TO MEMORIZE BUT TO LEARN AND BE UP-TO-DATE

MANDATORY TASKS

RESOLVE OPTIONAL EXERCISES AND MATERIAL

Evaluation?

FINAL MARK CALCULATION

ABOUT EXAMS

1 INTERMEDIATE TEST (UNITS 1 AND 2) - 10%

1 FINAL TEST (CONTINUOUS EVALUATION) - 30%

AVERAGING REQUIRES AT LEAST 40/100

YOU CAN BRING ALL THE MATERIAL WITH YOU, YOU CAN USE YOUR COMPUTERS, ETC.

BUT REMEMBER

INTRODUCTION

Main Objective

Learn how to deliver high-quality software

Let's break it down into small objectives:

1. What is high-quality software? What is quality?
2. How can we measure software quality (or related aspects)?
3. What activities can we do to improve software quality? Testing, Code Reviews, Failure Containment, Defect Prevention...
4. What activities can we do to improve how we build software? Source Code Management, Continuous Integration, Delivery...

Learn by example

How does a company that delivers great software products work?

SPOTIFY

Let's watch a couple of videos and think about all the activities they do to improve software quality

SO MANY THINGS TO LEARN...

• Release Train / Feature Toggle
• Failure Recovery vs. Failure Avoidance
• Issue is closed when learned from it
• Incident: Fix the process not only the product
• Limited Blast Radius
• Gradual Rollout
• Analyze Data
• Data Driven Decision
• Change whatever is needed
• Fail fast

WHY SOFTWARE QUALITY MATTERS?

Software Crisis (I)

The complexity of problems to be solved via software have outpaced the improvements in software development... Nowadays we're even talking about Artificial Intelligence

Software Crisis (II)

The demand of highly skilled software developers have surpassed the offer of good development resources

LIABILITY OF A BUG

LEGAL ASPECTS

TERMS AND CONDITIONS FROM A MICROSOFT PRODUCT

HAVING GOOD LAWYERS IS NOT ENOUGH

AT&T Bug

"The Jan. 1990 incident showed how bugs in self-healing software can bring down healthy systems, and the difficulty of detecting obscure load- and time-dependent defects in software."

1990: A mis-placed "break" clause in the code caused 50% of the AT&T calls fail during 9 hours ($60 million)

DIRECT ECONOMIC IMPACT

THE CODE

FACEBOOK BUG

RISK OF GDPR FINES AND REPUTATION

What is Software Quality?

What is Quality?

Easy to identify intuitively but difficult to explain, define or measure

... and highly influenced by the environment

What is quality for you?

Think about some examples of Good/Bad quality products (software and non-software)

Quality Definition

The standard of something as measured against other things of a similar kind;

The degree of excellence of something.

Quality of an essay?

Usually you know which one you like the most but it's difficult to explain why

Situation determines quality?

Think about having the same meal in a nice place, with nice silvers, fantastic service or having in an awful place with plastic fork, lame service...

Would you perceive the same meal quality?

Measuring Poetry?

Coding = Poetry?

Give ten novelists a plot summary and they will each write you a different story. Give ten programmers a functional specification and each one will produce something unique, an expression of their own voice as a programmer.

Software is poetry. It’s the expression of ideas in the most elegant form a programmer can devise. It is a creative act!

Quality in Software

But we are going to study metrics in Unit 2?

YES, but metrics do not measure directly quality, they can be correlated to it

Software Quality: View1

Formal Definition

Conformance to explicitly stated functional and performance requirements, explicitly documented development standards and implicit characteristics that are expected from all professionally developed software"

1. Software Requirements
2. Specified Development Standards
3. Implicit Requirements

Can you think about examples of any of these 3 types of requirements/standards ?

- + ?

+ - ?

complexity to check conformance?

https://engineering.naukri.com/2016/05/web-mobile-implicit-requirements/

Software Quality: View2

Human point of view

Product Quality is...

Implicit requirements... there is something more beyond what is explicitly required

USERS AND THEIR EXPECTATIONS

How much it changes the world for better

or

... the value to some person

The degree it meets:

- Specified Requirements

- User needs or expectations

INFORMALLY

FORMALLY (IEEE)

Are all of you going to perceive exactly the same quality in the same product?

As engineers you should not only care about requirements.

What else you should be thinking about?

Software Quality: View3

Internal vs External Quality

Does the software do what is supposed to?

Conformance

Reliability

Accuracy

Robustness

Usability

Is the software implemented in the way it was supposed to be?

Conformance

Standards

Best Practices

Some properties that are linked to internal quality are:

• Concision: the code just does what is supposed to be doing
• Cohesion: each module devoted to one purpose
• Low Coupling: reduce dependencies reduce error propagation
• Simplicity: Always as simple as possible so it's less error-prone
• Generality: use general solutions vs. specific ones ease maintenance
• Clarity: so it's easy to be understood and maintained

Sometimes external problems are symptoms of internal problems. This is specially true when the software is evolving and the need to change it is affected by the poor internal quality.

https://www.gridshore.nl/2008/03/29/how-wtfs-improve-code-quality-awareness/

But if our target is making software for end-users, should we care about internal quality?

http://blog.codinghorror.com/paying-down-your-technical-debt/

https://levelup.gitconnected.com/how-to-use-technical-debt-in-your-favor-98bae475ba68

IRRESPONSIBLE

INCOMPETENT

Software Quality: View4

ISO 9126

The Quality is determined by the Development Process, the Product itself and the usage of the Product

Software Quality: View4

ISO 9126

Does it work with the required performance for a period of time?

Does it work as expected?

Is it easy to use?

Does it  just use the required amount of resources but no more?

Is it easy to make modifications to the software?

Can it be re-used to other environments?

Are these all characteristics equally important for any product?

Summary:

Not a single definition of quality!

Quality of a product is extremely subjective and affected by the process for implementing it.

External quality is essential, but internal quality might be even more important as it affects software in the long-term and it's more difficult to be measured

Different products have different purposes, and hence, the key aspects for determining their quality should be different

KEY DEFINITIONS

KEY CONCEPTS

• Fault / Bug / Defect: Mistake in the software. I.e. something the developer did wrong.

• Error Situation: System state in which the system was not expected to be. I.e. it occurs when a buggy code is executed.

• Failure: Any deviation of the observed behaviour from the specified behaviour. I.e. something that the end-user notices.

Peter is driving his car towards Oxford. While he is driving, the road diverts into two different directions:
  1.  Left road to Oxford
  2.  Right road to Cambridge

By mistake, Peter takes the road to Cambridge. That is a fault that is committed by Peter.

Suddenly, Peter is in an error situation or state: Peter is heading 
Cambridge and not Oxford.

If Peter goes on and arrives to Cambridge, that would be a failure: 
Peter was planning to get to Oxford but he has arrived to Cambridge instead.

If Peter realizes of the error situation while he is driving Cambridge, returns to the junction and takes the right road to Oxford no failure would happen as Peter recovers from the error condition.

A REAL WORLD EXAMPLE

public static int numZero (int[] x) { 
  // effects: if x == null throw NullPointerException
  // else return the number of occurrences of 0 in x 
  int count = 0; 
  for (int i = 1; i < x.length; i ++) {
    if (x[i] == 0) { 
     count ++;
    }
  }
  return count;
}

A SOFTWARE EXAMPLE

Is there any bug?

In which circumstances will there be a failure?

public static int numZero (int[] x) { 
  // effects: if x == null throw NullPointerException
  // else return the number of occurrences of 0 in x 
  int count = 0; 
  for (int i = 1; i < x.length; i ++) {
    if (x[i] == 0) { 
     count ++;
    }
  }
  return count;
}

A SOFTWARE EXAMPLE

Is there any bug?

In which circumstances will there be a failure?

Function to count the number of Zeros in an array

Bugs

Error Situation

Failures

Source Code

Internal Exec Status

External Exec Status

The developer committed a mistake

... that led to a wrong situation

... which impacted the end-users

CAUSE - EFFECT

Some Examples

Therac-25

https://hackaday.com/2015/10/26/killed-by-a-machine-the-therac-25/

Therac-25

Therac-25 was a linear accelerator used in hospitals during the 80s to treat cancers. The machine had two possible configurations:

• Mode A: Radiate high energy on cells with cáncer without damaging the cells sourrounding them. There was no need to protect the patient in this mode.

• Mode B: Radiate X-rays with megavolts power that required filters and special protection to the patients.

People who operated the machine acquired great experience managing the machine and entering the command sequence to start the treatment, which they did very fast.

However, due to a programming bug, if during the process they made a sequence of operations in less than 8 seconds, the machine could use the wrong mode due to a race-condition. Because of this bug 5 people died and tens of them suffered the consequences of being exposed to a high radiation because treatment B was applied instead of mode A.

Zune Player

SOFTWARE QUALITY ASSURANCE

SQA TARGET

Improve Software Quality

What? Reducing two things:

#1 - Number of defects

#2 - defects' impact to users

How?

It depends! an answer typical from Spotify :-) way of working

Bugs

Failures

HOW?

Devs

What can we do to reduce #defects and their impact?

Error Situation

Prevent
Remove
Contain

Prevent

Actions intended to minimize the number of defects that are injected in the software by developers

Can you think about ways to prevent defects?

Remove

Actions intended to remove defects once they have been injected

Can you think about ways to remove defects?

Contain

Actions intended to avoid or minimise any impact defects have on end-users

Can you think about ways to contain defects?

Bugs

Failures

Devs

Defect Prevention

Fault Tolerance

Defect Removal

Error Situation

TAKING INTO ACCOUNT THE ECONOMIC IMPACT OF A BUG, WHICH OF THESE TYPE OF ACTIVITIES (PREVENT, REMOVE, CONTAIN) IS MORE OPTIMAL TO REDUCE THE ECONOMIC IMPACT OF A BUG?

COST OF QUALITY

AN OUNCE OF PREVENTION IS WORTH A POUND OF CURE

EXAMPLE: SAMSUNG GALAXY NOTE S7

How much had it cost if detected at an early stage?

Phones that exploded at the customers' hands: handling returns cost them $17billion + reputational impact

BUT THIS IS SIMPLISTIC APPROACH

• When was the defect injected? Was the explosion a design issue, a provider issue, etc.

• Are all the defects equally important? Explosion vs. a glitch

COST OF POOR QUALITY (COPQ)

SAMSUNG EXAMPLE

WHY DIDN'T SAMSUNG HAD THAT BATTERY CHECK SYSTEM BEFORE?

COST OF GOOD QUALITY

VERIFICATION VS. VALIDATION

VERIFICATION

Are we building the product right? I.e. does the product meet all the software requirements

VALIDATION

Are we building the right product? I.e. does the product meet the user needs

SOFTWARE QUALITY ENGINEERING

SQA = SOFTWARE QUALITY ASSURANCE

SQE = SOFTWARE QUALITY ENGINEERING

Techniques used to improve Software Quality: Testing, Code Reviews, Fault Tolerance, Defect Prevention, etc.

Processes that define how SQA Techniques are used in a software project. For instance:

• What is the Quality Target? E.g. Are we ready (quality-wise) to ship a product?
• What SQA activities should be used and when
• Are the SQA activities being executed right?
• Who should take decisions that affect quality

PLANNING

EXECUTION

ANALYSIS

EXAMPLE: AGILE

Some Observations

It's not lineal / sequential: We don't plan everything, execute all QA activities after and lastly assess all the results. Planning, execution and analysis is mixed

QUALITY PLANNING

DEFINE WHICH ACTIVITIES SHOULD BE EXECUTED

(HOW, WHEN, WHO...)

DEFINE QUALITY GOALS:

WHEN IS SOMETHING GOOD ENOUGH?

Target users, expectations, etc.

Metrics that can be helpful to correlate to user expectations 

Activities more helpful to achieve those metrics

Team skills and preferences

EXECUTION

THE EXECUTION OF ALL THE PLANNED QA ACTIVITIES

Execute Test Cases

Creation of Automatic Builds

Code Reviews

...

ANALYSIS

Assess if a user story is done or not

# Bugs detected per user story (per severity)

# User Stories Done

# Times regression tests failed

 QUALITY IMPROVEMENT PROCESS

Lot of bugs detected by QA Team

Review Acceptance Criteria in advance

Check how code reviews are done and do them in different ways

...

Releasing versions with bugs

Increase Regression Tests

Increase Number of Test Cases per User Story

Check how QA Engineers execute test cases

...

EXAMPLES

DEMING QUALITY CIRCLE

SOFTWARE QUALITY AND SOFTWARE ENGINEERING