DEFECT PREVENTION AND CODE REVIEWS
Bugs
Faults
Failures
Devs
Defect Prevention
Fault Tolerance
Defect Removal
REMEMBER OUR TARGET AS SOFTWARE ENGINEERS: REDUCE AND MINIMIZE IMPACT OF DEFECTS
Bugs
Devs
Defect Prevention
THIS SESSION IS ABOUT STUDYING A COUPLE OF TECHNIQUES RELATED WITH PREVENTING DEFECTS
- DEFECT PREVENTION: Identify key root causes behind defects and take actions to avoid engineers to repeat those type of defects again (e.g. training engineers, improving the process, etc.).
- CODE REVIEWS: Before merging any code in the upstream a review on the code must be performed. This helps to prevent some defects that otherwise would be part of the project.
TEACHING NOTES
1 - DEFECT PREVENTION
Why do you think it's important to prevent defects?
The best and cheapest way to reduce the number of defects is preventing them
"People underestimate the time they spend debugging. They underestimate how much time they can spend chasing a long bug. With testing, I know straight away when I added a bug. That lets me fix the bug immediately, before it can crawl off and hide. There are few things more frustrating or time wasting than debugging. Wouldn't it be a hell of a lot quicker if we just didn't create the bugs in the first place?"
Defect Causal Analysis (DCA)
- It's the most common way to implement a Defect Prevention Framework
-
It's composed mainly by two steps:
- Identify the causes of the defects
- Take actions to prevent those causes by improving the process
- In short: A systematic way to improve the process based on defects that have occurred in the past.
Defect Causal Analysis (DCA)
DEFECT IDENTIFICATION
DEFECT CLASSIFICATION
DEFECT
ANALYSIS
DEFECT PREVENTION
PROCESS IMPROVEMENT
Defects are found by QA activities specifically intended to detect defects such as Design review, Code Inspection, function and unit testing.
There are multiple ways and techniques to classify defects, for instance: Requirements, Design, Logical and Documentation.
These categories can be again divided in second and third levels depending on the complexity and size of the product.
After defects are logged and classified, the next step is to review and analyze them using root cause analysis (RCA) techniques.
Root-cause analysis is the process of finding the activity or process which causes the defects and find out ways of eliminating or reducing the effect of that by providing remedial measures. Defects are analyzed to determine their origins. A collection of such causes will help in doing the root cause analysis.
Once the causes of the defects are known it is key to identify actions that can be put in place to cut down these causes. This can be achieved, for intstance with meetings where all the possible causes are identified from the cause-and-effect diagram and debated among the team. All suggestions are listed and then the ones that are identified as the main reasons for causes are separated out. For these causes, possible preventive actions are discussed and finally agreed among project team members.
Once the preventive actions have been identified, they need to be put in place and verify their effectiveness, for instance by observing the Defect Density and comparing it with previous projects.
RCA Tools - Paretto Chart
Easy to find which type of issues occur the most and how can we solve % of the issues by focusing on some of them
- X-Axis items ordered by frequency
- Y-Axis:
- Left: Number of occurrences of the item
- Right: Cumulative % vs. total number of occurrences
In our case we show in the X-Axis the type of defects.
EXAMPLE
If we work on these type of defects we can address nearly 75% of the issues we are facing
RCA Tools - Fishbone Diagram
A graphical representation that sorts and relates factors that contribute to a given situation.
In the case of defects, it details the factors that could lead to that defect type.
- It should use the knowledge and experience of the team!
- It should be built in a way that helps implementing changes later on
A Paretto Chart gives the type of defect to focus in.
But we still need to understand why those type of defects happen -> Fishbone Diagram
EXAMPLE
DCA EXAMPLE: HP Software Failure Analysis
QUIZ ABOUT THE PAPER
ADDITIONAL READINGS
2 - CODE REVIEWS
Code Reviews: Intro
Why do you think a human being want to read others human being code?
"Any fool can write code that a computer can understand. Good programmers write code that humans can understand" (Martin Fowler)
Reasons to read code
- Learn how to solve a particular problem
- Learn how a programming language feature can be used
- Reuse the code
- Understand how a library you are relying on works
- Look for the reason for a bug
- Suggest how to improve the code
Today we are going to focus on the last one, but remember that it's not the only reason to read code!
The earlier defects are found
The best way to reduce defects is preventing them, the second best way is by conducting code reviews that detect the defects just while they are being injected.
... the lower the costs of correcting them
... the higher the probability of fixing them properly
If the issue is about when defects are detected, we could simply test early, but the results wouldn't be the same because:
Code Review Benefits
The earlier defects are found
The best way to reduce defects is preventing them, the second best way is by conducting code reviews that detect the defects just while they are being injected.
-
Testing can only detect External Quality Issues but cannot help to improve Internal Quality ones.
... the lower the costs of correcting them
... the higher the probability of fixing them properly
-
Code Reviews detect Defects while Test detect visible symptoms of a defect
If the issue is about when defects are detected, we could simply test early, but the results wouldn't be the same because:
Code Review Benefits
This kind of technique changes also the mind of developers. People reviewing code is also learning in parallel, people whose code is being reviewed is not only fixing defects but learning new techniques, paradigms, etc. that could be applied in future projects.
Formal Code Inspection
Walkthrough
Over-the-shoulder Review
Offline Review
Type of Reviews
FORMAL
INFORMAL
Formal Code Inspection
Team: 4 People (Moderator, Dev, QA, Architect)
Formal Process: Meeting scheduled, members, duration.
Formal Review: Dev shows the code statement by statement and a specific checklist is used
Formal Walkthrough
Similar to inspection but the review process is different, people in the meeting "play computer": some test cases are chosen and they are mentally executed; that is, the test data are "walked through" the logic of the program. The state of the program is monitored on paper or a whiteboard
The specific test cases chosen is something not critical
Team composition is slightly different as the required skills are different
Over the shoulder
Formal Reviews: Lot of effort and time from devs
Over-the-shoulder: a developer standing over the developer's computer while the author walks the reviewer through a set of code changes
It has been proven that informal methods could provide similar results
Advantages: Simple, encourages people to interact and cooperate, increases feedback
Issues: Require devs to be in the same place, no traceability, very author-driven
Offline Review
Offline because the developer submits the code to be reviewed (Zip file, URL, Pull Request...) so a peer can review it without the developer being directly involved while the review takes place.
Advantages: Simple, works well remotely, traceability
Issues: takes longer, misses a bit the human part of over-the-shoulder reviews
SCM Tools have made this kind of reviews very easy by integrating: notifications, recording review comments, combined display, discussion...
In general, we could say that offline code reviews, done properly integrated in an SCM gets a good balance between speed, effectiveness and traceability
Offline Review Examples
-
A simple one about detecting how to make the code better and more robust: The Bug Description: https://bugzilla.mozilla.org/show_bug.cgi?id=819898#c3 The Pull Request: https://github.com/mozilla-b2g/gaia/pull/6923 In this bug, the code review helped to make the code more robust and also helped the developer to learn a new thing about how to initialize a button in JavaScript Pay attention to traceability: A lot of information is available, the commit, the comments, the bug status...
-
A simple one in which nothing was changed in the code, but a double-check was helpful AND it helped to create team spirit: https://github.com/mozilla-b2g/gaia/pull/6461
-
But usually, code reviews are a bit more complex: https://github.com/mozilla-b2g/gaia/pull/6801
-
Or even more, more, more complex: https://bugzilla.mozilla.org/show_bug.cgi?id=767818
Recommended Readings
Practical Work
5.B.1 - Code Reviews ToDos and Not-ToDos
5.D.5 - Simple Code Review Snippets
Defect Prevention and Code Reviews
By Daniel Coloma
