Title Text

Subtitle

Salvador, Bahia, Brazil

Assistant Professor at the

Federal University of Bahia

Research interests

• Empirical software engineering
• Mining software repositories
• Release engineering
• Software quality

Rejected changes

Software changes

• Software has to change
• Not all changes are created equal
  • Good enough :)
  • Inappropriate :(

Inappropriate changes

Inappropriate changes

Data: Mozilla Foundation

• Main projects:
  • Firefox
  • Core (reusable code)
• Mercurial (commits) and
  Bugzilla (issues/bugs)
• All non-trivial code is reviewed
• All inappropriate commits are reverted
• Closed issues may be reopened

Inappropriate changes

how to detect them from existing repos?

Negative code reviews

• Very early rejection
• Problems in the code
  • coding style
  • design
  • performance
  • potential bugs
• Not all projects perform reviews on all changes

Reverted commits

• A commit that reverts a previous commit
• In Mercurial, revert = "backout"
• Can be early or late in the process
• Rejection due to problems found during automated or manual testing
• Very few projects actually revert commits

changeset: 0a93ae68184e
summary:   Bug 787078 - Backout 43dd8252f52d

Supplementary commits

• 2+ commits for the same issue
• Can be early or late in the process
• Not always a sign of problems
  • Multipart commits
• Easily detectable

changeset: 0a93ae68184e
summary:   Bug 1635 - add info button

changeset: 394bf8a8e9fa
summary:   Bug 1635 - fix button behavior

Issue reopening

• Problem found after issue was closed
• Late in the process
• Not always because of problems in the change itself
  • Tester used the wrong version
  • Issue was prematurely closed
    (e.g., duplicate, wontfix)
• In some projects, a new issue is created

supplementary commits

reverted commits

Core

supplementary commits

reverted commits

Core

reopened issues

commits reverted after issue is closed

Core

Inappropriate changes

how much rework do them cause?

rejected changes

negative reviews

reverted commits

reopened issues

21%

12%

8%

5%

(among changes associated with issues, reviews, and commits)

Core

5 rejections

per day

1 rejection

per day

How often are changes rejected?

Inappropriate changes

are they more likely under rapid releases?

Rapid Releases

Release approach

Rapid releases
and quality

Rapid releases
and time pressure

Traditional

Before March 2011

12+ months btw releases

Rapid

After March 2011

6 weeks btw releases

Transition to rapid releases

• Annoying update notifications!
• My extensions are broken!
• My company needs stability!

(users' viewpoint)

What about rejections?

... or is it?

.

.

.

Early vs late rejections

early reverts

problems are detected earlier

late reverts

You have to understand the process

main branch

Before

main branch

After

staging branch

Conclusions

• Naïve conclusion: with rapid releases...
  • developers are breaking the system more often
  • hence, they perform more rework
• After talking to developers: with rapid releases...
  • they perform less testing before pushing changes
  • the main branch is rarely broken
• For users:
  • late rejections decreased (fewer bugs are shipped)
  • updates arrive frequently
  • corporate users have extended support releases

References

R. SOUZA. Inappropriate Software Changes: Rejection and Rework (PhD Thesis)

http://rodrigorgs.github.io/files/rodrigo-thesis-final.pdf

SOUZA, R. ; CHAVEZ, C. ; BITTENCOURT, R. A.
Rapid Releases and Patch Backouts: A Software Analytics Approach
In: Software, IEEE, 2015.

http://online.qmags.com/ISW0315#pg91&mode2

Other related publications

http://rodrigorgs.github.io/publications

Move fast and break things?

The impact of rapid releases on the rejection of software changes

Rodrigo Souza

<rodrigo@dcc.ufba.br>

(extra slides)

Before

• All changes were pulled from and pushed to one single branch
• Each change triggered a build (compile + test), which lasted ~ 4h
• A broken build affected all developers
• The developers was responsible for reverting his/her commit

After

• Automated builds are still run and are still slow
• Changes are pulled from the main branch and pushed to a staging branch
• Upon a succesful build, sheriffs merge the two branches
• Upon a broken build, sheriffs revert the offending commits

Broad questions

• How to detect inappropriate changes before they reach end users?
• How to avoid creating inappropriate changes in the first place?

• what are the characteristics of changes that end up being rejected?
• what characteristics of the process lead to increased rejection?