Theories, tools and research methods in program comprehension: past, present and future

By Margaret-Anne Storey, 2006

Presented by Rodrigo Araujo

Paper goals

Summarize many cognitive theories and techniques applied to program comprehension

Paper goals

Summarize many cognitive theories and techniques applied to program comprehension

How developers understand code?

Paper goals

How these theories and techniques can lead to improvements in program comprehension tools

Paper goals

How it will change in the future

Paper goals

Cognitive models + development tools

Easier program comprehension

Terminology

Cognitive science

Study of the mind and its processes. It examines the nature, the tasks, and the functions of cognition.

Terminology

Mental model

Engineer's mental representation of the program to be understood

Terminology

Cognitive model

Cognitive processes and temporary information structures in the engineer's head used to form the mental model

Terminology

Programming plans

Generic fragments of code that represent typical scenario

Terminology

Delocalized plans

Programming plans implemented in disparate areas of the program

Terminology

Beacons

Recognizable, familiar features in the code that act as cues to the presence of certain structures

Comprehension models

Top-down comprehension

Reconstructing knowledge about the domain of the program and mapping it to the source code

Comprehension models

Top-down comprehension

Hypothesis on nature of the program

Comprehension models

Top-down comprehension

Hypothesis is refined in hierarchical fashion by forming sub hypotheses

Hypothesis on nature of the program

Comprehension models

Sub-hypotheses are refined and evaluated in depth-first

Top-down comprehension

Hypothesis on nature of the program

Hypothesis is refined in hierarchical fashion by forming sub hypotheses

Comprehension models

Group small chunks of statements

Bottom-up comprehension

Comprehension models

Group small chunks of statements

Bottom-up comprehension

Form higher level abstractions

Comprehension models

Group small chunks of statements

Bottom-up comprehension

Form higher level abstractions

Repeat until high-level understanding of the program is attained

Comprehension models

Knowledge-base model

Programmers as opportunistic processors capable of exploiting both bottom-up and top-down cues (Letovsky, 1986)

Comprehension models

Knowledge-base model

Knowledge base

Developer's expertise and background knowledge

Mental model

Assimilation process

Developer's current understanding of the program

How the mental model evolves using the developer's knowledge base + code + docs. Bottom-up or top-down

Comprehension models

Knowledge-base model

Knowledge base

Mental model

Assimilation process

+

Inquiries

Comprehension models

Knowledge-base model

Inquiries

Asking questions and conjecturing answers

Comprehension models

Knowledge-base model

Inquiries

Why conjectures

Questioning the role of a function or piece of code

How conjectures

What conjectures

What is the method for accomplishing a goal

What is the variable or one of the program functions

Comprehension models

Integrated metamodel

Everything together: It builds on the previous models. It uses knowledge base to support 3 comprehension processes

Comprehension models

Integrated metamodel

Top-down domain model

Invoked and developed using as-needed strategy, when code is familiar. It uses domain knowledge as starting point for formulating hypotheses

Program model

Situation model

Invoked when unfamiliar, it's a control-flow abstraction

Dataflow and functional abstractions in the program. Developed after a partial program model is formed

Comprehension models

Integrated metamodel

Knowledge base

Information needed to build the top-down domain model, program model and situation model

Comprehension models

Integrated metamodel

"Understanding is formed at several levels of abstractions simultaneously by switching between the three comprehension processes"

Current theories and tool support

Documentation

Should be designed to support top-down comprehension (Brooks, 1983). Important to document problem domain, programming concepts and domain knowledge

Searching/Querying

Support to search code by analogy and iterative searching. Should allow querying on the role of a variable, function, etc.

Current theories and tool support

Browsing/Navigation

Top-down process requires browsing from high level abstractions to lower levels details, taking advantage of beacons. Bottom-up process requires following control and dataflow. Both should be supported

Multiple views

Programming environments need to provide different ways of representing programs: textual representation, call graphs, classes and their relationship, etc. These views, if easily accessible, should facilitate comprehension, especially if combined and cross-referenced

Tool requirements

Concept assignment problem

Hard task of mapping the code to the requirements

Tool requirements

Concept assignment problem

Hard task of mapping the code to the requirements

#1: User indicates a starting point and then uses program slicing techniques to find related code

#2: Intelligent agent to scan code and search for candidate starting points

Tool requirements

Reverse engineering

Call trees, diff tools, browsing history and entity fan-in can support the top-down model

Tool requirements

Software visualization tool needs

Support the comprehension models by displaying how components interact

Tool requirements

Importance of search and history

Tools should support rediscovery. Developers quickly forget details on a specific part of the program when they move to a new location.

Tool requirements

Information needs for maintainers

7 questions are usually asked:

1. Where is a particular subroutine/procedure invoked?

2. What are the arguments and results of a function?

3. How does control flow reach a particular location?

4. Where is a particular variable set, used or queried?

5. Where is a particular variable declared?

6. Where is a particular data object accessed?

7. What are the inputs and outputs of a module?

Tools should try to answer these questions

Tool research

Extraction

Analysis

Presentation

Tool research

Parsers & data gathering tools to collect static/dynamic data.

Extraction

Analysis

Presentation

Tool research

Parsers & data gathering tools to collect static/dynamic data.

Extraction

Analysis

Presentation

Support clustering, concept assignment, feature identification, metrics. Dynamic analysis: code instrumentation

Tool research

Parsers & data gathering tools to collect static/dynamic data.

Extraction

Analysis

Presentation

Support clustering, concept assignment, feature identification, metrics. Dynamic analysis: code instrumentation

Code browsers, code editors, visualization tools

Programmer and program's trends: how will it be in the future?

Programmer and program's trends

More diversified programmers

"Programming, and hence program comprehension, is no longer a niche"

Programmer and program's trends

More diversified programmers

"Programming, and hence program comprehension, is no longer a niche"

Programmer and program's trends

Sophisticated users

Author bets that developers will use more complex UI to build software

Programmer and program's trends

Sophisticated users

Author bets that developers will use more complex UI to build software

?

Programmer and program's trends

Globally distributed teams

Programmer and program's trends

Globally distributed teams

Spot on!

Programmer and program's trends

Agile developers

Programmer and program's trends

Agile developers

Spot on!

Programmer and program's trends

Popularization of distributed and web applications

Programmer and program's trends

Popularization of distributed and web applications

"[...] more prevalent with technologies such as .NET, J2EE and web services. One programming challenge that is occurring now and is likely to increase, is the combination of different paradigms in distributed applications, e.g. a client side script sends XML to a server application"

Programmer and program's trends

Popularization of distributed and web applications

Programmer and program's trends

Improved and newer software engineering practices

We have problems. But we definitely improved over the past few years

How tools/theories will evolve in response to these changes

"Learning theories (Exton, 2002) will become more relevant to end-users doing programming-like tasks."

"Theories are currently being developed to describe the social and organizational aspects of program comprehension (Gutwin, 2004)"

How tools/theories will evolve in response to these changes

"The use of frameworks as an underlying technology for software tools is leading to faster tool innovations as less time needs to be spent reinventing the wheel"

How tools/theories will evolve in response to these changes

"The use of frameworks as an underlying technology for software tools is leading to faster tool innovations as less time needs to be spent reinventing the wheel"

Conclusion

Recommendation systems and search systems will evolve (i.e NavTracks). Visualization tools will evolve. Collaboration tools will evolve. Thus, Program Comprehension tools will evolve.

Conclusion

But how to seamlessly integrate all this to the developer's workflow without increasing complexity?

How can we avoid bloated IDEs?

Conclusion

But how to seamlessly integrate all this to the developer's workflow without increasing complexity?

Conclusion

How can we make more cognitive friendly software?

Paper pros/cons

Pros

Cons

Huge amount of ideas for developer tools

Great interdisciplinary study

Spreading awareness on how important cognitive models are

Evaluating tools is hard

Experience cannot always measure developers ability and creativity