Towards Supporting OnDemand Virtual Remodularization Using Program Graphs

1
Towards Supporting On-Demand Virtual
Remodularization Using Program Graphs

• David Shepherd, Lori Pollock, and K. Vijay-Shanker

University of Delaware
2
Overview and Rationale
//Activates the tool for the given view.
//Turn on dragtool by adding....
public void Tool.activate()
undo action
update drawing
on-demand remodularization
save drawing
activate tool
action oriented system
object oriented system
3
Overview and Rationale
public interface Storable...
//Store the fields in a file....
undo action
Key Insight Programmers leave natural language
clues that reference concerns
public void Circle.save()
update drawing
activate tool
save drawing
action oriented system
object oriented system
4
State of the Art in Feature Location/Aspect Mining
Low-level Clues

• Dynamic Information
• Wilde ICSM 00, Michail ICSE 02, Zhao ICSE
  04, Breu ASE 04, Ceccato ICSM 05...
• Program Structure
• FEAT, Sextant ICSM 05, Robillard FSE 05...
• Information Retrieval
• Marcus PhDThesis 03, ICSE 03
• Lexical Information
• Aspect Mining Tool, Aspect Browser

Complementary
Foundations
5
A Motivating Example
Recipe Writing Factory

• Recipe for Vegetable Soup
• Place asparagus and onion in a saucepan with 1/2
  cup vegetable broth. Bring the broth to a boil,
  reduce heat and let simmer until the vegetables
  are tender.
• Reserve a few asparagus tips for garnish. Place
  remaining vegetable mixture in an electric ...

Procedural instructions are easy to read
6
Choosing a Dominant Decomposition
Recipe Writing System 1.0
7
Choosing a Dominant Decomposition
Recipe Writing System 1.0

• asparagus
• Place asparagus in a saucepan.
• Reserve a few asparagus tips for garnish
• Garnish with reserved asparagus tips.

• flour
• Sprinkle flour into the butter. Do not let the
  flour brown.

• butter
• Melt butter in the pan that was used for
  simmering the asparagus and onionsstir.

• salt
• Sprinkle salt into the butter.

• soy milk
• Stir the milk into the saucepan.

• yogurt
• Whisk yogurt into the mixture

OO decomposition provides many benefits

• Recipe for Vegetable Soup
• List ingredients asparagus, onion...
• Bowl b new Bowl()
• while( ingredients.hasNext)
• Ingredient ingred ingredients.next()
• ingred.place(b)
• ...

8
Action-Oriented CCCs Appear

• asparagus
• Place asparagus in a saucepan.
• Reserve a few asparagus tips for garnish
• Garnish with reserved asparagus tips.

• flour
• Sprinkle flour into the butter. Do not let the
  flour brown.

• butter
• Melt butter in the pan that was used for
  simmering the asparagus and onionsstir.

• salt
• Sprinkle salt into the butter.

• onion
• Place onion in the saucepan

• yogurt
• Whisk yogurt into the mixture

OOP causes scattered actions
9
Evolution Causes Scattered Actions
//Place the Beef into the pan place(Pan p)
checkIfFitsInPan(p) actuallyPlaceInPan(p)
Beef place(p) checkIfFitsInPan(p) actuallyPlaceIn
Pan(p)
//Make sure the beef fits in the pan, //before
Placing the Beef into the pan checkIfItFitsInPan(P
an p) ...
//Place the beef into the pan actuallyPlaceInPan(
Pan p) ...
Evolution causes more scattered actions
10
Maintenance Tasks Involve Scattered Actions
Something goes wrong when I place ingredients in
the saucepan.
Maintenance tasks require action location
11
How can we find actions in a noun's world?

• Our Strategy
• Extract natural language clues
• Query clues and remodularize

12
Extract Clues with NLP
//Make sure the beef fits in the pan, //before
placing the beef into the pan Beef.checkIfItFitsIn
Pan(Pan p) ...
//Make sure the beef fits in the pan, //before
placing the beef into the pan Beef.checkIfItFitsIn
Pan(Pan p) ...
//Place the onion in the pan Onion.place(Pan
p) ...
place onion
place beef, check beef
//Place the beef into the pan Beef.place(Pan
p) checkIfFitsInPan(p)
actuallyPlaceInPan(p)
//Place the beef into the pan Beef.place(Pan
p) checkIfFitsInPan(p)
actuallyPlaceInPan(p)
//Place the beef into the pan Beef.actuallyPlaceI
nPan(Pan p) ...
//Place the beef into the pan Beef.actuallyPlaceI
nPan(Pan p) ...
place beef
place beef
Key Insight Use language clues to extract actions
13
Query Clues and Remodularize
Virtual Source File
//Make sure the beef fits in the pan, //before
placing the beef into the pan Beef.checkIfItFitsIn
Pan(Pan p) ...
simple query place
//Place the beef into the pan Beef.actuallyPlaceI
nPan(Pan p) ...
//Place the onion in the pan Onion.place(Pan
p) ...
//Place the beef into the pan Beef.place(Pan
p) checkIfFitsInPan(p)
actuallyPlaceInPan(p)
From OO code to Action-Oriented code!
14
A Closer Look at the Extraction Process...
15
What to Extract

• information about actions
• verbs
• direct-objects
• ex Place the item in the pan
• ex2 Place the beef in the pan

Something goes wrong when I place ingredients
into the saucepan.
Something goes wrong when I place beef into the
saucepan.
Our Strategy extract verb and direct object
pairs
16
How to extract clues

• Two types of extraction

class Beef //Chop the beef into cubes.
chop() unwrap() slice(CUBES)
Extraction from comments
Extraction from method signatures
17
Extracting Clues from Comments

• Part-of-speech tagging
• Chunking
• Role assignment

Remove an item from a cart
Removeltvgt anltdtgt itemltngt fromltpgt altdtgt cartltngt
Removeltvgt an itemltnoun phrasegt from a cart
ltnoun phrasegt ltprepositional phrasegt
Remove
item
rule when a verb is followed by a noun phrase,
extract
18
Extracting Clues from Signatures

• Process method name
• Use hierarchy of rules to classify
• Use classification to extract

class Beef //Chop the beef into cubes.
chop() unwrap() slice(CUBES)
19
Extracting Clues from Signatures

• Process method name
• Use hierarchy of rules to classify
• Use classification to extract

class Beef //Chop the beef into cubes.
chop() unwrap() slice(CUBES)
chopltvgt
chopltvgt the beefltnoun phrasegt
20
Extracting Clues from Signatures
Hand Tuned Rule Tree

• Process method name
• Use hierarchy of rules to classify
• Use classification to extract

class Beef //Chop the beef into cubes.
chop() unwrap() slice(CUBES)
Example Classification method has no parameters
and the method name is a verb
chopltvgt
21
Extracting Clues from Signatures

• Process method name
• Use hierarchy of rules to classify
• Use classification to extract

• Example Extraction Rule
• get verb from method name
• get DO from class name

class Beef //Chop the beef into cubes.
chop() unwrap() slice(CUBES)
chop beef
22
Storing the Clues

• Action-Oriented Identifier Graph (AOIG)

verb1
verb2
verb3
DO1
DO2
DO3
verb1, DO1
verb1, DO2
verb3, DO2
verb2, DO3
use
use
use
use
use
use
use
use
source code files
23
Storing the Clues

• Action-Oriented Identifier Graph (AOIG)

chop
grill
puree
onion
beets
steak
chop, onion
chop, beets
puree, beets
grill, steak
use
use
use
use
use
use
use
use
source code files
24
How Accurate is Extraction?

• Survey of AOIG-Builder's Accuracy

Avg. Precision 95
Avg. Recall 83
Good Precision and Recall
25
Costs of Building the AOIGper source file

• Space
• O(number of methods)
• minimal
• Time
• O(number of methods)
• 10 seconds for a large source file with lots of
  multi-line comments

Reasonable cost - can be done incrementally
26
Contributions...
27
Action-Oriented View

• High-level CCCs instead of low-level
• clear hypothesis of why actions are CCCs
• Useful type of CCC
• maintainers are looking for actions

Open Question How can we best deal with
Action-Oriented CCCs?
28
Contributions (continued)

• Action-Oriented View
• Combined NLPProgram Analysis (NLPA)
• Developed the AOIG for virtual remodularization
• Applied AOIG to
• Feature Location
• Aspect Mining
• Working Set Recovery

In this talk, we have shown how to virtually
remodularize code into actions using the AOIG
29
Questions?
save drawing
activate tool
update drawing
undo action
action oriented system
object oriented system
30
Conclusions

• NLP
• is sufficiently robust to use in SE tools
• NLP Program Analysis
• can extract more exact information from code
• can drive SE tools
• We have shown how to virtually remodularize code
  into actions using the AOIG

31
Demo of Current/Future Work

• Complete Feature Location Methodology
• sophisticated query expansion
• action-oriented code view for program
  understanding

32
Current/Future Work
33
generate
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
PowerGrid.timePassing()
PowerGeneration.constructor
PowerGrid.powerSurplus
PowerGeneration.getGeneratedPower()
PowerSource.getCurrentPower()
calls
StandardPowerSource.getCurrentPower()
SolarPowerSource.getCurrentPower()
PowerSource.getCurrentPower()
40
Experiments

• Subjects QLI projects
• Procedure
• Identify bugs or feature requests which require
  action location
• Use tool to locate and squash bugs
• For submission to ICSM Industry Track

41
User Feedback

• DIRECTIONS
• Place asparagus and onion in a saucepan with 1/2
  cup vegetable broth. Bring the broth to a boil,
  reduce heat and let simmer until the vegetables
  are tender.
• Reserve a few asparagus tips for garnish. Place
  remaining vegetable mixture in an electric
  blender and puree until smooth.
• Melt butter in the pan that was used for
  simmering the asparagus and onions. Stir while
  sprinkling flour, salt, and pepper into the
  butter. Do not let the flour brown. Allow the
  mixture to cook only 2 minutes. Stir in remaining
  1 1/4 cups vegetable broth and increase the heat.
  Continue stirring until the mixture comes to a
  boil.
• Stir the vegetable puree and milk into the
  saucepan. Whisk yogurt into the mixture, followed
  by lemon juice. Stir until heated through, then
  ladle into bowls. Garnish with reserved asparagus
  tips. Sprinkle with Parmesan cheese if desired.

Something goes wrong when I am sprinkling things
into the butter.
42
Recipe Developer, Fix it
43
Recipe Storage System 2.0
more complex system, actions are scattered further
Onion place(Pan p)
Person place(List ingredients, Pan p)
Placeable place(Pan p)
Vegetable place(Pan p)
Pepper place(Pan p)
Beef place(Pan p)
44
Clever Worker

• 1 pound fresh asparagus
• Place asparagus in a saucepan.
• Reserve a few asparagus tips for garnish
• Garnish with reserved asparagus tips.

2 tablespoons all-purpose flour
1. 2. 3. Sprinkle flour into the butter. Do
not let the flour brown.
1 tablespoons butter 1. 2. 3. Melt butter in
the pan that was used for simmering the asparagus
and onionsstir.
1 teaspoon salt 1. 2. 3. Sprinkle salt into
the butter.
1/2 cup yogurt 1. 2. 3. 4. Whisk yogurt
into the mixture
1 cup soy milk 1 2. 3. Stir the milk into
the saucepan.
45
Person sprinkleItems(List ingredients)
46
Clever Worker

• 1 pound fresh asparagus
• Place asparagus in a saucepan.
• Reserve a few asparagus tips for garnish
• Garnish with reserved asparagus tips.

2 tablespoons all-purpose flour
1. 2. 3. Sprinkle flour into the butter. Do
not let the flour brown.
1 tablespoons butter 1. 2. 3. Melt butter in
the pan that was used for simmering the asparagus
and onionsstir.
1 teaspoon salt 1. 2. 3. Sprinkle salt into
the butter.
1/2 cup yogurt 1. 2. 3. 4. Whisk yogurt
into the mixture
1 cup soy milk 1 2. 3. Stir the milk into
the saucepan.