Codeline Management for Evolutionary Development

1
Codeline Management for Evolutionary Development

• Anders JohnsonWIS Technologies

2
Evolutionary Development

• Traditional (big-bang)

• Evolutionary

100
100
Features
Quality
Completion ?
Completion ?
Features
Quality
0
0
Time ?
Time ?
3
So What?

• The literature says that it leads to
• Increased productivity
• Decreased risk
• Higher quality
• Simultaneously
• But be realistic
• Unlikely to halve the schedule on first try

4
Automated Regression

• Yes, you need it!
• Near-production test after every mainline change
• No way youre going to do that manually!
• Test at various levels of integration
• Dont need to worry about breaking anything that
  the tests cover

5
Who Farted?

• Syncing-in broken changes is extremely
  counterproductive
• Impossible to test unrelated changes
• But, you still need to share some changes before
  theyre functional

6
Branches or Labels?

• Branches (a.k.a. codelines) are better for
  evolutionary development
• Labels often fail to converge

OK (?)
Feature A
Broken
OK (?)
Feature B
Broken
Feature C
Broken
OK Label
L
Time ?
7
Development Codelines

• Contain changes that arent expected to work
• Multiple contributors can exchange early changes
• Use the mainline instead if you require something
  that actually works
• Integrate into mainline when it works

8
Distinguishing Codelines

• Each codeline has its own regression (or lack
  thereof)
• May or may not be driven by a submit trigger
• Each codeline has its own default view
• Client generation is scripted, such that the view
  is set automatically
• Codeline-name_at_change describes a source
  configuration definitively

9
Reining in the Chaos

• Each workspace is based on exactly one codeline
• Each codeline (except the mainline) has exactly
  one parent codeline
• Workspace location corresponds exactly to depot
  location (relative to codeline root)

10
Depot Structure
TRUNK
branch1/
branch2/
11
Extension Fields

• Extra fields (such as clients codeline and
  branchs parent codeline) piggy-back onto the
  Description field

Client anders_a4 Description Created
by anders. ----- Please do not delete
these lines from the Descripti on field -----
Branch a4/ Location
//depot/a4/top Root /home1/anders/anders_a4
12
Codeline Configuration

• Each codelines regression and default view is
  stored in a designated location in the file tree
• CFG/codeline-name/top/
• Changes propagate automatically
• Regression changes should be reviewed
• The regression failed, so I disabled it.

13
Sequential Regression

• If there is a regression, then every submission
  must be strictly sequential
• Submitters workspace (including unchanged files)
  must be up-to-date
• Otherwise, the regression might pass even if the
  depot would be broken
• Doing this reliably requires locking

14
.p4top

• Symbolic link in each workspace directory
  pointing to workspace root
• Portable among workspaces
• Can often be used where an absolute path would
  normally be required

cd anders_a4/foo/bar ls Alo lrwxrwxrwx 1
anders 5 .p4top -gt ../..
15
Staying in Sync

• Files can differ even if Perforce thinks that
  theres nothing left to merge
• For fully-merging codelines
• Integrate only between parent and child
• Child must contain all of parents changes
• Then, p4 integrate -dft and p4 resolve at into
  parent
• Not for vendor release codelines

16
Switching Codelines

• Changes sometimes wind up in the wrong workspace
• To avoid stranding them, need to create and apply
  patch files

17
Hierarchical Regression

• Every codeline can be regressing simultaneously

18
Reference Builds

• We cheat by running regressions inside the
  submitting workspace
• This occasionally causes a false positive or
  false negative
• Environmental dependencies
• Build system deficiencies
• Failure to p4 add

19
Reference Builds (contd.)

• Periodically clean-build as a pseudo-user with a
  baseline environment
• Regression should be deterministic
• Results can be cloned into new workspaces (in
  conjunction withp4 flush) to bypass
  clean-building
• Its practical to have tools under Perforce

20
Concurrent Related Projects

• SURPRISE Inter-file branching is not a good
  general solution for this
• Relies on merge-at-destination, which is bad
  because the originator is usually better equipped
  to merge
• Results in duplicated merging effort, especially
  if there are more than 2 projects

21
Concurrent Projects (contd.)

• Instead, make all the projects visible in the
  same tree, and share as much as you can
• Symbolic links
• ifdef
• Design patterns
• Etc.
• Works whether or not you branch

22
More Sharing Techniques

• Decompose to isolate variations
• Override files using a search path
• Read or include from same location
• Filter (e.g. Perl) to generate variations
• Leave unused code as-is
• Make the requirements more similar
• Inter-file branching (as a last resort)

23
Concurrent Projects (contd.)

• Copy-and-paste is bad code economy, and its
  essentially irreversible
• Indiscriminant inter-file branching is only
  somewhat better
• Try sharing first
• Youll like it
• If not, its relatively easy to branch later

24
Concurrent Projects (contd.)

• Conceptually, you now have a single
  hierarchically regressed super-project

25
Interoperability Testing

• Requires multiple versions in view
• Avoid mixed views
• Difficult to reproduce
• Instead, p4 integrate proj/..._at_1234 \
  proj/1234/...

26
Forward Integration Failures

• In case of conflicting assumptions, determine
  which codeline is at fault (WARNING may
  involve politics), and fix it
• In case of complex merging, submit extra merging
  changes to a grandchild codeline, and integrate
  back to child

27
A4 Codeline Automation

• Perl, mostly object oriented
• Reusable modules
• P4 form manipulation
• P4 command result parsing
• 5 person-months, 22,000 lines
• 55 Documentation and test code
• http//a-4.sourceforge.net

28
Conclusions

• Evolutionary development is good
• Use hierarchically regressed codelines to sustain
  incremental progress
• Use a shared code base for concurrent related
  projects
• Use A4 to automate codeline management