Using Tests as Agile Documentation

1
Using Tests as Agile Documentation

• Brian Button
• Principal Consultant

2
Introduction

• Welcome to my experiment!
• Not concrete yet, still taking shape
• Trying to prove an idea
• In Agile world, there is the claim that the tests
  are the documentation.
• What would it take to make that true???

3
Who is the Audience?

• Who is reading the documentation?
• Programmers (task oriented)
• Just want to get in and get out
• Maintainers (holistic understanding)
• Need to understand design
• Use tests as backup during maintenance
• Evaluators (kicking the tires)
• Setup, overview, simple tasks

4
My Target Audience

• This talk with focus on the Programmers using the
  library
• Most basic need
• If this need is not met, tests will not suffice
• Also focusing on projects where code is the output

5
What Problem are We Solving?
6
What is the Problem?

• We all need documentation
• But its expensive to
• Create
• Maintain
• Written as separate activity by writers or
  programmers
• And it needs to change as requirements change

7
What Can We Do?

• Nothing
• Limit changes to keep down costs
• Embrace Change

8
Status Quo

• Dont change
• Accept that documentation costs lots of money
• Were all comfortable with that
• But costs keep increasing, budgets keep
  shrinking, projects keep getting shorter
• Probably not the right answer

9
Limit Changes

• Let documentation costs lead design decisions
• Huh???
• Leads to missed opportunities
• After all, were building software, not manuals
• Also, probably not the way to go

10
The Agile Solution

• Embrace changes
• Because theyre gonna happen
• Adapt documentation process to allow for easy
  changes
• Leverage an existing, changing asset to keep
  documentation creation and maintenance costs low

11
Unit Tests!!!!
12
Unit Tests??

• Means different things to different people
• My definition
• Specific, targeted tests that focus on one path
  through code under test
• Not system-wide or complex transactions
• Simple, method-level tests

13
Example String Immutability
public void ConvertingStringToUpperCaseCreatesNewS
tringInstance() string
lowerCaseString "brian" string
upperCaseString lowerCaseString.ToUpper()
Assert.AreEqual("brian",
lowerCaseString) Assert.AreEqual("BRI
AN", upperCaseString)
14
Preconditions to Adoption

• Assumes Agile Development practices
• Must have Unit Tests for Everything
• Tests must be run all the time
• Tests must pass 100 all the time
• Assumes leeway and understanding from audience
• Its gonna be different!

15
Agile Documentation

• But if theyre open to it, we can really wow them
• We can create
• Simple statements of facts
• Give simple usage examples
• Document all behavior explicitly
• Guarantee code and docs are in sync

16
Standard Programming Docs

• Documentation for Stack in Visual Studio
• Class level
• Gives overview, class hierarchy, example
• Example takes some effort to read and is
  incomplete
• Peek()
• Gives overview, exception, more specific example
• Example gets worse as method gets more complex

17
Agile Docs for Stack
public class StackBehaviorFixture
public void StackIsEmptyAtCreation()
public void StackNotEmptyAfterItemPushed()
public void StackChangesBackToEmptyAfterPushAndPo
p() public void ValuePushedOntoStackIsSame
AsValuePoppedOffStack() public void
MultipleItemsPushedOntoStackArePoppedOffInReverseO
rder() public void PopFromEmptyStackThrows
InvalidOperationException() public void
PeekDoesNotRemoveItemFromStack() public
void PeekWillReturnItemPushedOnStack()
public void PeekWillAlwaysReturnTheSameItem()
public void PeekWillAlwaysReturnMostRecentPush
edItem() public void PeekOnEmptyStackThrow
sInvalidOperationException() public void
NullItemsCanBePushedOntoStack() public
void PopReturnsNullItemOffStack() public
void PeekReturnsNullItemOffStack()
18
Agile Docs for Stack

• Each test is statement of fact
• Human readable
• Simple usage example
• Would a Test List be better?

19
Creating Agile Documentation

• Generate a Test List
• List of candidate tests to flesh out interface
  and functionality.
• Grows over time as you learn more
• Task oriented
• Flows from simple to complex
• Implement tests

20
Anatomy of a Good Test

• Strong names, simplicity, clarity
• 3 As

public void PeekDoesNotRemoveItemFromStack()
Stack stack new Stack()
stack.Push("foo") stack.Peek()
Assert.IsFalse(stack.IsEmpty)
21
Whats Missing?

• What versus Why
• High-level UML diagrams
• Guidance on where to look

22
(Some) Documentation Necessary Evil

• Unit tests tell what not why
• For complex classes or subsystems, some level of
  written documentation is required
• But not at same level of detail as before
• Overview, design decisions, key UML diagrams
• Unlikely to change as details change

23
Test Map

• On complex project, there are lots of tests
• Key question is how to find right tests easily
• Is this harder than with regular documentation?
• Lots of paper, web pages, etc, versus lots of
  test
• Creation of a test map would go a long way to
  helping

24
What is a Test Map

• Idea of my own creation
• It simply maps from a method being tested to
  those tests that exercise that method
• Creates a manual page that lists methods and has
  hotlinks to tests that exercise them
• Use of good test names will help programmer find
  right test to look at

25
Test Maps and Automation

• Test Map would be hard to keep in sync
• Creation should be automated

Test TestFor("StackExample.Stack.
Push") TestFor("StackExample.Stack.Pop")
public void ValuePushedOntoStackIsSameAs
ValuePoppedOffStack() Stack
stack new Stack()
stack.Push("fred") object
poppedValue stack.Pop()
Assert.AreEqual("fred", poppedValue)
26
Downside of Markups

• Has to be maintained manually
• Failing test wont point you to incorrect markups
• Error prone
• Drives up cost
• Lessens value
• Where is the balancing point?

27
Conclusion

• Tests can replace much written documentation
• Some written docs and UML still needed
• Remaining documentation less change-prone
• Requires right audience and context
• Requires different mindset while writing tests