The UML Testing Profile

1
The UML Testing Profile

• Standardized by the Object Management Group (OMG)
• Adds test specification and execution concepts to
  UML
• Source OMG UML Testing Profile, version 1.0

2
Sections of the Testing Profile

• Test architecture
• Test behaviour
• Test data
• Time

3
Test Architecture

• Concepts
• Test context
• Test configuration
• Test component
• SUT
• Arbiter
• Scheduler

4
Testing Profile - Architecture
5
Test Context

• A UML testing profile stereotype.
• A collection of test cases together with a test
  configuration on the basis of which the test
  cases are executed.
• Attributes include exactly one instance of an
  Arbiter and exactly one instance of a Scheduler.
• The TestContext is the starting point for setting
  up testing with the UML testing profile.

6
Test Configuration

• The collection of test component objects, and of
  connections between the test component objects
  and the SUT.
• Test cases are separate from the test
  configuration.
• For connections and components
• Declares the number of instances when the test
  case is started.
• Declares the maximal number of instances
  permitted during test execution.

7
Test Component

• A UML testing profile stereotype for a class of a
  test system that is not part of the system under
  test.
• Has a set of interfaces via which it may
  communicate via connections with other test
  components, or with the SUT.
• Attributes include a timezone, for
  synchronization with other test components if
  applicable.

8
SUT

• SUT system under test
• As the profile addresses only black-box testing,
  the SUT is only assumed to provide a set of
  operations via publicly available interfaces.
• No internal information about the SUT can be
  assumed to be available.

9
Arbiter

• Predefined test profile interface
• Implementations determine the final verdict for a
  test case.
• Operations
• getVerdict(), setVerdict(Verdict)
• Every validation action should invoke the
  setVerdict() operation.
• Suggested semantics
• Verdicts can only be moved to the right in the
  listPass gt Inconclusive gt Fail gt Error
• Every test context must have an implementation of
  the Arbiter interface.
• Normally, a tool environment would provide such
  an implementation from its framework.

10
Scheduler

• Predefined test profile interface
• Purpose control the execution of various test
  components.
• Controls the creation and destruction of dynamic
  test components, and keeps information about
  which test components exist.
• Knows which components take part in each test
  case.
• Collaborates with an Arbiter to issue the final
  verdict.
• Operations
• startTestCase(), finishTestCase(TestComponent),
  createTestComponent(TestComponent)
• Every test context must have an implementation of
  a Scheduler.
• Normally, a tool environment would provide such
  an implementation from its framework.

11
Test Behaviour

• Concepts
• Test case
• Test invocation
• Test objective
• Stimulus
• Observation
• Coordination
• Default
• Verdict
• Actions Validation, Log, Finish
• Test Log

12
Testing Profile Test case and Test objective
13
Test Case (1)

• In the UML testing profile, TestCase is a
  stereotype applied to an Operation or Behaviour.
• Example ltltTestCasegtgt someMethod()
• Represents a complete technical specification of
  how the SUT should be tested for a given test
  objective.
• Belongs to a test context which includes test
  components and the system under test.
• Uses the test contexts Arbiter to return a
  Verdict.
• During execution, the SUT must reproduce all
  specified behaviour.
• A TestCase is invoked by the test contexts
  Scheduler.
• May invoke other test cases.

14
Test Case (2)

• Any UML operation / behaviour specification can
  be used to describe test case behaviour, as well
  as several constructs added by the testing
  profile.
• Defined in terms of sequences, alternatives,
  loops, and defaults for stimuli and observations
  to/from the SUT.
• Example sequence diagram, state chart, activity
  diagram, use case diagram, UML text code.

14
15
Testing Profile Interactions
16
Test Objective

• A named element describing what should be tested.
• Associated to a test case.
• Use is for documentation purposes the objective
  is not part of the behaviour specification for
  the test case.
• Verdicts should be assigned with respect to the
  test objective
• Pass and Fail verdicts indicate that the test
  objective was met / not met other verdicts used
  in other situations.

17
Test Invocation

• A test case can be invoked with specific
  parameters within a specific context by the
  Scheduler.
• Leads to the execution of a test case.
• Denoted in the test log.

18
Stimulus and Observation

• Stimulus
• Test data sent to the SUT in order to control it,
  and to make assessments about the SUT when
  receiving reactions to the stimuli.
• Observation
• Test data reflecting the reactions of the SUT
  used to assess SUT correctness.

19
Coordination

• Concurrent (and potentially distributed) test
  components have to be coordinated both for
  functionality and time, in order to assure
  deterministic and repeatable test executions
  resulting in well-defined verdicts.
• Done explicitly with normal message exchange
  between components, or implicitly with general
  ordering mechanisms.

20
Testing Profile Defaults
21
Default

• A behaviour triggered by a test observation that
  is not handled specifically by the test case.
• Defaults depend on the type of diagram used for
  test case behaviour specification.
• Example for a state machine, a default
  specification would describe what to do when an
  unexpected event arrives in the current state.
• Possibilities ignore and continue, fail, error,
  finish
• Normally specified by tool framework.

22
Verdict

• Predefined enumeration datatype.
• Assessment of the correctness of the SUT.
• Test cases yield verdicts.
• Minimum set of verdicts
• Pass test behaviour gives evidence for
  correctness
• Fail purpose of the test case has been
  violated.
• Error exceptions within the test system
• Inconclusive neither a verdict of Pass nor Fail
  can be given
• Additional user-defined verdicts are permitted.

23
Testing Profile Actions
24
Actions

• Validation Action
• An action to evaluate the status of the execution
  of a test case by assessing the SUT observations
  and/or additional characteristics of the SUT.
• Performed by a test component and sets the local
  verdict of that test component.
• Log Action
• An action to record information in the test log.
• Finish Action
• Action indicating that a test component has
  concluded its current behaviour specification
• Does not imply component termination.

25
Testing Profile Test Log
26
Test Log

• An interaction resulting from the execution of a
  test case.
• Represents the messages exchanged between the
  test components and the SUT and/or the states of
  the involved test components.
• Associated with a verdict.

27
MOF (Meta-object facility) specificationTest
Architecture and Test Behaviour
28
Test Data

• Concepts
• Wildcard
• Data selector
• Data pool
• Data partition
• Coding rule

29
Testing Profile Test Data
30
Wildcard

• Used for a loose specification of data to be
  returned from the SUT.
• UML 2.0 provides
• LiteralNull indicating an omitted value.
• Notation NULL.
• Testing profile extensions are
• LiteralAny value must be present, but have any
  value.
• Notation ?
• LiteralAnyOrNull any value or omitted.
• Notation

31
Data Selection

• All of the following are UML testing profile
  stereotypes that can be applied to class(ifiers).
• Data Selector
• An operation that defines how data values or data
  equivalences are selected from a data pool or
  data partition.
• Data Pool
• A set of values used to repeat invocations of the
  same Test Case behaviour, but with different data
  values.
• Data Partition
• A set of values used to indicate equivalence
  classes of data, from which one value can be
  selected.

32
Coding Rule

• Standard encoding schemes used by SUT interfaces.
• Examples XML, ASN.1, IDL, CORBA IIOP

33
MOF Specification Test Data
34
Time

• Timers
• Timezone

35
Time / Timer
36
Timer

• A mechanism that may generate a timeout event
  when a specified time value occurs.
• May be when a pre-specified time interval has
  expired, relative to a given instant (usually
  when the timer was started).
• Belongs to a Test Component.
• Operations
• start(Time)
• stop()
• read() return expiry time of timer
• isRunning() true if the timer is active

37
Testing Profile Timezone
38
Timezone

• A grouping mechanism for test components
• Each test component belongs to a specific time
  zone.
• Test components in the same time zone are
  synchronized.

39
Implementations of the UML testing profile

• Implemented in Telelogic Taus UML tool Tau.
• Implementation of Eclipse Test and Performance
  Tools Platform is based on a preliminary draft of
  the profile, as well as the Eclipse Modeling
  Framework (EMF).
• Basis for IBM/Rationals tool suite.
• An example implemented in Telelogic Tau follows.

40
SUT Classes and Interfaces

• Class to be tested is PingPong.
• One signal, Ping, can be sent to the class via
  port aPort.
• One signal, Pong, with an integer parameter, can
  be received from the class via port aPort.

41
SUT - Behaviour

• When the SUT receives a Ping signal, it will
  return a Pong signal.
• The first Pong signal will have parameter value
  1 subsequent Pong signals will increment the
  parameter value.

42
Using the UML profile for testing

• Steps
• Create a test context
• Make the test context aware of UML model elements
  for system under test
• Create test component(s)
• Choose classes to act for the stereotypes
  ltltTestComponentgtgt and ltltSUTgtgt.
• Create sequence diagrams for test cases
• Add sequence diagrams to test context.

43
Add in the test profile

• The UML test profile is a Tau add-in that must
  be activated for a project.
• When the add-in is activated, the profile is
  added to the libraries section of the model.

44
Test context

• When a new test context is created, and a class
  is selected for the system under test, the test
  configuration is created automatically.
• Class MyTestComponent is generated automatically,
  as well as the test architecture.

45
Test Configuration

• The class PingPongTest is stereotyped with
  ltltTestContextgtgt
• Inside this class is an instance of a class
  stereotyped with ltltTestComponentgtgt, along with
  an instance of a class stereotyped ltltSUTgtgt
• The connector c1 provides a path for signals to
  be sent to/from the SUT.

46
Test case specified asinteraction diagram
47
Test case that will fail
48
Run the test cases
49
Test Log