Automated Testing Environment

1
Automated Testing Environment

• Concepts required for testing embedded systems
  adopted in this course
• (quizzes, assignments and laboratories)

2
To be tackled today

• Why test, and what kinds of tests are there?
• Difference between
• TLD Test Last Development
• TDD Test Driven Development
  (Test First Development)
• How do you add tests?
• More details on the testing process E-TDD and the
  testing toll E-UNIT
• Other kinds of available tests time and access
• Design of custom TESTs and ASSERTS

3
Why test?

• Unit test
• I AM CURRENTLY BUILDING A NEW FUNCTION does it
  work the way I expect?
• Testing my and my partners understanding of the
  Design
• WE ARE DESIGNING A NEW PRODUCT-- If we cant
  explain (to ourselves via a test) the product
  ideas how do we know when product is working?
• Regression testing
• MY PARTNER ADDED A NEW FEATURE to a product how
  do I know it did not break something I wrote?
• System testing
• PROOF TO THE CUSTOMER that the product (Lab. or
  assignment) works the way that was requested.

4
What are the problems with this sort of test?
5
VisualDSP development environment(IDDE)

Lab. station (ICT318 / 320)
BF533 Evaluation Board
Each time we want to send a message (via printf(
) or cout) we must stop the processor from
doing what we want it to do, and then send
messages over the interface Slow, plus the
testing can impact on peripheral
performance. E.g. no sound while printing in
Assignment 2 and Lab. 1
6
More issues with this sort of test!

• Tests are mixed up with production code
• Must remove tests before release to customer
• Unreliable may result in test messages at wrong
  time in released code
• Difficult to add and remove tests
• Cant be automated therefore unlikely to have
  tests run often during development leads to
  unreliable product

7
E-TDD Tool

• E-TDD embedded test driven development
• Build-the-tests-first (during design) ideas for
  product development has been made popular by the
  Agile approach
• Requires an automated testing approach
• In this class, we are going to use the E-Unit
  tool for its automated behaviour
• Will use in both test last development (what
  most people currently do) and in test first
  development.
• I (the customer) will provide tests so that you
  can check that you have developed the right
  stuff at the high level
• You will build additional tests to check your own
  code at the low level

8
Test Environment

• Can download LabTestEnvironment.zip file from
  the web (Check the September webpages for exact
  name of file)
• LabX directory iswhere you keepyour final
  code
• LabXTests is whereyou keep the testsfor LabX
• E-UNITIncludes directory contains everything
  needed to make test environment work

E-UNITIncludes
9
Build a new VDSP project Assignment1Test and
place in AssignmentTests directory
E-UNIT name replaces E-TDD name

• Now add certain standard files to the project
  same set of files for all assignments and
  laboratory
• All found in E-UNITIncludes directory

07.dlb
07.dlb
07.h
07.h
Expect minor changes (names and dates) as we
improve E-UNIT as part of our own going
research work on embedded systems
10
Now add your Assignment1 subroutines files to the
Assignment1Test project

• Dont move or copy the files into the test
  directory just add them to the project
• Note the use of Assignment1.h which has the
  prototypes for all C and assembly code
  functions used (from Assignment dir.)

This files are added to theTest project and
NOT copiedfrom the Assignment1 directory
07.dlb
E-UNIT name replaces E-TDD name
07.dlb
11
Now we need to develop and add Testmain.cpp
E-UNIT name replaces E-TDD name
12
Now we need to develop and add Assignment1Tests.cp
p
1 2 3 4 5 6 7 8 9
../E-UNITIncludes/E-UNIT_Tests1Sept07.h
13
Tests have a standard format
1 3 4 8 9
../E-UNITIncludes/E-UNIT_Tests1Sept07.h
TESTGROUP NAME
TEST TYPE
TEST NAME
Add further tests at this location
TESTGROUP NAME,
TEST TYPE
14
Tests then need to be customized
1 2 3 4 5 6 7 8 9
../E-UNITIncludes/E-UNIT_Tests1Sept07.h
07h
15
Test Components

• The ACTUAL Test (This is a TEST MACRO)
• CHECK(result1 result2)
• Would provided better test messages if we
  self-documented the code
• CHECK(result_CPP result_ASM)
• Test CONTROL statements (ALL MACROS)
• TEST_CONNECT( test_name)
• LINK_TEST( test_name, test_type)
• TEST_LEVEL( which_test_level)

16
Stages to automating the tests

• Activate the E-TDD Gui (E-TDDIncludes directory)
• This causes the GUI to appear, and adds new
  menus to VisualDSP

17
Stages to automating the tests
2) Select E-UNIT Connection Refresh Connection
List This automatically builds the E-Unit Test
Info file
18
Now build and run the tests
Quick test failure visualization
Detailed test failure information
Clicking on the detailed test infoautomatically
takes you to the test that failed
19
Test information modes arecontrolled via
ActivateTests.cpp

• Show Failures only
• Show Failures and Successes

BTC_CONNECTION
BTC_CONNECTION
20
Add a separate test to show that result2(from
the ASM code) is equal to 7
../E-UNITIncludes/E-UNIT_Tests1Sept07.h
Add further tests at this location
21
Write the tests for self-documenting
messages(What does result1 result2 mean if
200 tests present?)
HAVE
WANT
22
Write the CPP code to get to workbut Write
the ASM for speed
../E-UNITIncludes/E-UNIT_Tests1Sept07.h
FAIL
SUCCESS
SUCCESS
23
Did the ASM code destroy or even use the
CPParray defined in CPPassignment1.cpp?
24
Available ASSERTS

• LONG INT tests 32 bit typical for C
• CHECK(condition) Success if passes
• CHECK_EQUAL(expected, actual)
• EXPECTED_FALSE(condition) Success if fails
• ARRAYS_EQUAL(expected, actual, length)
• CHECK_VALUES(expectedTemp, condition, actualTemp)
• DOUBLES_EQUAL(expected, actual, threshold)

25
ASSERTS for embedded systems

• Short int tests 16 bit typical for embedded
  systems
• SHORTS_EQUAL(expected, actual)
• SHORTS_CHECK(expected, condition, actual)
• USHORTS_EQUAL(expected, actual)
• USHORTS_CHECK(expected, condition, actual)
• Timing of routine Blackfin Specific
• MAXTIME_ASSERT(max_time, function)
• Useful utility
• MEASURE_EXECUTION_TIME(timetaken, function)
• Examples available in the tests associated with
  Lab. 1

26
New ASSERTS can be added to customize the tests
or speed the tests

• ASSERT FORMAT
• define CHECK_VALUES(expectedTemp, condition,
  actualTemp)\
• if (!((expectedTemp) condition (actualTemp))) \
• \
• FAILURE1(expectedTemp" !"condition"
  "actualTemp)\
• \
• else \
• SUCCESS1(expectedTemp""condition"
  "actualTemp)\

27
Practice Assert Design

• Design an assert that will determine whether the
  lower 8 bits of a 32-bit variable is identical to
  the lower 8 bits of a second 32-bitvariable.
• The two variables should remain unchanged by the
  assert (since they may be reused later)
• The upper 24 bits of each variable MUST be
  ignored in the assert
• Does it matter if the variables were defined in
  the .data section of an assembly code routine or
  in an C code routine?

28
Handled today some easy non-programming quiz
and midterm questions

• Why test, and what kinds of tests are there?
• Difference between
• TLD Test Last Development
• TDD Test Driven Development
  (Test First Development)
• How do you add tests for Assignment 1?
• More details on the testing process E-TDD and the
  testing tool E-UNIT
• Other kinds of available tests time and access
• Design of custom TESTs and ASSERTS