Parallel%20Testing%20and%20TestStand

1
Parallel Testing and TestStand 2.0

• James Grey
• TestStand Software Architect

2
Agenda

• Parallel testing
• Background information
• Parallel Process models
• Synchronization

3
Benefits of Parallel Test

• Increase throughput
• Improve hardware utilization

4
Benefits of Parallel Test (Cont..)

• Allows for a natural representation of concurrent
  operations
• Multi-UUT
• Batch
• Parallel production lines
• Simultaneous control of heterogeneous devices
• Stimulus and Measurement
• Transmit and Receive
• Environmental control during testing

5
Parallelism for Speed

• Non-parallel test software is typically I/O bound
• Serial Port, Ethernet, GPIB
• Hard disk, database
• Measurement devices
• UUT, fixture control, assembly line handlers
• CPU cycles are wasted while waiting for
  operations to complete

6
Techniques for Parallel Test

• How
• Multiple computers or operating system processes
• Useful when parallel tasks do not interact
• Minimal interaction allows for simpler
  implementation
• Can be inefficient or difficult to share
  resources
• Multiple threads
• Easier to share resources
• Highest performance on a single computer

7
Multiple Processes
Program2.exe
Program1.exe
Main Thread
Main Thread
Driver Instances
Data
Driver Instances
Data
8
Multiple Threads
Program1.exe
Thread 1
Thread 2
Driver Instances
Data
9
Parallel Test Challenges

• Test Executive must support parallelism
• O/S services can require low level coding
• Coordinating access to shared resources requires
  synchronization
• Perils of synchronization
• Deadlock
• Increased complexity!

10
Parallel Test With TestStand 2.0

• Features
• Asynchronous sequence calls (Easier than in TS
  1.0.x)
• Synchronization steps
• Lock, Queue, Notification, Wait, Batch, Thread
  Priority, Rendezvous, Semaphore
• Synchronization properties
• Lock, Batch
• Batch and Parallel process models
• Benefits
• Easy to configure, no low-level coding
• Flexible and integrated

11
Types of Parallel Testing

• Batch
• All test sockets operate on a group of UUTs
• Oven test
• Independent
• Each test socket operates independently
• End-of-line functional test
• Other
• Stimulus and Response, etc.
• ltyour situation heregt

12
Process Models for Parallel Testing

• What is a process model?
• Coordinates tests, UUT handling, reporting,
  logging,
• Two new process models
• Batch
• Parallel
• Both similar to existing sequential model
• Callbacks PreUUT, PostUUT, etc.
• Configuration options

13
Parallel vs. Batch Models
14
A Batch Starts and Ends as a Unit
Socket 1
Socket 2
Socket 0
Start testing all units in the batch
Wait until all units complete
15
Synchronized Sections in a Batch
Socket 1
Socket 2
Socket 0
Synchronized Section
Wait for all units
Wait for all units
16
Synchronized Sections (Cont.)

• Three Types
• Parallel
• Serial
• One-Thread-Only

17
Parallel Sections
Socket 1
Socket 2
Socket 0
Wait for all units
No synchronization within parallel section
Wait for all units
18
Serial Sections
Each test socket executes in series
Socket 1
Socket 2
Socket 0
Wait for all units
Shared resource
Wait for all units
Shared resource
Wait for all units
19
One-Thread-Only Sections
Socket 1
Socket 2
Socket 0
Wait for all units
Only one test socket executes the actions in the
section
Wait for all units
20
Batch Testing Demo

• (show model file)
• Configurable number of UUTS
• A separate thread tests each UUT
• (run demo)
• Batch reports

21
Parallel Demo
22
How do You Identify the Test Socket?

• New properties in TestStand 2.0
• RunState.TestSockets.MyIndex
• RunState.TestSockets.Count
• Batch and Parallel process models update these
  properties

23
Asynchronous Sequence Calls

• Start your own thread or execution, no process
  model required
• Optional setting on Sequence Call Dialog Box

24
Executions vs. Threads

• An execution is a group of threads
• Begins with a single thread
• Asynchronous sequence calls create additional
  threads
• All threads in execution suspend or terminate
  when the execution suspends or terminates
• An execution completes when all of its threads
  complete

25
Executions vs. Threads (Cont.)

• Additional Execution Traits
• Can run under a process model
• Report
• Separate Window (in NI GUIs)
• Batch and Parallel models create an execution for
  each test socket
• Separate report for each test socket
• Enables test sockets to independently terminate
  or suspend

26
Synchronization and Resource Sharing

• Old Way - Synchronize in code modules
• LabVIEW Mutex, etc.
• Win32 Critical Section, etc.
• New Way Synchronize in TestStand
• Synchronization Steps
• Synchronization Properties

27
Synchronization Step Types

• Step Types
• Lock, Rendezvous, Queue, Notification, Wait,
  Batch Synchronization,
• Common Features
• Names and references
• Timeouts
• Suspension/termination friendly
• Shareable among processes and machines
• Configurable object and operation lifetimes
• Example Acquire lock for duration of sequence

28
Locks
Socket 1
Socket 2
Socket 0
Shared Resource
Lock
29
Locks - Race Conditions
Problem 1 Your program does not do what you
specify

• Operation 1
• X X 1
• Move X ? Register
• Increment Register
• Move Register ? X

• Operation 2
• X X 1
• Move X ? Register
• Increment Register
• Move Register ? X

• X starts at 0
• X increments twice
• X is now 1, not 2! The value of X is corrupt

Registers are thread specific
30
Locks - Race Conditions (Cont.)
Problem 2 Your program does not do what you
intend

• Measure A
• Apply switch route A
• Read voltage

• Measure B
• Apply switch route B
• Read voltage

• Both measurements use route B!
• Multithread safe instrument drivers dont protect
  you!

31
Locks provide Thread Safety



32
Nested Locks

• Nested locks in same thread always succeed
• Threads own locks
• Balance nested locks with unlocks

33
Locks and Deadlock

• Lock DMM
• Lock Scope
• Test

• Lock Scope
• Lock DMM
• Test

• Deadlock occurs when two threads each wait
  forever to acquire a resource held by the other
  thread.
• Acquiring multiple locks one at a time can lead
  to deadlock

34
Locks and Deadlocks (Cont.)

• Deadlock prevention
• Always acquire locks in same order
• or
• Acquire all locks in one operation
• Specify an array of locks
• Deadlock detection
• Automatic
• TestStand identifies the threads and locks that
  cause the deadlock

35
Lock Step Configuration Dialog
36
Step Synchronization Properties
Convenient method for placing a Lock or Batch
Synchronized Section around an individual step
37
Batch Synchronization Step

• All test socket threads rendezvous at the
  beginning and end of the section
• Section Types
• Parallel
• Serial
• One-Thread-Only

38
Rendezvous Step

• Waits until a specific number of threads reach
  designated locations

39
Rendezvous Cont.
40
Queue Step

• Stores any type of data
• Can send data from one thread to another
• Queue commands for worker thread to execute
• Queue results or measurements for worker thread
  to process or store

41
Queue Step Cont.
Thread 1
Thread 2
Create Queue
Acquire Data
Enqueue
Loop
42
Wait Step

• Wait for
• Thread or Execution completion
• The Wait step reports the results, status and
  error state of the thread or execution
• Time interval
• Delay without wasting CPU cycles
• Next multiple of a time interval
• Useful to control the speed of a loop

43
Notification Step

• For sending and receiving events
• An event can contain data of any type
• To receive an event, Wait on a Notification
• To send an event, Set or Pulse a Notification.
  You can send events to
• All threads
• Only the first thread to wait on the Notification
• Only threads currently waiting on the
  Notification
• Only the first thread currently waiting on the
  Notification

44
Advanced Synchronization

• Thread Priority
• Specifies operating system thread priority
• Semaphore
• Similar to Lock
• Restricts access to a resource to a specific
  number of simultaneous users
• Nested locks can block
• Batch Specification
• Use in batch process models

45
Synchronization Demo

• Locks and asynchronous sequence calls

46
Summary

• Parallelism can improve performance and hardware
  utilization
• Multithreading enables parallel operations to
  interact efficiently
• TestStand 2.0 makes it easy to launch and
  synchronize threads
• The Parallel and Batch models provide
  implementations for common parallel test
  scenarios

47
Resources

• TestStand 2.0 User Manual
• Chapter 11, Synchronization Step Types
• Chapter 13 40, Module Adapters Sequence
  Adapter
• Chapter 14, Process Models
• ltTestStand20gt\Components\NI\Models\TestStandModels
  \TestStandProcessModels.PDF
• Developer Exchange
• www.ni.com/exchange