Emulation as Test Infrastructure

1
Emulation as Test Infrastructure

• Dan Voss, Varolii Corp.
• SASQAG
• May 15, 2008

2
Outline

• Emulation past, present, and future
• How a software emulator works
• Emulation benefits
• Demonstration vs. Hardware Target
• Automated Test in Emulation
• Automated Test demonstration
• Q A

3

• Emulation is the ability of one system to
  operate like another system.
• Then came microprocessors.
• (def.) Computers. Imitation (of a particular
  computer system) by using a software system,
  often including a microprogram or other computer
  that enables it to do the same work, run the same
  programs, etc., as the first. 1

4
Emulation Past

• Emulates hardware

1970s
2000s
1980s
1990s
5


An early hardware emulator 2
6

Emulation Past

• Emulates hardware
• Primitive but expensive

1970s
2000s
1980s
1990s
7

brain behind the emulator 3
8

Emulation Past

• Emulates hardware
• Primitive but expensive
• Few alternatives to chip makers systems

1970s
1980s
1990s
2000s
9
Emulation Past

• Problems mounted
• Faster speed vs. native mode emulation
• Primitive tools vs. larger code size
• Single-user systems vs. larger teams
• No role for Test

1970s
1980s
1990s
2000s
10
Emulation Present

• Breakthrough at 32-bit target system
• Lower cost tools on PC platform
• Standard desktop OS (Linux)
• Widely supported environment

1970s
1980s
1990s
2000s
11
Emulation Present (contd)

• Emulate software source files
• Tool environment is like the target
• Everybody can have one

1980s
1990s
2000s
1970s
12

Emulation Future

• Emulate the system
• Niche Windows apps on Mac
• Broad system virtualization

1980s
1990s
2000s
1970s
13
software emulation details

• if emulator build
• emulator code for this operation
• else
• target code for this operation
• endif
• Sprinkled throughout the code!

14
software emulator architecture

app code
app libraries
OS libraries
OS kernel
OS kernel

drivers
devices
15
Project Charter

• Contract on interface code design
• Aim for perfect emulation
• SDET commitment

16
Perfect emulation benefits

• All Everybody can have one
• Team groups work independently
• Test Immediate access to app code
• Lower integration risk
• Test is a full partner

17
cell handset (8/04)

• Emulator implemented
• keypad, extra keys
• display
• browser
• messaging
• skin interface
• battery monitor
• RF module signaling
• Missing voice channel
• ringtones

Curitel G400B GSM 4 band
18
WiFi Media Player (11/07)

• Emulator implemented
• keypad, extra keys
• display
• browser
• WiFi connection
• audio (limited formats )
• video (limited formats)
• battery monitor
• Missing Bluetooth, FM radio

Haier ibiza player 30 GB hard drive
19
Demo emulator vs. target

• Play same audio on both
• Play same video on both
• Display same picture on both

20
Emulation for automated test

• Measurement needs app state
• Examples key up/down WiFi connection
• screen location audio playback
• backlight video playback
• memory used input buffer
• Keep it lightweight

21
Varia Automated Test Artifacts

• Smoke test for continuous integration
• Functional test for stability
• Automated test cases (e.g., critical, tedious)

22
Automated Test Demonstration

Stress test example
23
Conclusions

• Emulation has come a long way
• Perfect emulation is a productive infrastructure
  to get a head start on challenging embedded
  applications
• Perfect emulation enables early testing
• Perfect emulation encourages automated test

24
References

• 1 Websters Encyclopedic Dictionary of the
  English Language, 1996
• 2 Intel MDS in-circuit emulator photo
• www.classiccmp.org/hp/mds-800/ice-80.jpg
• 3 Intel MDS main unit photo
• www.google.com/imgres?imgurl
• http//www.classiccmp.org/hp/mds-800/ice-80.jpg
• imgrefurlhttp//www.classiccmp.org/hp/mds-800/
• mds-800.htmh100w133sz60tbnid
• -6NLxJ5U9ocJtbnh100tbnw133prev
• /images3Fq3DintelsaXoiimage_result
• resnum1ctimagecd2

25
Q A

26
Emulation as test infrastructure

• Dan Voss, Varolii Corp
• SASQAG 5/15/2008
• Abstract
• Emulation tools are particularly suited to
  software for embedded applications.
• Early emulators were based on hardware, limited
  in resources, and proprietary.
• They were poor tools to develop code, and just
  about useless for testing.
• Processor performance has improved tremendously,
  however, and the situation
• is much better today. A prime reason is targets
  can now run a conventional desktop
• operating system. I will describe several
  important benefits from this for Development
• as well as Test, and what it takes to make
  infrastructure based on emulation workable
• There will be a demonstration comparing emulator
  operation with the real target for a
• portable entertainment product. Emulation
  encourages automated test, and an
• example from the same project will be
  demonstrated.