Testing Localized Applications

1
Testing Localized Applications

• From Text to Code

Rocío Chavarriaga, MA xldrummer_at_yahoo.com Present
ed at ATAs 43rd Annual Conference
2
Contents

• Understanding software quality assurance
• Testing in the localization process
• Skills and knowledge the linguist needs for
  testing localized software
• Resources hand-out

3
Software Quality Assurance

• Types of Testing
• Functionality test
• Patch/Fix verification
• Internationalization test
• Regression and acceptance test Usability
  test (end-user test)
• Performance/Load test
• Linguistic review
• Localized version functionality test
• Localized version regression,
• Acceptance and usability (delivery) tests

Source program
Localized program
4
Software Quality Assurance

• Internationalization (I18N) test
• Ideally, the set of tests performed on the
  original program prior to localization. Issues
  such as hard-coded strings, concatenation,
  character sets, encoding, and support for locales
  are examined. This is the program owners
  responsibility. Doing a thorough
  internationalization test will prevent problems
  during the localization phase.

5
Localization Quality Assurance
Localization (L10N) Testing

• - Linguistic review
• - Functionality testing
• - Test scripts
• Application used for test scripts
• Adaptation of test for localized product
• - Bug reports
• - Bug tracking
• - Regression testing
• - Acceptance and usability tests
• - Performance/Load testing

6
Localization Quality Assurance

• Linguistic review
• - Grammar, syntax (and proper place of
  variables), spelling
• - Word wrap hyphenation
• - Correct display of diacriticals and fonts
• - Limit of string length and abbreviations
• - Truncation
• - Consistency
• - Culturally appropriate graphics
• - Correct icons for localized OS/Locale
• - Detect, report and correct bugs

7
Localization Quality Assurance

• Functionality testing
• - Check the functionality of the user interface
• - Check the aesthetics of the user interface
• - Test directly on localized platform when
  available
• - Test databases and other supporting software
  and plugins
• - Verify hot keys and support for international
  keyboard layouts
• - Detect and report bugs
• - Verify that bugs can be reproduced in original
  version

8
Localization Quality Assurance

• Test scenarios and bug reports
• - Scenarios (controlled environment) for use of
  software program User actions and expected
  outcome.
• - Scenarios are part of a test of a particular
  component.
• - Pass or Fail grading.
• - Scenarios can be detailed and lengthy, or
  limited to simple hierarchical scripts. If the
  latter is the case, a separate test script might
  be needed for error messages.
• - Bug reports Tester needs to document the steps
  that lead to the detection of the problem. It is
  necessary to verify that bugs can be reproduced
  in the original version.

9
Localization Quality Assurance

• Bug TrackingOnce a set of tests is completed,
  the QA team must keep track of bugs and report
  periodically on the status of the bug. Bug
  reports are best kept in bug databases (as
  opposed to random messages, tables, or comment
  fields in test scenarios). A bug needs to be
  described in a meaningful yet concise way, and
  linguistic issues need to be explained (chances
  are the engineer in charge of the fix does not
  know the localized language).

10
Localization Quality Assurance

• And when the bug has been fixed...
• When the bug reported is of a linguistic nature,
  the linguist is asked to go through a test
  scenario once again to verify the fix.
• If the bug is of a functional nature, the QA
  team first verifies the fix and logs the bug as
  needing verification in the next round of
  testing.When a functional bug is encountered in
  the localized version of a product, the source
  language version and all other localized versions
  need to be verified as well.

11
Localization Quality Assurance

• Regression Testing After making changes to a
  software product, it is necessary to make sure
  the original code works with the new changes. To
  do this, prior to the release of a new version,
  the older tests are perfomed again to ensure the
  integrity of the original functionality.
  Similarly, localized versions need regression
  testing when there are changes or upgrades.

12
Localization Quality Assurance

• Acceptance and usability tests
• The original development team conducts an
  acceptance test, in which a general verification
  of the release is done.
• A person who fits the profile of the products
  end-user (usually somebody not involved in the
  product development or testing) is then asked to
  conduct a general test of the product, report
  problems, and make suggestions. This is the
  usability test, and in the localization process,
  the equivalent is usually known as the delivery
  test.

13
Localization Quality Assurance

• Performance/Load testing
• Certain complex applications, for instance,
  those that draw data from large databases at
  running time, need to be tested for the amount of
  data the application can process without
  malfunctioning (or the amount of users that can
  access an application at a given time).
• These testing phases may not apply to all kinds
  of software products and might not be a part of
  the localization process.

14
What the linguist needs to know

• Sample of a testing job ad
• ...organizes and executes the testing of
  complex areas of a product using existing test
  templates and automated scripts. Prepares test
  plans. Logs defects and prepares defect tracking
  reports for management. May develop simple
  automated scripts. May participate in user
  documentation reviews. Working closely with the
  internationalization and localization team
  members in the product group to understand
  specific requirements. Writing appropriate
  internationalization and localization test plans
  and test cases and updating them as needed.
  Testing all internationalization aspects of the
  product(s) developed by the product group to
  ensure it meets the stated internationalization
  requirements.
• (continues)

15
What the linguist needs to know
Job Ad, continued Testing all aspects of the
localized product(s) to ensure it meets the
stated localization requirements. Reporting
all test results to the relevant product
group. Must have 3-5 years experience in
internationalization and localization testing.
Knowledge of Japanese or Simplified Chinese.
Solid understanding of all aspects of
internationalization and localization testing,
including test and use case creation as well as
tools. Experience working with developers to
ensure all test reports are clearly understood.
Strong organizational skills, excellent
interpersonal skills, ability to communicate
effectively verbally and in writing.
16
What the linguist needs to know

• Must be familiar with different operating systems
• Must become familiar with the product being
  tested
• Must be familiar with general software quality
  assurance procedures
• Must be analytical
• Must understand what a test case is
• For applications with a GUI, must know how to
  make screen shots
• For other applications, the tester needs to be
  able to accurately identify specific functions

17
How to acquire some knowledge

• Suggested exercise
• 1. Download the localized version of an
  application you are completely unfamiliar with
  (obviously something compatible with your OS).
• 2. Look closely at every screen, from the
  installation to the license agreement to the main
  program screen. Read everything. Note down
  every problem you find in a bug report (no
  editing for style at this stage!).
• 3. If the program has a tutorial, do the
  tutorial. If not, create a new document or file
  (use a wizard when available) to get familiar
  with the interface.

18
How to acquire some knowledge

• 4. Follow the logical hierarchy of the software
  program Explore each option of every menu and
  click to see what happens.
• 5. After looking at all possible options,
  plug-ins included, examine the entire contents of
  the help files (clicking on every jump, link and
  pop-up).
• 6. Keep a log of all the activities performed
  and the location of the options used. Example
  Created a new doc. using a template in
  File/New/Templates/Business letter.

19
A screen shot with linguistic issues
In this screen there is a mix of Spanish and
Italian
20
Example of a bug report
In this example of a linguistic issues-only
remote test, the linguist is given a table format
report to submit bugs. Left to right The 1st
column identifies the actual test case (an
internal ref. ), the 2nd column contains a brief
description of the test, the 3rd column is the
actual sequence of actions required for the test,
and the 4th and 5th columns are for screen shots
and bug description.
21

• THANKS FOR DOWNLOADING THIS PRESENTATION!
• RESOURCES FOR LEARNING ABOUT SOFTWARE QUALITY
  ASSURANCE
• - Introduction to Software Quality Assurance
  http//louisa.levels.unisa.edu.au/se1/management-s
  qa/week7a96.htm
• - All about software QA in one site
  http//www.softwareqatest.com/
• - QA engineers share resources
  http//www.ics.uci.edu/cliu1/sqa/
• http//www.georgsehl.com/Profession/
• - A rather technical, but informative QA
  education site http//www.mtsu.edu/storm/
• - Microsoft's word on QA http//www.microsoft.com
  /GLOBALDEV
• - Unicode http//www.unicode.org
• - A magazine article Testing Multilingual
  E-commerce applications by Gabriella Spatolisano
  Multilingual Computing and Technology 31
  Volume 11 Issue 3.
• - Books
• Global Solutions for Multilingual applications
  by Chris Ott Wiley
• A Practical Guide to Localization by Bert
  Esselink - John Benjamins
• Testing Applications on the Web by Hung Q.
  Nguyen Wiley
• Software Quality Analysis and Guidelines for
  success by Capers Jones - ITP