University of Colorado at Colorado Springs

1
University of Colorado at Colorado Springs
Design of a Parametric Outlier Detection System

• Ronald Erickson
• as part of the requirements for the degree of
• Master of Engineering in Software Engineering

2
Example Integrated Circuit Test Floor
Configuration

• A typical ATE test floor example is shown above.
  This type of configuration allows each platform
  to continue to operate independent in the event
  of a network problem. A database server with
  outlier detection PC has been shown attached to
  this configuration.

3
Background Motivation Information

• To stay competitive in the global market,
  manufacturing, packaging, testing activities
  can be implemented anywhere. DfM can be used as a
  potential pre-test screen, but do not account for
  all failure mechanisms or the effects of
  commercial radiation hardened (CRH) implant.
  Ultimately electrical test and robust statistical
  screening of very large amounts parametric data
  must be accomplished to ensure the highest
  quality products are shipped to customers.
• By utilizing a fab-less manufacturing system with
  no capital costs, it allows the company stay
  price competitive. However by using different
  suppliers many differing failure variables can be
  introduced into the final integrated circuits.
• Integrated circuit defects are discovered during
  the electrical test (exercise) of the integrated
  circuit. These tests are performed on wafers and
  packaged devices
• Companies need a tool that processes large
  amounts of device parametric data and performs a
  robust statistical analysis of the parametric
  data in an effort to screen out of family
  parametrics within a lot, product line,
  technology, or fab.

4
Goals of this project

• The proposed tool would use parametric data, and
  perform a test for normality.
• After a parametric value distribution has been
  deemed normal. The 75th percentile analysis by
  quartile is completed
• If the test data distribution pass a normalcy
  test then the data will be presented in a
  histogram format for an engineer to intervene and
  set the lower and upper fences of the
  distribution.
• In the case of a bi-model or non-normal
  distribution the data will be presented as a
  histogram, an engineer will have decide which
  distribution to use as the correct
  representation.
• Engineer decisions into the histogram formatted
  fence boundaries decisions will be captured into
  the database for future use.

5

Challenges of this Project

• Automatic Distribution Modeling and
  implementation of very complex equations
• Anderson-Darling with quartile analysis of
  normalized data only ties OR
• Shapiro-Wilks with quartile analysis of
  normalized data only ties. OR
• Skewness-Kurtosis All with quartile analysis of
  normalized data only not robust.
• If the data set is a fails the normalcy tests due
  to bi-model or a non-normal distributions,
  including too many ties encountered.
• Can an alternative implementation be performed
  when false positive failing normalcy is
  implemented on Anderson-Darling tie related
  fails?
• Data set will be shown in a histogram format and
  require user intervention to pick the data
  modeling performed.
• Future enhancements on non-normal data sets will
  utilize machine learning to track the type of
  data modeling that was performed by user, device
  type, and distribution. All user interventions
  must be tracked for future analysis.

6
Tasks

• Already Complete
• Developed an application in C to analyze
  datasets per the reference Precision Estimates
  for AASHTO Test Method T308 and the Test Methods
  for Performance-Graded Asphalt Binder in AASHTO
  Specification M320
• In Progress Intent to complete in Spring 2011
  Semester
• Develop the normalcy tests
• Shapiro-Wilks
• Anderson-Darling
• Skewness-Kurtosis All
• Develop the quartile tests on the normalized
  data.
• Future Intent to complete in Spring 2011
  Semester
• Analyze the results of pre-determined datasets
  to the test-beds.
• Write the project report

7
Deliverables

• The outlier detection test-bed, including a
  device parametric data-log loaded into a database
  and a data modeling response that resembles a
  real product manufacturing scenario.
•  
• The outlier detection engine code that implements
  the Anderson-Darling, Shapiro-Wilk , or
  Skewness-Kurtosis All normalcy tests. Then if the
  data set is normal complete a 75th percentile on
  the data set.
• If the data set does not pass the normal
  distribution tests, then present the data in a
  histogram format for user intervention.
•  
• A masters project report documenting the outlier
  detection design and the results of implementing
  the data-log within the outlier detection design
  prototype.
• An analysis report describing the software
  engineering principles selected and how the
  selected techniques are applied in the outlier
  detection implementation.

8
References

• Anil Kumar Jain, M Narasimha Murty, Patrick
  Joseph Flynn Data clustering a review. ACM
  Computing Surveys Volume 31, Issue 3, Pages 264
  323, September 1999.
•  
• Ronald Holsinger, Adam Fisher, Peter Spellerberg,
  Precision Estimates for AASHTO Test Method T308
  and the Test Methods for Performance-Graded
  Asphalt Binder in AASHTO Specification M320.
  National Cooperative Highway Research Program,
  AASHTO Materials Reference Laboratory,
  Gaithersburg, Maryland, February, 2005
•  
• Joao Gama, Pedro Pereira Rodrigues, and Raquel
  Sebastiao Evaluating Algorithms that Learn from
  Data Streams. ACM SAC '09 Proceedings of the
  2009 ACM symposium on Applied Computing, March
  2009.
•  
• Jennifer G. Dy, and Carla E. Brodley Feature
  Selection for Unsupervised Learning. JMLR.org
  The Journal of Machine Learning Research , Volume
  5, December 2004.
•  
• Tony Jebara, Jun Wang, and Shih-Fu Chang Graph
  Construction and b-Matching for semi-Supervised
  Learning. ACM Proceedings of the 17th ACM
  international conference on Multimedia, October
  2009.
•  
• David Moran, Daria Dooling, Tom Wilkins, Ralph
  Williams, and Gary DitlowIntegrated
  Manufacturing and Development (IMaD). ACM
  Supercomputing '99 Proceedings of the 1999
  ACM/IEEE conference on Supercomputing, Jan 1999.
•  
• R.A Perez, J.T Lilkendey, and S. W Koh. Machine
  Learning for a Dynamic manufacturing Environment.
  ACM SIGICE Bulletin , Volume 19, Issue 3 ,
  February 1994.
•  
• 8 Khaled Saab, Naim Ben-Hamida, and Bozena
  Kaminska Parametric Fault Simulation and Test
  Vector Generation. ACM Proceedings of the
  conference on Design, automation and test in
  Europe, January 2000.

9
Prototype Application View 1

• This application performs histogram fence choice
  and quartile of the reference Precision
  Estimates for AASHTO Test Method T308 and the
  Test Methods for Performance-Graded Asphalt
  Binder in AASHTO Specification M320 see go 195

10
Prototype Application View 2

• This application performs histogram fence choice
  and quartile of the reference Precision
  Estimates for AASHTO Test Method T308 and the
  Test Methods for Performance-Graded Asphalt
  Binder in AASHTO Specification M320 see go 196

11
References Continued

• Soumenda Bhattachatya and Abhijit Chatterjee.
  Optimized Wafer-Probe and Assembled Package Test
  Design for Analog Circuits. ACM Transactions on
  Design Automation of Electronic Systems (TODAES)
  , Volume 10 Issue 2, April 2005.
•  
• Wei-Shen Wang and Michael Orshansky Robust
  Estimation of Parametric Yield under Limited
  Descriptions of Uncertainty. ACM ICCAD '06
  Proceedings of the 2006 IEEE/ACM international
  conference on Computer-aided design, November
  2006.
•  
• Anne Gattiker Using Test Data to Improve IC
  Quality and Yield. IEEE Press ICCAD '08
  IEEE/ACM International Conference on
  Computer-Aided Design, November, 2008
•  
• Ashish Kumar Singh, Murari Mani, and Michael
  Orshansky Statistical Technology Mapping for
  Parametric Yield. IEEE Computer Society ICCAD
  '05 Proceedings of the 2005 IEEE/ACM
  International conference on Computer-aided
  design, May 2005.
•  
• Kees Veelenturf The Road to better Reliability
  and Yield Embedded DfM tools. ACM Proceedings of
  the conference on Design, automation and test in
  Europe, January 2000.
• Erik Jan Marinissen, Bart Vermeulen, Robert
  Madge, Michael Kessler, Michael Muller Creating
  Value Through Test DATE '03 Proceedings of the
  conference on Design, Automation and Test in
  Europe - Volume 1, March 2003.