Measurement Systems Analysis

1
Measurement Systems Analysis
2
Dont Let This Happen To YOU!
3
VariationThink of Measurement as a Process
4
Definition

• Measurement
• The assignment of numbers to material things to
  represent the relationships among them with
  respect to particular properties.
• C. Eisenhart (1963)

5
Measurement Systems Analysis

• Basic Concepts of Measurement Systems
• A Process
• Statistics and the Analysis of Measurement
  Systems
• Conducting a Measurement Systems Analysis
• ISO - TC 69 is the Statistics Group
• Ensures high Data Quality (Think of Bias)

6
Course Focus Flow

• Measurement as a Process
• Mechanical Aspects (vs Destructive)
• Piece part
• Continuous (fabric)
• Features of a Measurement System
• Methods of Analysis
• Gauge RR Studies
• Special Gauging Situations
• Go/No-Go
• Destructive Tests

7
Place Timeline Here
8
The Target Goal
Continuous Improvement
Production
Pre-Launch
Prototype
USL
LSL
9
Key Words

• Discrimination
• Ability to tell things apart
• Bias per AIAG (Accuracy)
• Repeatability per AIAG (Precision)
• Reproducibility
• Linearity
• Stability

10
Terminology

• Error ? Mistake
• Error ? Uncertainty
• Percentage Error ? Percentage Uncertainty
• Accuracy ? Precision

11
Measurement Uncertainty

• Different conventions are used to report
  measurement uncertainty.
• What does 5 mean in m 75 5?
• Estimated Standard Deviation ?
• Estimated Standard Error ?m ?/vN
• Expanded Uncertainty of 2? or 3?
• Sometimes 1? (Why?)
• 95 or 99 Confidence Interval
• Standard Uncertainty u
• Combined Standard Uncertainty uc

12
Measurement Uncertainty

• Typical Reports
• Physici

13
Measurement as a Process

• Basic Concepts
• Components of the Measurement System
• Requirements of a Measurement System
• Factors Affecting a Measurement System
• Characteristics of a Measurement System
• Features (Qualities) of a Measurement Number
• Units (Scale)
• Accuracy
• Precision (Consistency or Repeatability)
• Resolution (Reproducibility)

14
Measurement Related Systems

• Typical Experiences with
• Measurement Systems

15
Basic Concepts

• Every Process Produces a Product
• Every Product Possesses Qualities (Features)
• Every Quality Feature Can Be Measured
• Total Variation
• Product Variation Measurement Variation
• Some Variation Inherent in System Design
• Some Variation is Due to a Faulty Performance of
  the System(s)

16
The Measurement Process

• What is the Product of the Measurement Process?
• What are the Features or Qualities of this
  Product?
• How Can We Measure Those Features?

17
Measurement Systems Components

• Material to be Inspected
• Piece
• Continuous
• Characteristic to be Measured
• Collecting and Preparing Specimens
• Type and Scale of Measurement
• Instrument or Test Set
• Inspector or Technician
• AIAG calls these Appraiser
• Conditions of Use

18
Where Does It Start?

• During the Design (APQP) Stage
• The engineer responsible for determining
  inspections and tests, and for specifying
  appropriate equipment should be well versed in
  measurement systems. The Calibration folks should
  be part of the process as a part of a
  cross-functional team.
• Variability chosen instrument must be small when
  compared with
• Process Variability
• Specification Limits

19
Typical Progression
Determine Critical Characteristic
Product Engineer
How will the data be used?
Determine Required Resolution
Product Engineer
Consideration of the Entire Measurement System
for the Characteristic (Variables)
Cross-Functional
Determine What Equipment is Already Available
Metrology
20
Measurement Systems Variables
Fixture Eyesight Air Pressure Air Movement Fatigue
These are some of the variables in a measurement
system. What others can you think of?
21
Determining What To Measure
External Requirements

• Voice of the Customer
• You Must Convert to Technical Features
• Technical Features
• Failure Modes Analysis
• Control Plan

Convert To
Internal Requirements
22
Voice of the Customer
Customer may specify causes rather than output

• External and Internal Customers
• Stated vs Real and Perceived Needs
• Cultural Needs
• Unintended Uses
• Functional Needs vs. Technical Features

23
Convert to Technical Features

• Agreed upon Measure(s)
• Related to Functional Needs
• Understandable
• Uniform Interpretation
• Broad Application
• Economical
• Compatible
• Basis for Decisions

Y
Functional Need
Z
Technical Feature
24
Failure Modes Analysis

• Design FMEA
• Process FMEA
• Identify Key Features
• Identify Control Needs

Critical Features are Defined Here!
25
Automotive FMEA
Leading to MSA. Critical features are determined
by the FMEA (RPN indicators) and put into the
Control Plan.
26
Control Plan / Flow Diagram

• Inspection Points
• Inspection Frequency
• Instrument
• Measurement Scale
• Sample Preparation
• Inspection/Test Method
• Inspector (who?)
• Method of Analysis

27
GM Process Flow Chart
28
Standard Control Plan Example
This form is on course disk
29
Fords Dimensional Control Plan (DCP)
30
Measurement as a System

• Choosing the Right Instrument
• Instrument Calibration Needs
• Standards or Masters Needed
• Accuracy and Precision
• Measurement Practices
• Where
• How Many Places
• Reported Figures
• Significant Figures Rule
• 2 Action Figures
• Rule of 10
• Individuals, Averages, High-Lows

31
Measurement Error
Measured Value (y) True Value (x)
Measurement Error
Deming says there is no such thing as a True
Value.
Consistent (linear)?
32
Sources of Measurement Error

• Sensitivity (Threshold)
• Chemical Indicators
• Discrimination
• Precision (Repeatability)
• Accuracy (Bias)
• Damage
• Differences in use by Inspector (Reproducibility)
• Training Issues
• Differences Among Instruments and Fixtures
• Differences Among Methods of Use
• Differences Due to Environment

33
Types of Measurement Scales

• Variables
• Can be measured on a continuous scale
• Defined, standard Units of Measurement
• Attributes
• No scale
• Derived Unit of Measurement
• Can be observed or counted
• Either present or not
• Needs large sample size because of low
  information content

34
How We Get Data

• Inspection
• Measurement
• Test

Includes Sensory (e.g.. Beer)
Magnitude of Quality
35
Operational Definitions

• Is the container Round?
• Is your software Accurate?
• Is the computer screen Clean?
• Is the truck On Time?

36
Different Method Different Results
Method 1
Method 2
In Spec
Out of Spec
37
Measurement System Variability

• Small with respect to Process Variation
• Small with respect to Specified Requirements
• Must be in Statistical Control
• Measurement IS a Process!
• Free of Assignable Causes of variation

38
Studying the Measurement System

• Environmental Factors
• Human Factors
• System Features
• Measurement Studies

39
Standards

• National
• In the US - Kept or Tracked by NIST
• Primary
• Copied directly from National Standard using
  State-of-the-Art Equipment
• Secondary
• Transferred from Primary Standard
• Working
• Used to calibrate laboratory and shop instruments

40
Environmental Factors

• Temperature
• Humidity
• Vibration
• Lighting
• Corrosion
• Wear
• Contaminants
• Oil Grease
• Aerosols

Where is the study performed? 1. Lab? 2. Where
used? 3. Both?
41
Human Factors

• Training
• Skills
• Fatigue
• Boredom
• Eyesight
• Comfort
• Complexity of Part
• Speed of Inspection (parts per hour)
• Misunderstood Instructions

42
Human Measurement Errors

• Sources of Errors
• Inadvertent Errors
• Attentiveness
• Random
• Good Mistake-Proofing Target
• Technique Errors
• Consistent
• Wilful Errors (Bad mood)
• Error Types (Can be machine or human)
• Type I - Alpha Errors risk
• Type II - Beta Errors risk

Unaware of problem
Good
Bad
OK!
beta
Accept
Training Issue
Reject
alpha
OK!
Process in control, but needs adjustment, False
alarm
43
Measurement System Features

• Discrimination
• Ability to tell things apart
• Bias per AIAG (Accuracy)
• Repeatability per AIAG (Precision)
• Reproducibility
• Linearity
• Stability

44
Discrimination

• Readable Increments of Scale
• If Unit of Measure is too course Process
  variation will be lost in Rounding Off
• The Rule of Ten Ten possible values between
  limits is ideal
• Five Possible Values Marginally useful
• Four or Less Inadequate Discrimination

45
Discrimination
46
Range Charts Discrimination
Indicates Poor Precision
47
Bias and Repeatability
Precise
Imprecise
Accurate
Bias
Inaccurate
You can correct for Bias You can NOT correct for
Imprecision
48
Bias

• Difference between average of measurements and an
  Agreed Upon standard value
• Known as Accuracy
• Cannot be evaluated without a Standard
• Adds a Consistent Bias Factor to ALL
  measurements
• Affects all measurements in the same way

Bias
Standard Value
Measurement Scale
49
Causes of Bias

• Error in Master
• Worn components
• Instrument improperly calibrated
• Instrument damaged
• Instrument improperly used
• Instrument read incorrectly
• Part set incorrectly (wrong datum)

50
Bias and QS9000

• Bias - The difference between the observed
  Average of measurements and the master Average of
  the same parts using precision instruments. (MSA
  Manual Glossary)
• The auditor may want evidence that the concept of
  bias is understood. Remember that bias is
  basically an offset from zero. Bias is linked
  to Stability in the sense that an instrument may
  be zeroed during calibration verification.
  Knowing this we deduce that the bias changes with
  instrument use. This is in part the concept of
  Drift.

51
Bias

• I choose a caliper (resolution 0.01) for the
  measurement. I measure a set of parts and derive
  the average.
• I take the same parts and measure them with a
  micrometer (resolution 0.001). I then derive the
  average.
• I compare the two averages. The difference is the
  Bias.

52
Repeatability

• Variation among repeated measurements
• Known as Precision
• Standard NOT required
• May add or subtract from a given measurement
• Affects each measurement randomly

Repeatability
Measurement Scale
5.15 99
Margin of Error Doesnt address Bias
53
Repeatability Issues

• Measurement Steps
• Sample preparation
• Setting up the instrument
• Locating on the part
• How much of the measurement process should we
  repeat?

54
Using Shewhart Charts I
Repeatability
55
Using Shewhart Charts II
56
Evaluating Bias Repeatability

• Same appraiser, Same part, Same instrument
• Multiple readings (n10 with 20 to 40 better)
• Analysis
• Average minus Standard Value Bias
• 5.15 Standard Deviation Repeatability
• or /- 2.575 ? 99 repeatability
• or /- 2 ? 95 repeatability
• Histogram
• Probability

AIAG
True
57
Repeatability Issues

• Making a measurement may involve numerous steps
• Sample preparation
• Setting up the instrument
• Locating the part, etc.
• How much of the measurement process should we
  repeat? How far do we go?

58
Bias Repeatability Histogram
Never include assignable cause errors
59
Linearity

• The difference in the Bias or Repeatability
  across the expected operating range of the
  instrument.

60
Plot Biases vs. Ref. Values
Linearity Slope Process Variation
0.13176.00 0.79 Linearity 100 Slope
13.17
61
Causes of Poor Linearity

• Instrument not properly calibrated at both Upper
  and Lower extremes
• Error in the minimum or maximum Master
• Worn Instrument
• Instrument design characteristics

62
Reproducibility

• Variation in the averages among different
  appraisers repeatedly measuring the same part
  characteristic
• Concept can also apply to variation among
  different instruments

Includes repeatability which must be accounted
for.
63
Reproducibility Example
64
Calculating Reproducibility (I)

• Find the range of the appraiser averages (R0)
• Convert to Standard Deviation using d2
• (m of appraisers g of ranges used 1)
• Multiply by 5.15
• Subtract the portion of this due to
  repeatability

65
Calculating Reproducibility
People variance
Times done
Trials
66
Stability

• Variation in measurements of a single
  characteristic
• On the same master
• Over an extended period of time
• Evaluate using Shewhart charts

67
Evaluate Stability with Run Charts
68
Stability
Both gages are stable, but.....
69
Importance of Stability

• Statistical stability, combined with
  subject-matter knowledge, allows predictions of
  process performance
• Action based on analysis of Unstable systems may
  increase Variation due to Tampering
• A statistically unstable measurement system
  cannot provide reliable data on the process

70
Methods of Analysis
71
Analysis Tools

• Calculations of Average and Standard Deviation
• Correlation Charts
• Multi-Vari Charts
• Box-and-Whisker Plots
• Run charts
• Shewhart charts

72
Average and Standard Deviation
73
Correlation Charts

• Describe Relationships
• Substitute measurement for desired measurement
• Actual measurement to reference value
• Inexpensive gaging method versus Expensive gaging
  method
• Appraiser A with appraiser B

74
Substitute Measurements

• Cannot directly measure quality
• Correlate substitute measure
• Measure substitute
• Convert to desired quality

75
Comparing Two Methods

• Two methods
• Measure parts using both
• Correlate the two
• Compare to Line of No Bias
• Investigate differences

Magnetic
Line of Perfect Agreement
Line of Correlation
Stripping
76
Measurements vs. Reference Data
77
Measurements vs. Reference Correlation
Disparity
78
Comparing Two Appraisers
79
Run Charts Examine Stability
80
Multiple Run Charts
More than 3 appraisers confuses things...
81
Multi-Vari Charts

• Displays 3 points
• Length of bar bar-to-bar Bar cluster to cluster
• Plot High and Low readings as Length of bar
• Each appraiser on a separate bar
• Each piece in a separate bar cluster

High Reading
Average Reading
Low Reading
82
Multi-Vari Type I

• Bar lengths are long
• Appraiser differences small in comparison
• Piece-to-piece hard to detect
• Problem is repeatability

83
Multi-Vari Type II

• Appraiser differences are biggest source of
  variation
• Bar length is small in comparison
• Piece-to-piece hard to detect
• Problem is reproducibility

84
Multi-Vari Type III

• Piece-to-piece variation is the biggest source of
  variation
• Bar length (repeatability) is small in comparison
• Appraiser differences (bar-to-bar) is small in
  comparison
• Ideal Pattern

85
Multi-Vari Chart Example
Normalized Data
86
Multi-Vari Chart, Joined
Look for similar pattern
87
Using Shewhart Charts

• Subgroup Repeated measurements,, same piece
• Different Subgroups Different pieces and/or
  appraisers
• Range chart shows precision (repeatability)
• Average chart In Control shows reproducibility
• If subgroups are different appraisers
• Average chart shows discriminating power
• If subgroups are different pieces
• (In Control is BAD!)

88
Shewhart Charts

• This is not a good way to plot this data
• Too many lines

89
Shewhart Chart of Instrument
90
Gage RR Studies
91
Gauge RR Studies

• Developed by Jack Gantt
• Originally plotted on probability paper
• Revived as purely numerical calculations
• Worksheets developed by AIAG
• Renewed awareness of Measurement Systems as Part
  of the Process
• Consider Numerical vs. Graphical Data Evaluations

92
Terms Used in RR (I)
Minimum of 5. 2 to 10 To accommodate worksheet
factors

• n Number of Parts 2 to 10
• Parts represent total range of process variation
• Need not be good parts. Do NOT use consecutive
  pieces.
• Screen for size
• a Number of Appraisers
• Each appraiser measures each part r times
• Study must be by those actually using
• R - Number of trials
• Also called m in AIAG manual
• g ra Used to find d2 in table 2, p. 29 AIAG
  manual

3
4
5
1
2
1 Outside Low/High 1 Inside Low/High Target
93
Terms Used in RR (II)

• R-barA Average range for appraiser A, etc.
• R-double bar Average of R-barA, R-barB
• Rp Range of part averages
• XDIFF Difference between High Low appraiser
  averages
• Also a range, but R is not used to avoid
  confusion
• EV 5.15 Equipment variation (repeatability)
• EV 5.15 Equipment variation
  (reproducibility)
• PV Part variation
• TV Total variation

Process Variation
94
RR Calculations
Left over Repeatability
Remember - Nonconsecutive Pieces
Left over Repeatability
Product Process Variation
Measurement System Variation
95
Accumulation of Variances
96
Evaluating RR

• RR100RR/TV (Process Control)
• RR100RR/Tolerance (Inspection)
• Under 10 Measurement System Acceptable
• 10 to 30 Possibly acceptable, depending upon
  use, cost, etc.
• Over 30 Needs serious improvement

97
Analysis of Variance I

• Mean squares and Sums of squares
• Ratio of variances versus expected F-ratio
• Advantages
• Any experimental layout
• Estimate interaction effects
• Disadvantages
• Must use computer
• Non-intuitive interpretation

98
Analysis of Variance II

• The nr measurements must be done in random
  sequence a good idea anyway
• Assumes that EV repeatability is normal and
  that EV is not proportional to measurement
  normally a fairly good assumption
• Details beyond scope of this course

99
Special Gauging Situations

• Go/No-Go
• Destructive Testing

100
If Gauges were Perfect
101
But Repeatability Means We Never Know The Precise
Value
102
So - Actual Part Acceptance Will Look Like This
103
The Effect of Bias on Part Acceptance
104
Go/No-Go gauges

• Treat variables like attributes
• Provide less information on the process, but...
• Are fast and inexpensive
• Cannot use for Process Control
• Can be used for Sorting purposes

105
Short Go/No-Go Study

• Collect 20 parts covering the entire process
  range
• Use two inspectors
• Gage each part twice
• Accept gauge if there is agreement on each of the
  20 parts
• May reject a good measuring system

106
Destructive Tests

• Cannot make true duplicate tests
• Use interpenetrating samples
• Compare 3 averages
• Adjust using vn

107
Destructive Tests Interpreting Samples
AIAG does not address
108
Summary
109
Measurement Variation

• Observed variation is a combination of the
  production process PLUS the measurement process
• The contribution of the measurement system is
  often overlooked

110
Types of Measurement Variation

• Bias (Inaccuracy)
• Repeatability (Imprecision)
• Discrimination
• Linearity
• Stability

111
Measurement Systems

• Material
• Characteristic
• Sampling and Preparation
• Operational Definition of Measurement
• Instrument
• Appraiser
• Environment and Ergonomics

112
Measurement Systems Evaluation Tools

• Histograms
• Probability paper
• Run Charts
• Scatter diagrams
• Multi-Vari Charts
• Gantt RR analysis
• Analysis of Variance (ANOVA)
• Shewhart Control Charts

113
Shewhart Charts

• Range chart shows repeatability
• X-bar limits show discriminating power
• X-double bar shows bias
• (if a known standard exists)
• Average chart shows stability
• (sub-groups overtime)
• Average chart shows reproducibility
• (sub-groups over technicians/instruments)

114
Conclusion

• Rule of Ten
• Operating Characteristic Curve
• Special Problems
• Go/No-Go Gages
• Attribute Inspection
• Destructive Testing