QA Sensory Evaluation Methodologies

1
Services Provided by LCBO Quality Assurance
VQA SENSORY EVALUATION METHODOLOGY
Leading Sensory Evaluation Services
2
VQA Appellation System
VQA Mandate

• Appellation of Origin System
• Quality Standard
• Control of use of specified terms, descriptions
  and designations associated with the VQA
  appellation system

3
Role of Sensory Evaluation Testing
Quality of Beverage Alcohol Products

• Sensory Quality
• Chemical Composition
• Microbiological Stability
• Packaging and Labelling Standards

4
Role of VQA Sensory Evaluation Testing
Scope

• Free of technical faults / defects
• Typicity of varietal character for varietal
  wines
• A wine bearing varietal designation shall be
  assessed to determine if the varietal designation
  for the wine exhibits the predominant character
  of a wine produced from the designated grape
  variety/varieties (VQA Rules)
• Typicity of the wine category Late Harvest,
  Icewine, Nouveau, Sparkling Traditional Method,
  Icewine Dosage, Botrytized Wine (VQA Rules)

5
Role of VQA Sensory Evaluation Testing

• A wine shall be deemed to have passed the taste
  test if a majority of the members of the Tasting
  Panel determine
• That the wines attributes fairly reflect the
  viticultural and oenological quality standards
  established in O.Reg. 406/00 (Rules) without
  defects or flaws and
• That the wine is representative of quality wines
  of the stated category (VQA Rules)

6
Sensory Panels

• Grading Panel 4 panel groups of 5 panellists
  each
• 28 Members (20 regular panellists, 8 alternate
  tasters)

7
Panel Member Selection Criteria

• Panel members are LCBO Product Consultants who
  work in retail stores within metropolitan Toronto
  area
• Continuous training/ development opportunities
• LCBO Product Knowledge I, I, III
• Certification from the Wine and Spirit Education
  Trust
• Master of Wine Certification

8
Panel Member Selection Criteria

• Panel members are subjected to annual
• Sensory evaluation testing
• Product knowledge testing
• 25 of the questions are VQA specific
• Demonstrated professionalism Sensory
  Evaluations Code of Conduct

9
Panels Composition

• Panel Groups Assembly Criteria
• Results from the annual testing
• Testing performance history
• Previous experience (as a Grading Panel member,
  from the industry)
• Performance monitoring data
• Sensory sensitivities

Consistent Performance
Reproducible Results
Balanced Panels
10
Panel Training
Training Elements

• Aroma, flavour recognition
• Product category recognition
• Varietal character recognition
• Regional character recognition
• Winemaking techniques
• Technological influences
• Defect identification

11
Sample Presentation

• Blind sample presentation
• Samples are presented without identifying
  markings
• Uniform sample presentation

• Sample information Varietal composition, vintage
  year, wine category, method of production (where
  applicable sparkling wines, icewine dosage, etc)

12
Sensory Evaluation Method

• Sensory Evaluation Grading System

13
Objectives
Sensory Evaluation Grading System

• Superior objectivity system design eliminates
  bias
• Wider system applicability system design equally
  applies to all wine categories
• Efficient and effective calculation of results
  time savings, accuracy
• Data Analysis Tools Traceability of results
  enabled by automated data collection and
  quantification of the results

Client Satisfaction
14
Sensory Evaluation Grading System

• Sensory evaluation grading system elements
• New sensory evaluation grading method

• Integrating technology
• Automated data collection
• Automated data quantification, analysis and
  reporting

15
Sensory Evaluation Grading System
Sensory characteristics applicable to all types
of beverage alcohol products.

• Appearance and Colour
• Aroma (primary and secondary) and Bouquet
• Taste
• Harmony

Appearance
Harmony
Colour
Bouquet
Aroma
Taste
16
Sensory Evaluation Grading System

• Sensory characteristics and attributes are
  evaluated using five grading categories.

Category Excellent Very Good Good Satisfactory Unsatisfactory
Quality Level Outstanding /Exceptional Superior/ Very Correct Typical Weak/Not at full potential Faulty / Defective
17
Grading Form
Characteristics Attributes Characteristics Attributes Grading Categories Grading Categories Grading Categories Grading Categories Grading Categories Comments
Characteristics Attributes Characteristics Attributes Excellent Very Good Good Satisfactory Unsatisfactory Comments
Appearance Colour Appearance Colour x
Aroma Bouquet Correctness x
Aroma Bouquet Intensity x
Aroma Bouquet Quality x
Taste Correctness x
Taste Intensity x
Taste Finish x
Taste Quality x
Harmony Harmony x
18
Integrated Sensory Evaluation Software
Innovative Technology

• Data collection
• Data quantification
• Data analysis
• Data reporting

Compusense Five
19
Data Collection Tablet PCs

• Automated
• Interactive
• Flexible-Wireless
• Pen or/and keyboard
• Handwritten notes

20
Grading Systems Comparison
21
QA of Sensory Evaluation Results
Quality Sensory Evaluation Result
Quality Assurance
System Controls
Proficiency Programs
Verification Tasting
Performance Monitoring
22
System Controls
QA of Sensory Evaluation Results

• VQA Tasting protocol (VQA Rules)
• 2nd Bottle tasting
• Grading System Controls
• Security of panellist registration
• Forced completeness of the assessment
• Ratings cannot be changed, once finalized
• Sample evaluation cannot be redone, once
  completed
• Validation questions in the grading questionnaire
• Calculation validation based on statistical
  measures
• Automated calculation and reporting

23
Verification Tasting
QA of Sensory Evaluation Results

• Verification Tasting - an independent sensory
  evaluation of the products in conditions
  identical to the tasting panel.
• The verification process has a quality assurance
  role and provides a reference value that is then
  compared with the panel results to identify any
  discrepancies in the assessment.
• Verifiers qualified Quality Assurance tasters.
  A tasting session may require up 2 Verifiers.
• Verifiers results are typically not included in
  the calculation, unless discrepancy situation.

24
Panellists Performance Monitoring
QA of Sensory Evaluation Results

• Panellists performance is monitored for each
  tasting session
• Measures
• Outliers Frequency, - measure of rating
  accuracy
• Rating Rank measure of rating bias (high, low,
  trends)
• Sensitivity data - missed defects, good
  detection, hyper-sensitivity
• Feedback on performance Feedback Report
• Provides tools to help identify training
  opportunities

25
QA of Sensory Evaluation Results
Panellist Feedback Report
26
Sensory Proficiency Programs
QA of Sensory Evaluation Results

• Internally designed to measure quality and
  consistency of the sensory evaluation assessments
• Designed to measure
• System performance
• Panel Performance
• Panellist Performance
• Typical measures repeatability, reproducibility,
  bias, defects identification, etc.

System
Panel
Panellist
27
Repeatability
Internal Proficiency Testing
28
Internal Proficiency Testing
Repeatability
29
Internal Proficiency Testing
Reproducibility

• No significant difference between reported
  results, i.e. median values (p-Value 0.649671)
  at a 95 confidence level

30
Internal Proficiency Testing
Reproducibility

• No significant difference between panels' (i.e.
  all 5 panels) reported results (i.e. median
  value), at a 95 confidence level (p-value
  0.99605)

31
Sensory Evaluation Grading System
Summary

• A superior method for performing sensory
  evaluations.
• Eliminates many sources of bias.
• Customized for use with all beverage alcohol
  products.
• Flexibility (data collection, quantification
  reporting).
• Provides significant time savings in data
  management.
• Provides tools to analyze panellist results to
  help identify training opportunities.

32
LC

• Leading the Sensory Evaluation of Beverage
  Alcohol