PS28C Psychometrics

1
PS28CPsychometrics

• Tutorial 1
• Introduction to Summated Rating Scale Construction

2
Introduction

• Purpose of tutorial is to provide practical
  experience developing summated rating scales
• Will work in groups of 3-4
• Group will create a rating scale and examine its
  psychometric properties

3
Introduction

• During these tutorials you will learn how to
• Develop a pool of items measuring a psychological
  attribute,
• Select a few items from this pool to construct a
  multi-item summated rating scale, and
• Assess the reliability and validity of your
  measure

4
Introduction

• Rules for the tutorials
• Turn off cell phones, pagers car alarms
• Attend each tutorial from start to finish
• Be on time for the tutorials

5
Introduction

• Keep a record of your intellectual contribution
• Portion of assignment grade will be based on your
  intellectual contribution
• Hand in participation sheet with your report

6
Overview of Tutorials

• Today will introduce the process of summated
  rating scale construction
• Second tutorial reviews APA style
• Third tutorial introduces hypothetical constructs
  and use of library resources
• Fourth tutorial presents correct question writing
  and layout

7
Overview of Tutorials

• Fifth tutorial teaches pilot-testing and
  think-aloud interviewing
• Sixth tutorial teaches how to construct a
  psychometric study
• Seventh tutorial involves estimating internal
  consistency reliability
• Eighth tutorial presents Factor Analysis

8
Overview of Tutorials

• Ninth tutorial describes how to estimate the
  validity of your instrument
• Final tutorial involves completion of Cattells
  16PF

9
Tutorial One

• Summated rating scales very popular in social
  sciences
• Roots in work of Likert
• Four key features
• Multiple items which are combined
• Each item quantitatively assesses a construct
• No correct or incorrect answers
• Items ask for ratings by respondents

10
Advantages of Rating Scales

• Produce measures that are reasonably reliable and
  valid
• Inexpensive and easy to develop
• Creating items is straightforward
• Only require 100-200 participants to develop and
  assess properties
• Relatively easy for respondents to complete

11
Disadvantages of Rating Scales

• Respondents must have the ability to read and
  write to answer
• Require a moderate amount of statistical
  sophistication and expertise

12
Why construct and use multi-item scales?

• Several problems with single item measures
• Produce very unstable or unreliable responses
• Lack precision in measurement
• Can not capture complexity of most psychological
  concepts

13
Why construct and use multi-item scales?

• Using multiple items to assess psychological
  concepts has many advantages
• Allows for many more aspects to be assessed
• Respondents given a wider range of possible
  responses
• Errors in responding average out over many items

14
Why construct and use multi-item scales?

• Errors in measurement are reduced
• Multiple items allow respondents to be more
  easily sorted into groups

15
What makes a good scale?

• Three key characteristics to a good scale
• The items must fully capture the psychological
  concept and its behaviour
• The scale must be reliable
• The scale must be valid

16
What makes a good scale?

• Other important attributes of a good scale
• The items should be clear, well written and
  contain a single idea
• The scale should be appropriate for the
  population to which it is applied
• The scale should be free from response biases
• The scale should not produce defensive responses

17
Steps in Creating Multi-Item Scales

• Five major steps
• Clearly define the psychological attribute to be
  measured
• Design and layout the draft scale
• Pilot-test the scale
• Collect data on the draft scale and examine
  properties of items
• Examine the validity of the scale and create norms

18
Theory Of Scales

• Development of scales based on classical test
  theory
• True score can be distinguished from observed
  score
• O T e
• Each item is an attempt to assess the true score

19
Theory Of Scales

• Single items unreliable due to large error
• Increasing the number of items reduces the total
  error in a scale
• Each item measures a separate aspect of the
  psychological concept
• Errors associated with each item average out
  reducing total error
• Increasing the number of items reduces the error

20
Formation of Working Groups

• Find two to three other people to work with
• Select a psychological attribute you would like
  to measure
• Attribute should not be some aspect of
  intelligence, ability or aptitude

21
Formation of Working Groups

• Your task is to
• 1) Decide what you wish to measure
• 2) Plan out a schedule of tasks and
  responsibilities
• Plan a timeline for the completion of your
  measure
• Report to me who is in your group, the attribute
  your group will measure, and your planned schedule