SLAT7806 Research Methods

1
SLAT7806 Research Methods

• Survey Research
• Week 12

2
(No Transcript)
3
Questionnaire.

• a set of questions on a given topic that do not
  test the respondent's ability, but rather elicit
  or measure their opinions, interests, personality
  characteristics, etc., as well as provide
  biographical information

4
Survey research

• Is a form of data-based research that can be both
  qualitative and quantitative.

5
Types of data collected with questionnaires

• facts
• interests
• attitudes
• opinions
• preferences

6
Elements of a questionnaire

• i. instructions to respondent
• ii. content questions
• iii. biodata information
• iv. layout

7
i. Instructions to the respondent

• - should be clear and easy to understand
• - should indicate that the survey is
  confidential, anonymous, voluntary

8
ii. Content questions

• - should be motivated
• - should be worded appropriately

9
iii. Biodata information

• - should elicit information needed to interpret
  results

10
iv. Layout

• - should be attractive and easy to complete
• - questions should be presented in a logical
  manner
• - key questions should be placed at the one-third
  point

11
Question types

• Closed response
• 3. Alternative
• 4. Checklist
• 5. Category
• 6. Ranking
• 7. Likert scale
• 8. Quantity

Open response 1. Fill-in 2. Short answer
12
Open versus closed response questions (1)

• Open response questions
• allow a wide range of possible answers
• are useful for exploratory research
• are relatively easy to write
• ltbutgt
• can be more difficult to answer
• are more likely to be skipped
• are more difficult to analyse
• difficult to show validity and
  reliability

13
Open versus closed response questions (2)

• Closed response questions
• allow uniformity across questions
• are easier to answer
• are easier to code and score
• are easier to analyse for validity
  and reliability
• ltbutgt
• permit a narrower range of possible
  answers
• are not useful for exploratory research
• are relatively difficult to write

14
1. Fill-in

• expected response is a specific piece of
  information
• example
• Nationality_________________
• First language________________

15
2. Short-answer

• expected response is a phrase or extended comment
  
• example
• What are your strongest personal qualities?

16
3. Alternative-answer

• - response selected from one of two alternatives
• Are you carrying any products of animal origin?
  Yes No
• Are you carrying more than AUS10,000 in
  currency? Yes No
• Are you a convicted felon? Yes No

17
4. Checklist

• - response selected from a list of alternatives
• example
• What leisure-time activities have you done in the
  last month? Check all that apply.
• ___ attend a movie
• ___ watch TV
• ___ play tennis
• ___ go dancing
• ___ go camping
• ___ attend a concert
• ___ play golf
• ___ go to the beach
• ___ rent a video
• ___ jog

18
5. Category

• response is only one of a set of categories
• example
• Choose the one area that best describes your
  research interests.
• discourse analysis
• phonology
• syntax
• sociolinguistics
• psycholinguistics
• pedagogy
• testing
• other

19
6. Ranking

• -response is a rank ordering of alternatives
• example
• Rank in personal importance the following things.
  (1 most important, 5 least important)
• __ wealth     __ fame     __ health     __
  friends     __ career satisfaction

20
7. Likert Scale

• - response is a rating of degree along a
  continuum
• example
• Please indicate the degree to which you agree
  with the following statement by making a mark on
  the line
• I review my previous lecture notes before class.
• never ------------------------------------------
  always
• (or)
• ..by circling the appropriate number
• never 1 2 3 4 5 always

21
8. Quantity

• - response is a number giving the amount of some
  characteristic
• example
• How many hours per week do you watch television?

22
Wording of questions

• Be clear, concise, and neutral.
• When writing questions, be careful to avoid a
  number of potential wording pitfalls
• ambiguous or imprecise wording
• Are the city buses too slow?
• Do students have too much free time?
• double questions
• Are the city buses too expensive and too slow?

23
Wording pitfalls to avoid Leading and loaded
questions

• leading questions that encourage respondents to
  respond in one way
• Given the demonstrated importance of lowering
  learner anxiety in the classroom, should teachers
  use specific activities to make students less
  anxious?
• loaded questions that use words that elicit an
  automatic or emotion-charged response
• Spanish is obviously the language to study if you
  live in California
• Agree or disagree?

24
Wording pitfalls to avoid hypotheticals and
assumptions about the respondent

• hypothetical questions
• If you won 5 million what would you do?
• making assumptions about the respondent
• How would you compare the Brisbane Courier-Mail
  and the Sydney Morning Herald in terms of quality?

25
Wording pitfalls to avoid Overly difficult or
inappropriate wording

• overly complex questions
• Does the recent proposal to revise the tax code
  to bring it in line with policy changes pose a
  challenge for the implementation of previously
  approved changes in non-tax spheres?
• the use of jargon or specialist terminology
• Do you think non-target-like usage in dyadic
  contexts by acquirers of a nonprimary language is
  evidence for the on-going development of the
  underlying approximative system?
• offensive questions based on gender, race,
  religion, etc
• ..

26
Doing a survey

• Survey construction ethical clearance.
• Sampling
• Data collection
• Data Analysis

27
Strategies for survey sampling (1)Source Cohen
Manion, 1985 (adapted)

• simple random Select subjects at random from a
  list of the population.
• systematic Select subjects in a systematic
  rather than random fashion (e.g., select every
  twentieth person).
• stratified Subdivide population into subgroups
  (e.g., male/female) and randomly sample from
  subgroups.

28
Strategies for survey sampling (2)Source Cohen
Manion, 1985 (adapted)

• Cluster Restrict one's selection to a
  particular subgroup from within the population
  (e.g., randomly selecting schools from within a
  particular school district rather than the entire
  state or country).
• Convenience Choose nearest individuals and
  continue the process until the requisite number
  has been obtained.
• Purposive Subjects are handpicked by the
  researcher on the basis of his/her own estimate
  of their typicality.

29
Data Collection

• 1. Run a pilot and revise as necessary.
• 2. Recruit respondents.
• 3. Try to optimise conditions in which the survey
  is completed. Make sure there is enough time and
  an adequate setting.
• 4. Be nice to the respondents. Show your
  gratitude.
• 5. Make a photocopy of the original data.

30
Data Analysis

• Qualitative
• Data coding
• Statistical
• t-unit
• chi-square
• factor analysis

31
A guide for questionnaire construction (1)
Source Selltiz, Wrightsman Cook, used in
Jackson, W. (1988). Research methods Rules for
survey design analysis. Scarborough, Ontario
Prentice Hall.

• A. Decisions about question content
• 1. Is the question necessary? Just how will it be
  useful?
• 2. Are several questions needed on the subject
  matter of this question?
• 3. Do respondents have the information necessary
  to answer the question?
• 4. Does the question need to be more concrete,
  specific and closely related to the respondent's
  personal experience?
• 5. Is the question content sufficiently general
  and free from spurious concreteness and
  specificity?
• 6. Do the replies express general attitudes and
  only seem to be as specific as they sound?
• 7. Is the question content biased or loaded in
  one direction, without accompanying questions to
  balance the emphasis?
• 8. Will the respondents give the information that
  is asked for?

32
A guide for questionnaire construction (2)

• B. Decisions about question wording
• 1. Can the question be misunderstood? Does it
  contain difficult or unclear phraseology?
• 2. Does the question adequately express the
  alternative with respect to the point?
• 3. Is the question misleading because of unstated
  assumptions or unseen implications?
• 4. Is the wording biased? Is it emotionally
  loaded or slanted towards a particular kind of
  answer?
• 5. Is the question wording likely to be
  objectionable to the respondent in any way?
• 6. Would a more personalised wording of the
  question produce better results?
• 7. Can the question be better asked in a more
  direct or a more indirect form?

33
A guide for questionnaire construction (3)

• C. Decisions about form of response to the
  question
• 1. Can the question best be asked in a form
  calling for check answer (or short answer or a
  word or two, or a number), free answer or check
  answer with follow-up answer?
• 2. If a check answer is used, which is the best
  type for this question - dichotomous,
  multiple-choice, or scale?
• 3. If a checklist is used, does it cover
  adequately all the significant alternatives
  without overlapping and in a defensible order? Is
  it of reasonable length? Is the wording of items
  impartial and balanced?
• 4. Is the form of the response easy, definite,
  uniform and adequate for the purpose?

34
A guide for questionnaire construction (4)

• D. Decisions about the place of the question in
  the sequence
• 1. Is the answer to the question likely to be
  influenced by the content of preceding questions?
• 2. Is the question led up to in a natural way? Is
  it in correct logical order?
• 3. Does the question come too early or too late
  from the point of view of arousing interest and
  receiving sufficient attention, avoiding
  resistance, and so on?

35
End of week 12 slat7806