Research Methods - PowerPoint PPT Presentation

About This Presentation
Title:

Research Methods

Description:

Research Methods Michael Wood michael.wood_at_port.ac.uk http://userweb.port.ac.uk/~woodm/rm/rm.ppt This file contains draft s which will be updated. – PowerPoint PPT presentation

Number of Views:604
Avg rating:3.0/5.0
Slides: 128
Provided by: Michael2205
Category:

less

Transcript and Presenter's Notes

Title: Research Methods


1
Research Methods
  • Michael Wood
  • michael.wood_at_port.ac.uk
  • http//userweb.port.ac.uk/woodm/rm/rm.ppt
  • This file contains draft slides which will be
    updated.
  • 30 November 2009

2
Reading
  • There are many books available e.g.
  • Saunders et al (2007)
  • Robson (2002)
  • Easterby-Smith et al (2002)
  • And many others browse in the library
  • These books vary a lot some are better on the
    practical aspects, others on the theoretical
    aspects. Sometimes you will get different advice
    from different sources, so you need to consider
    the rationale behind the advice. Robson is good
    on most aspects, although Saunders et al is
    probably more student-friendly

3
Contents
  • Overview of academic business research
  • What must be in a project plan and a project?
  • Formulating research aims
  • The design of research projects
  • Evaluating research
  • Statistical analysis for research
  • Qualitative data analysis
  • Analysing data and presenting results
  • Philosophy of research
  • Questionnaire design
  • Interview design and qualitative research
  • Reminders about the project
  • Interviews and qualitative research more detail
  • More on literature reviews

4
Overview of academic business research
  • Reading browse through a book on research
    methods e.g. Saunders et al (2007), Robson
    (2002)
  • These slides intended as a brief summary of the
    important points
  • Reread them when you are starting your project

5
Advice on research methods
  • Common sense dont forget this!
  • Articles and books reporting similar research
    should be discussed in the project
  • Books on research methods in general
  • Focus on chapters relevant to your project.

6
Purpose and characteristics of academic research
  • Purpose
  • Discover truth about something and/or
  • Find a good way of doing something
  • Must be
  • Systematic and as thorough and trustworthy as
    possible
  • Clearly written and with sufficient detail for
    readers to check credibility
  • Ethical

7
Types of research include
  • Large scale surveys (of people, organisations,
    events, etc) analysed statistically
  • Small scale surveys with emphasis on
    qualitative detail
  • Case studies (to see how something works in
    detail)
  • Experiments (change something to see what
    happens)
  • Models can be set up, tested and used for
  • Participant observation (observe as participant)
  • Action research (combine research and action)
  • Evaluation
  • and may other possibilities be imaginative!
  • Many projects combine several of these

8
Sources of data many possibilities
  • Interviews
  • Including focus groups, Delphi technique (Robson,
    200257), various approaches to eliciting
    comments (e.g. photo elicitation Sam Warren)
  • Questionnaires, including via email (be careful
    )
  • Documents (minutes of meetings, company reports,
    etc)
  • The web
  • Databases within organisation, of share prices,
    etc
  • Observations of various kinds
  • Etc . Be imaginative!
  • Sources of literature is a different issue
    (Judiths session is very important for this)

9
Experiments (randomised controlled trials)
  • Put people (or whatever you investigating) in
    randomly assigned groups, give the groups
    different treatments, and compare groups to see
    what differences emerge.
  • Used for testing drugs, diets (http//tinyurl.com/
    yp2t2o, http//tinyurl.com/489hns), educational
    methods, different designs for websites, social
    policies, etc. Lots of examples in Ayres (2007).
  • Advantages of experiments over non-interventionist
    research
  • Disentangle cause and effect. Can control
    variables you havent even thought of. If done
    well evidence can be very convincing.
  • Can investigate new things
  • Ayres, Ian. (2007). Super Crunchers how
    anything can be predicted. London John Murray.

10
But
  • Experiments are often impractical or unethical
  • Difficulties include
  • Hawthorne effect
  • Failure to assign groups at random (this matters
    a lot because )
  • So use less rigorous quasi-experiments instead
    (Grant Wall, 2008) e.g. in action research
    you may do a before and after comparison. This is
    a sort of crude experiment but it is not as
    convincing as a proper RCT.
  • Grant, A. M. Wall, T. D. (2008). The
    Neglected Science and Art of Quasi-Experimentation
    Why-to, When-to, and How-to Advice for
    Organizational Researchers. Organizational
    Research Methods (published online, July 18,
    2008).

11
Finding a suitable topic
  • Interest
  • Career
  • Feasibility
  • Usefulness

12
How to do research
  • Read about topic
  • Draft aims of research. Clear, simple, focused.
  • Draft literature review.
  • Draft research plan check it is really likely
    to meet your research aims. Check again.
  • Do research/analysis
  • Draft research/analysis and recommendations/conclu
    sions
  • Check it fits together and revise all sections
  • If it doesnt fit together revise aims and

13
Practical issues
  • Timing
  • Plan this remembering that your supervisor may
    suggest extensive changes.
  • Gantt chart may help.
  • Ethics (remember the form!)
  • Access to information.
  • Take care this is often difficult!

14
What must be in a project and a project plan?
  • Reading
  • Project guidelines
  • Proposal guidelines
  • Saunders et al (2007), or another similar book

15
What must be in a project?
  • Abstract (short summary of project including
    conclusions)
  • Background and aims (what youre trying to find
    out and why its important)
  • Literature review (of relevant previous research
    which you will build on or extend)
  • Research methods plan and justification (what
    you did to meet the aims, and why it was a
    sensible approach)
  • Analysis (in detail, to convince sceptical
    readers and impress examiners important tables,
    diagrams etc must be in the text, only details in
    appendix)
  • Results, conclusions, recommendations,
    limitations, further research
  • References (list works cited in text in
    alphabetical order)
  • Appendices Ethics form, extra details for the
    reader
  • Flexible designs can be more flexible but
    everything must be there!

16
Features of a good project
  • Obviously important and interesting
  • Difficult to disagree with because
  • Arguments and analysis detailed, clear and
    obviously valid
  • Possible objections considered and if possible
    answered
  • Fits together
  • Aims met by methods (check this in your project
    plan)
  • Conclusions follow from analysis

17
References and citations
  • You must give references to publications which
    you draw on or quote
  • Exact (word for word) quotes must be in and
    the reference must be given
  • Maximum about one paragraph
  • Use one of the standard referencing systems
    preferably the Harvard (see university website)
  • Copying word for word without and reference
    is treated as cheating and you will fail!

18
Harvard referencing system
  • Very important to use this (or another
    established system)
  • Seems easy to me, but causes a lot of difficulty
  • Check library website (search for Harvard) and/or
    copy an academic article or book.
  • All references in text like Smith (2001)
  • Then alphabetical list of references at the end.
    Should include everything referred to, and
    nothing else.

19
What must be in your project plan (proposal)?
  • See assignment description
  • You may be able to put parts of it in your
    project!
  • You should describe and justify your research
    methods in as much detail as possible

20
Writing style (1)
  • Keep it simple.
  • Short sentences
  • Clear, short paragraphs
  • Clear subheadings
  • Read it through to make sure you can follow it.
    Swap with a friend and check each others

21
Writing style (2)
  • I think the EMH was true in this situation
  • In my opinion the EMH was true
  • In the authors opinion the EMH was true
  • 4 The evidence suggests that the EMH was true
  • 5 This shows that the EMH was true
  • Use 4 or 5.
  • Avoid 1, 2 or 3 because it gives the impression
    that its just your opinion and that other, even
    wiser, people may see it differently.

22
Writing style (3)
  • I work for and the problems are / I
    interviewed three managers.
  • The author works for and the problems are /
    The author interviewed three managers.
  • Then problems of this organization are / Three
    managers were interviewed.
  • Opinions vary here. I (MW) prefer (1). Others
    prefer (2) or (3).
  • Check with your supervisor.

23
Formulating research aims
  • Reading most research methods books, e.g.
    Saunders et al, 2007

24
Research aims or questions
  • Usually start from vague idea
  • Then formulate a clear aim, or list of aims, that
    your research will achieve. Think of these as
    hoped-for outcomes.
  • Alternativelyformulate a clear question or list
    of questions.
  • This process may require some creative thinking
  • Techniques like brainstorming and mind maps may
    be useful

25
Aims, objectives, questions
  • You can formulate your research aims as aims (or
    objectives if you prefer that word) or questions.
  • These are different ways of saying the same
    thing. Doesnt matter which you use, but dont
    confuse things by having aims and questions
  • May be helpful to have a list or hierarchy of
    aims, but keep it simple

26
Hypotheses
  • Hypotheses are statements whose truth you want to
    test, or predicted answers to research
    questions (Robson, 2002)
  • Occasionally appropriate as a top level research
    aim
  • e.g. to test the hypothesis that Working at home
    improves quality of life
  • Usually best to avoid hypotheses when formulating
    main research aims because questions or aims tend
    to be more flexible
  • e.g. How does working at home affect quality of
    life?
  • Null hypotheses have a (controversial) role in
    some statistical analysis ( as you will see),
    but they are not relevant to formulating your
    overall research aims

27
Research aims or questions
  • Research aims or questions should
  • Be clearly and simply expressed
  • Fit together (so that you have a coherent
    project)
  • Clarify the intended outcome and scope of the
    research
  • Your research aims or questions should also
  • Be relevant to your degree
  • Be achievable
  • Present a reasonable level of challenge

28
Research aims or questions
  • Must be research aims, not business or personal
    aims.
  • However, business or personal aims may be part of
    the background motivating your research aims, and
    research aims would normally include the aim of
    making recommendations to people or
    organisations.
  • Should generally have a limited scope or focus.
  • The danger with general aims is that they lead to
    superficial research.
  • May relate to theoretical issues. You may be
    aiming to test, modify or create a theory

29
Theory
  • Theory includes models, explanatory frameworks,
    generalisations, recommendations
  • Examples .
  • Your research should link with any relevant
    theory. It may
  • Use a theory
  • Demonstrate that a theory is useful
  • Test a theory
  • Modify a theory or create a new theory

30
Also ask yourself
  • Is the research worth doing?
  • Are there any ethical or political problems?
  • Is it possible? Have you got access to the
    necessary data?

31
Is it really going to be useful?
  • What use do you want the results to be? This may
    be a practical use to find out how to make more
    money, or to make life easier or a contribution
    to theory, but it should be something that is
    really worth achieving. Must pass the so what?
    test.
  • May help to clarify your aims if you imagine
    youve done the research and write down what you
    think your conclusions and recommendations might
    be.
  • Then work backwards from what you want to achieve
    to the best methods to achieve it.

32
Example of research aims
  • The aims of this research are to
  • Describe the decision making strategies of small
    investors
  • Determine the effectiveness of these strategies
  • Any comments? Does this seem reasonable for a
    Masters project?

33
Another example of research aims
  • The aims of this research project are to
  • Evaluate Method X for planning mountaineering
    expeditions
  • If necessary propose and justify Amended Method X
    for planning mountaineering expeditions

34
Another example of research aims
  • What are the important quality problems in
    Company X?
  • How serious are these problems?
  • What is the best strategy for reducing these
    problems?
  • Any comments? Does this seem reasonable for a
    Masters project? Does it matter that they are
    expressed as questions?

35
Three more examples of research aims
  • The aim of this research is to investigate the
    role of the internet in banking.
  • This research project aims to explain activity
    based costing.
  • The aim of this project is to
  • Test the efficient market hypothesis for the
    Athens stock exchange, and
  • Determine how global warming will influence share
    prices.
  • Any comments? These are not reasonable for an
    Masters projects! Why not?

36
Possible research topics
  • Research in a specific organisation
  • Best if they are likely to implement any
    recommendations
  • Take care you have adequate access to data
  • Easier if you have a recognised / paid job there
    and / or know key players well.
  • Research based on publicly available data
  • Eg share prices, the www, published statistics
  • Research based on surveys of the public
  • These are just some possibilities. There are more

37
Design of research projects
  • Design means deciding on the methods and
    approaches which will best achieve your aims
  • Needs thinking out carefully starting from your
    aims
  • Check the proposed design will achieve all your
    aims
  • The design may require the use of a theoretical
    framework which should be explained and its use
    justified
  • May incorporate several approaches (e.g. earlier
    slide)
  • Some advocate flexible designs (E.g. Robson,
    2002)
  • E.g. Poppy Jamans summary. Any comments?
  • E.g. check aims and designs of these projects.

38
Designing research is not easy!
  • Think about how you can best achieve your aims
  • Consider all possible types of research
  • Be imaginative
  • Think about it again
  • and again
  • Check youve found the best way you can for
    meeting all your aims

39
Group exercise
  • Design a research plan for one of the projects
    below, and do a pilot study for part of it. (You
    may find you need to make the aims / questions
    more precise.)
  • Michaels project. The provisional aims are
  • To evaluate the suitability of the PBS website
    for prospective PhD students
  • To suggest improvements to the website from this
    perspective
  • Alisons project on the impact of a Blackberry on
    family/work-life balance. What are the problems
    and opportunities, and what would you recommend?
  • or

40
Email project
  • How much time do people spend on emails, is it
    time well spent, and if not how can things be
    improved?
  • Provisional method Survey to find how much time
    is spent on emails, and respondents opinions on
    whether this is time well spent, and on
    recommendations (is this a satisfactory method?)
  • And / or other possibilities ?

41
A general design for a typical Masters degree
project
  • If the aim is to find a good strategy to
    "improve" X in org Y, then a possible design may
    be
  • Survey/case studies of Org Y to investigate
    problems and opportunities
  • Survey/case studies to see how other
    organisations do X and which approaches work well
  • Based on (1), (2), the literature, and perhaps
    creative inspiration, consultations within the
    organisation, simulation or modelling, devise a
    strategy likely to improve X
  • Try/test/pilot/monitor the proposed strategy,
    probably in a limited domain

42
Take care with opinion surveys
  • Suppose your research is about risk management
    and its effectiveness. You decide to investigate
    by means of a questionnaire and come up with
  • 70 of people in the organisation think our risk
    management is unsatisfactory
  • 60 think Method X is the best way of improving
    it
  • You then present this as the rationale behind
    your recommendations to improve risk management.
  • But how do they know?
  • Surely the researcher should find out by rigorous
    and sensible methods, rather than asking people
    who may neither know nor care?

43
Exercise
  • There are many problems with interviews and
    questionnaires. Your respondents may
  • Not know the answers
  • Not understand the questions
  • Be too lazy to think about the issues
  • Want to deceive you
  • Try to design the methods for a research project
    without using interviews or questionnaires. (This
    is not usually a good idea but it should help you
    to consider alternatives.)

44
Then
  • Having designed your research get someone to act
    as a devils advocate and tell you
  • Whats wrong with it why it may fail to deliver
    what you are aiming for
  • What may go wrong
  • Would they trust the answer?

45
Evaluating research
  • Relevant to
  • Planning your own research. Use the following
    slides to
  • Check your proposal
  • Check your final project
  • Critically reviewing published research
  • These slides are intended as a checklist for your
    research and others

46
Good research should be
  • As User-friendly as possible
  • Simple as possible given the message?
  • As Uncritisable (trustworthy) as possible
  • Trustworthiness or credibility is particularly
    important. Can you trust the conclusions? Do you
    believe them? Are there any flaws? Essential to
    give readers enough detail to check.
  • As Useful or interesting as possible
  • Clear implications for future? New results?

47
In groups
  • Choose one of the articles you have been given
  • Assess its
  • User-friendliness
  • Trustworthiness (pay particular attention to
    this)
  • Usefulness
  • Brief feedback session, then we will compare your
    critiques with my slides

48
Trustworthiness of research main things to check
  • C
  • R
  • I
  • T
  • I
  • C
  • Each letter represents an issue you should
    consider

49
Jargon
  • Most of these checks are covered by technical
    jargon, concepts and techniques e.g. lots of
    types of validity (internal, external, construct,
    face ), lots of types of reliability, ideas
    about test and scale construction (see Robson,
    2002), etc
  • Read up only those areas which you think are
    relevant. I have largely avoided jargon here.
  • Always check sampling always necessary to
    consider whether your sample is likely to be
    representative of your area of interest.

50
Deciding what is Cause and what is effect
  • Important to try to work out what causes what,
    and how strongly and under what circumstances, so
    that you know what you should change to achieve a
    particular effect.
  • Take care may be more complicated than it
    appears (ISO 9000 and profitability drinking and
    thinking, storks and babies).
  • Variable you havent thought of may be the
    important cause!
  • Experiments (randomised controlled trials) for
    definitive answers, but may be difficult, so
  • Quasi-experiments (e.g. a before/after comparison
    of a trial of a new innovation) insead, but
  • May be lots of causes. Be suspicious of simple
    explanations (see Taleb, 2008).

51
Deciding what is Cause and what is effect more
examples
  • A survey of organizations showed that those that
    used the balanced scorecard were more profitable
    than those that didnt.
  • Does this show that the balanced scorecard makes
    firms more profitable?
  • A survey showed that the average job satisfaction
    score for a department rose substantially and
    significantly between 2006 and 2008. In 2007
    everyone was sent on a weeks computer course in
    the Seychelles.
  • Would you recommend a computer course for other
    departments?
  • Does high staff turnover cause poor performance
    or vice versa? (Glebbeek and Bax, 2004). Does
    extraversion help people get promoted, or vice
    versa (Moutafi et al, 2007). Does it matter?
  • What caused the fall of the Berlin Wall?

52
To ensure results Representative check Sampling
  • Decide what youre interest in often called the
    population or target population.
  • Usually this is too big to look at everything so
    take a sample. Normally we want the sample to be
    representative of the population or wider
    contextso you must check if this is likely.
  • Need to consider how the sample is selected and
    its size. Badly chosen samples can be biased and
    give very misleading results.
  • E.g. TV audience research, word length, NRE,
    non-response bias in surveys, survivor bias in
    stock price samples

53
How to sample
  • Clarify target population (the whole group of
    interest)
  • May be a population of people, organisations or
  • Decide sampling approach. There are many methods
    of taking a sample from your target population,
    including
  • Random
  • Stratified
  • Purposive
  • Convenience (or opportunity, haphazard,
    accidental)
  • Cluster, snowball, quota, etc (see a book)
  • Decide size of sample need to balance cost with
    information obtained. If you analysis is
    statistical, statistical theory can help

54
Random sampling
  • Make a numbered list of the target population (a
    sampling frame)
  • Use random numbers to choose sample
  • Each member of population has the same chance of
    being selected (and its independent of any
    biases)
  • Each member of sample selected independently
  • In practice, likely that some members of the
    sample cant be found or wont help, so the
    sample may be biased. Difficult to deal with this
    possibilities
  • The principle is to ignore all variables and
    choose at random. This allows for all noise
    variables.

55
Which sampling method?
  • Usually random samples are best for large
    samples, and purposive samples for small samples
    analysed qualitatively.
  • Done properly, with a large enough sample, random
    or stratified samples (probability samples)
    should be reasonably representative of the
    population. Cant assume this about purposive or
    convenience samples (non-probability samples)
    because these are selected by factors that are
    likely to bias the result in one direction or
    another.

56
Sampling in practice
  • Many samples are biased and so will not give a
    good idea of the population regardless of
    sample size.
  • E.g. NRE, non-response bias in surveys, survivor
    bias in
  • Ideal for large samples is random sampling, but
    this is often difficult to do properly.
  • E.g. Iraq war death rate (see http//www.iraqbodyc
    ount.org/ for another approach), TV audience
    research.
  • Be suspicious of statistical results from
    purposive or convenience samples
  • Need to be especially careful with small,
    purposive samples for detailed analysis
    consider the purpose and choose accordingly

57
Three surveys to check accuracy of NRE phone
service which is right?
  • A Consumers Association survey used a sample of
    60 calls, mainly about fares. The worst mistake
    was when one caller asking for the cheapest fare
    from London to Manchester was told 162 instead
    of the cheaper 52 fare which was available via
    Sheffield and Chesterfield. The percentage
    correct was
  • 32
  • A reporter rang four times and each time asked
    for the cheapest route from London to Manchester.
    The proportion of the four answers which were
    correct was
  • 25
  • An NRE sponsored survey found that the answers
    were
  • 97 correct
  • (Source Breakfast programme, BBC1 TV, April 30
    2002.)

58
More sampling problems
  • An MBA student sends out 100 questionnaires to
    100 organisations asking if they would be
    interested in a particular service. Twenty are
    returned, and of these 6 indicated they may be
    interested in the service
  • There are 650 firms in the relevant industry
    sector. How big is the market for the service?
    Are you sure?
  • Suppose you wanted to find out how common it is
    for women aged 30-40 to enjoy running.
  • How would you choose a sample to ask?
  • Other examples and exercises attached

59
Measurements (Indicators)
  • If you want to find out whether customer
    satisfaction, or quality or profits have improved
    you must have a sensible way of measuring them.
  • Moreno-Luzon (1993) used managers perceived
    achievement of objectives as a measure. Can you
    see any problems with this?
  • How would you measure quality of service in a
    casino?
  • How would you check if your proposed measure is
    valid / reliable / right / accurate?

60
Things to remember with measurements (1)
  • Conventional to distinguish between validity (are
    you measuring the right thing?) and reliability
    (consistency)
  • If possible use an existing measurement system
    (with acknowledgement / permission). This has two
    advantages there may be evidence validating it,
    and you can compare your results with previous
    results.
  • Remember that some informants may be biased, or
    too lazy to give good answers, or just ignorant.
  • If possible use triangulation (check with
    information from different sources)
  • Ask yourself whether your proposed method of
    measurement really measures the right thing

61
Things to remember with measurements (2)
  • Be especially careful with measures of value.
    This may have several dimensions (Keeney, 1992).
    E.g. the success of a firm might depend on
    profitability, worker satisfaction, contribution
    to the community
  • If you are measuring the success of a change,
    remember there may be several different criteria.
    E.g.
  • May be useful to use the average (mean) response
    to a series of questions. Use your common sense
    to see if this is reasonable, or if they should
    be kept separate. (See literature on Tests and
    scales e.g. Robson, 2002 292-308).
  • Keeney, R. L. (1992). Value-focused thinking a
    path to creative decisionmaking. Cambridge,
    Massachusetts Harvard University Press.

62
Reliability of measurements
  • Same answer at different times?
  • If anything depends on subjective judgments,
    check agreement between different judges
  • Eg marking projects
  • If youre asking a number of questions to get at
    the same information, check the relationship
    between answers to these questions with two
    questions use a correlation coefficient, with
    more than two use Cronbachs Alpha (if you are
    keen on stats!) see http//www.statsoft.com/text
    book/stathome.htm

63
Exercise how would you measure
  • ??

64
Theoretical assumptions
  • If the research uses a theory, is the theory
    right for the purpose? And is it a valid
    theory? (Some theories, of course, are stupid or
    wrong!) You need a critical evaluation in your
    literature review.
  • A questionnaire or interview plan may be based on
    assumptions about what is relevant. Are these
    assumptions OK?

65
Is the research sufficiently Imaginative?
  • Imagination helpful in
  • Thinking of hypotheses to explain things
  • Thinking of new methods for researching
  • Thinking of new ways of doing things
  • Many recommendations for boosting imagination and
    thinking creatively e.g.
  • Brainstorming
  • Doing something else and coming back to the task
  • etc

66
Making sure that you are not being misled by
Chance
  • Could your results just be due to chance?
  • Have you taken account of sampling error? (If you
    repeated your research with another sample are
    you sure the answer would be the same?)
  • Is the sample large enough?
  • Null hypothesis tests or confidence intervals can
    be used to answer these questions.
  • Are the measurements reliable?

67
The first CRITIC
  • Cause and effect assumptions OK?
  • Representative sample?
  • Indicators (measurements) OK?
  • Theoretical assumptions OK?
  • Imaginative enough?
  • Chance ruled out as explanation?
  • (Checks needed are mostly common sense except
    for Chance.)

68
The second CRITIC
  • C Claim?
  • R Role of the claimant?
  • I Information backing the claim?
  • T Test?
  • I Independent testing?
  • C Cause proposed?
  • Teaching skepticism via the CRITIC acronym and
    the skeptical inquirer
  • Skeptical Inquirer,  Sept-Oct, 2002  by Wayne R.
    Bartz

69
Two extra checks
  • Use of a devils advocate or critical friend.
    Remember the problem of confirmation bias you
    are likely to be more enthusiastic about evidence
    that confirms your pet ideas than about evidence
    that undermines it! Get someone to try and be
    critical and find difficulties with your research
    then fix or (if unfixable) discuss the
    problems.
  • Triangulation compare results from different
    sources. Applies to data, methods, observers,
    theories (Robson, 2002 174).

70
Anything else?
  • Is this list complete?
  • Does it address all the flaws you noticed in the
    paper you looked at?
  • What would you add or change?

71
Checklist the 3 Us, the CRITIC and Extra checks
  • User-friendly?
  • UnCRITICisable (trustworthy)?
  • CRITIC
  • Useful?
  • Extra checks
  • Triangulation
  • Devils advocate (critical friend)

72
Another measurement problem
  • Andy had answers from lots of questions on a SD,
    D, N, A, SA scale
  • He wanted a measure to tell him which questions
    produced responses which gave a a clear overall
    view (COV) from his respondents
  • His defined his measurement as
  • COV abs(SDDASA) N
  • (where SD is the number of SD responses, etc, abs
    absolute value)
  • He then highlighted questions for which COV gt 0
  • Do you think this is a sensible measurement?

73
Critique of an article
  • Do you accept what the article says, or are there
    flaws in the research?
  • Think about the article! Use your common sense.
  • Check the CRITIC.
  • Is it worth publishing? Could you do better?
  • Read round the subject e.g. other research on
    the same theme.
  • Would the research benefit from some qualitative
    work, p values or confidence intervals, case
    studies, different perspectives, experiments

74
Statistical data analysis
  • Go to http//userweb.port.ac.uk/woodm/stats/StatN
    otes0.ppt

75
Qualitative data analysis
  • Aim is detail and depth of understanding
  • Demonstrate and understand possibilities, but not
    how frequently they occur
  • Use direct quotes () as evidence and to reduce
    danger of imposing your perspective
  • Sometimes may be helpful to
  • Summarise in a table or similar
  • Use coding scheme to analyse statistically (but
    be careful if the sample is very small!)
  • Further possibilities in Saunders et al, Robson,
    www.qual.auckland.ac.nz/, Thorpe and Holt (2008)

76
Analysing data and presenting results
  • Questionnaires and interview plans, and possibly
    some data, in appendix
  • Graphs and tables and important quotes from
    interviewees etc in the main text
  • Focus on your research aims, not the questions in
    your questionnaire
  • Readers want an analysis which shows how your
    aims are met. They dont want to know the answers
    to all the questions in your questionnaire!
  • Use appropriate summaries e.g. tables of
    averages, or of main points from interviews

77
Literature review
  • See Saunders et al (2003) Chapter 3
  • Focus on relevant books, articles and theories
  • Brief on general points
  • More detailed on research of particular relevance
    to your project you will need to search for
    articles using the library databases
  • Critical
  • Should lead into your method and analysis
  • Must be properly referenced!

78
Philosophy of research
  • In the textbooks you will find discussions of
    positivism, social constructivism, phenomenology,
    etc, etc.
  • In my view, Robson (2002) is the best research
    methods text for philosophical concepts.
  • Almost all concepts and distinctions here open to
    serious criticism see Robson (2002). Most
    management research articles dont mention
    philosophy.
  • I wouldnt suggest focusing on these ideas unless
    you are interested in which case be critical of
    what you read!
  • If you do want to go into philosophy, use a book
    like the Penguin Dictionary of Philosophy
    (Mautner, 2005) or Thorpe and Holt (2008) to
    check what the words mean.
  • Also note that there are arguments against being
    prescriptive about research methods and
    philosophy in books with titles like
  • After method mess in social science research
    (Law, 2004), and
  • Against method outline of an anarchistic theory
    of knowledge (Feyerabend, 1993)

79
Further reading and references
  • http//userweb.port.ac.uk/woodm/qualquant.pdf

80
Some ideas which are worth mulling over
  • Detailed study of a small sample vs less detailed
    study of a large sample
  • Induction vs the Hypothetico-deductive method
    (Popper) vs Following a framework / paradigm /
    theory vs Deduction
  • Subjective vs Objective Facts vs Values

81
Some misguided platitudes
  • The following are often assumed (I think
    wrongly)
  • There are two distinct kinds of research
  • Quantitative (positivisthypothetico-deductive),
    and
  • Qualitative (phenomenologicalinductive).
  • Instead
  • Positivist research (only) starts from
    hypotheses.
  • Instead ...
  • Academics tend to disagree about many of these
    issues. If you do decide to go into them, please
    think hard, and dont accept everything you read
    in the textbooks uncritically!

82
Qualitative vs quantitative
  • Quantitative usually means statistical often
    with largish samples
  • Qualitative means focusing on qualities usually
    with smallish samples studied in depth
  • Disadvantage with statistical approaches is that
    the data on each case is often very superficial
  • Disadvantage with qualitative approaches is that
    case(s) studied may be untypical and cant be
    used for statistical generalisation
  • Often best to use both approaches. This is known
    as mixed methods search for this keyword in
    library.
  • This distinction often confused with other
    distinctions

83
Regrettable tendency to reduce things to a simple
dichotomy
  • If youre a soft and cuddly person
  • Soft and cuddly (e.g. interpretivist,
    qualitative, inductivist ) is good
  • Hard and spiky (e.g. positivist, quantitative,
    deductivist, ) is bad
  • But if you are a hard person you will probably
    reverse good and bad above. There are really many
    different dichotomies. Reducing them all to one
    is neither right nor useful.

84
And
  • To hard and spiky people, soft and cuddly
    research is lacking in rigour
  • To soft and cuddly people, hard and spiky
    research is naïve and lacking in richness

85
Induction vs hypothetico-deductive method
  • Generalise from the data without preconceptions
    (induction)
  • Grounded theory. Rigour is in process used to
    generate theory from data
  • Versus
  • Use data to test hypotheses or theories
    (hypothetico-deductive method)
  • Karl Popper. Rigor is in the testing.
  • Theory building vs theory testing is much the
    same distinction (see Saunders et al, 2007, pp
    117-119)
  • However, I dont think these are the only choices

86
Other useful approaches besides induction and
hypothetico-deduction
  • Use a framework or theory or paradigm (Kuhn,
    1970) to define concepts, questions, and
    measurements, but without trying to test the
    theory
  • Arguably what most scientists do most of the time
    (c.f. Kuhn, 1970). Rigour is in ensuring the
    theory is a good one, and in using it properly.
  • Deduction from data, theories and framework. E.g.
    the differences between two quality standards can
    be deduced.
  • Rigour is in checking the deduction and the info
    you start with
  • Differs from the hypothetico deductive method in
    that the result is the deduction itself, not a
    confirmation, rejection or revision of a
    hypothesis or theory
  • Note that this contradicts the assumption in
    Saunders et al (2007 117-119) that there are
    just two approaches deductive and
    inductive. I think they mean
    hypothetico-deductive, and they omit the two
    very important possibilities above.

87
An example
  • How would these four approaches work with a
    project of interest to you

88
Karl Poppers ideas (1)
  • Science works by putting forward bold theories
    (or hypotheses) and then testing them as
    thoroughly as possible
  • Provisionally accept theories that have withstood
    this testing process
  • Theories must be sufficiently precise to be
    falsifiable otherwise not proper science (eg
    Freuds theories are too vague)

89
Karl Poppers ideas (2) - eg
  • Einsteins theory of general relativity predicts
    that light from a distant star will be bent by a
    small amount by passing close to the sun.
    Newtons theory predicts the light will not be
    bent.
  • Only possible to check during a total eclipse of
    the sun. In an eclipse in 1918 light was bent as
    Einsteins theory predicted
  • Newtons theory is falsified Einsteins lives on
    and seemed much more credible.

90
Karl Poppers ideas (3)
  • Theories can come from anywhere guesswork,
    intuition, other theories, etc
  • The process of criticising theories and trying to
    show they are wrong is vital for science
  • The method applies to both natural and social
    sciences
  • How would you apply Poppers ideas to a
    management research project? in practice, has
    elements in common with a critical attitude

91
Critical attitude
  • Try to anticipate and discuss criticisms
  • Get a friend to act as a devils advocate
  • Your work should be so convincing that it cant
    be disputed!
  • Think of any criticisms you have of articles you
    have read. Make sure the same faults dont apply
    to your work.
  • Word critical sometimes used in a slightly
    different, more specific, sense.

92
Questionnaire design summary
  • Read a (chapter of a) book on questionnaires
  • Develop a pilot. Remember questionnaires are far
    more difficult to design than they appear! Check
    with your pilot respondents
  • Is it clear?
  • Is it interesting / appealing / user-friendly /
    not too long? Would you answer it?
  • Does it provide (only) the information you want?
  • Are you still sure a questionnaire is a good idea?

93
Questionnaire design (1)
  • Write down what you want to find out
  • Closed questions
  • Tick boxes
  • Rating (Likert) scales
  • Open questions
  • Pros and cons of each
  • Check your questions will enable you to find out
    what you need to for your research

94
Questionnaire design (2)
  • Covering letter
  • Pilot it
  • 3-4 nice friendly people to tell you whats wrong
    with it
  • Pilot the analysis too
  • Consider sample to send it to
  • Anonymity / confidentiality
  • How to send it / get it back (email?)
  • What to do about non-response?

95
Questionnaire design (3)
  • Far too many questionnaires about - many of them
    very silly. What is the response rate likely to
    be? Would you fill it in?
  • Are you sure a questionnaire is necessary???
  • Many companies have a policy of not responding to
    questionnaires
  • Are there any alternatives?
  • Check with your supervisor before sending it out

96
Take care with opinion surveys
  • You can ask someone
  • What she did last week
  • What she does in general terms
  • Her opinion of what she does
  • What she thinks other people do
  • Her opinion of what she thinks other people do
  • How she thinks things can be improved
  • What she thinks about particular suggestions
    about how things can be improved
  • What she likes / wants / values
  • Etc
  • Think about what type of question you are using
    and whether it is really useful

97
Interview design in brief (1)
  • Read a (chapter of a) book on interviews
  • Follows, or precedes questionnaire, or stands
    alone
  • Be clear what you want to find out
  • Consider telephone interviews
  • Small sample. Dont do too many interviews.
  • Plan your questions. Should be open-ended and
    flexible, and aim for a detailed understanding
  • Probes and prompts

98
Interview design (2)
  • Ask for permission to tape record
  • Transcribe interesting bits to get quotes for
    your project
  • Get interviewee relaxed. Anonymity /
    confidentiality (take care here!)
  • Check youve covered everything
  • Send interviewee transcript afterwards?
  • Some transcripts or parts of transcripts in
    appendix?

99
Reminders about the project
  • Research aims should be simple, explicit,
    focussed, motivated, useful
  • Literature review should focus on relevance to
    your project
  • References should be complete and in order
  • Methods should be the best which are feasible.
  • Analysis chapter should show how hard and
    skilfully youve worked, and why readers should
    believe you. You need to convince a sceptical
    reader who may want to know details of how your
    data was obtained e.g. source of samples,
    location of interviews (pub or office?), etc, etc
    and analyzed.
  • Conclusions and recommendations should summarise
    what you have found, and clearly meet the
    research aims. Also discuss limitations.
  • Changing your mind is to be expected if
    necessary rewrite aims after doing the research!

100
Reminders (2)
  • Docs/links at http//userweb.port.ac.uk/woodm/pro
    jects
  • Keep to the 15,000 word limit. You can get a good
    mark with 13,000 words but not with 17,000 words.
  • Remember the ethics form no form, no pass!
  • Be particularly careful about NHS ethics
    clearance
  • Make use of your supervisor (see Project
    Guidelines)
  • Plan the timescale (Gantt chart) allow time for
    delays
  • Allow time at the end for your supervisor to read
    it for you to make any necessary amendments
  • If its good, consider publishing a summary in a
    journal. Ask your supervisor.

101
When starting your project you should
  • Have a clear aim, and a rough idea of your
    methods and the relevant literature, and a few
    ideas about problems
  • Make an appointment with your supervisor and
    discuss what you will do and the timescale. Take
    your proposal and comments
  • Remember your supervisor may have a holiday
    planned agree when you will meet / email. Usual
    to send drafts of chapters when completed
  • Remember the deadline and plan back from this.
    Send your supervisor a draft of the project at
    least a month before the deadline
  • Project guidelines at http//userweb.port.ac.uk/w
    oodm/rm
  • Practical guidelines on statistical analysis at
    http//userweb.port.ac.uk/woodm/stats/statnotes0.
    pdf
  • Any questions to michael.wood_at_port.ac.uk

102
Interviews and qualitative research more detail
  • I am grateful to Alan Rutter for the next few
    slides, some of which I have edited slightly

103
Primary data collection interviewing
  • Useful for accessing peoples perceptions,
    meanings, definitions of situations, eliciting
    their constructions of reality, etc.
  • Alternative types
  • structured
  • semi-structured
  • in-depth
  • Ethical considerations

104
Forms of qualitative interviews
F F
f
Qualitative interviews
One to one
One to many
Focus group interviews
Face to face interviews
Telephone interviews
After Saunders, et al, 2000
105
Interview respondents
  • Who will be interviewed and why?
  • How many will be interviewed and how many times?
  • When and for how long will each person be
    interviewed?
  • Where will each person be interviewed?
  • How will access to the interview situation be
    organised?

106
Sampling for small sample qualitative research
  • Usually best to use theoretical (purposive)
    sampling - the selection of individuals who you
    think will best contribute to the development of
    a theory
  • Results apply to immediate situations
  • May be tentatively generalised, but the small
    sample means

107
Difficulties with interviews
  • Mistrust by respondents
  • e.g. researcher is a management spy
  • Loyalty to organisation/colleagues
  • Adherence to stereotypical views rather than
    their own inner feelings and knowledge
  • Complete indifference
  • An opportunity for respondent to sell their
    ideas

108
Managing the interview
  • Preparation for the interview
  • the interview schedule
  • Beginning the interview - establishing rapport
  • Communication and listening skills
  • Asking questions
  • sequence and types of questions
  • Closing the interview

109
Verifying interview data
  • Body language
  • Material evidence
  • e.g. company/factory tour
  • Writing notes
  • as soon as possible after interview
  • Use informant verification and secondary sources

110
Remember
  • Need to demonstrate rigour
  • Good research acknowledges bias and the need to
    expose it.
  • Devils advocates are useful for revealing bias
    and other problems, but are seldom used.
  • Is all research is biased?

111
More on Literature reviews
  • I am grateful to Andreas Hoecht for the next 16
    slides
  • Dont forget the literature should be clearly
    focused on your research aims, and it should be
    critical in the sense that you should point out
    strengths and weaknesses where appropriate

112
Research methods writing a literature review
(Andreas Hoecht)
  • 1.Finding material
  • 2. Mapping relevant literatures
  • 3. Evaluating literature
  • 4.Some practical hints

113
Writing a literature reviewFinding material
  • There is no prescribed number of sources you
    should use, it depends on the topic
  • Be wary if you feel that you are drowning in
    material you found for your topic, it probably
    means you have not narrowed it down enough
  • Be wary if you find no sources or very little
    sources. You normally need some academic sources
    to be able to write a meaningful literature review

114
What secondary sources should you use?
  • Books
  • Use textbooks only to get an overview over a
    topic
  • Academic monographs (edited books with chapters
    by different authors) can be very useful. They
    often explore a topic from different angles or
    cover different aspects of a topic
  • Dont use airport bookstall books as serious
    sources

115
Secondary sources
  • Journals
  • Peer-reviewed academic journal articles should
    normally be the backbone of your literature
    review
  • They provide up-to date discussions of topics and
    are usually more narrowly focused than textbooks
  • Trade journals (non peer-reviewed) can provide
    good introductions to topics and overviews of
    developments but carry considerably less academic
    weight than academic journals.

116
(Secondary) sources
  • Sometimes you may be able to find article titles
    like A review of the literature in academic
    journals. They can save you lots of work
  • Internet
  • Make sure you are able to distinguish between
    credible sources and Joe Blocks unsubstantiated
    views
  • Reputed organisations websites can be good
    sources of information (but may have a
    bias/self-interest). (gov. Agencies, internat.
    Organisations)

117
(Secondary) sources
  • Dissertations and PhDs
  • Checking dissertations stocked in the library may
    help you to get a feel for what is expected in a
    dissertation as well as provide information on a
    topic
  • Government reports/EU reports/other
    organisations reports can be very useful (but
    are sometimes biased).

118
Searching for literature
  • The key is the use of electronic databases
  • Some databases are full text (you can download
    articles directly), others are bibliographic
    databases (you need to check with library or use
    inter-library loan requests)
  • Business Sources Premier/Emerald Full
    Text/Econlit/Science Direct are all recommended
  • Be patient and creative in the use of keywords

119
Searching for literature
  • CD-Rom newspaper databases (FT, Economist) can be
    useful tools
  • Financial Data and Marketing databases mainly
    provide primary data

120
Mapping out relevant literatures
  • Dont put everything you find or everything you
    read in your literature review
  • Time spent on familiarising yourself with and
    assessing literature for relevance is never
    wasted
  • Only after you have gained a good overview over
    the literature will you be able to decide on your
    particular angle and your research questions

121
Mapping out relevant literature
  • Your database search should tell you how much and
    what type of literature is available
  • For some well-researched topic you will be able
    to concentrate on the literature directly dealing
    with your specific topic
  • For other research ideas, you may need to think
    about related areas or similar experiences in
    other industries or possible insights from other
    subject disciplines for enlightenment

122
Mapping out relevant literature
  • An simple example If you are interested in TQM
    and small firms you may wish to
  • Look at the TQM literature in general for the
    pros and cons, constraints and motives
  • Identify success and failure factors from the TQM
    literature
  • Check the small business literature for general
    business conditions and constraints
  • Check the small business literature to find out
    if these success factors apply there

123
Mapping out relevant literature
  • You can draw this as a conceptual map of
    overlapping circles or as a flow diagram if this
    suits your learning style
  • Brainstorming and drawing conceptual maps is best
    done after you have gained a feel for the
    literature from your literature search

124
Evaluating literature
  • This becomes easier with experience
  • When reading literature
  • identify the key arguments that are made
  • The reason(s) for the conclusions drawn
  • They should be either derived from logical
    deduction (a conclusion following necessarily
    from premises) and /or based on empirical
    evidence

125
Evaluating literature
  • Check the logic of the arguments made
  • Does this necessarily follow?
  • Check the supporting evidence
  • Is this data relevant? Is it meaningful and
    accurate? Could it be interpreted in another way?
    Which data would I need to challenge this?
  • Check for flaws tautologies, simplistic
    analogies, redefinition of terms, moral
    judgements (ought to)

126
Some practical hints
  • Make sure you refer to key texts that are
    frequently cited in the literature
  • Find out whether there are different schools or
    camps in the literature and cover their
    positions.
  • Use your research questions to structure your
    literature review
  • Check the validity (logic, empirical evidence) of
    arguments made
  • Make clear on what basis you decide to side with
    a camp or author or why you remain unconvinced
    or oppose a judgement

127
Some practical hints
  • Dont overstate your case and be realistic about
    what you can conclude
  • Be particularly fair to views and arguments you
    dont agree with (avoid to be seen as biased)
  • Dont be shy to critique established trade
    names(academic gurus)
  • Write your literature review for non-specialists
    and avoid jargon
  • Write it well structured and easy to read
Write a Comment
User Comments (0)
About PowerShow.com