Title: Research Methods
1Research Methods
- Michael Wood
- michael.wood_at_port.ac.uk
- http//userweb.port.ac.uk/woodm/rm/rm.ppt
- This file contains draft slides which will be
updated. - 30 November 2009
2Reading
- There are many books available e.g.
- Saunders et al (2007)
- Robson (2002)
- Easterby-Smith et al (2002)
- And many others browse in the library
- These books vary a lot some are better on the
practical aspects, others on the theoretical
aspects. Sometimes you will get different advice
from different sources, so you need to consider
the rationale behind the advice. Robson is good
on most aspects, although Saunders et al is
probably more student-friendly
3Contents
- Overview of academic business research
- What must be in a project plan and a project?
- Formulating research aims
- The design of research projects
- Evaluating research
- Statistical analysis for research
- Qualitative data analysis
- Analysing data and presenting results
- Philosophy of research
- Questionnaire design
- Interview design and qualitative research
- Reminders about the project
- Interviews and qualitative research more detail
- More on literature reviews
4Overview of academic business research
- Reading browse through a book on research
methods e.g. Saunders et al (2007), Robson
(2002) - These slides intended as a brief summary of the
important points - Reread them when you are starting your project
5Advice on research methods
- Common sense dont forget this!
- Articles and books reporting similar research
should be discussed in the project - Books on research methods in general
- Focus on chapters relevant to your project.
6Purpose and characteristics of academic research
- Purpose
- Discover truth about something and/or
- Find a good way of doing something
- Must be
- Systematic and as thorough and trustworthy as
possible - Clearly written and with sufficient detail for
readers to check credibility - Ethical
7Types of research include
- Large scale surveys (of people, organisations,
events, etc) analysed statistically - Small scale surveys with emphasis on
qualitative detail - Case studies (to see how something works in
detail) - Experiments (change something to see what
happens) - Models can be set up, tested and used for
- Participant observation (observe as participant)
- Action research (combine research and action)
- Evaluation
- and may other possibilities be imaginative!
- Many projects combine several of these
8Sources of data many possibilities
- Interviews
- Including focus groups, Delphi technique (Robson,
200257), various approaches to eliciting
comments (e.g. photo elicitation Sam Warren) - Questionnaires, including via email (be careful
) - Documents (minutes of meetings, company reports,
etc) - The web
- Databases within organisation, of share prices,
etc - Observations of various kinds
- Etc . Be imaginative!
- Sources of literature is a different issue
(Judiths session is very important for this)
9Experiments (randomised controlled trials)
- Put people (or whatever you investigating) in
randomly assigned groups, give the groups
different treatments, and compare groups to see
what differences emerge. - Used for testing drugs, diets (http//tinyurl.com/
yp2t2o, http//tinyurl.com/489hns), educational
methods, different designs for websites, social
policies, etc. Lots of examples in Ayres (2007). - Advantages of experiments over non-interventionist
research - Disentangle cause and effect. Can control
variables you havent even thought of. If done
well evidence can be very convincing. - Can investigate new things
- Ayres, Ian. (2007). Super Crunchers how
anything can be predicted. London John Murray.
10But
- Experiments are often impractical or unethical
- Difficulties include
- Hawthorne effect
- Failure to assign groups at random (this matters
a lot because ) - So use less rigorous quasi-experiments instead
(Grant Wall, 2008) e.g. in action research
you may do a before and after comparison. This is
a sort of crude experiment but it is not as
convincing as a proper RCT. - Grant, A. M. Wall, T. D. (2008). The
Neglected Science and Art of Quasi-Experimentation
Why-to, When-to, and How-to Advice for
Organizational Researchers. Organizational
Research Methods (published online, July 18,
2008).
11Finding a suitable topic
- Interest
- Career
- Feasibility
- Usefulness
12How to do research
- Read about topic
- Draft aims of research. Clear, simple, focused.
- Draft literature review.
- Draft research plan check it is really likely
to meet your research aims. Check again. - Do research/analysis
- Draft research/analysis and recommendations/conclu
sions - Check it fits together and revise all sections
- If it doesnt fit together revise aims and
13Practical issues
- Timing
- Plan this remembering that your supervisor may
suggest extensive changes. - Gantt chart may help.
- Ethics (remember the form!)
- Access to information.
- Take care this is often difficult!
14What must be in a project and a project plan?
- Reading
- Project guidelines
- Proposal guidelines
- Saunders et al (2007), or another similar book
15What must be in a project?
- Abstract (short summary of project including
conclusions) - Background and aims (what youre trying to find
out and why its important) - Literature review (of relevant previous research
which you will build on or extend) - Research methods plan and justification (what
you did to meet the aims, and why it was a
sensible approach) - Analysis (in detail, to convince sceptical
readers and impress examiners important tables,
diagrams etc must be in the text, only details in
appendix) - Results, conclusions, recommendations,
limitations, further research - References (list works cited in text in
alphabetical order) - Appendices Ethics form, extra details for the
reader - Flexible designs can be more flexible but
everything must be there!
16Features of a good project
- Obviously important and interesting
- Difficult to disagree with because
- Arguments and analysis detailed, clear and
obviously valid - Possible objections considered and if possible
answered - Fits together
- Aims met by methods (check this in your project
plan) - Conclusions follow from analysis
17References and citations
- You must give references to publications which
you draw on or quote - Exact (word for word) quotes must be in and
the reference must be given - Maximum about one paragraph
- Use one of the standard referencing systems
preferably the Harvard (see university website) - Copying word for word without and reference
is treated as cheating and you will fail!
18Harvard referencing system
- Very important to use this (or another
established system) - Seems easy to me, but causes a lot of difficulty
- Check library website (search for Harvard) and/or
copy an academic article or book. - All references in text like Smith (2001)
- Then alphabetical list of references at the end.
Should include everything referred to, and
nothing else.
19What must be in your project plan (proposal)?
- See assignment description
- You may be able to put parts of it in your
project! - You should describe and justify your research
methods in as much detail as possible
20Writing style (1)
- Keep it simple.
- Short sentences
- Clear, short paragraphs
- Clear subheadings
- Read it through to make sure you can follow it.
Swap with a friend and check each others
21Writing style (2)
- I think the EMH was true in this situation
- In my opinion the EMH was true
- In the authors opinion the EMH was true
- 4 The evidence suggests that the EMH was true
- 5 This shows that the EMH was true
- Use 4 or 5.
- Avoid 1, 2 or 3 because it gives the impression
that its just your opinion and that other, even
wiser, people may see it differently.
22Writing style (3)
- I work for and the problems are / I
interviewed three managers. - The author works for and the problems are /
The author interviewed three managers. - Then problems of this organization are / Three
managers were interviewed. - Opinions vary here. I (MW) prefer (1). Others
prefer (2) or (3). - Check with your supervisor.
23Formulating research aims
- Reading most research methods books, e.g.
Saunders et al, 2007
24Research aims or questions
- Usually start from vague idea
- Then formulate a clear aim, or list of aims, that
your research will achieve. Think of these as
hoped-for outcomes. - Alternativelyformulate a clear question or list
of questions. - This process may require some creative thinking
- Techniques like brainstorming and mind maps may
be useful
25Aims, objectives, questions
- You can formulate your research aims as aims (or
objectives if you prefer that word) or questions.
- These are different ways of saying the same
thing. Doesnt matter which you use, but dont
confuse things by having aims and questions - May be helpful to have a list or hierarchy of
aims, but keep it simple
26Hypotheses
- Hypotheses are statements whose truth you want to
test, or predicted answers to research
questions (Robson, 2002) - Occasionally appropriate as a top level research
aim - e.g. to test the hypothesis that Working at home
improves quality of life - Usually best to avoid hypotheses when formulating
main research aims because questions or aims tend
to be more flexible - e.g. How does working at home affect quality of
life? - Null hypotheses have a (controversial) role in
some statistical analysis ( as you will see),
but they are not relevant to formulating your
overall research aims
27Research aims or questions
- Research aims or questions should
- Be clearly and simply expressed
- Fit together (so that you have a coherent
project) - Clarify the intended outcome and scope of the
research - Your research aims or questions should also
- Be relevant to your degree
- Be achievable
- Present a reasonable level of challenge
28Research aims or questions
- Must be research aims, not business or personal
aims. - However, business or personal aims may be part of
the background motivating your research aims, and
research aims would normally include the aim of
making recommendations to people or
organisations. - Should generally have a limited scope or focus.
- The danger with general aims is that they lead to
superficial research. - May relate to theoretical issues. You may be
aiming to test, modify or create a theory
29Theory
- Theory includes models, explanatory frameworks,
generalisations, recommendations - Examples .
- Your research should link with any relevant
theory. It may - Use a theory
- Demonstrate that a theory is useful
- Test a theory
- Modify a theory or create a new theory
30Also ask yourself
- Is the research worth doing?
- Are there any ethical or political problems?
- Is it possible? Have you got access to the
necessary data?
31Is it really going to be useful?
- What use do you want the results to be? This may
be a practical use to find out how to make more
money, or to make life easier or a contribution
to theory, but it should be something that is
really worth achieving. Must pass the so what?
test. - May help to clarify your aims if you imagine
youve done the research and write down what you
think your conclusions and recommendations might
be. - Then work backwards from what you want to achieve
to the best methods to achieve it.
32Example of research aims
- The aims of this research are to
- Describe the decision making strategies of small
investors - Determine the effectiveness of these strategies
- Any comments? Does this seem reasonable for a
Masters project?
33Another example of research aims
- The aims of this research project are to
- Evaluate Method X for planning mountaineering
expeditions - If necessary propose and justify Amended Method X
for planning mountaineering expeditions
34Another example of research aims
- What are the important quality problems in
Company X? - How serious are these problems?
- What is the best strategy for reducing these
problems? - Any comments? Does this seem reasonable for a
Masters project? Does it matter that they are
expressed as questions?
35Three more examples of research aims
- The aim of this research is to investigate the
role of the internet in banking. - This research project aims to explain activity
based costing. - The aim of this project is to
- Test the efficient market hypothesis for the
Athens stock exchange, and - Determine how global warming will influence share
prices. - Any comments? These are not reasonable for an
Masters projects! Why not?
36Possible research topics
- Research in a specific organisation
- Best if they are likely to implement any
recommendations - Take care you have adequate access to data
- Easier if you have a recognised / paid job there
and / or know key players well. - Research based on publicly available data
- Eg share prices, the www, published statistics
- Research based on surveys of the public
- These are just some possibilities. There are more
37Design of research projects
- Design means deciding on the methods and
approaches which will best achieve your aims - Needs thinking out carefully starting from your
aims - Check the proposed design will achieve all your
aims - The design may require the use of a theoretical
framework which should be explained and its use
justified - May incorporate several approaches (e.g. earlier
slide) - Some advocate flexible designs (E.g. Robson,
2002) - E.g. Poppy Jamans summary. Any comments?
- E.g. check aims and designs of these projects.
38Designing research is not easy!
- Think about how you can best achieve your aims
- Consider all possible types of research
- Be imaginative
- Think about it again
- and again
- Check youve found the best way you can for
meeting all your aims
39Group exercise
- Design a research plan for one of the projects
below, and do a pilot study for part of it. (You
may find you need to make the aims / questions
more precise.) - Michaels project. The provisional aims are
- To evaluate the suitability of the PBS website
for prospective PhD students - To suggest improvements to the website from this
perspective - Alisons project on the impact of a Blackberry on
family/work-life balance. What are the problems
and opportunities, and what would you recommend? - or
40Email project
- How much time do people spend on emails, is it
time well spent, and if not how can things be
improved? - Provisional method Survey to find how much time
is spent on emails, and respondents opinions on
whether this is time well spent, and on
recommendations (is this a satisfactory method?) - And / or other possibilities ?
41A general design for a typical Masters degree
project
- If the aim is to find a good strategy to
"improve" X in org Y, then a possible design may
be - Survey/case studies of Org Y to investigate
problems and opportunities - Survey/case studies to see how other
organisations do X and which approaches work well - Based on (1), (2), the literature, and perhaps
creative inspiration, consultations within the
organisation, simulation or modelling, devise a
strategy likely to improve X - Try/test/pilot/monitor the proposed strategy,
probably in a limited domain
42Take care with opinion surveys
- Suppose your research is about risk management
and its effectiveness. You decide to investigate
by means of a questionnaire and come up with - 70 of people in the organisation think our risk
management is unsatisfactory - 60 think Method X is the best way of improving
it - You then present this as the rationale behind
your recommendations to improve risk management. - But how do they know?
- Surely the researcher should find out by rigorous
and sensible methods, rather than asking people
who may neither know nor care?
43Exercise
- There are many problems with interviews and
questionnaires. Your respondents may - Not know the answers
- Not understand the questions
- Be too lazy to think about the issues
- Want to deceive you
- Try to design the methods for a research project
without using interviews or questionnaires. (This
is not usually a good idea but it should help you
to consider alternatives.)
44Then
- Having designed your research get someone to act
as a devils advocate and tell you - Whats wrong with it why it may fail to deliver
what you are aiming for - What may go wrong
- Would they trust the answer?
45Evaluating research
- Relevant to
- Planning your own research. Use the following
slides to - Check your proposal
- Check your final project
- Critically reviewing published research
- These slides are intended as a checklist for your
research and others
46Good research should be
- As User-friendly as possible
- Simple as possible given the message?
- As Uncritisable (trustworthy) as possible
- Trustworthiness or credibility is particularly
important. Can you trust the conclusions? Do you
believe them? Are there any flaws? Essential to
give readers enough detail to check. - As Useful or interesting as possible
- Clear implications for future? New results?
47In groups
- Choose one of the articles you have been given
- Assess its
- User-friendliness
- Trustworthiness (pay particular attention to
this) - Usefulness
- Brief feedback session, then we will compare your
critiques with my slides
48Trustworthiness of research main things to check
- C
- R
- I
- T
- I
- C
- Each letter represents an issue you should
consider
49Jargon
- Most of these checks are covered by technical
jargon, concepts and techniques e.g. lots of
types of validity (internal, external, construct,
face ), lots of types of reliability, ideas
about test and scale construction (see Robson,
2002), etc - Read up only those areas which you think are
relevant. I have largely avoided jargon here. - Always check sampling always necessary to
consider whether your sample is likely to be
representative of your area of interest.
50Deciding what is Cause and what is effect
- Important to try to work out what causes what,
and how strongly and under what circumstances, so
that you know what you should change to achieve a
particular effect. - Take care may be more complicated than it
appears (ISO 9000 and profitability drinking and
thinking, storks and babies). - Variable you havent thought of may be the
important cause! - Experiments (randomised controlled trials) for
definitive answers, but may be difficult, so - Quasi-experiments (e.g. a before/after comparison
of a trial of a new innovation) insead, but - May be lots of causes. Be suspicious of simple
explanations (see Taleb, 2008).
51Deciding what is Cause and what is effect more
examples
- A survey of organizations showed that those that
used the balanced scorecard were more profitable
than those that didnt. - Does this show that the balanced scorecard makes
firms more profitable? - A survey showed that the average job satisfaction
score for a department rose substantially and
significantly between 2006 and 2008. In 2007
everyone was sent on a weeks computer course in
the Seychelles. - Would you recommend a computer course for other
departments? - Does high staff turnover cause poor performance
or vice versa? (Glebbeek and Bax, 2004). Does
extraversion help people get promoted, or vice
versa (Moutafi et al, 2007). Does it matter? - What caused the fall of the Berlin Wall?
52To ensure results Representative check Sampling
- Decide what youre interest in often called the
population or target population. - Usually this is too big to look at everything so
take a sample. Normally we want the sample to be
representative of the population or wider
contextso you must check if this is likely. - Need to consider how the sample is selected and
its size. Badly chosen samples can be biased and
give very misleading results. - E.g. TV audience research, word length, NRE,
non-response bias in surveys, survivor bias in
stock price samples
53How to sample
- Clarify target population (the whole group of
interest) - May be a population of people, organisations or
- Decide sampling approach. There are many methods
of taking a sample from your target population,
including - Random
- Stratified
- Purposive
- Convenience (or opportunity, haphazard,
accidental) - Cluster, snowball, quota, etc (see a book)
- Decide size of sample need to balance cost with
information obtained. If you analysis is
statistical, statistical theory can help
54Random sampling
- Make a numbered list of the target population (a
sampling frame) - Use random numbers to choose sample
- Each member of population has the same chance of
being selected (and its independent of any
biases) - Each member of sample selected independently
- In practice, likely that some members of the
sample cant be found or wont help, so the
sample may be biased. Difficult to deal with this
possibilities - The principle is to ignore all variables and
choose at random. This allows for all noise
variables.
55Which sampling method?
- Usually random samples are best for large
samples, and purposive samples for small samples
analysed qualitatively. - Done properly, with a large enough sample, random
or stratified samples (probability samples)
should be reasonably representative of the
population. Cant assume this about purposive or
convenience samples (non-probability samples)
because these are selected by factors that are
likely to bias the result in one direction or
another.
56Sampling in practice
- Many samples are biased and so will not give a
good idea of the population regardless of
sample size. - E.g. NRE, non-response bias in surveys, survivor
bias in - Ideal for large samples is random sampling, but
this is often difficult to do properly. - E.g. Iraq war death rate (see http//www.iraqbodyc
ount.org/ for another approach), TV audience
research. - Be suspicious of statistical results from
purposive or convenience samples - Need to be especially careful with small,
purposive samples for detailed analysis
consider the purpose and choose accordingly
57Three surveys to check accuracy of NRE phone
service which is right?
- A Consumers Association survey used a sample of
60 calls, mainly about fares. The worst mistake
was when one caller asking for the cheapest fare
from London to Manchester was told 162 instead
of the cheaper 52 fare which was available via
Sheffield and Chesterfield. The percentage
correct was - 32
- A reporter rang four times and each time asked
for the cheapest route from London to Manchester.
The proportion of the four answers which were
correct was - 25
- An NRE sponsored survey found that the answers
were - 97 correct
- (Source Breakfast programme, BBC1 TV, April 30
2002.)
58More sampling problems
- An MBA student sends out 100 questionnaires to
100 organisations asking if they would be
interested in a particular service. Twenty are
returned, and of these 6 indicated they may be
interested in the service - There are 650 firms in the relevant industry
sector. How big is the market for the service?
Are you sure? - Suppose you wanted to find out how common it is
for women aged 30-40 to enjoy running. - How would you choose a sample to ask?
- Other examples and exercises attached
59Measurements (Indicators)
- If you want to find out whether customer
satisfaction, or quality or profits have improved
you must have a sensible way of measuring them. - Moreno-Luzon (1993) used managers perceived
achievement of objectives as a measure. Can you
see any problems with this? - How would you measure quality of service in a
casino? - How would you check if your proposed measure is
valid / reliable / right / accurate?
60Things to remember with measurements (1)
- Conventional to distinguish between validity (are
you measuring the right thing?) and reliability
(consistency) - If possible use an existing measurement system
(with acknowledgement / permission). This has two
advantages there may be evidence validating it,
and you can compare your results with previous
results. - Remember that some informants may be biased, or
too lazy to give good answers, or just ignorant. - If possible use triangulation (check with
information from different sources) - Ask yourself whether your proposed method of
measurement really measures the right thing
61Things to remember with measurements (2)
- Be especially careful with measures of value.
This may have several dimensions (Keeney, 1992).
E.g. the success of a firm might depend on
profitability, worker satisfaction, contribution
to the community - If you are measuring the success of a change,
remember there may be several different criteria.
E.g. - May be useful to use the average (mean) response
to a series of questions. Use your common sense
to see if this is reasonable, or if they should
be kept separate. (See literature on Tests and
scales e.g. Robson, 2002 292-308). - Keeney, R. L. (1992). Value-focused thinking a
path to creative decisionmaking. Cambridge,
Massachusetts Harvard University Press.
62Reliability of measurements
- Same answer at different times?
- If anything depends on subjective judgments,
check agreement between different judges - Eg marking projects
- If youre asking a number of questions to get at
the same information, check the relationship
between answers to these questions with two
questions use a correlation coefficient, with
more than two use Cronbachs Alpha (if you are
keen on stats!) see http//www.statsoft.com/text
book/stathome.htm
63Exercise how would you measure
64Theoretical assumptions
- If the research uses a theory, is the theory
right for the purpose? And is it a valid
theory? (Some theories, of course, are stupid or
wrong!) You need a critical evaluation in your
literature review. - A questionnaire or interview plan may be based on
assumptions about what is relevant. Are these
assumptions OK?
65Is the research sufficiently Imaginative?
- Imagination helpful in
- Thinking of hypotheses to explain things
- Thinking of new methods for researching
- Thinking of new ways of doing things
- Many recommendations for boosting imagination and
thinking creatively e.g. - Brainstorming
- Doing something else and coming back to the task
- etc
66Making sure that you are not being misled by
Chance
- Could your results just be due to chance?
- Have you taken account of sampling error? (If you
repeated your research with another sample are
you sure the answer would be the same?) - Is the sample large enough?
- Null hypothesis tests or confidence intervals can
be used to answer these questions. - Are the measurements reliable?
67The first CRITIC
- Cause and effect assumptions OK?
- Representative sample?
- Indicators (measurements) OK?
- Theoretical assumptions OK?
- Imaginative enough?
- Chance ruled out as explanation?
-
- (Checks needed are mostly common sense except
for Chance.)
68The second CRITIC
- C Claim?
- R Role of the claimant?
- I Information backing the claim?
- T Test?
- I Independent testing?
- C Cause proposed?
- Teaching skepticism via the CRITIC acronym and
the skeptical inquirer - Skeptical Inquirer, Sept-Oct, 2002 by Wayne R.
Bartz
69Two extra checks
- Use of a devils advocate or critical friend.
Remember the problem of confirmation bias you
are likely to be more enthusiastic about evidence
that confirms your pet ideas than about evidence
that undermines it! Get someone to try and be
critical and find difficulties with your research
then fix or (if unfixable) discuss the
problems. - Triangulation compare results from different
sources. Applies to data, methods, observers,
theories (Robson, 2002 174).
70Anything else?
- Is this list complete?
- Does it address all the flaws you noticed in the
paper you looked at? - What would you add or change?
71Checklist the 3 Us, the CRITIC and Extra checks
- User-friendly?
- UnCRITICisable (trustworthy)?
- CRITIC
- Useful?
- Extra checks
- Triangulation
- Devils advocate (critical friend)
72Another measurement problem
- Andy had answers from lots of questions on a SD,
D, N, A, SA scale - He wanted a measure to tell him which questions
produced responses which gave a a clear overall
view (COV) from his respondents - His defined his measurement as
- COV abs(SDDASA) N
- (where SD is the number of SD responses, etc, abs
absolute value) - He then highlighted questions for which COV gt 0
- Do you think this is a sensible measurement?
73Critique of an article
- Do you accept what the article says, or are there
flaws in the research? - Think about the article! Use your common sense.
- Check the CRITIC.
- Is it worth publishing? Could you do better?
- Read round the subject e.g. other research on
the same theme. - Would the research benefit from some qualitative
work, p values or confidence intervals, case
studies, different perspectives, experiments
74Statistical data analysis
- Go to http//userweb.port.ac.uk/woodm/stats/StatN
otes0.ppt
75Qualitative data analysis
- Aim is detail and depth of understanding
- Demonstrate and understand possibilities, but not
how frequently they occur - Use direct quotes () as evidence and to reduce
danger of imposing your perspective - Sometimes may be helpful to
- Summarise in a table or similar
- Use coding scheme to analyse statistically (but
be careful if the sample is very small!) - Further possibilities in Saunders et al, Robson,
www.qual.auckland.ac.nz/, Thorpe and Holt (2008)
76Analysing data and presenting results
- Questionnaires and interview plans, and possibly
some data, in appendix - Graphs and tables and important quotes from
interviewees etc in the main text - Focus on your research aims, not the questions in
your questionnaire - Readers want an analysis which shows how your
aims are met. They dont want to know the answers
to all the questions in your questionnaire! - Use appropriate summaries e.g. tables of
averages, or of main points from interviews
77Literature review
- See Saunders et al (2003) Chapter 3
- Focus on relevant books, articles and theories
- Brief on general points
- More detailed on research of particular relevance
to your project you will need to search for
articles using the library databases - Critical
- Should lead into your method and analysis
- Must be properly referenced!
78Philosophy of research
- In the textbooks you will find discussions of
positivism, social constructivism, phenomenology,
etc, etc. - In my view, Robson (2002) is the best research
methods text for philosophical concepts. - Almost all concepts and distinctions here open to
serious criticism see Robson (2002). Most
management research articles dont mention
philosophy. - I wouldnt suggest focusing on these ideas unless
you are interested in which case be critical of
what you read! - If you do want to go into philosophy, use a book
like the Penguin Dictionary of Philosophy
(Mautner, 2005) or Thorpe and Holt (2008) to
check what the words mean. - Also note that there are arguments against being
prescriptive about research methods and
philosophy in books with titles like - After method mess in social science research
(Law, 2004), and - Against method outline of an anarchistic theory
of knowledge (Feyerabend, 1993)
79Further reading and references
- http//userweb.port.ac.uk/woodm/qualquant.pdf
80Some ideas which are worth mulling over
- Detailed study of a small sample vs less detailed
study of a large sample - Induction vs the Hypothetico-deductive method
(Popper) vs Following a framework / paradigm /
theory vs Deduction - Subjective vs Objective Facts vs Values
81Some misguided platitudes
- The following are often assumed (I think
wrongly) - There are two distinct kinds of research
- Quantitative (positivisthypothetico-deductive),
and - Qualitative (phenomenologicalinductive).
- Instead
- Positivist research (only) starts from
hypotheses. - Instead ...
- Academics tend to disagree about many of these
issues. If you do decide to go into them, please
think hard, and dont accept everything you read
in the textbooks uncritically!
82Qualitative vs quantitative
- Quantitative usually means statistical often
with largish samples - Qualitative means focusing on qualities usually
with smallish samples studied in depth - Disadvantage with statistical approaches is that
the data on each case is often very superficial - Disadvantage with qualitative approaches is that
case(s) studied may be untypical and cant be
used for statistical generalisation - Often best to use both approaches. This is known
as mixed methods search for this keyword in
library. - This distinction often confused with other
distinctions
83Regrettable tendency to reduce things to a simple
dichotomy
- If youre a soft and cuddly person
- Soft and cuddly (e.g. interpretivist,
qualitative, inductivist ) is good - Hard and spiky (e.g. positivist, quantitative,
deductivist, ) is bad - But if you are a hard person you will probably
reverse good and bad above. There are really many
different dichotomies. Reducing them all to one
is neither right nor useful.
84And
- To hard and spiky people, soft and cuddly
research is lacking in rigour - To soft and cuddly people, hard and spiky
research is naïve and lacking in richness
85Induction vs hypothetico-deductive method
- Generalise from the data without preconceptions
(induction) - Grounded theory. Rigour is in process used to
generate theory from data - Versus
- Use data to test hypotheses or theories
(hypothetico-deductive method) - Karl Popper. Rigor is in the testing.
- Theory building vs theory testing is much the
same distinction (see Saunders et al, 2007, pp
117-119) - However, I dont think these are the only choices
86Other useful approaches besides induction and
hypothetico-deduction
- Use a framework or theory or paradigm (Kuhn,
1970) to define concepts, questions, and
measurements, but without trying to test the
theory - Arguably what most scientists do most of the time
(c.f. Kuhn, 1970). Rigour is in ensuring the
theory is a good one, and in using it properly. - Deduction from data, theories and framework. E.g.
the differences between two quality standards can
be deduced. - Rigour is in checking the deduction and the info
you start with - Differs from the hypothetico deductive method in
that the result is the deduction itself, not a
confirmation, rejection or revision of a
hypothesis or theory - Note that this contradicts the assumption in
Saunders et al (2007 117-119) that there are
just two approaches deductive and
inductive. I think they mean
hypothetico-deductive, and they omit the two
very important possibilities above.
87An example
- How would these four approaches work with a
project of interest to you
88Karl Poppers ideas (1)
- Science works by putting forward bold theories
(or hypotheses) and then testing them as
thoroughly as possible - Provisionally accept theories that have withstood
this testing process - Theories must be sufficiently precise to be
falsifiable otherwise not proper science (eg
Freuds theories are too vague)
89Karl Poppers ideas (2) - eg
- Einsteins theory of general relativity predicts
that light from a distant star will be bent by a
small amount by passing close to the sun.
Newtons theory predicts the light will not be
bent. - Only possible to check during a total eclipse of
the sun. In an eclipse in 1918 light was bent as
Einsteins theory predicted - Newtons theory is falsified Einsteins lives on
and seemed much more credible.
90Karl Poppers ideas (3)
- Theories can come from anywhere guesswork,
intuition, other theories, etc - The process of criticising theories and trying to
show they are wrong is vital for science - The method applies to both natural and social
sciences - How would you apply Poppers ideas to a
management research project? in practice, has
elements in common with a critical attitude
91Critical attitude
- Try to anticipate and discuss criticisms
- Get a friend to act as a devils advocate
- Your work should be so convincing that it cant
be disputed! - Think of any criticisms you have of articles you
have read. Make sure the same faults dont apply
to your work. - Word critical sometimes used in a slightly
different, more specific, sense.
92Questionnaire design summary
- Read a (chapter of a) book on questionnaires
- Develop a pilot. Remember questionnaires are far
more difficult to design than they appear! Check
with your pilot respondents - Is it clear?
- Is it interesting / appealing / user-friendly /
not too long? Would you answer it? - Does it provide (only) the information you want?
- Are you still sure a questionnaire is a good idea?
93Questionnaire design (1)
- Write down what you want to find out
- Closed questions
- Tick boxes
- Rating (Likert) scales
- Open questions
- Pros and cons of each
- Check your questions will enable you to find out
what you need to for your research
94Questionnaire design (2)
- Covering letter
- Pilot it
- 3-4 nice friendly people to tell you whats wrong
with it - Pilot the analysis too
- Consider sample to send it to
- Anonymity / confidentiality
- How to send it / get it back (email?)
- What to do about non-response?
95Questionnaire design (3)
- Far too many questionnaires about - many of them
very silly. What is the response rate likely to
be? Would you fill it in? - Are you sure a questionnaire is necessary???
- Many companies have a policy of not responding to
questionnaires - Are there any alternatives?
- Check with your supervisor before sending it out
96Take care with opinion surveys
- You can ask someone
- What she did last week
- What she does in general terms
- Her opinion of what she does
- What she thinks other people do
- Her opinion of what she thinks other people do
- How she thinks things can be improved
- What she thinks about particular suggestions
about how things can be improved - What she likes / wants / values
- Etc
- Think about what type of question you are using
and whether it is really useful
97Interview design in brief (1)
- Read a (chapter of a) book on interviews
- Follows, or precedes questionnaire, or stands
alone - Be clear what you want to find out
- Consider telephone interviews
- Small sample. Dont do too many interviews.
- Plan your questions. Should be open-ended and
flexible, and aim for a detailed understanding - Probes and prompts
98Interview design (2)
- Ask for permission to tape record
- Transcribe interesting bits to get quotes for
your project - Get interviewee relaxed. Anonymity /
confidentiality (take care here!) - Check youve covered everything
- Send interviewee transcript afterwards?
- Some transcripts or parts of transcripts in
appendix?
99Reminders about the project
- Research aims should be simple, explicit,
focussed, motivated, useful - Literature review should focus on relevance to
your project - References should be complete and in order
- Methods should be the best which are feasible.
- Analysis chapter should show how hard and
skilfully youve worked, and why readers should
believe you. You need to convince a sceptical
reader who may want to know details of how your
data was obtained e.g. source of samples,
location of interviews (pub or office?), etc, etc
and analyzed. - Conclusions and recommendations should summarise
what you have found, and clearly meet the
research aims. Also discuss limitations. - Changing your mind is to be expected if
necessary rewrite aims after doing the research!
100Reminders (2)
- Docs/links at http//userweb.port.ac.uk/woodm/pro
jects - Keep to the 15,000 word limit. You can get a good
mark with 13,000 words but not with 17,000 words. - Remember the ethics form no form, no pass!
- Be particularly careful about NHS ethics
clearance - Make use of your supervisor (see Project
Guidelines) - Plan the timescale (Gantt chart) allow time for
delays - Allow time at the end for your supervisor to read
it for you to make any necessary amendments - If its good, consider publishing a summary in a
journal. Ask your supervisor.
101When starting your project you should
- Have a clear aim, and a rough idea of your
methods and the relevant literature, and a few
ideas about problems - Make an appointment with your supervisor and
discuss what you will do and the timescale. Take
your proposal and comments - Remember your supervisor may have a holiday
planned agree when you will meet / email. Usual
to send drafts of chapters when completed - Remember the deadline and plan back from this.
Send your supervisor a draft of the project at
least a month before the deadline - Project guidelines at http//userweb.port.ac.uk/w
oodm/rm - Practical guidelines on statistical analysis at
http//userweb.port.ac.uk/woodm/stats/statnotes0.
pdf - Any questions to michael.wood_at_port.ac.uk
102Interviews and qualitative research more detail
- I am grateful to Alan Rutter for the next few
slides, some of which I have edited slightly
103Primary data collection interviewing
- Useful for accessing peoples perceptions,
meanings, definitions of situations, eliciting
their constructions of reality, etc. - Alternative types
- structured
- semi-structured
- in-depth
- Ethical considerations
104Forms of qualitative interviews
F F
f
Qualitative interviews
One to one
One to many
Focus group interviews
Face to face interviews
Telephone interviews
After Saunders, et al, 2000
105Interview respondents
- Who will be interviewed and why?
- How many will be interviewed and how many times?
- When and for how long will each person be
interviewed? - Where will each person be interviewed?
- How will access to the interview situation be
organised?
106Sampling for small sample qualitative research
- Usually best to use theoretical (purposive)
sampling - the selection of individuals who you
think will best contribute to the development of
a theory - Results apply to immediate situations
- May be tentatively generalised, but the small
sample means
107Difficulties with interviews
- Mistrust by respondents
- e.g. researcher is a management spy
- Loyalty to organisation/colleagues
- Adherence to stereotypical views rather than
their own inner feelings and knowledge - Complete indifference
- An opportunity for respondent to sell their
ideas
108Managing the interview
- Preparation for the interview
- the interview schedule
- Beginning the interview - establishing rapport
- Communication and listening skills
- Asking questions
- sequence and types of questions
- Closing the interview
109Verifying interview data
- Body language
- Material evidence
- e.g. company/factory tour
- Writing notes
- as soon as possible after interview
- Use informant verification and secondary sources
110Remember
- Need to demonstrate rigour
- Good research acknowledges bias and the need to
expose it. - Devils advocates are useful for revealing bias
and other problems, but are seldom used. - Is all research is biased?
111More on Literature reviews
- I am grateful to Andreas Hoecht for the next 16
slides - Dont forget the literature should be clearly
focused on your research aims, and it should be
critical in the sense that you should point out
strengths and weaknesses where appropriate
112Research methods writing a literature review
(Andreas Hoecht)
- 1.Finding material
- 2. Mapping relevant literatures
- 3. Evaluating literature
- 4.Some practical hints
113Writing a literature reviewFinding material
- There is no prescribed number of sources you
should use, it depends on the topic - Be wary if you feel that you are drowning in
material you found for your topic, it probably
means you have not narrowed it down enough - Be wary if you find no sources or very little
sources. You normally need some academic sources
to be able to write a meaningful literature review
114What secondary sources should you use?
- Books
- Use textbooks only to get an overview over a
topic - Academic monographs (edited books with chapters
by different authors) can be very useful. They
often explore a topic from different angles or
cover different aspects of a topic - Dont use airport bookstall books as serious
sources
115Secondary sources
- Journals
- Peer-reviewed academic journal articles should
normally be the backbone of your literature
review - They provide up-to date discussions of topics and
are usually more narrowly focused than textbooks - Trade journals (non peer-reviewed) can provide
good introductions to topics and overviews of
developments but carry considerably less academic
weight than academic journals.
116(Secondary) sources
- Sometimes you may be able to find article titles
like A review of the literature in academic
journals. They can save you lots of work - Internet
- Make sure you are able to distinguish between
credible sources and Joe Blocks unsubstantiated
views - Reputed organisations websites can be good
sources of information (but may have a
bias/self-interest). (gov. Agencies, internat.
Organisations)
117(Secondary) sources
- Dissertations and PhDs
- Checking dissertations stocked in the library may
help you to get a feel for what is expected in a
dissertation as well as provide information on a
topic - Government reports/EU reports/other
organisations reports can be very useful (but
are sometimes biased).
118Searching for literature
- The key is the use of electronic databases
- Some databases are full text (you can download
articles directly), others are bibliographic
databases (you need to check with library or use
inter-library loan requests) - Business Sources Premier/Emerald Full
Text/Econlit/Science Direct are all recommended - Be patient and creative in the use of keywords
119Searching for literature
- CD-Rom newspaper databases (FT, Economist) can be
useful tools - Financial Data and Marketing databases mainly
provide primary data
120Mapping out relevant literatures
- Dont put everything you find or everything you
read in your literature review - Time spent on familiarising yourself with and
assessing literature for relevance is never
wasted - Only after you have gained a good overview over
the literature will you be able to decide on your
particular angle and your research questions
121Mapping out relevant literature
- Your database search should tell you how much and
what type of literature is available - For some well-researched topic you will be able
to concentrate on the literature directly dealing
with your specific topic - For other research ideas, you may need to think
about related areas or similar experiences in
other industries or possible insights from other
subject disciplines for enlightenment
122Mapping out relevant literature
- An simple example If you are interested in TQM
and small firms you may wish to - Look at the TQM literature in general for the
pros and cons, constraints and motives - Identify success and failure factors from the TQM
literature - Check the small business literature for general
business conditions and constraints - Check the small business literature to find out
if these success factors apply there
123Mapping out relevant literature
- You can draw this as a conceptual map of
overlapping circles or as a flow diagram if this
suits your learning style - Brainstorming and drawing conceptual maps is best
done after you have gained a feel for the
literature from your literature search
124Evaluating literature
- This becomes easier with experience
- When reading literature
- identify the key arguments that are made
- The reason(s) for the conclusions drawn
- They should be either derived from logical
deduction (a conclusion following necessarily
from premises) and /or based on empirical
evidence
125Evaluating literature
- Check the logic of the arguments made
- Does this necessarily follow?
- Check the supporting evidence
- Is this data relevant? Is it meaningful and
accurate? Could it be interpreted in another way?
Which data would I need to challenge this? - Check for flaws tautologies, simplistic
analogies, redefinition of terms, moral
judgements (ought to)
126Some practical hints
- Make sure you refer to key texts that are
frequently cited in the literature - Find out whether there are different schools or
camps in the literature and cover their
positions. - Use your research questions to structure your
literature review - Check the validity (logic, empirical evidence) of
arguments made - Make clear on what basis you decide to side with
a camp or author or why you remain unconvinced
or oppose a judgement
127Some practical hints
- Dont overstate your case and be realistic about
what you can conclude - Be particularly fair to views and arguments you
dont agree with (avoid to be seen as biased) - Dont be shy to critique established trade
names(academic gurus) - Write your literature review for non-specialists
and avoid jargon - Write it well structured and easy to read