Methodologies

1
Methodologies

• Setting up a Study

2
Review

• Problem Statement
• Problem
• Originality
• Direction
• Significance
• Literature Review Theoretical Framework
• Logical Structure
• Research Questions
• Population Sampling

3
Methodologies

• Specific systems/ tools/ approaches to gathering
  and analyzing data.
• Structure of the research
• Builds on and draws from previous sections
  problem statement, research questions, literature
  review, theoretical framework
• Influences analysis
• How to choose an appropriate method?

4
Methodologies Options

• Quantitative
• Numbers, statistics
• Qualitative
• Words, narrative

• Mixed Methods
• Both Quantitative and Qualitative
• Triangulation
• More than one method, may be two of the same type

5
Methods Options
Quantitative Qualitative Mixed
Predetermined Instrument-based questions Performance data, attitude data, observational data, and census data Statistical analysis Emerging Open-ended questions Interview data, observation data, document data, and audiovisual data Text and image analysis Predetermined and emerging Open- and close-ended questions Multiple forms of data Statistical and textual analysis
-Creswell, J.W.
6
Methodologies Options

• Quantitative
• Surveys
• Structured Interviews
• Questionnaires
• Bibliometrics
• Transaction Log Analysis

• Qualitative
• Case Studies
• No Contact
• Examining Documents
• Citation Analysis
• Diaries
• One-to-one
• Unstructured Interviews
• Think-Aloud
• Ethnographic methods
• Observations
• Group Interaction
• Focus Groups
• Delphi Method
• Concept Mapping

7
Quantitative Bibliometrics

• Quantitative study of literature
• Patterns of publishing within a field or body of
  literature
• Quantitative study of information associated with
  published works authorship, publishers,
  citations

8
Quantitative Transaction Log Analysis

• Quantitative study of user behavior as exhibited
  through computer logs
• Domains (.gov, .edu, .com)
• Hits
• Paths followed
• Time spent (searching, viewing, downloading)
• Errors
• Applications used

9
Qualitative Case Study

• In-depth review of a single situation program,
  process, phenomenon

10
Qualitative Examining Documents

• Systematic review of text/ images content
  analysis
• Classifies textual or visual material
• Uses analytical constructs or rules to draw
  inferences about recurring aspects of text.

11
Types of Documents?

• Policy Manuals
• Digital reference transcripts
• Comment/ complaint cards
• Job ads
• Published Literature
• Blogs, listserv postings, etc.
• Open-ended responses

12
Evaluation Research

• Published
• Books
• Journal articles
• Conference proceedings
• Theses/dissertations
• Web
• ALA. ACRL. CIP Notes
• CLIR. www.clir.org/pubs/pubs.html
• ARL
• Other
• Peer Review

• Unpublished
• Internal reports
• Local use only reports

13
Qualitative Analysis Examining Documents

• Focus on
• Word choice
• Word frequency
• Word sequence
• Intensity of feeling/ expression (how to
  measure?)
• Key word in context
• Typology of concepts/ categories?

14
Example

• Regional Accreditation Organizations Treatment
  of Information Literacy

15
Qualitative Study Citation Analysis

• Systematic review of bibliographies/ references
  within published literature.
• Focus on
• Authorship
• Form of publication (periodical, monograph, etc.)
• Class of material (primary, secondary, etc.)
• Language
• Subject
• Currency

16
Citation Analysis

• What can citations tell us?
• How scholarly is the cited literature?
• How current are the citations?
• How research-oriented is it?
• How interdisciplinary is it?
• How writes the literature? How collaborative?
• Where does the literature appear?

17
Citation Analysis

• Growth of literature on a subject
• Dispersion of writings on a subject across form
  and journals
• Obsolescence of literature
• Scholarly networks who cites whom?
• Publishing productivity

18
Citation Analysis

• Advantages
• Shows what is cited
• Does not involve interaction with subjects
• Profiles a literature
• Shows changes in a field over time?

• Disadvantages
• How complete is the work from which citations are
  drawn?
• How accurate are citations?
• Are all materials cited?
• Choosing easy to find/ retrieve items over better
  quality?
• How easily retrievable are works?

19
Citation Analysis Issues

• From where are citations drawn? How far can you
  generalize findings?
• Does citation reflect use?
• Self-citations and/ or gratuitous citations
• Half-life
• Impact factor (to what extent are recent articles
  in journals cited?)
• Calculated annually divide the number of current
  citations a journal receives to articles
  published in the previous two years by the number
  of articles published in those same years

20
Web Citations

• How prestigious are different online sources-
  ejournals, open access, etc.
• References to and from a Web site
• Retrievability

21
Qualitative Study Diaries

• Participants record activities, thoughts,
  reactions, etc. daily (weekly, etc.) over a set
  period of time.
• Blogs- equivalent?
• When to use?
• Issues
• How faithful are participants? How can you help
  ensure full participation?
• How much guidance to give?
• Too much/ too little detail.
• Advantages and disadvantages?

22
Students World

• Nicole Henning/ Photo Diary Study (MIT)
• 16 students tracked their information seeking
  behavior for one week.
• Used diaries and screen shots to record their
  thoughts and actions

23
Qualitative Study Interviews

• Attempt to gain in-depth knowledge on a topic
• In a less structured format, interviewer may act
  more as facilitator, asking open ended questions
  and drawing the participant out.
• Structured, semi-structured, or unstructured?
• Format- in person, telephone, email?

24
Interviews

• Types of questions
• Experience/ behavior
• Feeling
• Opinion/ value
• Knowledge
• Sensory
• Demographic

25
Interviews

• Issues
• Logistics
• Building trust
• Delicate questions/ situations- ensuring honesty
• Accuracy of transcripts
• Advantages and disadvantages?
• Appropriate uses?

26
Qualitative Study Think-Aloud

• Asks respondent to verbalize their thoughts while
  performing an assigned activity or task
• Attempt to gain more insight into thought
  processes
• Correct for mistakes/ assumptions of observer

27
Think Aloud

• Requires participants that are highly verbal in
  nature
• Adding this layer of feedback may affect their
  cognitive processes and behaviors- more
  self-concious.
• How to transcribe while observing? Accuracy?
• Appropriate use of clarifying questions?
• Appropriate uses?

28
Qualitative Study Ethnographic

• Tools developed in the field of cultural
  anthropology
• Attempt to better understand people/ behavior by
  observing it within natural setting
• Can study observable material items, individual
  behaviors and performances, or ideas
• Requires careful attention to detail within
  cultural context

29
Ethnographic

• Possible Tools
• Observation in context
• Key informant interviewing
• Drawing pictures
• Taking photos
• Using maps to track activities
• Videotaping

30
Ethnographic Examples

• Susan Gibbons and Nancy Fried Foster of
  University of Rochester Understanding Users to
  Develop Better Library Services (ACRL/NEC 2006)
• User-centered studies of library use
• Mapping Diaries
• Photo Elicitation Interviews

31
Photo Elicitation
32
Mapping Diaries
33
Sweeping Studies

• A type of spatial data analysis
• Useful for mapping out the physical spaces of a
  library and investigating how people use those
  spaces

34
Qualitative Study Observation

• Attempt to understand activities/ behaviors
• Obtrusive or Unobtrusive
• Role of the observer
• Complete observer
• Observer and participant
• Participant with individual being observed

35
Qualitative Study Focus Groups

• Attempt to learn about the attitudes/ beliefs/
  feelings of groups and how those influence
  behavior
• Why groups? One individuals comments can
  trigger important responses from others.
• Can explore large ranges of topics

36
Focus Groups

• Issues
• How structured?
• Importance of facilitators role- how well
  trained?
• Building trust
• Finding reliable representative volunteers
• Ensuring accuracy of transcription- audio/ video
  tape? Outside observation?
• Facilities

37
Qualitative Study Delphi Method

• A systematic interactive forecasting method.
• Involves interviewing/ surveying of experts
  within a field- generally 9 to 99
• Experts are provided with hypotheses, trigger
  statements, scenarios, etc. and asked to respond.
• First round responses are shared with the group
  anonymously, so participants can revise earlier
  statements, react to responses.

38
Delphi Method

• After several rounds, hypotheses may be refined,
  group may reach consensus
• Facilitator reviews responses to each round- can
  filter out irrelevant content, choose
  presentation of information, ask questions.
• Advantages and disadvantages?

39
Qualitative Study Concept Mapping

• Any process that represents ideas in pictures of
  maps.
• A method of organizing the ideas and thoughts of
  a group to form a common framework
• Can be used to integrate ideas from less
  structured activities such as brainstorming

40
Concept Mapping

• Typical steps
• Focus determine desired outcomes/ questions to
  be addressed
• Generating ideas Brainstorm, use trigger
  statements, ask questions
• Analysis Sort ideas into large sets
• Unstructured idea sorting ask individuals to
  sort ideas into groups and label
• Sorting by stakeholders organize ideas by group
  that generated those ideas
• Ratings assign values to ideas (importance,
  feasibility, etc.)

41
Concept Mapping

• Map analysis generate map based on idea sorting
  (could use statistical software, or do by hand)
• Interpret map Share with other groups to obtain
  understanding

42
Example IR and IL