Research Design

1
Research Design

• Procedures

2
Research Design

• Forming your action plan
• Deciding on the Who and When
• Defining all concepts and terms

3
Research Design

• Three purposes for research
• Exploration
• Description
• Explanation
• Or
• Descriptive- existing conditions
• Normative- observed vs. intended
• Impact- can it be attributed to programs?

4
Exploration

• To gain familiarity with a topic
• Typically done to
• Satisfy a researchers curiosity and desire to
  understand
• Test the water for a more extensive study
• Develop methodology to be used in a subsequent
  study

5
Description

• Provide context for situations and events
• Typically based on observation and reporting
• Observation is systematic
• Example U.S. Census

6
Explanation

• Attempts to address the question of why
• Tries to get at reasons and underlying causes
• Example Not would you vote for McCain but why
  would(nt) you vote for McCain?

7
GAO- Designing Evaluations

• Considerations
• Kind of information to be acquired
• Sources of information (i.e. types of
  respondents)
• Methods of sampling (i.e. random)
• Methods of collecting data (i.e. interviews,
  surveys)
• Timing and frequency of information collection
• Basis for comparing outcomes
• Analysis plan

8
Research Design Experiments

• Experiments aim to control one variable or set of
  variables in order to determine their
  relationship to and/ or impact on another set of
  variables.
• Types of studies
• Experiment
• Uses a random sample
• Quasi-experiment
• Does not use a random sample, must try to correct
  for error through statistical tests

9
Impact of a Program/Service Involving an
Experimental Design

• IMPACT
• Outcome of experimental group receiving the
  treatment compared to control group
• Information Literacy Instruction

10
Classic Design

• Two group pre- and post- test
• One experimental group
• One untreated control group
• Compare outcomes to assess impact
• Problems with this?

11
Solomon 4 Group Test

• Classic Design expanded to include two sets
• One set has experimental group and control group
  who both receive pre- and post-tests
• One set has experimental group and control group
  who receive only post- tests.
• Advantages over classic model?

12

• Solomon Four Group Design
• Before Treatment After
• Group 1 no yes yes
• Control 1 no yes yes
• Group 2 yes yes yes
• Control 2 yes no yes

13
Time Series Design

• Repeats testing twice (or more) to establish a
  trend in the data independent of the experiment

Pretest Treatment Posttest Pretest
Treatment Posttest REPEAT Experiment yes yes
yes yes yes yes Control
Group yes no yes
yes no yes
14
EXAMPLES

• Pretest/posttest design with control group
• pretest treatment posttest
• Experiment yes yes yes
• Control Group yes no yes
• Pretest/posttest design without control group
• pretest treatment posttest
• Experiment A yes yes yes
• Experiment B yes yes yes

15
Case Study

• Basis of selection representative, typical,
  cluster, probability, etc.
• Multiple methods of data collection

16
Populations and Statistical Sampling
17
Populations

• Population- the entire group/ universe under
  study
• Sample- a portion of a population of possible
  information sources
• Sampling- methods for selecting these sources

18
Research Design Action Plan (continued)

• Who is studied
• population
• Sample
• Is sample reflective of population?
• Where
• Sampling?
• When
• Sampling?

19
Who (or What) is being studied?

• Units of analysis the what or whom being
  studied. In social research the most typical
  units of analysis are individual people.
• Can be individuals, groups, organizations,
  social interactions, social artifacts
• Examples
• Library Users or Non-Users
• First-year Students
• Senior Citizens
• Public Libraries
• Also
• ILLs
• Biographies, Mysteries, Audiobooks (i.e.
  collections)
• Web sites

20
Beware Pitfalls of Analysis

• Ecological Fallacy
• Something learned about a group says something
  about the individuals making up that group.
• Reductionism
• An attempt to explain phenomena in terms of
  limited or lower-order concepts.

21
Who is being studied

• How to select a sample?

22
Sampling- Three Options

• Census- collecting information from the entire
  group making up a population
• Like the decennial census
• Judgment sampling- making conscious choices
• Convenience Sampling- whats available
• Probability/ Statistical Sampling- left to
  chance, each member of a population has an equal
  chance of being chosen

23
Sampling Purpose

• Representativeness
• Sample has roughly the same distribution of
  characteristics as the population from which it
  is drawn.
• Nevertheless, each sample will differ from each
  other, as well as from the population
• Can determine the amount of error

24
Probability Sampling (1)

• Random sampling Each member of the population
  has an equal and known probability of being
  selected
• Systematic sampling Each member of the
  population is either assembled or listed, a
  random start is designated and then members of
  the population are selected at equal intervals
  nth intervals

25
Probability Sampling (2)

• Stratified Each member of the study population
  is assigned to a group or stratum, then a simple
  random sample is selected from each group or
  stratum

26
Probability Sampling (3)

• Cluster Each member of the study population is
  assigned to a geographically-defined group or
  cluster. Clusters are then selected at random,
  and members of a selected group are represented
  in the sample.
• http//www.claritas.com/MyBestSegments/Default.jsp
  
• Role of GIS and TIGER files
• http//www.census.gov/geo/www/tiger/index.html

27
Non-probability Sampling

• Convenience selecting cases based on their
  availability
• Typical cases selecting cases already known
  and not considered extreme
• Snowball group members identify additional
  members to be included in sample
• Quota sample is in same proportion as population

28
The Sample

• How selected
• Sample size

• Determine the actual individuals or things
  included

29
Sample Size

• A larger sample does not necessarily mean better
  results, but
• Too small a sample can lead to error

30
Sample Size- 3 Considerations

• Precision (sampling error)- the range in which
  the true value is estimated to be 5
• Confidence Level (Central Limit Theorem)- when a
  population is repeatedly sampled, the average
  value is to the true value, and values in each
  survey will be normally distributed 95
  confidence level
• Degree of Variability- distribution of attributes
  within a population. The more homogenous the
  population, the smaller the sample size.

31
Sample Sizehttp//edis.ifas.ufl.edu/PD006TABLE_1
32
Sampling Customers (Example)

• Present
• Lost
• Never-gained
• Nonuser

33
Users/Uses of Electronic Resources (More Examples)

• Home page users in general
• Users of a database

34
Questions of When

• How might time effect our study?
• How do we choose a time frame?
• What is an appropriate time frame based on the
  research problem?
• Address time through
• Cross-sectional studies
• Longitudinal studies

35
Cross-Sectional Studies

• A study based on observations representing a
  single point in time.
• A snapshot
• Best for exploratory and descriptive studies
• U.S. Census
• Explanatory cross-sectional studies aim at
  drawing causal relationships over time based on
  observations made at one time.
• Issues?

36
Longitudinal Study

• Permits observations of the same phenomena over
  an extended period.
• Researcher may follow a group over time
• Researcher may become part of a group
• Researcher may study artifacts developed over time

37
Types of Longitudinal Studies

• Trend Studies
• Type of longitudinal study that examines changes
  within a population over time
• Cohort Studies
• Examines a specific subpopulation (cohort) as
  they change over time- often based on age.
• Panel Studies
• Examines the same set of people over time

38
The Learning Organization

• http//www.lib.umd.edu/groups/learning/learningorg
  .html

39
Group Activity Selecting a Sample

• Archive/ Special Library
• Public Library
• Academic Library/ School Library
• Identify a research question
• Define your population
• Describe how you would select a sample
• Could you design an experiment around this
  project?