Comprehensive Exam Review

1
Comprehensive Exam Review
Click the LEFT mouse key ONCE to continue
2
Research and Program Evaluation Part 1
Click the LEFT mouse key ONCE to continue
3
Types of Research
4
Theoretically, basic research is conducted solely
for the purpose of developing and/or refining
theory.
Theoretically, applied research is conducted
solely for the purpose of evaluating the
application of theory for the solution of
problems.
This distinction is not, however, a particularly
helpful way to consider research methodologies,
because basic and applied are actually extremes
on a continuum of application and
generaliza-bility to other situations.
5
The two most commonly discussed types of research
are qualitative and quantitative.
Technically, qualitative and quantitative are
different approaches to inquiry rather than
distinctly different research methodology
categories.
However, the qualitative / quantitative
differ-entiation facilitates explanation of
different research methodologies.
6
Qualitative Research
7
Qualitative research involves collection of
extensive data on many variables over an extended
period of time in a naturalistic setting.
Qualitative research methodologies are grounded
in the belief that behavior is significantly
influenced by the environment in which it occurs.
Qualitative researchers seek to find holistic,
in-depth understanding of the phenomenon observed.
8
Qualitative research is inductive, and its
primary goal is to promote greater understanding
by explaining how and why people behave the way
they do.
Measurement data collection, analysis, and
interpretation techniques and research design
and method in qualitative research are flexible
and evolve as the research process proceeds.
9
Qualitative research may involve interactive
strategies, such as participant observation, or
noninteractive strategies, such as review of
documents.
Participant or nonparticipant observation are the
most frequently used research strategies in
qualitative research, particularly in the social
sciences.
10
The primary distinguishing characteristic of
qualitative research is that the actual
methodology used cannot be determined a priori
the methodology emerges quite literally during
the process and often involves both interactive
and noninteractive strategies.
Qualitative research usually involves pur-poseful
sampling, as opposed to probabilistic sampling,
to ensure that the right person(s) and/or
situation(s) are examined.
11
Generalizability of the results is not a major
concern in qualitative research (in accord with
nonprobabilistic sampling), and usually
generalizability is very restricted.
A qualitative research case study design involves
focus on one phenomenon, such as a person, a
concept, a process, a group, or a program.
12
Qualitative case study designs are often used to
describe and analyze a situation, event,
or process,
develop a concept or model,
evaluate a program,
investigate social and cultural beliefs, or
serve as a prelude to quantitative research.
13
Ethnographic research is sometimes con-sidered
synonymous with qualitative research and
sometimes as a subtype of qualitative research.
In the latter perspective, ethnographic research
is usually interpreted to mean the study of a
culture.
In this context, a culture is any group of
people who regularly associate with one another
and who develop characteristic ways of behaving
and thinking.
14
Other key concepts in qualitative research
include
Participant observation, in which the re-searcher
literally becomes a participant in a situation
and makes research observations while
participating.
Observer effect, in which the presence of the
participant observer alters the nature of the
situation.
15
Observer bias, in which the participant observer
makes inaccurate and/or invalid interpretations
of the phenomenon being observed.
Fieldnotes, which are the actual data resulting
from the participant observation process.
Grounded theory, which is theory (or com-ponents
of theory) developed from the data collected in
real world settings.
16
Peer debriefer, which is a colleague who works
with the primary researcher to generate meaning
from the data collected.
Audibility, which is the process of maintaining a
record of data management techniques that
document the decision trail used.
Key informant, which is a person other than the
researcher who provides information (data) about
the phenomenon being studied.
17
Low inference descriptors, which are concrete,
precise, and almost literal descriptions of
phenomena.
Comprehensive sampling, in which all members of
an entire group are selected as the units of
study.
Maximum variation sampling, in which objects of
observation are selected because they represent
disparate examples of the phenomenon being
studied.
18
Critical case sampling, in which a unit is
selected for study because it represents a
dramatic example of the phenomenon being studied.
Network (sometimes known as snowball) sampling,
in which future successive participants are
identified by prior participants.
Negative case / discrepant data, in which a unit
is not behaving within the parameters of the
emerging pattern.
19
Quantitative Research
20
Quantitative research involves deductive logic,
focused and specific questions, collection and
analysis of empirical data, and generalization of
results.
To facilitate discussion, quantitative research
methods may be divided into descriptive and
experimental research.
21
Descriptive research does not involve
manipulation of variables, whereas experimental
research does involve manipulation of one or more
variables.
Descriptive research is conducted to provide a
characterization of a situation or circumstance,
or a complete account of a process.
Descriptive research does not alter whatever is
being studied.
22
Descriptive research methods include historical,
case study, field study, survey, developmental,
causal comparative, and correlational research.
Historical research is the systematic search for
facts relating to questions about the past and
the interpretation of those facts.
23
Historical research involves specific definition
of the time period addressed within the research.
It encompasses use of primary and secondary
sources.
Primary sources are first-hand accounts or
original source documents, whereas secondary
sources involve interpretations of primary source
data or information.
24
Case study research is the study of a single
individual for a specified period of time.
Case study research involves assessment, but not
manipulation, of a variety of variables that
potentially contribute to the persons current
situation.
Case study research also may be conducted through
use of a qualitative research paradigm.
25
Field study research is the simultaneous study of
a small number of people.
Field study research is sometimes conceived of as
a multiple case study, but includes
consideration of the members interactions.
Field study research also may be conducted
through use of a qualitative research paradigm.
26
Survey research is the collection of data from
members of a population to determine the current
status of the population with respect to one or
more variables.
Survey research is often a precursor to other
types of research, most commonly correlational or
experimental research.
27
Survey research typically includes self-reported
data.
Survey research methods often include use of
questionnaires, surveys, observations,
interviews, or sociometry.
28
Developmental research is the study of one or
more variables in a group of people over a
relatively long period of time (i.e., long enough
for potential developmental changes to occur).
Developmental research is usually considered
synonymous with the term longitudinal research.
Developmental research may be either
cross-sectional or longitudinal in nature.
29
Cross-sectional research involves studying cross
sections (i.e., subgroups) of a population
presumed to be at different developmental levels
to determine if develop-mental patterns or trends
exist as predicted.
Longitudinal research involves studying the same
group of people over a relatively long period of
time to determine if developmental patterns or
trends exist as predicted.
30
Other types of developmental research include
trend studies, in which a given general
population is sampled at each data-collection
point in time.
cohort studies, in which a specifically-defined
population is followed over time.
panel studies, in which the same, presumably
representative, panel (i.e., relatively small
group) is assessed periodically.
31
Causal comparative research is usually considered
synonymous with the term ex post facto research.
Causal comparative research is an attempt to
attribute causation without experimental
manipulation of a variable.
Causal comparative research is based on the
premise that both the effect and the alleged
cause exist at the time the research is conducted.
32
Correlational research includes studies having as
the purpose to determine the relationships
between or among variables through the use of
correlational statistics.
Correlational research has the advantage of
allowing study of many variables simultaneously.
Correlational research is probably the most
common type of research in the counseling and
development professions.
33
Experimental research is conducted to determine
if differences result from the interjection of
some phenomenon into peoples lives.
Experimental research always involves intentional
manipulation of one or more variables.
34
Experimental research also involves
comparing conditions under various stages of the
treatment (e.g., pre-post).
systematic manipulation of experimental
conditions in which extraneous influences are
controlled or eliminated.
application of Kerlingers MAXMINCON principle.
35
Kerlingers MAXMINCON principle is that
experimental research should
maximize the independent variables effects on
the dependent variable (i.e., maximize treatment
variance).
minimize error factors and/or variance.
control extraneous variance.
36
True experimental research means that the
researcher (theoretically) has control over all
the relevant variables.
True experimental research is usually conducted
in a laboratory or other highly controlled
settings.
True experimental research is relatively rare in
the counseling and development profes-sions due
to the extensive control required.
37
Quasi-experimental research approximates true
experimental research except that complete
control of all relevant variables is not possible.
Quasi-experimental research is usually
differentiated from true experimental research by
the inability to assign subjects to groups
randomly (e.g., intact groups are used) or lack
of a control group.
38
Single-subject experimental research involves
studying an individual in both treatment and
nontreatment conditions and evaluating
performance on the dependent variable in both
conditions.
Single-subject experimental research should not
be confused with a case study in single-subject
research, considerable effort is given to
controlling extraneous variables.
39
Action research is an attempt to solve a
specific, immediate, and concrete problem in a
local setting.
Action research is not concerned with
generalization to any significant degree.
Action research often is used to test the
effectiveness of new skills or methods.
Action research often lacks general credibility
due to vague definitions and controls.
40
Experimental Research Designs
41
The following are commonly used abbreviations in
the depiction of experimental research designs
R random assignment to groups
NR nonrandom assignment to groups
E experimental (or treatment or tx ) group
C control group
O observation (i.e., measurement)
X treatment (i.e., intervention)
42
Experimental research designs can be divided into
two types, depending upon the nature of the
comparison or type of effect to be evaluated
between groups and within groups designs.
Between Groups designs involve comparison of the
variable(s) across (i.e., between) two or more
tightly-controlled conditions (e.g., experimental
or control).
43
Within Groups designs involve each subject being
exposed to each treatment condition, but under a
randomly assigned sequence of treatment
presentations (i.e., each subject is his/her own
control).
Because within subjects designs, such as the
crossover, counter-balanced, and Latin Squares
designs, are used relatively rarely, the focus
here will be on the between groups designs.
44
In general, experimental research designs may be
divided into three categories based on the degree
of control over extraneous variance (i.e., the
degree to which a difference can be attributed
solely to the treatment)
Pre-Experimental Designs
Quasi-Experimental Designs
True-Experimental Designs
45
Pre-Experimental Designs

• One-Group Posttest-Only Design

Assign- ment Group Pre
Treatment Post NR E X O
46
Pre-Experimental Designs

• Nonequivalent Groups Posttest Only Design

Assign- ment Group Pre
Treatment Post NR E X O
NR C O
47
Pre-Experimental Designs

• Nonequivalent Groups Alternate Treatment
  Posttest-Only Design

Assign- ment Group Pre
Treatment Post NR E1 X1 O
NR E2 X2 O
48
Quasi-Experimental Designs

• Nonequivalent Groups Pretest-Posttest Design

Assign- ment Group Pre
Treatment Post NR E O X O
NR C O O
49
Quasi-Experimental Designs

• Single-Group Interrupted Time Series Design

Assign- ment Group Pre
Treatment Post NR E O1
O2 O3 O4 X O5 O6 O7 O8
50
Quasi-Experimental Designs

• Control-Group Interrupted Time Series Design

Assign- ment Group Pre
Treatment Post NR E O1
O2 O3 O4 X O5 O6 O7 O8 NR
C O1 O2 O3 O4 O5 O6 O7 O8
51
True-Experimental Designs

• Pretest-Posttest Control Group Design

Assign- ment Group Pre
Treatment Post R E O X O
R C O O
52
True-Experimental Designs

• Posttest-Only Control Group Design

Assign- ment Group Pre
Treatment Post R E X O
R C O
53
True-Experimental Designs

• Solomon Four-Group Design

Assign- ment Group Pre
Treatment Post R E1 O X O1
R C1 O O2 R E2 X O3 R C2 O4
54
True-Experimental Designs continued
Factorial experimental designs contain more than
two independent variables.
In the language of factorial designs (which
should not be confused with the statistical
process of factor analysis), a factor is an
independent variable.
Each factor in a factorial design has at least
two levels.
55
Levels in this context is synonymous with
categories.
Levels in this context is really a poor choice
of words, because no hierarchal order of
levels/categories in the variables is implied or
necessary.
For example, the variable gender in a factorial
design has two levels (i.e., male and female),
but clearly no hierarchy exists in these
categories.
56
The notation for a factorial design is a set of
numbers that identifies how many independent
variables are involved and the number of levels
in each of the variables.
For example, an experiment in which the
inde-pendent variables are group (experimental or
control), gender (male or female), and secondary
school grade level (9, 10, 11, or 12) would be
denoted as a
57
The number of numbers is the number of factors,
and the numbers themselves are the number of
levels in each of the respective factors.
In the 2 x 2 x 4 factorial design example
presented, there are three factors (independent
variables) and the first factor has two levels
(categories), the second factor has two levels,
and the third factor has four levels.
58
The order of the numbers in a factorial design
notation is not fixed by any rule.
The example presented could have been denoted as
a 2 x 4 x 2 factorial design, in which the first
factor was gender, the second was grade level,
and the third was group.
59
The (factorial experimental design) diagram for
the example presented would be
60
This concludes Part 1 of the presentation on
RESEARCH AND PROGRAM DEVELOPMENT