NACADA Research Committee

1
Conducting Academic Advising Research A Primer

• NACADA Research Committee
• Sharon Aiken-Wisniewski, University of Utah
• Joyce Buck, Pennsylvania State University
• Kathleen Smith, Florida State University
• October 18, 2006

2
Why research in advising

• Hot topics on campus
• Topics becoming research questions
• Research vs. program evaluation

3
Research vs. program evaluation

• Goals of research
• Creation of new knowledge
• Testing hypotheses
• Documenting a novel phenomenon

• Goals of program evaluation
• Accountability
• Management
• Decision making and budgeting

4
Methods from Social Science

• Similarities
• Use of objective and systematic methods
• Range from such subjective methods as field
  observation to objective experiments

• Differences
• Research systematic observations of phenomena
  manipulation of ex-perimental variable and random
  sample
• Evaluation of non-experimental observation

5
Measures

• Questionnaires, observation, interviews, content
  analysis, ratings similar in both research and
  program evaluation

6
Results

• Research/discovery
• Generalization to others
• Contribution to new knowledge
• New hypo-theses/questions raised

• Program evaluation
• Site specific
• Improved program delivery
• Increased response to constituents
• Cost savings
• Questions answered

7
Audiences

• Research, discovery, inquiry for ourselves and
  other professionals in the discipline
• Program evaluation for administration

8
Developing a research project

• Identify three hot topics on campuses. What
  intrigues you or others?
• Topic one
• Topic two
• Topic three

9
Developing a research project

• Put your hot topics into question form.
• Question 1.
• Question 2.
• Question 3.

10
The basics core skills

• Identify the question/problem
• Review the information
• Formulate the research question
• Select a research design
• Collect and analyze data
• Draw conclusions

11
Core skills--the problem

• Identify the question/problem
• What did you identify as your hot topic?
• Is there an answer available?
• Can you identify likely factors at play?

12
Core skills--information

• Review the information
• Review the existing literature.
• Ask what others have said on the topic.
• Look for a model or theory that may explain the
  problem.
• Consider the information in light of your own
  observations and reading of the literature.

13
Core skills--information

• The literature review--why
• To improve your knowledge
• To build upon previous research efforts
• To generate new ideas
• To look for models of good methodology
• To become familiar with publication formats
• To find arguments to support your efforts

14
Core skills--information

• The literature review--how
• Identify sources
• Books, journals, bibliographies, indices,
  conferences presentations and proceedings,
  database searches
• ERIC, Social Science Citation Index, NACADA
  Journal, relevant literature from various
  disciplines
• Take notes

15
Core skills--information

• The literature review--what
• Previous research on same topic
• Previous research on related topics
• Research populations
• Research methods/techniques
• Research materials
• Theoretical frameworks
• Trends motivating interest in the topic

16
Core skills--information

• The literature review--when
• At the start
• Throughout the research process
• Immediately before submitting your paper for
  review
• During the revision if asked
• Nearing final publication

17
Core skills--the question and design

• Formulate the research question
• Summarize your thoughts and clarify
  relationships. Identify the explicit question.
  State your hypothesis.

• Select a design and subjects
• Review designs and methods used to test questions
  similar to yours and select one.
• Identify subjects/participants and seek human
  subjects approval.

18
Core skills--design

• Qualitative methods result in data described in
  words, such as responses to open-ended questions,
  observations, and interviews

• Quantitative methods result in data described in
  numbers--statistics, probabilities, graphs

19
Core skills--design

• Qualitative methods may be used when
• Little may be known about the topic or
• Detailed observations are desired from individual
  participants

• Quantitative methods may be used when
• Participants are not available for extensive
  interactions or/and
• Time and funds are limited or/and
• Cause and effect are to be determined.

20
Core skills--design

• One use of qualitative research may be to develop
  new theory
• Identify a topic
• Choose qualitative method
• Select sample
• Collect, categorize and analyze data
• Discover, expand, revise theoretical
  relationships
• Formulate new theory

21
Core skills--design

• One use of quantitative research may be to test
  existing theory
• Identify a topic
• Develop a hypothesis
• Choose a quantitative method and appropriate
  sample
• Select/develop appropriate measure
• Collect, analyze and interpret data
• Discover, expand, revise theoretical
  relationships
• Formulate new or revised theory.

22
Core skills--design

• Common designs used in academic advising research
• Experimental
• Ex-post facto
• Survey
• Historical
• Ethnographic

23
Core skills--design

• Characteristics of good advising research
• Valid
• Measure of what thing measuring--internal
  validity
• Results generalizable to another group--external
  validity

• Reliable
• Methods and measures replicable throughout
  study--internal reliability
• Work replicable elsewhere--external reliability

24
Core skills--participants

• Selecting subjects/participants
• Identify total population
• Select a random sample (ideally)
• For empirical study assign to control and
  experimental group, and control for confounding
  and random variables
• Identify existing group for comparison for
  ex-post facto, historical, or ethnographic study
• Conduct the research to identify differences
  between groups or to describe the group.

25
Core skills--participants

• Human subjects review
• Any organization that receives federal money must
  have a person or committee to review research
  using humans as subjects/participants. They guard
  against abuse of subjects/participants.
• Human subjects committee
• Institutional review board

26
Core skills--participants

• Human subjects review
• Must obtain clearance from institutional review
  entity and from perhaps board of any other
  cooperating institution. Submit early!
• Approval also needed from the federal government
• NACADA research grant proposal must include the
  clearance to be considered.

27
Core skills--data analysis

• Data analysis
• Codify the data to look for patterns in answers
• Test your explanation
• Revise if necessary and re-analyze

28
Core skills--data analysis

• Descriptive statistics
• Techniques both analytical and graphic used to
  paint a picture of a data set
• Mean
• Median
• Mode
• Percentages

• Inferential statistics
• Techniques used to conclude or infer something
  about a large group of subjects
• Multiple regression
• Chi-square
• Analysis of variance

29
Core skills--conclusions

• Draw conclusions
• Consider the observed data and how you arrived at
  them. Draw conclusions from your results. They
  may
• Support your hypothesis
• Justify or prove the effectiveness of a program
• Refine an existing theory
• Help to develop a new theory

30
Developing a research project

• Select one of your questions. Which common
  research method would provide valid answers to
  it?
• Method
• Given the method selected,what group(s),
  phenomena or records should be evaluated? What
  comparison sample is needed to make a valid
  comparison?
• Participants sample Comparison sample

31
Developing a research project

• Where should you look to review the existing
  literature? How will you find valid measures?
• What method of data analysis will you use?
• How will you secure institutional or other
  support for this project?

32
Conclusion

• Commit yourself to the time and resources needed.
• Seek support of administrators.
• Ask for help from colleagues and collaborate with
  others.
• Follow sound research practices from the start.

33
Advising Research Reference List

Resources are available on the NACADA Web site at
http//www.nacada.ksu.edu/Clearinghouse/Research_
Related/index.htm
34
Thank You!

• NACADA Research Committee