Assessing the Impact of Faculty Advising: Implications for a Peer Advising Program

1
Assessing the Impact of Faculty Advising
Implications for a Peer Advising Program
Peter Swerdzewski Sara J. Finney Anna Lynn Bell
JAMES MADISON UNIVERSITY
Using assessment to determine if students lack
the knowledge and confidence that would
necessitate a peer advising program
Step 1 Assessment Design and Creation of
Objectives
Step 4 Results

• Simply surveying students about the universitys
  academic advisors may reveal perceptions about
  the advising program, but may not necessarily
  reveal what students know or feel confident in
  doing with respect to academic planning.
• Four outcomes of the universitys advising
  program were identified that address what
  students should know as a result of their
  advising experiences.
• One outcome of the universitys advising program
  was identified that addresses how confident
  students are in their abilities to complete the
  advising tasks expected of them by the
  university.
• All outcomes were crafted by a team consisting of
  the individual responsible for advising at the
  institution, a member of the Student Government
  Association, and liaisons from the universitys
  assessment center.

• Outcome 5 Increased student confidence in
  fulfilling the graduation requirements and
  utilizing academic tools and resources without
  the help of their faculty adviser.
• Five items that address students confidence in
  the following
• Completing graduation requirements
• Ability to interpret degree progress report
• Process for requesting an override
• Use of Web site to gather requirement information
• Navigating eCampus (student registration Web
  portal)
• All students indicated a confidence level
  somewhere between a fair amount of confidence
  and much confidence to fulfill the graduation
  requirements and utilize academic tools and
  resources without the help of their faculty
  adviser.

• Outcome 1 Increased student knowledge of
  academic resources
• Four items that address knowledge of the location
  of resources, including
• Registration dates and deadlines
• Financial aid
• The Registrars Office
• The Career Planning Office
• Assessment Day students answered 66.07 of the
  items correctly. Make-up students answered 64
  of the items correctly.

Step 2 Instrument Development

• Sample Multiple-Choice Item
• What is the minimum cumulative grade point
  average a student must maintain to be in good
  academic standing?
• ? 1.0 ? 1.5 ? 2.0 ? 2.5 ? 3.0
• Sample Confidence Item
• How confident are you in your ability to
  interpret your degree progress report?
• ? No confidence at all
• ? A little confidence
• ? A fair amount of confidence
• ? Much confidence
• ? Very much confidence
• Complete confidence
• Sample Attitude Item
• To what extent would you trust the information
  from a student peer adviser? Select the
  statement that is most characteristic of you.
• ? I would trust information from a peer adviser
  more than I
• would a faculty adviser.
• I would trust information from a peer adviser
  as much
• (equally) as I would a faculty adviser.

• The committee created indicators for each of the
  five outcomes. These items were constructed to
  represent the breadth and depth of each of the
  five outcomes.
• Rather than asking if students thought they knew
  specific knowledge related to the institutions
  advising procedures, we created a multiple-choice
  test so we would actually have evidence of
  students knowledge.
• Items were crafted under the assumption that a
  student with between 45 and 70 credit hours
  should correctly respond to them.
• In addition to the multiple choice items,
  students also responded to Likert-type items
  addressing their confidence to complete advising
  tasks (self-efficacy).
• Traditional survey-type and open-ended items were
  included in order to provide a profile of
  students attitudes toward current advising
  programming and a new peer advising initiative.
• This information was ancillary the primary focus
  was the assessment of student knowledge and
  self-efficacy related to academic planning, not
  their attitude about their advisor or the
  universitys advising program.

• Ancillary Analyses
• Although empirical evidence indicates that
  students knowledge and confidence related to
  advising at JMU is quite high (especially given
  the low-stakes testing environment in which the
  data was collected), approximately 30 of the
  Assessment Day sample (N 120) and 40 of the
  make-up sample (N 26) indicated that they are
  not satisfied with the advising they have
  received at JMU.
• Primary reasons stated for this lack of
  satisfaction include the belief that advisers
  lack competence or quality in the information
  they provide, and scheduling issues detract from
  the quality of advising.
• Students suggest that advisers become more
  knowledgeable in advising, and that the
  scheduling of meetings improve.
• Importantly The negative attitudes expressed
  toward advising were not due to students lack of
  knowledge of academic planning (students had the
  knowledge, but there were still complaints about
  advising).
• Furthermore Those students who may be most in
  need of a peer advising center (e.g., have
  avoidant or adversarial attitudes toward the
  university, such as students who skip the
  required Assessment Day and must attend make-up
  sessions) tend to indicate they would not feel
  comfortable approaching a peer advisor and may
  not trust a peer advisor.

• Outcome 2 Increased understanding of the
  students role in academic advising
• Six items that address students understanding of
  their responsibility in academic planning,
  including
• Course overrides
• Changing majors
• Scheduling preparing for a meeting with advisor
• Legitimate expectations of advisor
• Assessment Day students correctly answered 65.98
  of the items correctly, indicating that students
  understand their role in academic advising.
  Make-up students correctly answered 60.74 of the
  items correctly.

• Outcome 3 Greater knowledge of the nuts and
  bolts of academic advising like how to use
  e-campus, how to register for classes and how to
  make a four-year plan
• Nine items that address students understanding
  of the policies and processes related to academic
  planning, including
• Credit requirements (General Education,
  graduation, etc.)
• Grade point requirements
• Course withdrawal rules
• Which academic requirements apply to various
  situations
• Assessment Day students correctly answered 62.06
  of the items correctly, indicating a good
  understanding of the nuts and bolts of academic
  advising. Make-up students correctly answered
  60.99 of the items correctly.

Step 5 Use of Results
Step 3 Administration and Samples

• Given the empirical evidence from this study, it
  is apparent that students are knowledgeable about
  advising-related procedures and are confident in
  their abilities to carry out advising-related
  tasks. When synthesizing all the findings, it
  appears that some students are not satisfied with
  some specific advisors, and that there are some
  specific areas of the advising program that could
  be improved, but that investing in a peer
  advising program may not be the most judicious
  use of resources due to the high levels of
  knowledge and confidence reflected in the
  assessment results. Based on these assessment
  results, the university will take/has taken the
  following actions
• Assessment results were and continue to be
  disseminated to key university stakeholders
  (university president, deans, faculty senate,
  student government).
• A decision was made that it is premature to
  commit funding to the peer advising initiative
  until further studies can be conducted.
• Major Advising Programs, with student input
  through the Student Government Association, will
  work to expand the advising website to address
  some of the students advising knowledge
  deficiencies highlighted in the assessment
  results.
• Because students perceive that advisors have a
  lack of knowledge about the curriculum and
  related advising issues, Major Advising Programs
  will initiate conversations with the Center for
  Faculty Innovation to brainstorm ways of
  delivering more comprehensive training to faculty
  advisers.

• The Administration
• The test was administered at the universitys
  annual Assessment Day, a required day in which
  students with between 45 and 70 credit hours are
  asked to take a three-hour battery of instruments
  used to assess the institutions general
  education and student affairs programs.
• The Advising Assessment was administered via
  computer in a proctored session.
• The Students
• A random sample of all students at the university
  with between 45 and 70 credits was required to
  take the advising assessment (Regular Sample N
  401).
• This sampling provided results that are readily
  generalizable to the population of students at
  the university with 45 to 70 credit hours.
• Additionally, a sample of those students who did
  not attend the required Assessment Day were
  randomly assigned to take the advising assessment
  during make-up sessions (Make-Up Sample N
  65).
• These students were administered the Advising
  Assessment because it was thought that if they
  intentionally skipped the required Assessment
  Day, they would also be the type of student who
  may not be competent in navigating the
  advising-related tasks expected of them by the
  university.

The Samples
Regular Sample (N 401) Make-Up Sample (N 65)
Avg. Age 19.72 yrs 20.26 yrs
Female 66.1 60.0
White 80.0 73.8
Transfers 12.2 16.9
In-State 67.6 69.2
Avg. of Credits 53 55
Avg. SAT Score 1148 1165
Most common major Interdisciplinary Liberal Studies Marketing

• Outcome 4 Increased awareness of special
  opportunities like study abroad, internships, and
  competitive scholarships
• 84.29 of the Assessment Day students correctly
  answer the single item that served as the
  indicator for the outcome. 64.62 of make-up
  students correctly answered the single item
  correctly.
• One should be cautious in making inferences from
  this single item to the overall outcome because
  the item clearly does not cover the breadth of
  the outcome.