Accreditation in the West Indies

1
Building an Accrediting Body in the Caribbean

Organizational Meeting Chandrabhan Sharma The
University of the West Indies Moshe Kam IEEE
Educational Activities
April 2008 San Juan, Puerto Rico
Version 001
2
Welcome and Introductions
3
Introduction and General Aims
4
History of Recent Efforts
Objectives and Goals

• March 2006 initial discussions of the future of
  accreditation in the University of the West
  Indies
• April 2007 IEEE holds an accreditation workshop
  in UWI
• September 2007 IEEE holds an accreditation
  training workshop in Port of Spain
• Followed by a meeting of potential participants
  on the creation of a new accrediting body

5
Emergent Consensus
Objectives and Goals

• There is enough interest in creating an
  accrediting body for engineering, technology and
  computing in the Caribbean
• The overall objective is high-quality, impartial
  and recognized body that will become part of the
  Washington Accord
• We will start with programs provided in English

6
Scope
Objectives and Goals

• ETC(?) programs provided in English in the
  Caribbean
• An effort requiring 3-6 years
• and the consent and participation of all major
  constituencies
• The accrediting body will replace all accrediting
  activities currently conducted by external
  agencies

7
Desired Final Outcome
Objectives and Goals

• A fully functional stable accrediting body
  operating wherever there is a higher education
  institute in the Caribbean
• Run by educators and practitioners from the
  Caribbean
• Member in the Washington Accord

8
This Meeting Desired Outcomes
Objectives and Goals

• Agreement on establishment of a new accrediting
  body
• Develop a declaration during or after the
  meeting
• Establishment of an expanded Founding Committee
  with subcommittees on
• Accreditation model and accreditation process
• Administrative structure and finances
• Dissemination, implementation and rollout
• Establishment of preliminary timetable and
  milestones

9
Possible Dangers
Objectives and Goals

• Not enough interest
• Failure to include all major stakeholders
• Especially industry
• Poor finances
• Competition/meddling by outside accrediting
  bodies
• Political infighting

10
The Plan and Objectives of this Meeting
Objectives and Goals

• including participant expectations

11
Proposed Agenda Monday AM
Objectives and Goals
12
Proposed Agenda Monday PM
Objectives and Goals
13
Breakout Groups
Objectives and Goals

• Accreditation model and accreditation process
• Administrative structure and finances
• Dissemination, implementation and rollout

14
Proposed Agenda Tuesday AM
Objectives and Goals
15
Proposed Agenda Tuesday Late AM/PM
Objectives and Goals
16
Parameters that Need to be Determined (1)
Objectives and Goals

• Why are we doing this?
• What is wrong in the current situation?
• Why can we not rely on external accrediting
  bodies (IET, ABET)?
• Who wants to do this?
• Supporters inside and outside the region
• What are the main impediments/ opposition/
  obstacles?

17
Parameters that Need to be Determined (2)
Objectives and Goals

• Scope
• Engineering/ engineering technology/ computing/
  applied science?
• Schools and programs
• BS? MS? Other degrees?
• Number of evaluators and other volunteers needed
• Accreditation philosophy
• Administration and governance
• How do we guarantee impartiality?

18
Parameters that Need to be Determined (3)
Objectives and Goals

• Role of governments
• Relationships with existing laws
• Legal status and standing
• Institutional licensing
• Regional accreditation?
• Role of Industry
• As participants, as financial contributors

19
Parameters that Need to be Determined (4)
Objectives and Goals

• Role of professional societies
• And who they are
• Role of international organizations
• Role of external organizations and individuals
• In governance of the accrediting body
• As evaluators and team chairs

20
Parameters that Need to be Determined (5)
Objectives and Goals

• How much will this cost to establish?
• How much will this cost to maintain?
• Mock budget

21
Parameters that Need to be Determined (6)
Objectives and Goals

• How do we achieve buy-in?
• Whose buy-in is needed?
• How do we phase the new accrediting body in?
• Time table and Milestones
• Opportunities for collaboration

22
What Parameters or Issues Have We Missed?
Objectives and Goals
23
Charging the Breakout Groups
Breakout group charge
24
Accreditation Model and Accreditation Process
Breakout group charge
25
Accreditation Model and Accreditation Process (1)
Breakout group charge

• Scope
• Which programs? Which degrees?
• Accreditation philosophy
• Available options (pros and cons)
• Two most likely approaches to pursue
• Relationship with licensing
• Desired, expected

26
Accreditation Model and Accreditation Process (2)
Breakout group charge

• Governance
• Who is sitting in decision making capacity?
• Who is advising and observing?
• Functional (non-administrative) structure

27
Accreditation Model and Accreditation Process (3)
Breakout group charge

• Structure
• Board of Directors
• Commission(s)
• Appeals committees?
• Criteria, accreditation manuals
• Training
• Advisory Boards
• International participation
• The role of regional accreditation and
  institutional licensing

28
Administrative Structure and Finances
Breakout group charge
29
Administrative Structure and Finances (1)
Breakout group charge

• What is the administrative structure needed for
  the support staff?
• How much will the accrediting body cost?
• Administrative staff
• Training
• Governance
• Accreditation activities
• External evaluators/ advisors/ participants
• Transportation and communications

30
Administrative Structure and Finances (2)
Breakout group charge

• Who will pay for the activities of the
  accrediting body?
• Schools
• Professional associations (who?)
• Governments
• Industry (who? How would we reach them?)
• What legal entity does the new accrediting body
  constitute?

31
Administrative Structure and Finances (3)

• Mock budget
• To start the organization
• To maintain the organization

32
Dissemination, Implementation and Rollout
Breakout group charge

• Focus on TRAINING and EVALUATOR POOL

33
Training and Evaluator Pool (1)
Breakout group charge

• Who will serve as evaluator?
• Who makes the selection?
• Who will serve as team chair?
• How do we guarantee impartiality and rotation?
• What is the role of evaluators from outside the
  Caribbean?
• And how will they be trained?

34
Training and Evaluator Pool (2)
Breakout group charge

• What is the size of the evaluator pool?
• Do we pay evaluators for their work?
• What staff support would be needed?
• How do we assess evaluators?
• Including mentoring, feedback and removal

35
Training and Evaluator Pool (3)
Breakout group charge

• How do we start? Who will conduct the first
  training?
• Who is responsible for training in the steady
  state?
• The accreditation body? The professional
  associations?
• Mock budget for training
• Initial, steady state

36
Structure and Principles of Operation

• Moshe Kam
• IEEE Educational Activities Board
• Committee on Global Accreditation Activities

37
What does an accrediting body do? (1)

• Establishes and maintains regulations for
  accreditation
• Criteria and procedures
• Establishes and maintains an administrative
  structure to support the accreditation process
• A pool of program evaluators
• Decision maker and committee structure
• Communication tools

38
What does an accrediting body do? (2)

• Accepts and acts upon requests for accreditation
• Reviews and assesses self study and other
  preparatory documents
• Organizes and conducts accreditation visits
• Reviews recommendation of evaluator teams on
  accreditation action
• Makes and announces accreditation outcomes

39
Stakeholders

• Academic institutions
• Presidents, provosts, chancellors, deans
• Industry
• Major employers of engineers, computer
  scientists, and technologists
• Professional associations
• Local, local sections of transnational
  organizations, transnational organizations with
  local sections
• Governments
• Ministries of education and industry,
  accreditation oversight bodies
• Other Civic Organizations

40
What is required of an accrediting body?

• Based on the requirements of the Council on
  Higher Education Accreditation (CHEA)

41
Recognition Standards the AB

• Advances academic quality
• Demonstrates accountability
• Encourages, where appropriate, self-scrutiny and
  planning for change and for needed improvement
• Employs appropriate and fair procedures in
  decision making
• Demonstrates ongoing review of accreditation
  practices
• Possesses sufficient resources

42
Requirements from the Accrediting Body (1)

• A majority of the institutions and programs
  accredited by the organization grant higher
  education degrees
• The recognition process will place increasing
  emphasis on the effectiveness of accrediting
  organizations in assuring academic quality of
  institutions or programs
• AB is non-governmental

43
Requirements from the Accrediting Body (2)

• AB accredits institutions that have legal
  authority to confer higher education degrees
• AB accredits institutions or programs at
  generally accepted higher education levels

44
Written Public Procedures

• AB has written procedures that describe,
  officially and publicly
• the organizations decision-making processes,
  policies, and procedures that lead to
  accreditation actions
• the scope of accreditation that may be granted
• evaluative criteria (standards or
  characteristics) used
• levels of accreditation status conferred

45
Self Evaluation, Visits, Independence

• AB has procedures that include a self-evaluation
  by the institution or program and on-site review
  by a visiting team, or has alternative processes
  that were determined to be valid
• AB demonstrates independence from any parent
  entity, or sponsoring entity, for the conduct of
  accreditation activities and determination of
  accreditation status

46
AB is Operational

• AB is operational, with more than one completed
  accreditation review, including
• action by the accreditation decision-making body
  at each degree level, or for each type of program
• identified in the statement of proposed
  recognized scope of accreditation

47
AB Needs to Demonstrate

• a clear statement of proposed scope of
  accreditation activity
• a clear statement of the accrediting
  organizations purposes and why those purposes
  are in the public interest
• a description of the accrediting organization and
  its activities
• the quality, pertinence, and value of those
  activities
• the ways in which those activities serve higher
  education and the public interest

48
AB must demonstrate that the statement of
proposed scope

• addresses the types of institutions, the programs
  to be reviewed, degree levels, and the geographic
  boundaries of accreditation activity
• is consistent with organizational mission
  statements, charters, bylaws, candidacy
  requirements, and other requirements for
  accreditation and affiliation
• and the accrediting organization has had
  consultation with appropriate constituencies

49
Who should govern the accrediting bodies?

• Professional associations
• Academic institutions
• Industry
• Institutions from the three sectors should be
  invited to become Members of the accrediting body
• Voting Members in the annual/bi-annual assembly
  of Members
• Governments should be invited to observe and
  advise

50
ABETs Corporate Structure
.
Member Societies
Board of Directors
Industrial Advisory Board
Executive Committee
COMMISSIONS
Applied Science
Engineering
Technology
Computing
Program Evaluator Teams
51
The Structure of JABEE
52
Possible Structure (1)

• Member Assembly
• Meets every year or every other year
• Board of Directors
• 9 members
• 3 professional associations
• 3 academia
• 3 industry representatives

53
Possible Structure (2)

• Accreditation Commissions
• Engineering
• Computing
• Technology
• Program evaluators and Team Chairs

54
Staff

• Executive Director
• Director of Accreditation and Training
• Assist Staff (2-3)

55
Member Assembly
Board of Directors
STAFF
Engineering
Computing
Technology
COMMISSIONS
Program Evaluators
56
How should the accreditation body be financed?

• Participation fees
• From professional associations and industry
• Accreditation fees
• From participating institutions
• Grants and gifts
• For special projects and research

57
Accreditation Models
58
Different Approaches and Styles of Accreditation

• The Minimal Model
• The Regulatory Model
• The Outcome-Based Model
• The Peer-Review model
• The Program Club model

59
The Minimal Model

• Ascertains basic characteristics of the school
  and program
• Often numeric and law-based
• Does the school satisfy basic legal requirements?
• Does the school have enough budget,
  infrastructure and reserves to conduct the
  program?
• Ascertains existence of the fundamental basics in
  the school and program
• Physical conditions, size and skill base of the
  faculty, coverage of basic topics in the
  curriculum
• Provides a prescription for a minimal core and
  very general parameters for the rest of the
  curriculum

60
Reflections on the Minimal Model

• It is easy to install and maintain as long as it
  adheres to the minimal philosophy
• Not a bad way to start an accrediting body
• Does not encourage continuous improvement
• The biggest danger is mission creep
• More and more requirements

61
The Regulatory Model

• Requires strict adherence to a core curriculum
• E.g., defines the minimum requirements for a
  Software Engineering curriculum
• Specifies parameters for the rest of the
  curriculum
• E.g., at least 6 credit hours of post WWII
  history
• Often involving direct prescriptions of
  curriculum and faculty composition
• E.g., at least three faculty in manufacturing
  are required if the body of students exceeds 120

62
Reflections on the Regulatory Model

• Makes the accrediting process uniform and
  potentially fair
• Criteria are unambiguous and often numeric
• Difficult to establish and update
• Leads to endless strife over what the core
  means
• Relatively easy to maintain
• The key to success is adherence to clear rules
• Was shown to stifle innovation and creativity in
  the curriculum
• This was the philosophy of the pre-2000 ABET
  model

63
The Outcome-Based Model

• Prescribes a small core and basic requirements
• Prescribes basic parameters for the goals of the
  program
• But does not specify the specific goals of the
  program
• Focuses on the goals and objectives of the
  program
• E.g., to maximize the number of graduates who
  continue to Medical or Law school
• E.g., to maximize the number of graduates who
  become program managers in the construction
  industry
• Requires evidence of measurement of goals
• Requires evidence of using the measurements to
  feed a quality improvement process

64
Reflections on the Outcome-Based Model

• Provides for significant diversity in goals and
  objectives
• Very different from the regulatory model
• Puts a lot of responsibility and risk in the
  hands of the program leaders
• E.g., some programs may try to achieve goals that
  are unattainable
• Sophisticated and hard to evaluate
• Very difficult to avoid complaints on
  inconsistent evaluations
• This is the basic philosophy of the current ABET
  EC2000 and TC2000 criteria

65
A Word of Caution Outcome-Based
Accreditation

• While outcome-based accreditation is the most
  popular paradigm for accreditation, it is not
  problem-free
• The prescriptive nature with respect to course
  content can be replaced by a prescriptive process
  with respect to assessments
• Too much data may be collected and analyzed in
  order to prove that methods were assessed
• Adherence to the process by zealous program
  evaluators may cause strong disagreements about
  methodology
• E.g., the debate about Direct Assessment

66
Collection of Data
67
Use of Data for Improvement
68
The Peer Review Model

• A coalition of schools organizes in group of
  peers
• Schools select their peers and all bi-directional
  selections are accepted
• Members from other constituencies are added
• Government, Industry, professional associations
• The peer groups conduct the review in evaluator
  teams
• Model requires an arbitrator and facilitator
• Ideally a professional association

69
Reflections on the Peer Review
Model

• This is the way accreditation was done in the US
  in the early 20th Century
• E.g., Princeton and Johns Hopkins come to visit
  the College of Engineering at Drexel University
  in 1904
• Difficult to organize
• Considered less confrontational and more
  collegial
• Risk a drift in the direction of unpublished
  mandates
• Risks clashes of philosophies and program rivalry
• Selection/acceptance of peers may be painful

70
The Program Club Model

• Group of peer institutions create a program
  club
• Use a common website for communication
• Programs that wish to join create a website with
  requested information
• Programs report continually on progress and
  experimentation in education
• New ideas are discussed and tried by members of
  the club
• Few on site visits

71
Reflection on the Program Club Model

• Continuous accreditation model
• Difficult to organize
• Considered less confrontational and more
  collegial
• Risks clashes of philosophies and program rivalry
• Selection/acceptance of peers may be painful
• High maintenance

72
A Note on the Use of Information Technology

• In general accrediting bodies have been slow to
  make full use of modern information technology
• If program data is displayed, updated and
  analyzed automatically significant efforts can be
  saved
• The new accrediting body should strive for
  maximal review and data processing that are
  automatic and continuous

73
Questions or Comments?