The Online Assessment Advantage at Tyler Junior College

1
The Online Assessment Advantage at Tyler Junior
College

• Ken Craver
• Director of Distance Education

2
What We Will Cover

• Who we are
• Why Questionmark
• What we do
• Our Server Configuration
• Assessment Creation and Scheduling
• Assessment Delivery
• Download-ables (http//TJC-QM.4shared.com/)
• These slides
• Questionmark Bridge
• Questionmark Bridge Users Guide
• More documents available upon request

3
Who We Are

• Multi-campus junior college
• 10000 students
• 3700 / semester in distance education
• 60 faculty use Questionmark Perception in 90
  different courses
• 4 support persons
• Proctored testing in TJC Testing Center

4
Why Questionmark Our Students

• are computer learners
• desire flexibility in schedules
• can benefit from more and varied assessment
  opportunities
• like immediate feedback on their grades and test
  questions missed
• can be anywhere in the world

5
Why QuestionmarkOur Faculty (why they like it)

• do not lose valuable class time to giving
  assessments (4-6 class periods / semester)
• can provide more effective assessments using a
  variety of question types
• can categorize and tag questions to allow more
  intelligent question selection
• can give proctored exams in our testing center
• can give practice exams students can take from
  anywhere
• can use assessments as learning tools
• can obtain a wide variety of statistical reports
  on assessment results.

6
Why QuestionmarkSome History

• Originally championed by our Director of Testing
• I was paid a stipend to administer the server
  along with training and supporting faculty
• Also paid faculty member a stipend to train and
  support faculty
• 10-15 computer testing stations
• Tables for paper tests

7
Why Questionmark We Use Perception for

• Pre-course Assessments
• Practice Quizzes
• Exams
• Surveys

8
Why Questionmark Question Types

• Adobe Captivate
• Adobe Flash
• Drag and Drop
• Essay
• Explanation
• File Upload
• Fill in Blanks
• Hotspot
• Likert
• Matching
• Matrix

• Multiple Choice
• Multiple Response
• Numeric
• Pull-down List
• Ranking
• Select a Blank
• Spoken Response
• Text Match
• True/False
• Yes/No

9
Server ConfigurationQuestionmark Version 4.4
10
Assessment Process Our Process 1

• Item bank creation
• Create topic structure
• Import multiple-choice questions using
  Questionmark Bridge (in-house program)
• Assessment Creation
• Assessment Publishing (just a copy from
  development repository to production repository)
• Group Creation

11
Assessment Process Our Process 2

• Participant Creation
• Import into group using Questionmark Bridge
• All class rolls are sent as text files to a
  secure FTP site
• Rolls are downloaded and participants are
  imported and linked to groups
• Exam scheduling form completed
• Online form completed by each instructor that
  includes course, Exams to be given, deadlines
• Assessments scheduled according to exam
  scheduling form
• Results checked by instructor

12
Assessment Process Repository Structure
TJC.qmr Shared Production repository
TJC-Dev.qmr Shared Development repository
Local Development repositories (on faculty
computers)
13
Assessment ProcessSupporting Faculty

• Support Team
• 2 instructors, 1 staff, 1 part-time employee
• Semester stipend equal to 3 hour overload
  contract
• Responsibilities
• Create question topic structure and import/create
  questions sent by faculty
• Create assessments
• Create groups and import participants
• Schedule assessments
• Provide support
• Faculty Designers (Do-it-Yourself-ers)
• Do some or all of their own design,
  implementation and scheduling work
• Faculty Users
• Deliver questions to the support team and specify
  the content of their exams
• Complete online scheduling form for the exams
  they want scheduled
• Check student scores using Enterprise Manager
  web-based tool

14
Assessment Process Item Bank Creation

• We import multiple choice questions
• Format questions in a text file as follows
• 1. Question stem containing the text of the
  question.
• a. Answer 1
• b. Answer 2
• c. Answer 3 (correct answer)
• d. Answer 4
• Use the Questionmark Bridge program to create an
  import file.
• Import the import file to an Authoring Manager
  topic.

15
Assessment Process Questionmark Bridge
Text file answers marked
Prepare text file Questionmark Bridge (import.txt
)
Import import.txt imported into Authoring
Manager
16
Assessment ProcessAuthoring Manager
17
Assessment Process Assessment Creation

• Created in TJC-Dev development shared repository
• Copied to TJC production repository when ready to
  be scheduled
• All assessments use same custom template
• No changes allowed

18
Assessment Process Assessment Publishing
19
Assessment Process Group Creation

• Group typically class section
• Use naming standard to make groups easy to find
• Delete old groups at start of each semester

20
Assessment Process Participant Creation 1

• Download all class rolls as text files from
  internal FTP site that is populated automatically
  from our Banner system
• Done second week of semester
• Support team or faculty developer downloads roll
  from FTP site
• Roll is prepared for import using Questionmark
  Bridge

21
Assessment Process Participant Creation 2

• Import.txt file is imported into Questionmark
  and participants linked to a group.

22
Assessment Process Exam Scheduling Form and
Database
Zoho Creator form - www.zoho.com (Free!)
23
Assessment Process Exam Scheduling Database

• Available to all of our support team and our
  Testing Center personnel
• If deadline for an exam changes faculty will
  submit the change via the scheduling form

24
Assessment Process Assessment Scheduling

• Scheduled according to faculty input on
  scheduling form
• Scheduled to groups
• Deadlines may or may not be set on the schedule.

25
Assessment Process Results Checking

• Instructors check their own results using
  Enterprise Reporter.
• Common Reports
• Score List with Coaching Report
• Question Statistics Report
• Gradebook Report
• Transcript Report
• Test Analysis Report used by Nursing Instructors

26
Delivering AssessmentsOur Testing Center

• 50 testing stations
• Test over 33 subjects
• Average 3000 tests / month (long semester)
• Peak of 600 tests / day
• 4 Full time staff
• 12 Part time staff

27
Delivering AssessmentsOur Testing Center
Entry Point
28
Delivering Assessments Early Testing Center
29
Delivering AssessmentsCurrent Testing Center
Two types of Nova desk testing stations
30
Delivering AssessmentsOur Testing Center
Nova desks with hoods contain computer with a
monitor that is viewable through the glass top
31
Delivering AssessmentsOur Testing Center
Security Cameras
32
Delivering AssessmentsOur Testing Center
Security Monitor at front desk
33
Delivering Assessments Testing Center Procedures
1

• Check scheduling database (Zoho) report for
  instructors using QuestionMark, special
  instructions, courses and their deadlines
• Notify front desk at testing center of courses,
  instructors, and special instructions for
  QuestionMark users
• Students must be on the classroll, on the
  QuestionMark roster with their personal
  identification number to be able to test
• Students must show a valid photo ID and sign in
  before testing
• All electronic devices must be turned off before
  entering the testing center
• A proctor will take the students ID, select a
  computer, and enter the student into their test.
• During the testing, the proctors monitor the
  testing area looking for any unusual activities.

34
Delivering AssessmentsTesting Center Procedures 2

• Student will print a copy of their results if
  feature enabled by instructor
• Constant visual monitoring by inside proctors and
  video cameras
• Able to view any activities after the fact and
  download them if needed.
• Give computer based test for any type of class
• Paper based tests are allowed (but not
  encouraged) for online classes only
• Also, use Blackboard, SAMS, Word, and Excel for
  testing purposes
• Give Virtual College of Texas (VCT) exams and
  correspondence exams. Fee based
• Off-site students - send Questionmark Secure
  (browser add-on) to be installed on test station.
  

35
Delivering Assessments QM Skills for Testing
Center Manager

• Can add participants and link them to a group
• Can re-activate an expired exam.
• Can check exam scores

36
Delivering AssessmentsFuture

• Satellite Testing Centers
• Software to enable remote access to testing
  computers.

37
Delivering Assessments Security Options

• Open access for low stakes
• Secure access for high stakes
• Cameras in operation
• User name and password validated for monitored
  assessments
• Against Perception database
• Only proctor knows password
• QuestionMark Secure browser add-in to prevent
  printing and browsing
• Cannot access any other program once browser
  started
• Cannot close browser
• Can only take assessment

38
Thank You!

• Ken Craver
• 903-510-2591
• kcra_at_tjc.edu
• Downloads Web Site
• http//TJC-QM.4shared.com