STUDENT SATISFACTION WITH WEB-BASED EDUCATIONAL RESOURCES FOR COMPUTER - PowerPoint PPT Presentation

1 / 48
About This Presentation
Title:

STUDENT SATISFACTION WITH WEB-BASED EDUCATIONAL RESOURCES FOR COMPUTER

Description:

... business, chemistry, engineering, health sciences, history, information ... mathematics, music, physics, psychology, teacher education, and world languages. ... – PowerPoint PPT presentation

Number of Views:41
Avg rating:3.0/5.0
Slides: 49
Provided by: sysma
Category:

less

Transcript and Presenter's Notes

Title: STUDENT SATISFACTION WITH WEB-BASED EDUCATIONAL RESOURCES FOR COMPUTER


1
STUDENT SATISFACTION WITH WEB-BASED EDUCATIONAL
RESOURCES FOR COMPUTER INFORMATION SCIENCE
Teaching with Merlot Collection
  • Emrah Orhun, Ph.D.
  • Troy State University Montgomery
  • Alabama, USA

2
Outline
  • Introduction
  • Learning and Teaching Materials in The Merlot
    Collection
  • Students Satisfaction with Merlot Materials in
    CIS Classes
  • Conclusions

3
Introduction
  • Difficulties in locating relevant and affordable
    materials of high quality
  • World Lecture Hall at the University of Texas
  • 83 categories to browse and search utilities to
    locate materials
  • WebCT (2002)
  • 1,250 publisher course content packs in more than
    50 subject areas
  • customizable materials around which to build
    courses combined with the functionality of
    WebCT's course management software.

4
Quality issue
  • World Lecture Hall materials
  • courses offered at accredited colleges and
    universities around the world
  • WebCT packs
  • editorial standards of the publisher and author
    of the text

5
MERLOT (Multimedia Educational Resource for
Learning and Online Teaching)
  • Web-based open resource designed primarily for
    faculty and students in higher education
  • emphasizes the quality issue

6
MERLOT repository
  • Over 8,000 free learning materials and support
    materials that can be integrated within the
    context of a larger course

7
Searching and Browsing in MERLOT
8
Figure 1. Entries in the Applications
sub/subcategory of Information Technology
subcategory under Science and Technology category
9
Figure 2. Search results for TECHtionary
10
Support resources
  • sample assignments
  • evaluations of the learning materials
  • discipline panel peer reviews
  • individual user reviews

11
Peer Review Process
  • Discipline communities
  • Biology, business, chemistry, engineering, health
    sciences, history, information technology,
    mathematics, music, physics, psychology, teacher
    education, and world languages.

12
  • Discipline teams
  • (MERLOT Editorial Boards)
  • selected by the MERLOT partner organizations in
    accordance with the MERLOT guidelines.
  • develop evaluation standards,
  • conduct peer reviews of learning materials within
    the discipline, and
  • make connections with their professional
    organizations

13
Peer review process
  • Evaluated by at least two team members who have
    relevant expertise.
  • Review process stages
  • triage review to identify worthy candidates
  • review of candidates
  • composite review report sent back to the authors
    for feedback and permission to post the review on
    MERLOT.
  • approved review is posted to the MERLOT Web site
    by the discipline leader

14
  • The evaluation criteria
  • quality of content,
  • potential effectiveness as a teaching tool,
  • and ease of use.

15
  • Each dimension evaluated separately (on a scale
    of 1-5 stars, with 5 being the highest), and an
    overall rating is computed
  • A review must average three stars in order to be
    posted to the site.

16
Figure 3(a). Peer Review of TECHtionary (Part a)
17
Figure 3(b). Peer Review of TECHtionary (Part b)
18
Figure 3(c). Peer Review of TECHtionary (Part c)
19
Figure 4. Member Comments for TECHtionary
20
Current state of MERLOT
21
Table 1 Growth of MERLOT materials, members, and
peer reviews (May 28, 2002)
  Total Added during last 365 days Added during last 30 days
Materials 6863 1991 229
Members 8613 4142 600
Peer Reviews 479 361 20
22
Table 1 Growth of MERLOT materials, members, and
peer reviews (February 27, 2003)
Total Added during last 365 days Added during last 30 days
Materials 8331 2338 284
Members 12637 5527 712
Peer Reviews 778 86 2
Member Comments 1790 305 18
Assignments 322 59 1
23
Table 1 Growth of MERLOT materials, members, and
peer reviewsfor IT Category
Total Added during last 365 days Added during last 30 days
Materials 559 274 21
Members 524 259 30
Peer Reviews 36 3 0
Member Comments 218 99 6
Assignments 11 9 0
24
Ratio of Peer Reviews, Member Comments, and
Assignments to Materials
May 25, 2002 September 26, 2002 February 27, 2003
Materials 6863 7395 8331
Peer Reviews 7.0 8.5 9.3
Member Comments 21.2 20.8 21.4
Assignments 3.4 3.8 3.9
25
Problem
  • Percentage of reviewed materials low

Cause
  • Editorial peer review process too slow

Potential Solution
  • Peer review by community of learners

26
STUDENTS SATISFACTION WITH MERLOT MATERIALS IN
CIS CLASSES
27
Goal of Study
  • To find out to what extent MERLOT provides
    relevant and quality materials for the CIS
    courses.

28
Questions
  • Is the quality and quantity of learning materials
    in the MERLOT collection sufficient to satisfy
    CIS students knowledge/learning needs in
    specific courses?
  • Is the MERLOT collection usable by the
    students?
  • Does the MERLOT collection induce the students to
    become members of learning communities?
  • Can peer review by students substitute for peer
    review by editorial boards?

29
Questions re-formulated
  • How do students rate their satisfaction with
    MERLOT collection?
  • How do students rate the learning materials as
    reviewers? Do students ratings of the learning
    materials differ from the ratings given by
    Discipline Panel members?

30
Teaching with the MERLOT Collection in CIS
Courses
  • Summer-Fall 2002 semesters in the following CIS
    classes at Troy State University Montgomery
  • CIS 5547 CAR Applied Systems Analysis
  • CIS 5547 CBR Applied Systems Analysis
  • CIS 3332 Software Engineering I
  • CIS 3357 Logical Structures of Computer
    Design
  • Overall, 41 students participated in the
    evaluation. A total of 252 evaluations were made.

31
 
Student Assignment
  • Evaluation of web-based learning materials  
  •  
  • 1.  Visit MERLOT (www.merlot.org) site and
    register as a member.
  • 2.       For each chapter covered in the course
  • a.      Find a relevant learning material in the
    IT subcategory.
  • b.      Review the material using the guidelines
    provided by MERLOT. In the table provided below,
    state what learning outcomes are supported by the
    material and post your review as a Member Comment
    on MERLOT.
  • c.      Rate your satisfaction with MERLOT
    collection on a scale 1-5, with 5 implying that
    you were fully satisfied with the quality and
    quantity of learning materials you found in
    connection with your task, and 1 implying that
    you were not satisfied at all.
  • d.      Find a relevant online learning material
    not contained in MERLOT, post it to MERLOT and
    add a Member Comment.

32
Example of Relevant MERLOT Material
Chapter Topic Title URL Learning outcome Satisfaction with MERLOT collection for this task 5 very good 1very poor
1 Players in the System Game Enterprise Resource Planning Supersite http//www.erpsupersite.com Provides analysis on ERP trends 3
2 Information System Building Blocks Interface Design for Educational Multimedia http//www.cdl.edu.rachel/PDF/intdesmm.pdf Describes the principle of interface design 4
3 Information System Development Risk analysis and management for Projects (RAMP) http//www.ramprisk.com/frontindex.htm Comprehensive outline of RAMP  5
33
Example of New MERLOT material posted
Chapter Topic Title URL Learning outcome
1 Players in the System Game New responsibilities Fall Heavy on IT Workers www.informationweek.com Talks about the new job responsibilities of current and upcoming IT employees.
2 Information System Building Blocks TP Middleware keeping up with Mainframe Revival www.adtmag.com Explains the functionality of the product TP middleware on mainframe systems.
3 Information System Development Tips for a successful Software Distribution Project. www.computerworld.com Shares other companies success stories of in relation to Software Distribution Project.
34
How do students rate their satisfaction with
MERLOT collection?
35
Students mean satisfaction ratings for MERLOT
collection
36
CIS3357 Students satisfaction with MERLOT
collection
37
CIS3332 Students satisfaction with MERLOT
collection
38
CIS5547 Students satisfaction with MERLOT
collection
39
CIS5547 Students satisfaction with MERLOT
collection (contd.)
40
How different are students evaluations from
Editorial Peer Reviews?
41
Distribution of materials in IT
Type
Tutorial File System Design for an NFS File Server Appliance   Cascading Style Sheets, Level1 Electronic Commerce Tutorial   FTP Tutorial Learning Microsoft Word 2000   Web-Database Integration   Mulder's Cascading Style Sheets Tutorial   Sharky's Netscape Frames Tutorial   Tje ABCs of Building a Web Site Data Communications   ADSL   Computer Knowledge Virus Tutorial
Reference Material Study Guides and Strategies Tips for Using Blackboard
Animation TECHtionary Warriors of the net
42
Sample of Materials from IT Category
Material Type
Tutorial     ADSL
Reference Material Web-Database Integration
Animation TECHtionary  
Distribution of materials according to type and rating (excluding materials with no user comments with 3 categories)
43
Sample of Students
  • CIS3332 SOFTWARE ENGINEERING (16)
  • CIS5547 SYSTEMS ANALYSIS (27)

44
Students Ratings
Case Editorial Board Rating Member Rating Students Rating (mean)
TECHtionary 3.0 3.75
Web-Database Integration 3.67 3.82
45
Students ratings for TECHtionary
Mean Std. Dev. Std. Error Variance Coef.Var. Count
3.754 .421 .086 .177 11.218 24
Minimum Maximum Range Sum Sum Squared Missing
2.5 4.4 1.9 90.1 342.33 0
Editorial Board Rating 3.00
46
Students ratings for Web-Database Integration
Mean Std. Dev. Std. Error Variance Coef. Var. Count
3.829 .545 .119 .297 14.238 21
Minimum Maximum Range Sum Sum Squared Missing
2.8 4.9 2.1 80.4 313.76 0
Editorial Board Rating 3.67
47
Is the mean student rating different from the
rating given by the Editorial Board?
  • TECHtionary
  • Different (one-group t-value 8.773, p0.0001)
  • Web-Database Integration
  • Insufficient evidence (one-group t-value 1.333,
    p0.1975)

48
Conclusions
  • Students are satisfied with the quality and
    quantity of learning materials they found in the
    IT category.
  • Current data suggest that students rate materials
    differently than editorial boards.
  • The present limited study does not provide
    evidence for the notion that peer review by
    students can substitute for peer review by
    editorial boards.
Write a Comment
User Comments (0)
About PowerShow.com