Development and Evaluation of a LMS: A Pilot Study

1
Development and Evaluation of a LMS A Pilot
Study

• R. Orçun Madran
• Baskent University, Turkey
• omadran_at_baskent.edu.tr
• Yasemin Gülbahar
• Baskent University, Turkey
• gulbahar_at_baskent.edu.tr
• Filiz Köse
• Baskent University, Turkey
• filizk_at_baskent.edu.tr

2
Introduction

• E-learning Web Internet
• Learning Management Systems are tools that makes
  easier the components of teaching-learning
  process by the integration of technology
• the functions of such systems vary tremendously
• meet expectations in different situations in
  accordance with the needs of both learners and
  instructors

3
Features of LMS

• User, course and resource management
• Online courses with multimedia rich content
• Online communication tools
• Self-tracking system with various report options
• Evaluation options like exams, e-portfolios,
  tests, projects etc.
• Interactivity (Purnell, 2005 Al Madran, 2004
  Türkiye Bilisim Vakfi, 2003).

4
Purpose of the Study

• Reason is to have full control over the system in
  terms of both content and coding
• Purpose is to develop a learning management
  system and evaluate its usability upon a pilot
  implementation

5
Method

• Participants 18 students (9 female and 9 male)
  (3 freshmen, 2 sophomores, 12 juniors and 1
  senior)
• Instruments
• The Questionnaire for Evaluation the Usability of
  a Learning Management System
• composed of four parts addressing content, visual
  adequacy, technical adequacy and support tools
  (course, communication and evaluation)
• The questionnaire composed of 6 questions for
  demographics, 16 questions for content, 10
  questions for visual adequacy, 13 questions for
  technical adequacy and 15 questions for support
  tools.
• Perceptions on E-Learning
• Six open-ended questions

6
Research Questions

• What are the general thoughts of learners about
  e-learning and Sanal Kampüs (Virtual Campus)?
• How the participants evaluated the learning
  management systems usability in terms of
  content, visual adequacy, technical adequacy and
  support tools (course, communication and
  evaluation)?
• What are the perceptions of participants about
  the learning management system in terms of
  content in general, top three features favored,
  suggestions for being more effective and barriers
  of usage?

7
The Learning Management System

• Sanal Kampüs (version 1.0) is originally
  developed by the researchers
• http//bote.baskent.edu.tr
• The system composed of two distinct structures
• Management part is mainly based on PHP and MySQL
• Content presentation part is based on Flash and
  XML

8
Sanal Kampüs Home Page
9
Sanal Kampüs Home Page
10
Sanal Kampüs Courses Page
11
Sanal Kampüs Sample Syllabus
12
Sanal Kampüs Sample Course Content
13
Results General Thoughts of Learners About
E-Learning And Sanal Kampüs
14
Results - Evaluation of Usability

• Mean of items related with the content was 4,07.
  The most valued ones are
• A clear language was used to present content.
  (M4,22)
• The narrations in videos were parallel with the
  content. (M4,33)
• Visual elements used in the system were parallel
  with the topic. (M4,44)

15
Results - Evaluation of Usability

• Mean value for items related with visual adequacy
  was found to be 4,22. The following items were
  most valued.
• There is no readability problem in the system.
  (M4,56)
• Font styles used in the system can easily be
  read. (M4,44)
• Font sizes used in the system can easily be read.
  (M4,50)

16
Results - Evaluation of Usability

• Mean value for items related with technical
  adequacy was found to be 3,94. The most valued
  items are
• The system positively affects students
  achievement. (M 4,28)
• The system runs without any faults. (M4,17)
• The system provides users easy navigation. (M
  4,22)

17
ResultsPerceptions of participants about Sanal
Kampüs

• How did you feel while using e-learning method?
• Some participants stated that they feel
  themselves independent and free, whereas some
  other feel anxiety. Some stated that they were
  happy that they engage in a real life experience.
  Few of the participants pointed out that they
  felt responsible themselves while using the
  system

18
ResultsPerceptions of participants about Sanal
Kampüs

• What are your personal opinions about the
  strongest and weakest points of e-learning
  method?
• Strongest points unbounded information, taking
  individual responsibility, learn at their own
  pace, being independent from time and place,
  having multimedia features, accessibility,
  communication tools and appropriate content
• Weakest points speed of Internet connection,
  technical problems, socialization, keeping the
  system up-to-date, organizing effective
  discussions, lack of knowing each other, system
  security and lack of active participation

19
ResultsPerceptions of participants about Sanal
Kampüs

• What are your thoughts on the content presented
  on the e-learning environment?
• Some participants favored the content due to
  having important points and tips, whereas others
  stated that they found the content as a summary
  and asked more detailed information

20
ResultsPerceptions of participants about Sanal
Kampüs

• What were the top three features of Sanal
  Kampüs that you favored?
• From mostly favored to least as effective
  visuals, understandable content, easy of use,
  simple interface, effective navigation,
  accessibility and note-taking feature

21
ResultsPerceptions of participants about Sanal
Kampüs

• What are your suggestions for making this
  training more effective?
• detailed content, reward mechanisms,
  audio-visual communication, become widespread,
  active participation, course diversity,
  accessibility from different platforms,
  accessible information about participants,
  increased interactivity and evaluation options

22
ResultsPerceptions of participants about Sanal
Kampüs

• Have you ever faced difficulties while using
  Sanal Kampüs? Please state them.
• No major problems stated. Just few of them
  maintained some technical problems like low speed
  Internet connection and personal hardware
  problems

23
Conclusion

• The Sanal Kampüs project, designed and
  developed by the researchers, is in its first
  phase.
• There will be many other phases in order to
  enrich the system with many features to make it
  more effective.
• Thus, the developments and updates to the system
  will continue upon the findings from this
  research and feedbacks gathered during future
  implementations.

24
References

• Al, U. Madran, O. (2004). Web Tabanli Uzaktan
  Egitim Sistemleri Sahip Olmasi Gereken
  Özellikler ve Standartlar. Bilgi Dünyasi, 5(2),
  259-271.
• Carr-Chellman , A. A. (2006). Desperate
  Technologists Critical Issues in E-Learning and
  Implications for Higher Education. Journal of
  Thought, 41(1), 95-119.
• Hardaway, D. E. Scamell, R. W. (2005). Use of a
  Technology-Mediated Learning Instructional
  Approach For Teaching an Introduction to
  Information Technology Course. Journal of
  Information Systems Education, 16(2), 137-145.
• Horton, W. (2000). Designing Web-based Training.
  USA John Wiley Sons, Inc.
• Huang, H. (2002). Student Perceptions in an
  online mediated environment. International
  Journal of Instructional Media, 29(4), 405-422.
• Jung, I. (2001). Building a theoretical framework
  of web-based instruction in the context of
  distance education. British Journal of
  Educational Technology, 32(5), 525-534.
• Lim, C. P. (2004). Engaging learners in online
  learning environments. TechTrends, 48(49, 16-23.
• Purnell, J. (2005). E-Learning Virtual Schools
  Giving Students and Teachers an Edge. Media
  Methods, 42(1), 28-29.
• Türkiye Bilisim Vakfi (TBV) (2003). e-Ögrenme
  Kilavuzu, Sürüm 1. Retrieved from
  http//www.tbv.org.tr/TBV/Documents/EgitimveBilisi
  m/UzaktanEgitimKlavuzu.pdf on 10 August, 2006.
• Yildirim, A. Simsek, H. (1999). Sosyal
  Bilimlerde Nitel Arastirma Yöntemleri. Ankara
  Seçkin Yayinevi.