HyLighter Interactive Annotation System

1
HyLighter Interactive Annotation System

• Shine a New Light on Teaching and Learning
• FETC 2005 - Orlando, FL

2
HyLighter Presenters

• Dale W. Lick, Ph.D.
• University Professor, Florida State University
• David G. Lebow, Ph.D.
• President, HyLighter, Inc.
• Hope Hartman, Ph.D.
• Professor of Educational Psychology, City College
  of New York the City University of New York
  Graduate Center
• Edie LeBas, M.A.T.
• English Instructional Team Leader, University of
  Miami Online High School

3
on-line annotation programs spreading across
the digital landscape

• Many types of annotation systems largely reflect
  the many purposes for reading itself.
• Thousands of websites offer some form of
  annotation capability for public/social
  functions, collaboration, and education.

4
HyLighter supports learning, assessment, and
knowledge-building

• 1. Makes thinking that is ordinarily hidden,
  become transparent and available for
  self-reflection and feedback.
• 2. Allows users to continuously compare their
  developing understanding to others, monitor
  progress, and assess performance.
• 3. Provides new, technology-enhanced ways to
  organize and synthesize ideas from multiple
  sources and create meaning from multiple
  perspectives.

5
The HyLighter Difference

• draws broadly on the learning sciences (e.g.,
  expert performance, cognitive load theory,
  self-regulated learning)
• merges electronic markings on digital text and
  other "information objects" made by any number of
  readers
• generates composite displays showing convergence
  and divergence of highlighting and comments

6
highlight a section of text
7
add a comment
8
view a comment
9
click the Compare button
10
view map of group highlighting
11
view merged highlighting and comments the
"collective annotation model"
12
The Interactive Annotation Spreadsheet
13
Task Perspective Elements
14
(No Transcript)
15
Interactive Annotation for Argument Persuasion

• Introductory training sequence.
• Participants read and annotate material.
• Participants compare views and generate group
  views.
• Instructor performs assessment activities, leads
  whole class discussion, models active reading
  strategies, provides feedback to students, and
  modifies instruction as necessary.

16
HyLighter and Metacognitive Teaching

• Includes awareness of the sources and
  characteristics of students' misconceptions,
  selection of instructional strategies to overcome
  students' misconceptions, and monitoring/
  evaluating progress.

17
HyLighter Interactive Annotation System Manager

• Malleable to meet needs of different user groups
• Integration with other technologies
• Assessment instrumentation for complex cognitive
  skills
• Research across the curriculum and training
  contexts

18
HyLighter Use by CCNY Grad Students in Ed Psych
Courses Fall 2004

• Overview
• Two different types of applications of HyLighter
  Case Studies Explanations of Answers on
  Multiple Choice Tests.
• Students were working on their masters degrees in
  teaching.
• Approximately half of the students were born
  abroad.
• Assignments in Preparation for Use of HyLighter
• Text-based annotations preceded online
  annotations.
• Online case annotations preceded multiple choice
  explanations.
• Data Sources
• Case Studies of Beginning Teachers
• Class Discussions
• Multiple Choice Test Answer Explanations
• End of Multiple Choice Test Comments
• Essays on HyLighter

19
Case Study Results

• Many students commented on how using HyLighter
  made them more active readers that the acts of
  annotating comparing annotations increased
  meaningfulness memorability of what they read.
  
• A few students didnt like the highlighting but
  liked writing comments.
• Students thought about what they were
  highlighting for the first time.
• Students had thought that any and all
  highlighting was of equal value and there were no
  right or wrong ways to highlight - it was just a
  matter of style or personal preferences.
• One student didnt highlight anything in the case
  because she viewed everything in it as equally
  important. She and others didnt realize it was
  important to be selective.

20
Case Study Results cont.

• 6. Teachers and selected students annotations
  can be used as models to scaffold students
  development of effective highlighting skills.
• 7. Some students were relieved that they could
  highlight a section of text and didnt have to
  make a comment despite the appearance of the
  dialogue box.
• 8. Students liked comparing their highlights
  comments with other students the teacher so
  they could see if they missed any important
  information and if they included irrelevant
  details.
• 9. One student failed to use HyLighter when she
  heard there was a bug in the program because she
  thought that meant you would get a virus by going
  there (from Sweden).

21
Case Study Results cont.

• 10. Several students complained about the small
  size of the comment box making it hard to monitor
  your own comment writing.
• 11. Students persisted and enjoyed the HyLighter
  experience despite initial problems using it
  because of browser or other problems.
• 12. I identified and corrected students
  misconceptions through analyzing responding to
  comments in the case studies.
• 13. Students were fascinated by the differences
  between their highlights and comments
• 14. Small group discussions about using HyLighter
  and students background influences were
  extremely rich and enlightening, especially
  because about half of the class were born abroad.
  

22
Credit for Explanations for Selected Wrong
Answers

• Item/Credit None Partial Full
• 8 29 57 14
• 12 75 25
• 14 37.5 12.5 50
• 15 10 80 10
• 17 29 71
• 19 25 75
• Mean 30.08 45.08 24.83

23
Summary of Comments Re HyLighter at the End of
Multiple Choice Test

• Negative
• I wasn't exactly sure to what extent we were to
  explain ourselves and that might have been
  helpful to know.
• One criticism is that the comment box is too
  small and very fiddly. If you lose your train of
  thought, you then have to mess around until you
  regain it
• The test took much longer than I had expected. I
  wonder if an essay section is even necessary
  after all this writing.

• Positive
• I thought this worked very well. I liked the fact
  that it made one think in depth about the answers
  try to justify them.
• It allowed me to see gaps in my thinking, and
  knowledge, and I think it would have been nice to
  have had this opportunity on the first test.
• I like this a lot! I would definitely be excited
  to do something like this with my students.

24
Explanations of Multiple Choice Selections

• FINDINGS
• Students had more knowledge about content
  targeted in test items than their wrong answers
  implied, showing flaws in test items or their
  interpretation.
• 90 of students in course benefited from
  analyzing their response choice explanations by
  receiving at least partial credit for some wrong
  answers.
• Almost 50 of these students got at least partial
  credit for ALL wrong answers in the set of 6
  items analyzed.

• FREQUENT CATEGORIES
• 1. Explanation/Description
• 2. Elaboration
• 3. Examples
• 4. Problem with Terminology
• 5. Part/Whole
• 6. Misconception
• 7. Thought Report
• 8. Personal Opinion
• 9. Misapplication
• 10. Problematic Item

25
Essays on HyLighter Across Three Graduate Courses

• Impact of Using HyLighter on Attitudes about
  Technology
• Impact of Using HyLighter on Thoughts about
  Technology
• Cognitive and Metacognitive Benefits for Students
• Highlighting as a Reading Strategy
• Teaching
• Assessment
• Technology

26
Past, Present and Future of asynchronous
discussions in an online school AP Lit class
27
Pedagogical Challenges
Students live in different cities in different
time zones so asynchronous discussions are their
only ways to discuss literature with each other.
Focused and interactive discussions are an
integral part of the Advanced Placement
Literature classroom to develop students
understanding of texts and literary elements.
28
Past Apex Stranded Discussion Forum

• Students have productive discussions about
  characters and plot development in novels, modern
  plays, and short stories.
• However, students lack focus in responding to
  discussion questions about a specific passage in
  a difficult text or in analyzing stylistic
  features of a text.
• In addition, students voice difficulty with
  answering multiple-choice questions about poetry.

29
Sample Cry for Help in a Stranded
Discussion Post about the Romantic Poets
To be honest, I did not really enjoy this module.
I like reading poems because it is easy to come
up with your own interpretation of it. However,
when it comes to answering the questions on the
quiz, I get most of them wrong because I view
things differently. It's only after I see the
answer that I understand it completely. I just
want to know your opinion on reading these poems
and how well you do when taking the quizzes.
30
Before students begin study of Twelfth Night by
Shakespeare
I decide to ask the HyLighter people for help
because of AP Lit students frustrations and
difficulties with multiple-choice quizzes on
Romantic poets, I receive approval from Apex
Learning to use their multiple-choice quizzes on
the HyLighter site.
31
How students use the four multiple-choice
quizzes in HyLighter

• Students highlight their first choices and
  submit a thought report about how they decided
  on their answers
• Students email teacher their first choices for
  feedback as to whether right or wrong. Teacher
  responds quickly.
• Students are allowed to select a second choice
  for each incorrect answer as long as they provide
  a thought report.
• Students compare their thought reports with
  others and then respond to questions about their
  choices and their reports. Students make
  suggestions for improvement.

32
Sample Student Thought Reports for First Quiz

• I think that the answer is 7A. Viola is
  referring to Olivia's physical complexion. Viola
  is comparing apples to apples. The only other
  choice that made sense to me was Choice D, but it
  compares two unlike things, apples and oranges,
  so i went with the choice A.

• For this question I struggled to pick between A
  and D. I eliminated D as my choice though because
  I think the statement was talking more about her
  physical beauty and not how she acted. For that
  reason it was logical to pick A because Viola is
  complimenting Olivia, saying that even if she was
  as ugly as the devil, she would still be
  beautiful.

33
Thought Report for Third Quiz Identifying a
speaker of a passage
Malvolia is thrilled to find out that his
mistress has declared her love for him. Olivia
supposedly sends him a letter (in reality, the
letter is written by Maria) and in the letter she
tells him that she has fallen in love and that
she wishes him to wear the yellow stockings. To
get to this conclusion, I disregarded answers B,
D, and E. It is not B because Malvolio is simply
Olivia's steward. It is not D because Maria is
always finding ways to amuse herself by making
fun of Malvolio, so there is no reason why he
would want to get Maria's attention. The answer
is not D because Malvolio puts on the yellow
stockings before they put him in jail. Finally
the answer is not A because Toby and Andrew are
always making fun of Malvolio and not the other
way around. The answer is C.
34
Discussion Post after Third Quiz Note below the
change in thinking pattern of an apple and
oranges student
I didn't miss any questions the first time, but
by re-reading the questions that I narrowed down
to two choices I was able to better understand
why I selected the correct answer. For example,
question ten was pretty confusing for me. I
narrowed it down to Feste and Toby. Feste was the
fool, and I didn't know why Toby would consider
himself a fool, and at first thought, I just
guessed Toby because it didn't sound like Feste.
After re-reading the quote and looking at the two
choices, I understood not only the quote but also
remembered the scene that it was from, and this
helped me to understand the correct answer.
35
What I as a teacher observed from this first
trial

• On the first quiz, some students struggled with
  having to write a thought report and wanted to
  summarize the text
• in order to give an answer.
• By the second quiz, most students had begun to
  model their thought reports after viewing other
  students thought reports, especially the ones
  who had correct answers.
• Students who started out using test-prep apples
  and oranges thinking began to reread the text
  and come up with their answers based on their
  reading a second time.

36

• What students said about HyLighter
• More interactive

One of my favorite options of this new system
is that we get to compare and see how our
classmates might have had a different view. This
is definitely more interactive and enhances the
learning process. Instead of just having an
overall discussion, the compare option gives it
greater depth by being more specific. It also
gives me the opportunity to see how different
people might have seen it in a different way.
37

• What students said about HyLighter
• More focused

• I think it helps without a doubt. I enjoy seeing
  how other people approach the questions as well
  as having the opportunity to actual learn how to
  approach the question so I don't just continue to
  make the same mistakes. Having to discuss each
  question helps too because it keeps the
  discussion focused so it ends up helping instead
  of talking about pointless things.

38

• What students said about HyLighter
• Shows errors in thinking

• I didn't get any questions wrong on my first
  attempt at the quiz, but looking at how some of
  the other kids answered their questions wrong was
  interesting. It was helpful to see how they could
  interpret some of the quotes incorrectly, but be
  able to defend their reasoning well.

39
What students said about HyLighter

• Showed more efficient way
• to reach the right answer
• I found it really interesting to read the
  response to question ten. This was the question I
  was having the most trouble with, and it helped
  me to learn that other students also had the same
  problem. It also helped me to read the way they
  reached their answer. By reading these answers, I
  was able to find a shorter and more efficient way
  to reach the right answer.

40
What students said about HyLighter

• As part of their discussion questions, students
  were asked to make suggestions about how
  HyLighter could better serve their needs.
• Based on their suggestions, HyLighter is now
  designing a new version to meet the needs
  expressed by University of Miami Online High
  School students on the following slides.

41

• What students said to add to HyLighter
• Provide direct quotes from text
• and have students explain them

• The format overall is very helpful. I do think it
  could be beneficial to be able to post
  discussions on certain elements of the play that
  may be harder to understand. It's important for
  me to be able to see how my other classmates are
  thinking, and this becomes hard because we are
  not in a typical classroom-format. Any way that
  make taking AP English Lit online more like a
  classroom would be helpful to anyone.

42

• What students said to add
• Provide the freedom to write an
• overall comment about a passage

• I like this format. Yes, I do think having
  difficult passages would be helpful not only to
  learn different approaches but also to see how
  different people interpret writing, especially
  Shakespeare. It's very personal and to see others
  perceptions would be nice. I think the best part
  of writing is that there is so much freedom in
  it. I miss that one part about a regular
  classroom so that feedback would be very helpful.

43
What students said to add

• Provide a way to respond
• to other students ideas
• I think it would be a good idea to create more
  questions that cover the style of writing and
  important soliloquies. It would also be a good
  idea to be able to respond to other students.
  Similar to the Apex system, HyLighter should
  include the option of being able to have
  discussions with our classmates on the play as a
  whole.

44
What students said to add

• Ask questions about the authors
• style in important passages
• I think it would be a good idea to create more
  questions that cover the style of writing and
  important soliloquies. It would also be a good
  idea to be able to respond to other students.
  Similar to the Apex system, HyLighter should
  include the option of being able to have
  discussions with our classmates on the play as a
  whole.

45
What the teacher said to add

• Based on the success of UMOHS students, HyLighter
  will provide the following
• features in their new version
• Computer-graded multiple choice tests
• Computer-controlled feature that allows students
  to compare after completion of test, rather than
  wait on teacher to open up compare feature. This
  feature will be available on both multiple-choice
  tests and student discussions about the text.

46
Dale W. Lick Dale_at_hylighter.com
David G. Lebow David_at_hylighter.com Hope
Hartman gravity1_at_mindspring.com Edie
LeBas elebas_at_umohs.org
PowerPoint Presentation www.hylighter.com/FETC200
5.ppt