School of Computer Science and Software Engineering

1
School of Computer Science and Software
Engineering
Monash University

• Design Issues in Human Visual Perception
  Experiments on Region Warping

Yang-Wai Chow Ronald Pose Matthew Regan
2
Overview

• Background
• Address Recalculation Pipeline
• Priority Rendering
• Large object segmentation
• Scene tearing artefacts
• Region Warping
• Related Work
• Image/Video compression
• Level-of-Detail (LOD) management
• Visual attention models
• Design Issues
• Conclusions

3

• Background
• Address Recalculation Pipeline
• Priority Rendering
• Large object segmentation
• Scene tearing artefacts
• Region Warping

4
The Address Recalculation Pipeline (ARP) was
designed to reduce the end-to-end latency due to
user head rotations for immersive Head Mounted
Display (HMD) virtual reality systems

• Latency is a major factor that plagues the
  designing of immersive Head Mounted Display (HMD)
  virtual reality systems
• End-to-end latency is defined as the time between
  a users actions and when those actions are
  reflected by the display

5
Priority Rendering allows different section of
the scene that surrounds the users head to be
rendered onto separate display memories and
therefore can updated at different update rates

• Different section of the scene can be rendered
  onto separate display memories before being
  combined to form an image of the whole scene

6
Segmentation of large virtual world object in
conjunction with Priority Rendering can further
reduce the overall rendering load

• Fractal terrain example A fractal terrain
  typically consists of thousands of polygons. If
  the terrain were to be segmented for priority
  rendering, different sections of the fractal
  terrain could be updated at different update rates

7
However the implementation of object segmentation
in conjunction with priority rendering introduces
a potential scene tearing problem

• Scene tearing artefacts will completely destroy
  the illusion of reality, and therefore have to be
  addressed before object segmentation can be used
  effectively with priority rendering

8
Region Warping was designed to hide the scene
tearing artefacts resulting from object
segmentation with Priority Rendering

• Region Warping essentially involves the
  perturbation of object vertices in order to hide
  the tearing artefacts

9
An example showing the scene tearing effect
10
The exact same frame, this time with Region
Warping
11
Initial Region Warping experiments and analysis
do not portray the human perception or response
to the warping

• Some initial experiments on region warping have
  been conducted
• (Please refer to our paper entitled Region
  Warping in a Virtual Reality System with Priority
  Rendering IADIS Applied Computing 2005)
• However the analysis that was performed were
  mathematical analysis of the distortions and did
  not portray the human visual perception to region
  warping
• Therefore further experimentation involving human
  subjects are necessary

12

• Related Work
• Image/Video compression
• Level-of-Detail (LOD) management
• Visual attention models

13
A number of other computer graphics techniques
have also been developed that take advantage of
the limitations in the psychophysical aspects of
the human visual perception

• The human visual system has a finite perception
  to detail
• A variety of computer graphics techniques attempt
  to reduce the computational effort and time
  required in the rendering of scenes or generating
  of images without greatly affecting a users
  perception of the displayed image quality
• Examples of human visual perception exploits can
  be found in lossy image/video compression
  techniques
• JPEG and MPEG coding are lossy compression
  techniques that attempt to decrease the amount of
  data in the representation of an image/video

14
Level of Detail (LOD) management used in computer
graphics attempts to decrease the amount of
detail in a scene without the user noticing the
difference

• Level of Detail (LOD) management have been used
  by degrading the peripheral detail in a HMD
  virtual reality systems display, with no adverse
  effects to the user task performance
• LOD methods also include the use of object meshes
  with varying degrees of complexity
• A number of criteria for switching between
  varying LODs have been proposed and implemented
• Simpler object meshes can be switched into the
  display without the user noticing any alterations
  in the scene

15
Others have experimented with visual attention
models to selectively concentrate computational
effort on important sections of a scene

• A user will typically focus on prominent or
  conspicuous objects in a scene and will pay less
  attention to the details contained in the rest of
  the environment
• This is a fundamental feature of the human visual
  system known as inattention blindness
• Some researchers have conducted visual perception
  experiments based on visual attention models in
  order to predict where the user will be looking
  in a scene and focus rendering effort on those
  sections of the scene

16

• Design Issues
• Stereoscopy versus Monoscopy
• Distance from the user
• Head tracked displays
• Scene complexity
• Virtual environment characteristics
• Scene familiarity
• Ability to interact with surrounding environment

17
As part of the human visual system, stereo vision
is one of the main factors that contribute to a
users perception of realism within a virtual
world

• The human visual system is based on stereo vision
• A human perceives information such as depth based
  on the disparity between left and right eyes
• It is therefore conceivable that a user might
  perceive differently the effects of the
  distortions caused by region warping in a virtual
  world viewed through a stereoscopic display or a
  monoscopic display

18
A human perceives less detail in objects that are
located further away compared to closer objects

• Objects that are further from the users will
  appear to be smaller from the users viewpoint
• Human binocular disparity/stereo perception fails
  after a distance of more than 30 meters
• It is therefore possible that distortions that
  are concentrated at a distance away from the user
  will be less noticeable

19
Upon user head movements or rotations, based on
feedback from the motoric senses the human brain
will expect the scene to change in certain ways

• The human eye is less sensitive to details that
  move rapidly over the retina
• Perception to distortion might vary between head
  tracked virtual reality systems and non-head
  tracked systems

20
The human visual system depends on the retina to
cope with the wealth of information contained
within the scene

• The user visual attention will typically be focus
  or drawn to certain interesting areas of the
  scene
• High levels of detail contained within the scene
  will distract and possibly overwhelm the users
  perception

21
The makeup of the virtual world might also
potentially affect a users perception to the
region warping distortions

• Distortions will appear less obvious on scenes
  containing random nature objects compared to
  scenes containing rigid structured architectures
• The reason for this is because highly structured
  scenes contain straight lines can might
  noticeably bend as a result of the warping

22
The human brain might unconsciously correct
perceived errors when presented with a familiar
scene

• The human visual system is governed by real world
  experiences, as the brain interprets images
  presented to the retina based on these
  experiences
• The human brain might overlook errors contained
  in familiar scene
• Alternatively the converse might apply, in that
  errors might appear more prominent when presented
  with a familiar scene

23
A users focus of attention will be divided with
allowed to interact with the surrounding
environment

• Interactability with the surrounding environment
  adds to the users feeling or sense of presence
  in the virtual world
• When a user is immersed in a virtual world,
  his/her mindset and focus of attention will be
  drawn towards trying to manipulate objects or to
  navigate through the virtual world

24
Conclusions

• Region warping attempts to take advantage of the
  limitations in the human visual perception, to
  reduce the overall rendering load of a ARP
  virtual reality system with priority rendering
• However while region warping appears to works
  well on computer monitors and perform well by
  analytical measures, the real test can only be
  with real human perception
• Therefore before this technique can be used
  effectively, it is important to characterize and
  understand the level of distortions that might be
  perceivable to the user
• This paper has outlined our work and future
  approaches

25

• Questions
• or
• Suggestions?