Perception And Performance Measurements

1
PERCEPTION AND PERFORMANCE MEASUREMENTS
2

• As web pages continue to grow in size and users
  expect ever-faster-loading pages, measurements
  that accurately reflect the users experience are
  critical to understanding web performance. This
  had led to what some may call the search for the
  elusive unicorn metric. Many people with an
  interest in web performance are looking for a
  metric that will accurately measure page
  performance and user experience. This simply
  doesnt exist today, and it might never exist,
  but that wont stop us from searching for it.
• It isnt only about how fast all the elements on
  the page load, but rather how the human user
  perceives the page load. I dont care how long it
  takes all the analytics and tracking widgets to
  load, I care about when the information I am
  looking for is available.
• According to Jakob Nielsen, there are three main
  time limits to consider when it comes to web
  performance and user perception.
• 0.1 second limit for users to feel that user
  interface items in the page can be manipulated
• 1 second limit for users to feel they can
  navigate without having to wait for the computer
• 10 seconds limit to keep a users attention
• Users need to feel something is happening on the
  page to stay engaged. Delays over one second are
  noticeable. A page has to have some sort of
  noticeable change in less than a second for a
  user to not be frustrated. In a previous post I
  wrote about objective vs subjective time  these
  thresholds need to be considered more for
  subjective rather than objective timings, but for
  the most part we focus on objective measures.

3
Traditional web performance metrics such as time
to first byte (TTFB) or page load time (PLT) are
commonly used to create baselines for determining
how fast a page loads. Unfortunately these
metrics do not provide insight into the users
perception. TTFB is the time it takes between
issuing a request for a resource and receiving
the first byte of content. When the first byte of
content is received by the browser, the user is
still looking at a blank screen (or the previous
page they were on). This metric does not help
determine how the user is actually perceiving the
page, as they still dont even know that data is
starting to be received. On the opposite end of
the spectrum, PLT measures how long it takes for
every element on the page to load even if they
are below the fold or otherwise invisible to
the end user. Users may navigate away from a page
before it has fully loaded, so this isnt helpful
in determining if a user perceives a page as
being slow. Whenever I read a paper or report on
how page load times are improved by a certain
percentage, I immediately look to see what metric
is being reported. In the majority of instances,
it is page load time. This disappoints
me. Luckily, user-centric measurements have
emerged and are becoming more the norm than the
exception in the web performance community.
Instead of measuring PLT or TTFB, measurements
such as start render, Speed Index, critical
resources index, and Perceptual Speed Index have
come to the forefront. These measurements aim to
give us insight into how users perceive page
load, which is more important than the actual
milliseconds or seconds it took for the document
to complete. Start Render Start render measures
the moment when the first item was displayed on
the screen its size doesnt matter, nor does
its relevance to the end user. This is when the
user is no longer staring at a blank white screen
or at the previous screen. Start render is good,
but this does not take into account whether the
content displayed is relevant to the end user.
There may be something on the page, but the page
may still not be usable to the end user.
Referring back to Nielsens considerations, the
user still cannot manipulate the UI.
4
Speed Index Speed Index was added to WebPagetest
in April 2012, as a way of measuring how quickly
content above the fold is populated. Speed Index
goes beyond tracking a specific timing milestone
in the web application loading process and
focuses on items that are most critical to the
end user. Speed Index computes the visual
completion of the application-loading process
based on pairwise comparisons of pixel histograms
from time lapse video snapshots, such as the
snapshots in the strip below. Essentially
each frame of the video is compared with the end
frame to estimate how much visual completion has
been achieved with respect to the fully-loaded
frame. Critical Resources Index Critical
resources index, developed by Steve Souders,
combines two of the main blockers to rendering,
style sheets and synchronous scripts, into a
single metric. Measuring these critical
resources as a single metric makes it easier to
measure the combined impact on performance.
Critical resources index is reported in HTTP
Archive, as one of the metrics in the Highest
Correlation to Render Time charts. Look for the
CSS and Sync JS label
5
Perceptual Speed Index Perceptual Speed Index
(PSI) is a complementary measurement to Speed
Index that accounts for layout stability. Speed
Index answers the question of how fast did
above-the-fold content load? while PSI answers
the question how fast did above-the-fold content
load without jitter? Stability includes
performance concerns like whether there are
multiple reflow and repaints, and whether there
is visual interference by pop-up ads, but can
also penalize intended page changes like animated
GIFs and auto-running carousels (which some may
argue should be penalized). You cant improve
what you cant measure. But, you also need to
make sure you are measuring the right thing. To
learn more about this, download the Measuring Web
Performance white paper from ZK Research. Thus
far we havent found the unicorn metric/index of
web performance measurements. Maybe one of these
metrics can bring us closer to ending the
search. Next Steps Taking human visual
perception into account is a critical factor in
estimating the quality of experience of a web
application. There is still work to be done to
find measurements that accurately measure user
perception of when a page becomes responsive. I
look forward to the day when one of these metrics
or possibly another one replace TTFB and page
load time.
6
Instart Logic is the worlds first endpoint-aware
application delivery solution that makes websites
and applications fast, secure, and easy to
operate.
Interested in learning more? Preview our image
optimization capabilities in the Playground
Contact Sales