Advanced Paid Search Track: Multivariate Testing

1
Advanced Paid Search TrackMultivariate Testing
Nicholas Abramovic company zoosk.com blog
manybodytheory.com
Advanced Search Track Multivariate Testing
2
Building a Proper Testing EnvironmentNicholas
Abramovic Zoosk, Inc.

• Part 1 Understanding User Behavior
• Part 2 Setting up a Proper Test
• Part 3 Statistical Testing
• Part 4 Additional Resources

Advanced Search Track Multivariate Testing
3
Part 1 Understanding user behavior

• We assume that prior user behavior is
  characteristic across all of the landing pages
  that we are testing to build a statistical model
• Why? All landing pages will receive users from
  the same source (keywords, bids, geographic,
  language ad copy)
• Give the same users the same test
• Do not have Test A in Germany and Test B in
  France. Run both Test A and Test B in Germany for
  accurate data

Advanced Search Track Multivariate Testing
4
Same users. Same intent.
share with friends
Connect with Friends Join Today for Fun Meet,
Chat, Reconnect www.socialnet.com URL
Facebook.com
post pictures
Search on Google for social network
Connect with Friends Join Today for Fun Meet,
Chat, Reconnect www.socialnet.com URL MySpace.com
Advanced Search Track Multivariate Testing
5
Part 2 Setting up a proper test
There are a lot of tools Most tools can only
get you so far. Zoosk needs top of the line
tracking and analytics that few can provide. --
More page views than Digg -- More active users
than Twitter Here are the next steps
Advanced Search Track Multivariate Testing
6
Track the entire funnel

• There are options, but many times most are not
  enough
• You need more than just basic conversion tracking
  that the search engines provide
• Track click gtgt lead gtgt signup gtgt order gtgt follow
  up orders

You need to track all of this in real-time
Advanced Search Track Multivariate Testing
7
Do not set yourself up for failure

• If you copy and paste existing ads and just
  change the destination URL. Google can favor
  your older ad/url because of the quality score
  and history
• Your original destination url (older ad) may have
  a higher position than your new destination url
  (new ad) so you get different quality traffic
• Dont copy and paste you ads. Instead, use one ad
  and have an interstitial page to redirect users
  for your test
• Click One goes to Page A, Two goes to B, Three to
  C, Four goes to A,

Redirect visitors and track on your backend
Advanced Search Track Multivariate Testing
8
Use interstitial pages for your ads

• A great solution for PPC tracking is
  Tracking202.com
• Made for internet marketers in mind. Free version
  and Pro version -- affordable for anyones budget

For performance-based marketers Web-based
solution They can provide hosting Also available
for download Get stats on your iPhone
http//www.Tracking202.com
Advanced Search Track Multivariate Testing
9
Commercial tools for click tracking

• You need to integrate all your paid marketing
  AdWords, Yahoo, media buys, affiliates
• HitPath.com (highly suggested)
• Very reliable
• Great customer support
• Excellent replacement for the basic conversion
  tracking that search engines provide
• Easy to integrate with your paid online marketing
  efforts
• Track multiple conversions (lead, signup, order)

Advanced Search Track Multivariate Testing
10
After we setup our tracking
how do we know what landing page is working?
Advanced Search Track Multivariate Testing
11
Part 3 Statistical testing

• A basic test you can do yourself is G-Test
• G-Test calculator is available online
• http//manybodythoery.com/smxlondon
• Very effective for search marketing
• Based off of spending millions of dollars in paid
  search
• Find out more info at http//manybodytheory.com

More info http//en.wikipedia.org/wiki/G-Test
Advanced Search Track Multivariate Testing
12
Looking at actual data
Only use clicks and conversions when testing
landing pages

• Cost too high? Lower your keyword bids
• Never make a decision based off of your CPA
• Remember You can not change the bid of a
  destination URL
• CTR too low? Update your ad creative

Advanced Search Track Multivariate Testing
13
Now that you have your data
Originally ran as an A/B/C test but later split
the results to help make it easier on the eyes
(for this presentation) G-Test concludes Test
A as Winner
G-Test Calculator found at www.ManyBodyThoeory.com
/SMXLondon
Advanced Search Track Multivariate Testing
14
You need to have real data

• Always use a reasonable size data set
• Golden Rule No more than 10 of your days can
  have under 5 conversions to ensure some level of
  confidence
• A one week test needs 5 days with at least 5
  conversions per day(5 days x 5 conversions 25
  minimum plus data from other two days)

too little data
G-Test Calculator found at www.ManyBodyThoeory.com
/SMXLondon
Advanced Search Track Multivariate Testing
15
Never stop testing

• Always let the data speak for itself. Do not get
  caught up with what you wanted the outcome to be
• If you expected Test B to succeed but data shows
  Test A look at the difference between the pages
  and expand
• Be realistic. Having two clicks with two
  conversions is nice but can you maintain a 100
  conversion rate?
• Always have a question that you want to answer
  before you begin your test (ie What color font
  should we use)

Advanced Search Track Multivariate Testing
16
Part 4 Additional Resources
Find this guy for more info

• My blog (lots of cool stuff)
• www.ManyBodyTheory.com
• Personal email address (for QA)
• nick_at_manybodytheory.com
• Want to work with Zoosk?
• nicka_at_zoosk.com

I am here today and tomorrow at SMX London )
dont be afraid to walk up and ask a question!
Advanced Search Track Multivariate Testing