Measuring Success

This is reprint of an article I wrote last month for www.journalism.co.uk .
It came about from a discussion of what is the most accurate method for measuring visitor activity.

Content: This is a multi-page post. You are on Page: 1 2 3

1. Page 1: Summary – there is no accuracy debate!
2. Page 1: Introduction – why most of the web is junk and what role web analytics plays
3. Page 2: On-site versus Off-site web analytics tools and how they work
4. Page 3: Discrepancies – what’s accurate and how can accuracy be improved

4. Discrepancies – what’s accurate?
As you can see, the differences in methodology of each of these techniques are significant and this leads to very different results. Even for the same metric, basic website numbers, such as the number of visitors a website receives, the total number of pageviews etc, can vary dramatically and this is a constant and exasperating problem for site owners, media buyers and marketers alike.

So which method produces the more accurate data?

The truth is, all web analytics solutions have their limitations as shown below:

Off-site analytics
Advantages:

• Demographic information available
• Can track competitors and related sites e.g. visitors first went to competitor A, then your site, then onto Site B
• No website required – can track trends irrespective of a web presence

Disadvantages:

• Inferred data – not real visitors
• Small sample sizes limit accuracy – requires a significant level of traffic to your website to be viable
• Extrapolation errors – analogous to polling
• US-centric data
• Expensive

On-site analytics
Advantages:

• Real visitors measured
• Tracks engagements and conversions
• Available for any web site, regardless of size
• Inexpensive tools available

Disadvantages:

• No demographics available (unless you ask for it)
• Cannot track competitors/related sites
• Visitors can block*, loose and delete cookies

* The blocking of first party cookies by visitors is considered to be very low and of the order of 3-5 per cent

As long as your interest is about trends, for example, our site experienced a 10 per cent increase in visits week-on-week, and you use the same measurement tool throughout, then those trends will be accurate.

However, problems arise when you start to compare different tools in order understand the underlying absolute numbers, as these will vary widely. The key is to use the right tool for the job.

How accuracy can be improved by using the right tools

It is important to realise that different data collection methodologies bring different metrics to the table. It is these differences that are their strength as they help you build a bigger picture of the website in question. Think of it as building a jigsaw – one piece alone can be very misleading, multiple pieces together provide a clearer picture.

The solution to maintaining your sanity is to combine on-site and off-site web analytics data in a way that compliment each other, rather than providing conflicting data points.

Here are some recommendations:

1. Use off-site metrics when considering the launch or relaunch of a website. For example, what terminology and semantics are being used on the search engines – blue widgets or blue gadgets?
2. Use off-site metrics to understand your visitor demographics as proportions of the total. For example, 65 per cent of our traffic is female in the age bracket 25-34. Does this match your customer base?
3. Use off-site metrics to understand what websites your visitors go to just prior to yours and just after they visit your site.
4. When using off-site panel based data, bear in mind that the data is more reflective of a US home audience. If your target is more international or a business audience, use ISP data
5. Use on-site metrics if you wish to know how many people visit your website. This should always supersede off-site data
6. Use on-site metrics if you wish to know whether visitors are engaging with your content
7. Use on-site metrics to gauge if the user-experience of your visitors is good or bad
8. Use on-site metrics if you wish to know where your visitors are being referred from – which search engine, keyword, email or banner campaign etc.

This is a multi-page post. You are on Page: 1 2 3

As always, I am interested in your feedback (or simply rate this article by clicking on the stars at the top). Do you have to battle with reconciling numbers from different tools, or never ending accuracy questions from senior management?

1. Why counting uniques is meaningless
2. Accuracy Whitepaper for web analytics
3. Understanding Web Analytics Accuracy – Whitepaper
4. What is ABCE?
5. Google is Like a Bank
6. Should you focus on website visitors as individuals?

This is a multi-page post
Page: 1 2 3