Comments on: Accuracy Whitepaper for web analytics http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/ Companion site for the book Advanced Web Metrics with Google Analytics by Brian Clifton Thu, 11 Mar 2010 19:36:48 +0000 http://wordpress.org/?v=abc hourly 1 By: Martijn Staal http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/comment-page-1/#comment-7507 Martijn Staal Mon, 25 Aug 2008 13:52:20 +0000 http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/#comment-7507 Brian, Thanks for the whitepaper. Interesting view on the accuracy of web analytics. Some time ago we also discovered that there are a lot of differences between the metrics that are used by web analytics software. For example, Google Analytics measures a unique visitor in a different way then DART or Sitestat. In January I also posted an article about the differences between analytics software (unfortunately in Dutch): http://www.marketingfacts.nl/berichten/20080115_hoe_betrouwbaar_zijn_website_statistieken/ Together with multiple marketing agencies we are now working on a pilot to compare the desired analytics functionality against the functionality that is being provided by current analytics software. We know about some other research at http://www.seomoz.org/blog/how-reliable-is-google-analytics, http://www.stonetemple.com/articles/analytics-report-may-2007.shtml and http://www.idealware.org/blog/2007/06/google-analytics-underreporting-or-just.html. Am curious whether you know any more research projects in which the differences between analytics software have been investigated. Do you, or any of the other visitors here, know about similar research? Brian, Thanks for the whitepaper. Interesting view on the accuracy of web analytics.

Some time ago we also discovered that there are a lot of differences between the metrics that are used by web analytics software. For example, Google Analytics measures a unique visitor in a different way then DART or Sitestat. In January I also posted an article about the differences between analytics software (unfortunately in Dutch): http://www.marketingfacts.nl/berichten/20080115_hoe_betrouwbaar_zijn_website_statistieken/

Together with multiple marketing agencies we are now working on a pilot to compare the desired analytics functionality against the functionality that is being provided by current analytics software.

We know about some other research at http://www.seomoz.org/blog/how-reliable-is-google-analytics, http://www.stonetemple.com/articles/analytics-report-may-2007.shtml and http://www.idealware.org/blog/2007/06/google-analytics-underreporting-or-just.html.

Am curious whether you know any more research projects in which the differences between analytics software have been investigated. Do you, or any of the other visitors here, know about similar research?

]]> By: Ryan http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/comment-page-1/#comment-7476 Ryan Fri, 22 Aug 2008 18:34:29 +0000 http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/#comment-7476 Great whitepaper! Thanks. Great whitepaper! Thanks.

]]> By: Brian Clifton http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/comment-page-1/#comment-2291 Brian Clifton Sat, 01 Mar 2008 22:37:32 +0000 http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/#comment-2291 <strong>Steve/Chris</strong> - thanks for the feedback. I hope its of use at providing answers to those never ending questions about accuracy and data alignment. Its helped me a great deal to collate this into one document - I wish I had written it years ago. In fact its been bouncing around for a least two years in one form or another... <strong>Jim</strong>: No marketer works with accurate numbers, so it should not be a great surprise that web analytics is not accurate. In fact, web analytics is several orders of magnitude more accurate than the offline world, its just not 100% accurate, which is a common misconception for the reasons I give. <strong>Sara</strong>: You ask a good (and difficult) question. Let me ponder this for a while with some fictitious examples and post a reply. Steve/Chris – thanks for the feedback. I hope its of use at providing answers to those never ending questions about accuracy and data alignment. Its helped me a great deal to collate this into one document – I wish I had written it years ago. In fact its been bouncing around for a least two years in one form or another…

Jim: No marketer works with accurate numbers, so it should not be a great surprise that web analytics is not accurate. In fact, web analytics is several orders of magnitude more accurate than the offline world, its just not 100% accurate, which is a common misconception for the reasons I give.

Sara: You ask a good (and difficult) question. Let me ponder this for a while with some fictitious examples and post a reply.

]]> By: Sara http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/comment-page-1/#comment-2153 Sara Wed, 20 Feb 2008 21:58:02 +0000 http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/#comment-2153 I am happy to see that an analyst is talking about trends and although I agree with the fact that accuracy is important and of course ideal - it is the truth and a fact that we have to live with. Looking at it from a marketer perspective I know that there are way to many sources of data or studies that don't actually even predict what the result will be. Many times we conduct focus group studies just to see... what likelihoods we have of hitting the right target group with the right product or the right message. We know accuracy only comes down to sales and this is where we need to look for trends. Bottom line is what is important and to make decisions in a marketing world full of not only web analytics but tons of data sources we need to follow the trends. Most sites are not conducting e-commerce and we need to spend our resources analysing the lead generations through a combination of the online traffic to the other trend tools available. A question for you: - In your opinion..how would you best go about to study a trend when you have to aggregate additional resources of information from various tools (all with their accuracy problems)? I am happy to see that an analyst is talking about trends and although I agree with the fact that accuracy is important and of course ideal – it is the truth and a fact that we have to live with.

Looking at it from a marketer perspective I know that there are way to many sources of data or studies that don’t actually even predict what the result will be. Many times we conduct focus group studies just to see… what likelihoods we have of hitting the right target group with the right product or the right message. We know accuracy only comes down to sales and this is where we need to look for trends. Bottom line is what is important and to make decisions in a marketing world full of not only web analytics but tons of data sources we need to follow the trends.

Most sites are not conducting e-commerce and we need to spend our resources analysing the lead generations through a combination of the online traffic to the other trend tools available.

A question for you: – In your opinion..how would you best go about to study a trend when you have to aggregate additional resources of information from various tools (all with their accuracy problems)?

]]> By: Jim http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/comment-page-1/#comment-2126 Jim Mon, 18 Feb 2008 14:08:56 +0000 http://www.Advanced-Web-Metrics.com/blog/2008/02/16/accuracy-whitepaper/#comment-2126 Hi Brian - Thanks for the whitepaper, looks like a good summary of common accuracy issues. My concern is that although the ideal of precision being more important than accuracy is one that all analysts (in particular when faced with conflicting data) should espouse, this doesn't always work in the real world. Out in the trenches, stakeholders who rely on analytics data to make business decisions often need to make these decisions based on numbers that need to be accurate, for example when pricing CPM ads or developing business models around traffic numbers. Explaining to them that the numbers may not be accurate but they are precise is going to get you some very odd looks, to say the least... Hi Brian – Thanks for the whitepaper, looks like a good summary of common accuracy issues. My concern is that although the ideal of precision being more important than accuracy is one that all analysts (in particular when faced with conflicting data) should espouse, this doesn’t always work in the real world. Out in the trenches, stakeholders who rely on analytics data to make business decisions often need to make these decisions based on numbers that need to be accurate, for example when pricing CPM ads or developing business models around traffic numbers.

Explaining to them that the numbers may not be accurate but they are precise is going to get you some very odd looks, to say the least…

]]>