Comments on: The need for a measurement summit http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/ by Jeff Jarvis Fri, 22 Aug 2008 02:01:42 +0000 http://wordpress.org/?v=2.5.1 By: "Community Project: Roundtable on Social Media Measurement" from Pro PR http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-365819 "Community Project: Roundtable on Social Media Measurement" from Pro PR Wed, 26 Dec 2007 17:40:45 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-365819 [...] Bell, Flemming Madsen, Geoff Livingston, Katie Paine, David Brain, Brendan Cooper, Brian Solis and Jeff Jarvis have made valuable contributions to our emerging understanding of social media measurement and [...] [...] Bell, Flemming Madsen, Geoff Livingston, Katie Paine, David Brain, Brendan Cooper, Brian Solis and Jeff Jarvis have made valuable contributions to our emerging understanding of social media measurement and [...]

]]> By: Time for a new PageRank? | terrainnova.org http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-354797 Time for a new PageRank? | terrainnova.org Wed, 18 Jul 2007 12:54:07 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-354797 [...] more encompassing proprietary PageRank replacement (or I should say upgrade). Others suggest that a summit (!) is necessary to decide on open standards for measuring influence. I’d go with openness [...] [...] more encompassing proprietary PageRank replacement (or I should say upgrade). Others suggest that a summit (!) is necessary to decide on open standards for measuring influence. I’d go with openness [...]

]]> By: Jeremiah Owyang http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-324016 Jeremiah Owyang Fri, 09 Feb 2007 10:39:23 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-324016 Jeff Many of our clients at PodTech have a need to measure success (esp since corporations that are hesitant to adopt). I've spent quite a bit of time researching this information (and Clint Ivy points to me) Please check out some of my thoughts Defining Engagement http://www.web-strategist.com/blog/2007/02/01/defining-engagement/ Discussions with Nielsen Buzz Metrics CEO http://www.web-strategist.com/blog/2007/01/26/social-media-measurement-industry/ I worked with Factiva/Dow Jones to run a workshop with some of the top minds in social media and the topic, an invite should have been extended to you: http://www.web-strategist.com/blog/2006/12/06/factiva-social-media-roundtable-helps-to-answer-what-should-we-measure/ You can see all my thoughts that are tagged "Social Media Measurement" http://www.web-strategist.com/blog/category/social-media-measurement/ I was at Hitachi, and running the Social Media program, it was my job to measure success and impact. I did it using a variety of tools as well as a vendor, so I come from experience in the 'trenches'. I'm interested in learning how to participate at your event. In the end, I hope to be an influencer, as I'll be recommendations to our clients, both in person as well as from my blog. Jeff

Many of our clients at PodTech have a need to measure success (esp since corporations that are hesitant to adopt).

I’ve spent quite a bit of time researching this information (and Clint Ivy points to me)

Please check out some of my thoughts

Defining Engagement
http://www.web-strategist.com/blog/2007/02/01/defining-engagement/

Discussions with Nielsen Buzz Metrics CEO
http://www.web-strategist.com/blog/2007/01/26/social-media-measurement-industry/

I worked with Factiva/Dow Jones to run a workshop with some of the top minds in social media and the topic, an invite should have been extended to you:
http://www.web-strategist.com/blog/2006/12/06/factiva-social-media-roundtable-helps-to-answer-what-should-we-measure/

You can see all my thoughts that are tagged “Social Media Measurement”
http://www.web-strategist.com/blog/category/social-media-measurement/

I was at Hitachi, and running the Social Media program, it was my job to measure success and impact. I did it using a variety of tools as well as a vendor, so I come from experience in the ‘trenches’.

I’m interested in learning how to participate at your event.

In the end, I hope to be an influencer, as I’ll be recommendations to our clients, both in person as well as from my blog.

]]> By: Ian Houston http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-323825 Ian Houston Fri, 09 Feb 2007 05:23:26 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-323825 Hi Jeff, Interesting post. A subject near and dear to my heart and one I've spent a fair amount of personal research on the past couple years. The organization you should check out is the <a href="http://www.webanalyticsassociation.org" title="Web Analytics Association" rel="nofollow">Web Analytics Association</a> which enjoys participation from a major portion of the major analytics vendors. There is also an associated conference called <a href="http://www.emetrics.org" title="Emetrics Summit" rel="nofollow">Emetrics Summit</a> where myself and others have presented directly about this topic as well as Web 2.0 measurement issues a whole. If you are interested in looking at a somewhat simple example of RSS event based tracking I recommend the book <a href="http://www.webanalyticsdemystified.com/about_wsmh.asp" title="Web Site Measurement Hacks" rel="nofollow">Web Site Measurement Hacks</a> by Eric Peterson where I contributed a hack with a basic perl example using the "snoopers" you mention above. My point in all this is there is a fairly substantial community out there already talking, meeting and working on these issues that includes a major percentage of companies that build the tools you would use to do this tracking, the people who use them and a "niche" of bloggers. Perhaps we should all get together for coffee sometime. Maybe in May, in San Fransisco during the Emetrics Summit? Best regards, -Ian Hi Jeff,

Interesting post. A subject near and dear to my heart and one I’ve spent a fair amount of personal research on the past couple years.

The organization you should check out is the Web Analytics Association which enjoys participation from a major portion of the major analytics vendors. There is also an associated conference called Emetrics Summit where myself and others have presented directly about this topic as well as Web 2.0 measurement issues a whole.

If you are interested in looking at a somewhat simple example of RSS event based tracking I recommend the book Web Site Measurement Hacks by Eric Peterson where I contributed a hack with a basic perl example using the “snoopers” you mention above.

My point in all this is there is a fairly substantial community out there already talking, meeting and working on these issues that includes a major percentage of companies that build the tools you would use to do this tracking, the people who use them and a “niche” of bloggers. Perhaps we should all get together for coffee sometime. Maybe in May, in San Fransisco during the Emetrics Summit?

Best regards,

-Ian

]]> By: hey jeff jarvis » Instant Cognition » web analytics http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-323633 hey jeff jarvis » Instant Cognition » web analytics Fri, 09 Feb 2007 02:00:29 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-323633 [...] …and anyone else in the blogosphere who thinks that they are the only ones thinking about how to measure “…social and niche media online.” [...] [...] …and anyone else in the blogosphere who thinks that they are the only ones thinking about how to measure “…social and niche media online.” [...]

]]> By: Steve Broback http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-323319 Steve Broback Thu, 08 Feb 2007 18:28:34 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-323319 The blog business summit team would be interested in the organizing/hosting the "blog measurement summit." I know several good potential venues. Jeff, ping me if you want to discuss. My email address is attached to this post. The blog business summit team would be interested in the organizing/hosting the “blog measurement summit.” I know several good potential venues. Jeff, ping me if you want to discuss. My email address is attached to this post.

]]> By: Adrian Cockcroft http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322609 Adrian Cockcroft Thu, 08 Feb 2007 01:48:25 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322609 Edward says: >Use uniques if you want to measure “audience reach'’; >Use page impressions to check out “volume'’ of traffic; >Use visits to gauge “frequency'’. I'd add one more metric - stickiness or time spent on a site, since a fleeting ad impression is worth less than one that gets a lot of exposure. This could help capture some of the AJAX based sites, where the ad in a surrounding frame gets a lot of viewing while the rest of the page is being modified. If you do get a summit together, let me know and I'll do what I can to get eBay involved (I'm in the research labs at eBay). Edward says:
>Use uniques if you want to measure “audience reach’’;
>Use page impressions to check out “volume’’ of traffic;
>Use visits to gauge “frequency’’.

I’d add one more metric - stickiness or time spent on a site, since a fleeting ad impression is worth less than one that gets a lot of exposure. This could help capture some of the AJAX based sites, where the ad in a surrounding frame gets a lot of viewing while the rest of the page is being modified.

If you do get a summit together, let me know and I’ll do what I can to get eBay involved (I’m in the research labs at eBay).

]]> By: Todd Zaki Warfel http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322293 Todd Zaki Warfel Wed, 07 Feb 2007 20:26:30 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322293 This is definitely needed. With the influx of RIAs and the slow death of the page, metrics is changing. Most of our clients are trying to use things like Google Analytics, WebTrends, FireClick, and Omniture. None of these work great for RIA situations. We have done some customer tracking with Flash-based sites that track individual MCs. So, we can track states, which is more granular, but this is a different model than what the industry has been used to in the past. This is definitely needed. With the influx of RIAs and the slow death of the page, metrics is changing. Most of our clients are trying to use things like Google Analytics, WebTrends, FireClick, and Omniture. None of these work great for RIA situations. We have done some customer tracking with Flash-based sites that track individual MCs. So, we can track states, which is more granular, but this is a different model than what the industry has been used to in the past.

]]> By: edward roussel http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322239 edward roussel Wed, 07 Feb 2007 19:51:55 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322239 Greetings, The page impression is far from dead - just speak to the media buying agencies who still consider it to be the best available measure of volume of traffic (as oppose to number of readers or frequency of visits). There will never be a consensus about a single metric. Yes, ABC has now made uniques their mandatory measure; but that doesn't mean that other measures matter less. The significance of a particular metric depends on what it is that an advertiser (or layman) is trying to measure. Indeed, ABC offers this useful guide: http://www.abce.org.uk/ Boils down to this: Use uniques if you want to measure ``audience reach''; Use page impressions to check out ``volume'' of traffic; Use visits to gauge ``frequency''. For most people, the above three measures provide more than enough information about the performance of a website. Edward Greetings,

The page impression is far from dead - just speak to the media buying agencies who still consider it to be the best available measure of volume of traffic (as oppose to number of readers or frequency of visits).

There will never be a consensus about a single metric. Yes, ABC has now made uniques their mandatory measure; but that doesn’t mean that other measures matter less. The significance of a particular metric depends on what it is that an advertiser (or layman) is trying to measure. Indeed, ABC offers this useful guide: http://www.abce.org.uk/

Boils down to this:

Use uniques if you want to measure “audience reach”;
Use page impressions to check out “volume” of traffic;
Use visits to gauge “frequency”.

For most people, the above three measures provide more than enough information about the performance of a website.

Edward

]]> By: Katie Delahaye Paine http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322231 Katie Delahaye Paine Wed, 07 Feb 2007 19:45:42 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322231 As someone who's been measuring CGM for a decade, I'd love to be at a Blog Measurement Summit. A decade ago we were having this same conversation at a summit about how to measure web pages -- and unique visitors was the best we could come up with at the time. Now at least, there are real business outcome metrics available. It would be great if we didn't have to wait ten years for good CGM metrics. And I totally agree with the importance of connectedness, and level of conversation, but too many in the agency world are still obsessed with quantity of eyeballs, and the loudness of their screams, rather than on the actual business outcomes. As someone who’s been measuring CGM for a decade, I’d love to be at a Blog Measurement Summit. A decade ago we were having this same conversation at a summit about how to measure web pages — and unique visitors was the best we could come up with at the time. Now at least, there are real business outcome metrics available. It would be great if we didn’t have to wait ten years for good CGM metrics.
And I totally agree with the importance of connectedness, and level of conversation, but too many in the agency world are still obsessed with quantity of eyeballs, and the loudness of their screams, rather than on the actual business outcomes.

]]> By: Serge Lescouarnec http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322100 Serge Lescouarnec Wed, 07 Feb 2007 18:53:16 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322100 Jeff It seems it is hard to get a clear picture. For my blog 'Serge the Concierge', I now use the Stats provided by Feedburner which give a good picture but not a perfect one. Maybe this is due to different companies and services using different methods. As for advertising, I think it works better if the ads are closely related to the content and audience of the site regardless of the audience size and also of a certain quality. Some sites run the risk of looking like billboards with a few stories. My few cents. Serge Blog: http://www.sergetheconcierge.com Biz: http://www.njconcierges.com Jeff

It seems it is hard to get a clear picture.

For my blog ‘Serge the Concierge’, I now use the Stats provided by Feedburner which give a good picture but not a perfect one.

Maybe this is due to different companies and services using different methods.

As for advertising, I think it works better if the ads are closely related to the content and audience of the site regardless of the audience size and also of a certain quality.

Some sites run the risk of looking like billboards with a few stories.

My few cents.

Serge
Blog:
http://www.sergetheconcierge.com
Biz:
http://www.njconcierges.com

]]> By: Olivier http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-322014 Olivier Wed, 07 Feb 2007 18:10:27 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-322014 Hi Jeff, I think that the Summit you are looking for already exists. You might want to check the <a href="http://www.esomar.org/index.php/wm3-07-overview.html" rel="nofollow"> ESOMAR Worldwide Multi Media Measurement Summit</a> (WM3) which will take place in Dublin in June and gather researchers and all media to discuss about measurement challenges (it was held last year in Shanghai). I believe that the Program Committee is still receiving submissions for papers... Olivier Hi Jeff,

I think that the Summit you are looking for already exists. You might want to check the ESOMAR Worldwide Multi Media Measurement Summit (WM3) which will take place in Dublin in June and gather researchers and all media to discuss about measurement challenges (it was held last year in Shanghai).

I believe that the Program Committee is still receiving submissions for papers…

Olivier

]]> By: Jim Louderback http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-321186 Jim Louderback Wed, 07 Feb 2007 02:39:08 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-321186 Absolutely. This is one of the biggest problems we have today for video on the web, along with podcasts, as much of it is downloaded, vs. streamed, and impossible to track. I'm there! Absolutely. This is one of the biggest problems we have today for video on the web, along with podcasts, as much of it is downloaded, vs. streamed, and impossible to track.

I’m there!

]]> By: Matt Sollars http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-321049 Matt Sollars Tue, 06 Feb 2007 23:23:45 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-321049 Such a discussion is even more important with politicians moving their campaigns online. Hopefully, we can develop a way to judge a candidate's online popularity a bit more accurately, and without the apparent spin, than stories like <a href="http://www.politico.com/news/stories/0207/2621.html" rel="nofollow">this one at politico.com yesterday</a> that declared Hillary the most popular online Dem. She may well be, but the numbers used in the story aren't all that convincing. If we are to make judgments on candidates based on online metrics, then we really need to understand what those metrics are and what they mean. Such a discussion is even more important with politicians moving their campaigns online. Hopefully, we can develop a way to judge a candidate’s online popularity a bit more accurately, and without the apparent spin, than stories like this one at politico.com yesterday that declared Hillary the most popular online Dem. She may well be, but the numbers used in the story aren’t all that convincing.

If we are to make judgments on candidates based on online metrics, then we really need to understand what those metrics are and what they mean.

]]> By: Mike Orren http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-321040 Mike Orren Tue, 06 Feb 2007 23:08:21 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-321040 While I agree with James Lewin's sentiment above, the advertisers want to see traffic numbers -- and breaking them of that habit will take some time. I'd absolutely volunteer to participate from the niche/local side. This undermeasurement problem is particularly acute on local sites. While I agree with James Lewin’s sentiment above, the advertisers want to see traffic numbers — and breaking them of that habit will take some time.

I’d absolutely volunteer to participate from the niche/local side. This undermeasurement problem is particularly acute on local sites.

]]> By: Chip Griffin: Pardon the Disruption http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320919 Chip Griffin: Pardon the Disruption Tue, 06 Feb 2007 20:56:50 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320919 <strong>Measuring Blogs</strong> Jeff Jarvis today issued a call for a blog measurement summit. That's a great idea. As he points out, current panel based solutions aren't as accurate as many would like to see, especially when it comes to niche blogs. Fortunately, Measuring Blogs

Jeff Jarvis today issued a call for a blog measurement summit. That’s a great idea. As he points out, current panel based solutions aren’t as accurate as many would like to see, especially when it comes to niche blogs. Fortunately,

]]> By: Glenn Fannick http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320795 Glenn Fannick Tue, 06 Feb 2007 19:21:03 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320795 Dow Jones's Factiva group is one of those companies actively investigating better measurement approaches. We held a <a href="http://fannick.blogspot.com/2006/12/day-after-my-heads-still-spinning.html" rel="nofollow">roundtable discussion</a> in Palo Alto last year to start some ideas flowing. (A white paper is forthcoming) We want to do more in this area in 07. It's badly needed. Dow Jones’s Factiva group is one of those companies actively investigating better measurement approaches. We held a roundtable discussion in Palo Alto last year to start some ideas flowing. (A white paper is forthcoming) We want to do more in this area in 07. It’s badly needed.

]]> By: heri http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320694 heri Tue, 06 Feb 2007 16:46:16 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320694 i think what we need actually is a buzz tracker, or a tool which can track audience for a niche site, where they are coming from, what users are asking for, measuring their potential interest etc. I agree that panels and old metrics don't work any more. i think what we need actually is a buzz tracker, or a tool which can track audience for a niche site, where they are coming from, what users are asking for, measuring their potential interest etc. I agree that panels and old metrics don’t work any more.

]]> By: Andy Dickinson.net » Blog Archive » Quickies: Measurement and web video http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320690 Andy Dickinson.net » Blog Archive » Quickies: Measurement and web video Tue, 06 Feb 2007 16:40:42 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320690 [...] Jeff Jarvisexplaisn why he thinks there is a need for a measurement summit I think it’s time for a measurement summit: Bring together the measurement companies, the advertisers and their agencies (buyers), the sites’ reps (sellers), the media sites, and technology companies and let’s hammer out some standards and methods for measurement. [...] [...] Jeff Jarvisexplaisn why he thinks there is a need for a measurement summit I think it’s time for a measurement summit: Bring together the measurement companies, the advertisers and their agencies (buyers), the sites’ reps (sellers), the media sites, and technology companies and let’s hammer out some standards and methods for measurement. [...]

]]> By: James Lewin http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320688 James Lewin Tue, 06 Feb 2007 16:34:23 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320688 Jeff Worrying too much about page views, authority, speed, etc, statistics seems to be horseless-carriage thinking. If a company is advertising on the web, they need to be measuring the results that campaigns deliver. This is something companies can and should be doing now, but few are. On the other hand, it will be a long time before there's a consensus on measurement of penetration, etc. Jeff

Worrying too much about page views, authority, speed, etc, statistics seems to be horseless-carriage thinking.

If a company is advertising on the web, they need to be measuring the results that campaigns deliver. This is something companies can and should be doing now, but few are.

On the other hand, it will be a long time before there’s a consensus on measurement of penetration, etc.

]]> By: Jay Bryant http://www.buzzmachine.com/2007/02/06/the-need-for-a-measurement-summit/#comment-320634 Jay Bryant Tue, 06 Feb 2007 15:23:18 +0000 http://www.buzzmachine.com/index.php/2007/02/06/the-need-for-a-measurement-summit/#comment-320634 Thanks for the post, I would agree that we need to bring the measurement business out into the open with standards that are industry supported. Let me know if you need space for the summit. Jay Bryant Thanks for the post, I would agree that we need to bring the measurement business out into the open with standards that are industry supported. Let me know if you need space for the summit.

Jay Bryant

]]>