The analogy goes further: we can simply port the open source definition to create an open data definition or, more broadly, an open knowledge definition:

http://www.opendefinition.org/.

I think this is a nicer way of going about defining open data than talking about ‘recognizing the value implicit in user data’ which is fairly vague (plus what about all the others types of data from geographic to genomic?). Even in the context of companies providing various data services to users it seems to me the main point of open data is to reduce lock-in — not necessarily to recognize to the value inherent in the data itself.

With things like text analytics, the key is going to be proving the accuracy of the method. It is one thing to extract a bunch of company names or ticker symbols from social media, but how accurate is it? Is there a bias to one type of blog over another? Convincing people of this is a key challenge.

As for opening up data, a number of institutions including Intelliseek/BuzzMetrics and TREC have offered data sets for analysis in (academic) research contexts. Again, one needs to consider the ‘owning’ of the data with the cost of aggregation and distribution. Sure, we all own our blog posts, but I don’t own the infrastructure and distribution channels that various institutions invest in to acquire and analyse that data. Thus I can’t perform induction on my ownership and claim ownership of all blog data, the infrastructures that aggregate it and so on.

(BTW, I’ve worked with Kate in the past but won’t damn her with praise here

We do give away a huge amount of data and analytics, primarily through our Blogpulse.com site, but also through presentation at conferences like the one you are at with Kate (who, by the way, is quite bright and definitely “gets it”), open publication in academic journals, or informal blog posts like the ones you referenced on Matt’s blog.

In fact, we put many of our analytic techniques on Blogpulse before we put them into client products. Floodgate is a good example of this; the first use of the Floodgate technology was in the “Blogpulse Live” application debuted a number of months ago on Blogpulse.com. We have a whole second iteration of that technology planned for a forthcoming update of Blogpulse, and all of this will happen before those technologies are ever used in client deliverables. Many of our approaches for influencer/social network analysis have also been debuted through Blogpulse.

Jonathan Carson
CEO
Nielsen BuzzMetrics

I’m one of many who have created mashups based on their web service. Take a look at http://sws.clearforest.com and poke around (and look at the cool mashups listed on the right hand side!).