NewsTrust
I have been criminally negligent in not linking to and blogging about NewsTrust.net, the laudable effort by Fabrice Florin and Rory O’Connor to get the public to rate news sources. I missed linking to the launch — my fault — then wanted to spend more time getting my head around the thing.
Fabrice and Rory’s effort is much-needed in our new, loose, distributed media world: They want to answer the question, Whom do we trust? Ratings, reputation, and trust are critically important elements in an open medium where all can and do publish. But, of course, it’s not a one-size-fits-all answer anymore; we each trust different sources for different information and different reasons: I may trust you for your music reviews but not your scientific explanations. But before I know you, I still need to know where best to start when looking for or judging information. And there is also a benefit in judging sources, to improve them or make them more responsive or keep them honest.
And so NewsTrust set about having us rate stories and sources. When I first heard about their effort, I liked the idea but also cautioned that the self-selecting nature of having anyone rate whatever they want would skew the value of the results; it will amplify agendas as one person comes in to praise those with whom he agrees and the next comes in to blast those with whom she disagrees. Fabrice and Rory worked to fix that by having juries of judges — for example, classes of journalism students — judge articles assigned to them. But it’s still the case that anyone can rate anything.
So the issue remains that in a small set of judges, there is an almost-inevitable skewing of the judgments. Fabrice and Rory tried extremely hard to recruit people from all ends of the political spectrum, but the early days of NewsTrust clearly lean left: The top-rated magazine is Mother Jones; the no. 3-rated newspaper is The Independent; the no. 1-rated neutral source is NPR; The New York Review of Books is no. 3 under magazines and no. 8 under all sources; Juan Cole is no. 24. And the ratings show a scientific bent: The Union of Concerned Scientists is the No. 1 news source of all; New Scientist, Science, and Scientific American all make the top list.
This is not an intractable issue. As more people use NewsTrust, its worldview is likely to widen (though it is also possible it could contract). What that says is that volume matters; there is a critical mass of both the numbers of judges and the variety of their viewpoints that is necessary to make NewsTrust valuable; until then, it is the view of a small crowd and in an endeavor such as this, a small crowd is less wise than a large crowd. That is the chicken-and-egg issue facing NewsTrust: It will be more valuable when it has more judges, but it will have more judges when it is more valuable.
There are a few solutions to this. The first is that lots of you should go to NewsTrust and start rating stories and sources; the more the better.
But until then, I am coming to think that we should view NewsTrust differently: as a recommendation service. It is Digg with a difference: the people there explain why they like what they like and try to judge news and sources on a scale. In that sense, it is a group blog with one aim: not life stories but links. And that is valuable.
I’d find it more valuable if I could sort those recommendations by worldview. As in the ill-fated LA Times wikitorial, I suggested that they fork the effort: pit one side against the other so we can judge the best of both views. I’d like to see the stories that liberals and conservatives each consider the best on various issues.
And along the way, I think, the end-product NewsTrust started to create — data on trustworthiness in media — will be a byproduct that will grow over time if more and more people use the service.
January 2nd, 2007 at 10:53 am
[...] Buzz Machine [...]
January 2nd, 2007 at 4:13 pm
Jeff,
thanks for the pointer. Without having checked it out yet… Are they really speaking to trust or to popularity? “I like…” is very different from “I trust…” and yet I don’t think most people make that judgement, esp when you toss in ” I agree with.. ” into the mix.
It seems the issue with this type of effort is that anyone can vote and all votes are equal. Usually a good thing, but the person who is very knowledgeable about a field and has read a given source of information in that field for months or years has the same weight as someone who just read an article from that source for the first time. By giving both of these people the same influence you’re masking the very different sources of the votes - what you usually end up with in these cases is what most people who are rating things like.
Somehow we have to separate agreement, like and trust… All three are valuable, but they’re very different. If I’m looking for trustworthy information on, say, a medical condition or where to buy long term care insurance I want a trust rating. If I’m looking for other things that people like me find interesting, I want a “People like you also like… ” rating. In some cases agreement will be the same as this… maybe in most cases.
I’m not sure how to solve this… but whoever does will have something very valuable andnot just in monetary terms
January 2nd, 2007 at 6:04 pm
Yes, it’s been a bad year for news trust with the LA Times as one of the worst:
http://patterico.com/2007/01/02/5595/patterico%e2%80%99s-los-angeles-dog-trainer-year-in-review-2006/
And, AP is still protecting the bogus Captain Hussein :
http://www.iraqslogger.com/index.php/post/486/The_APs_Jamil_Hussein_Scandal
The MSM, in its present form, needs to reform and fast or die faster. Their arrogance, duplicity and general low quality products have finally caught up with them. With no trust or goodwill left between them and their readers, diminishing quality of content, offensive smarmy politically driven agendas as a turnoff, just what do they have left?
January 2nd, 2007 at 9:20 pm
As more people use NewsTrust, its worldview is likely to widen (though it is also possible it could contract).
Well, you’ve got the bases covered, Jeff.
This site is already a leftist echo chamber and I don’t see why or how that will change. Even if a large number of non-leftists made a concerted effort to provide balance to a site that ranks Counterpunch among its top 10 web sites and the Boston Globe as the world’s best newspaper, I imagine the “progressives” would object to the hijacking of their site and would leave. And I think they would have a point.
I don’t think we will ever see a single balanced site that evaluates news and opinion on the web, and I don’t see the need. It doesn’t cost more than electrons for the user to access a wide variety of branded options online. Haven’t you long promoted choice and transparency over balance? I’ve always agreed with that.
January 2nd, 2007 at 11:16 pm
Jeff, thanks so much for your insighful perspective on what we’re doing at NewsTrust.
Your point is well taken that we need a larger crowd to make this work, and your call to action is much appreciated - indeed, the more people rate the news on NewsTrust, the more we can learn from each other, and the smarter we’ll get as a community.
For now, you’re quite right that NewsTrust tends to attract members that lean left, skewing our current line-up of trusted sources. But we’ve only been public for a few weeks and are working hard to invite people with widely different viewpoints to join our non-partisan effort. As we reach out to a broader consistuency, we hope to see our members become more representative of our nation as a whole.
We also plan to introduce this spring a new type of “balanced rating” that would correct such imbalances in our membership. These balanced ratings would be culled from ad-hoc panels composed of representative numbers of members from the left, center and right.
I also appreciate your clever observation that NewsTrust could be used as a way to compare recommendations from different worldviews. We hope to offer that feature as soon as we have a critical mass of diverse viewpoints on our site. At that point, we plan to break down ratings by political affiliation, so you could see what ratings were given to any story from members that self-identify with liberal, moderate or conservative viewpoints.
We believe that the three remedies mentionned above, used in combination, can address the very reasonable concerns you (and ronbo) were kind enough to point out.
And to address Rick Gregory’s comment above, we work hard to rate the news based on journalistic quality, not just popularity. Our methodology was designed to help reviewers go past their initial gut reaction (e.g.: whether or not they agree with views presented in the story), and judge a story on its own merits. Our research studies with the University of Michigan suggest that by asking our members to answer a few more rating questions, we’re able to identify quality journalism a lot more effectively than the single thumbs-up/thumbs-down votes offered by most first-generation social news sites.
Lastly, here are a few of the other ways we build trust on our site:
* reviewers are identified by their real names
* we encourage reviewers to rate each other’s work
* our reviewer’s ratings are weighed based on member level
* member levels are based on activity, experience, ratings and transparency
* reviewers are rewarded for disclosing political viewpoints
* your ratings count more if you have a higher member level
If you think we’re on to something, we’d be grateful if you could help us broaden our community, by participating actively on our site - perhaps even inviting people whose political viewpoints you do NOT share. Getting folks from different backgrounds to collaborate in this way could be quite valuable in our over-polarized nation. It’s about time we started talking to each other, and gaining an appreciation for reasonable viewpoints from the “other side.”
And Jeff, thanks again for your constructive comments, which are always so helpful to us - and for this wonderful opportunity to discuss our experiment with other readers. Onward!
January 2nd, 2007 at 11:53 pm
ronbo has nailed it. Having checked out NewsTrust, what’s balanced or unique there? It’s an aggregate of left-leaning articles, an inventory of the most objectionable and agenda driven usual suspects.
It’s digg it by ad hoc committee.