Tell us about NewsTrust

Since I have been using NewsTrust for a while, I will relate my experiences. I can't even remember how I found NewsTrust. At first I went there to see the news that others had put there and rated. Finally I got a login and reviewed a few stories. Then I found some news articles that I wanted to see on NewsTrust, so I submitted my first story. By the next day my story had been reviewed by others, and could be found on NewsTrust. Cool. So I kept doing it.

At some point Fabrice Florin emailed to ask if I would become a host, the inner circle of NewsTrust. Sure, I said. It turned out to be a bigger inner circle than I thought. It wasn't until Aldon Hynes said it in his blog that I realized the real reason I am involved in NewsTrust: because I was asked. Sure, you don't have to be a host to find NewsTrust useful or to contribute. But it helps, and Fabrice obviously has been asking people to join. There is a community there, and we are getting to know each other. Being able to talk about the principles of journalism with people who know the subject is very helpful.

NewsTrust tries to take advantage of the particular expertise that each person brings. For example, the review form asks you to rate your own knowledge on a topic, and this is used to weight some reviews higher than others. There are also topic hosts, people who watch over a particular news topic and keep it going smoothly. I volunteered to host the "energy" and "election reform" topics. These were a good fit because I am a local election official and I also run an energy task force for my city. Among other things I try to make sure the pipeline of stories on those topics stays filled. With my own little corner of NewsTrust to watch over I started reviewing even more stories. Soon I will have reviewed 1000 stories in about a year. It's addictive.

OK, so here I am adding stories to NewsTrust that I care about, that I think are important. I suppose that is an agenda. But there are some controls. If no one else cares about my stories, or if I submit too many, other people simply won't review them. If my stories don't get the minimum number of reviews, they will not bubble up to the top where people who want to read the news will see them. Also if I go "dive bombing" through the system, giving stories I like a 5 and ones I don't a 1, without writing a good, thoughtful review, then other people will give my reviews a low rating. Not only will that limit the effect of my reviews on those stories, it will lower my "member level" rating, reducing my effect on all stories. Likewise, high ratings fo my reviews increase my member level. So the NewsTrust system is somewhat self-policing.

Sometimes when I give someone's review a low rating, I wish I could tell them why. NewsTrust only let's me click a number 1 to 5. I wish others could tell me about my reviews too. In a way, they can; my NewsTrust profile page has my email address and other information about me. In fact this also increases my member level. NewsTrust tries to encourage transparency. At a minimum, registered users have to provide their real name.

All of this said, there are some people using NewsTrust to pursue an agenda, and their reviews are too often not about journalism. NewsTrust needs to be better at filtering them out, the idea being that they will eventually drop out because they are having little effect, or preferably make their reviews more about journalism. Being in that inner circle, I know the ideas are in place. But the need to avoid censorship calls for making conservative changes.

Prior to NewsTrust I spent a few years rating Yahoo News stories. For each story I would give 1 star for crap that I would never want anyone to see, or 5 stars for good stories that everyone should see. There was no point in giving any other rating. I simply would not rate stories that fell in the middle, which was most stories. On NewsTrust I use all 5 settings for the various ratings. It's easier to do this because each rating is about one thing. For example, I can sometimes give a middle score like 3 if a story has multiple viewpoints, but not all that it should. NewsTrust has changed the way I read journalism. I can't say that for the others.

Hi. I am a Host and early Member on NewsTrust. Addressing just a couple of your questions:

Compared to Alter-Net and Dissident Voice, I find many of the same writers represented on NewsTrust, submitted by a variety of people.

As far as which stories "rise to the top", in our situation that phrase is really not appropriate. The point of NewsTrust is that there is a wide variety of valid stories and viewpoints, deserving of our attention at any one point in time. We are fed at least a dozen stories per day, to scan and comment on, as we see fit. The stories are on several topics, and topics are selected through a group-decision making process. The sources of the stories submitted also represent a variety of funding sources and talent pools. So I would say which stories "spiral in the circle" or "get into the loop", rather than "rise to the top".

Hello. I would like to share with you a few observations about my personal experience with NewsTrust.

About five months ago I began lurking around NewsTrust, interested in the media submissions, comments and reviews, but I didn't consider myself tech savvy enough to ever consider doing anything more advanced online than surfing the net. After a very short time though, I decided to review an article on the site and saw how my review changed the standing of that story on the home page. Within the first week I was hooked. After a few weeks of reviewing articles I received an email message from Fabrice Florin (NewsTrust's Executive Director) inviting me to become a host because of my participation. Easily flattered, I agreed and began posting articles for review by others. Now with 493 story reviews under my belt, I have come to know and trust the staff and many of the members, reviewers, hosts and fellow editors (I was recently promoted!) of the NewsTrust community. I am firmly committed to this "little experiment" as Fabrice refers to "NewsTrust".

I am constantly in awe of the talented members of our team who have extensive backgrounds in journalism and the media. As a media consumer, I've learned so much from them in the short time I've been involved. Yet, I have never felt as though my opinion matters less just because I'm not a trained journalist. My member level is currently 4.6 (out of 5 - not bad!). As the site explains: "All members on the NewsTrust site are given a member level, indicated by one to five yellow bullets. Member levels are the average of all level values for a given member, and are based on member activity, overall experience, transparency, ratings from other members and profile validation. A one-bullet icon indicates a low member level and a four or five–bullet icon indicates a high member level. This member level is determined by five different sources: your site activity records (Activity and Transparency), your own self-assessment (Experience), ratings from other members (Ratings), and NewsTrust staff (Validation)."

Generally, I can submit and review an article (after reading it of course) in as little as five to ten minutes using the short submission form. This form provides an opportunity to comment about the journalistic quality of the story and to rate the story (from 1 to 5 stars) on 12 different journalistic aspects (balance, style, sources, importance, etc.). In the beginning, and from time to time still, I complete the longer review form. This multi-page system provides an excellent mechanism (especially for those of us without a background in journalism) to evaluate the quality of a piece based on the "stats" (i.e. the number of sources - named and anonymous, the number of viewpoints, opinions stated as fact, number of stakeholders, stakeholders quoted, the number of derogatory words and the number of complimentary words in the article). This review process includes research tools, an option to provide ideas on how the story could be improved and an option to post related links for others who want to learn more.

In addition to the above, the comments of the other reviewers quickly taught me to check my biases at the door and to review each story on the quality of its content and not on how closely its author's biases match mine. For me, and for most people I believe that is a tall order, but for the most part the reviews on NewsTrust meet this standard. When a review creeps in with an agenda, any member can rate the review itself and give it a low rating. This lowers that reviewer's standing in the community and more importantly reduces the weight their review has on the average rating of the story. If a member gets enough low ratings, their reviews are effectively made irrelevant.

Recently I hosted a discussion on "Atheism and Faith" on the site. The points of view and substantiating articles provided were insightful and respectful. The feedback from the staff and those who participated was very positive. We successfully discussed this bombastic topic without any bloodshed! My personal point of view moved from that of a religious agnostic to that of a secular humanist during that weeklong discussion, while my respect for the beliefs of those with religious conviction grew dramatically. This has had a positive impact on my ability to deal with my own fundamentalist Christian family with more compassion.

Bi-weekly conference calls with the staff give hosts and editors the opportunity to provide input and hear about progress on the business side of NewsTrust. This provides a sense of ownership that is crucial to maintaining volunteer involvement and helps create community beyond the online experience. It also provides management with valuable feedback, resulting in improvements to the site. Having spent years in a big corporate environment, this almost instant gratification of seeing ideas become reality so easily is possibly one of the most rewarding aspects of my involvement. No bureaucracy here!

NewsTrust has introduced me to so many high quality online media sources and has at the same time taught me how to better evaluate information provided by these sources, and mainstream media, on journalistic merits. I believe this makes me not just a better media reviewer, but also a much better citizen. NewsTrust is a five star site!

Regards,
Dale Penn
Miami Beach, FL

I've been contributing to the Newstrust for about 3 months.

The greatest value of Newstrust for me is that I have been able to find high-quality reports on topics that I'm interested in. This has led me to sources that I wasn't aware of earlier. Read/Write Web is an example of a source that I discovered from Newstrust. I could mention Cnet's Elinor Mills as a reporter who I've found when analyzing articles for Newstrust.

The second most important value of Newstrust for me is the site's educational and focus-seeking impact. When I pick up stories for review, I have to analyze them (on the fly and very quickly, but still) and decide if they are rubbish or valuable. I write for my own publication and freelance for other publications. The exercise of reviewing high (and low)-quality stories is educating and is helping me to stay focused on what's important in writing.

The third value of Newstrust for me is the people who create the community. This is related to the real-identity model Newstrust is using. It was a vital reason for me to be able to view who the other people really were when I joined the community. In fact, I wouldn't be surprised if sites like Wikipedia would have to reconsider their model (before someone else does it).

The big issue for me with Newstrust is that it is very much focused on the U.S. politics. It maybe unintentional, but it is very visible, for instance, when Spotlight stories and Weekly topics are being selected. If the focus is intentional, it should be clearly spelled out. Then, contributors who are not into politics could go somewhere else. If the focus on U.S. politics is unintentional, Newstrust currently may risk losing contributors who are not interested in the topic.

It's a business decision if Newstrust will focus on U.S. politics, on another topic, or on wider range of topics. I hope the decision is made in the near future.

I'm from Europe. So far, I've found so much value at Newstrust that I believe it has a great future as a site and as a community.

I have been a contributor to NewTrust since early on.(December 2006) and have been "promoted" to Host and into the Editor's group. I have reviewed about 270 stories (as of 19April2007).

First, my observations as a contributor. The NewsTrust review process is very comprehensive and if you, as reviewer of the story, take the time to do something more than click on a summary review value (1 to 5, 1 is awful, 5 if fabulous), you add to a crowd-based evaluation that is often quite powerful. The flip side is that it can take a while to read the story, click on the value buttons, write a short review, find a site or two that augment or contrast, and submit. I find that all I can handle is about four or five stories a day. But it feels like I'm not "doing my job" when I do much less than that. And I really appreciate the other contributors who take the time to put in the effort to add to the detail of their stories.

Turning to my feelings about how Newstrust is coming along, still as a participant, I'm struggling with the existential questions of the meaning of it all [half-joking]. The premise of the site is that a crowd can, with a loosely shared set of values and a single objective, rank news stories objectively. The working assumption is that if enough people rate a story, the rating wil be approximately accurate (to a very close degree). During the conference calls, we frequently discuss the intended meanings of the rating questions. The issue is that we need reviewers to share those meanings and to understand the standard of excellence for "journalistic quality". I suspect we all fear that there's a lot to do to educate reviewers and improve the understanding of the intent.

Another thing we all discuss is that we think our reviewers are probaby biased to the Left (liberal leaning) and are concerned that may be skewing the reviews. But it shouldn't matter if we're not reviewing the bias of the material. However, most of us agree that our bias probaby does skew our ratings by some amount (+- 1/2 a point? ... as a guess). So the answer is, as would come as no surprise to crowdsourcing folks, more reviewers. The larger the number of (thoughtful) reviews, the higher the likelihood we're getting a valid cross section of real opinion. We need reviewers from the Right and who understand that transparency, balance, breadth of views, and the like are important. If they come and do the reviews in the spirit of the site, they'll have as much fun as the rest of us.

The last thing to note as a contributor and collaborator on direction is that I'm still looking for ways to increase the power of the message that gets created by the reviewers. We're revieweing a relatively small number of stories and we're reviewing them "in isolation". There are two dimentions missing from this approach IMHO (and they're really hard to design solutions for or to imagine that real humans have time and energy to fix): comparative quality on the same story amoung sources and the track-record of quality over the evolution of the story. The first is obvious. It is the question of whether source A's piece is better than source B's as measured on the crowd-driven scale(s) of NewsTrust. The seond is more a measure of intellectual honesty and consistency of viewpoint as we watch outlets change their positions or revise the facts as reported one or two days (hours, minutes) ago without transparency. Some subset of the NewsTrust team has discussed this at one time or another but these are hard things to do.

As a user, I stop by the site and watch what rises to the top. I'm convinced the idea is working. Those who really get active get the spirit and contribute generally good information. This isn't a site that tells you what people enjoy or think is cool. It is a site that tells you what's worth reading to stay current and have the depth of information and exposure to the range of viewpoints to be a responsible member of civil society. The reviewers seem to promote decent stories and leave others in our version of the cutting room floor. The experience (the design of the site) limits the amount we can see and that's a challenge for the future. I keep thinking that something like the approach taken by Google's news pages might be an approach but who knows.

Turning to what I see on the site, I must admit I'm not yet impressed with the breadth of what's getting submitted and reviewed. It's getting better as more folks sign up and get active. This criticism is also an expression of personal guilt as I am feeling like I should be submitting a much larger number of stories.

I'm reading 102 newsfeeds of which at least 40 or fifty are "news related". I'm reading feeds from alJazeera English, Haaretz and Ynetnews.com (Israel), Russia, and Turkey and am looking for feeds out of China and India and South America. The US mass media is over-represented, predictably, on NewsTrust and has left me (guiltily) feeling that NewsTrust needs broadening. The community is trying to pick up more blog entries and the like and that's a good thing, but it tends to be responsive to US media and to miss the broader flow of global news. If (when) I get over my emotional/workflow roadblocks, perhaps I will be submitting some of the interesting and relevant news from outside the US and the West (Europe outside the UK, another large group of valuable voices we are not yet seeing on NewsTrust).

Finally, I am convinced that the rating system (1-5) is forcing the ratings into too narrow a band of differences. One feels simply terrible giving something a 1. It's such a horrific slam ;-) ... and likewise, most of us reserve 5s for really stellar (and rare) stuff. That leaves our ratings to fall somewhere between 2.3(ish) and 4.6(ish). I keep meaning to act like I know something about statistics and doing distribution analysis on the results and proposing a way to translate that into a more useful metric ... but life, work, and the dog interfere.

Newstrust is in the early stages. In general, judging from the several Host conference calls I've participated in, the thought-leaders have a good sense of where it "needs" to go. It needs to become the work of a bigger crowed and it represents a way of looking at "the media" (from the paid journalists published by the big outlets to the bloggers reporting, synthesizing, and sharing opinions on their own ... like me) and making a reasonably objective judgement about the quality of the work being presented. I think this is very important. I think that NewsTrust has a chance of growing in stature (by virtue of its representing a broad spectrum of educated reviewers across the political spectrum) and becoming a real voice to help "the media" clean up its act.

Greetings!

I'm Executive Director for NewsTrust.net, and we're very honored to be included by newassignment.net for this investigation of social news sites.

We have high hopes for the Pro-Am journalism practiced by newassignment.net. At NewsTrust, we're also finding that when experienced journalists actively collaborate with concerned citizens, everyone wins. I also appreciate the fact that you're inviting the subjects of this story to chime in -- and participate actively in this conversation.

With that in mind, I would like to point out some highlights on the work under way at NewsTrust:
http://www.newstrust.net/

NewsTrust.net helps people find good journalism online. We created this nonprofit service to help citizens make informed decisions about their lives and governments. We've based our social trust network on a few key principles, which seem to be working well for us, even though we're still in beta stage.

Here are some unique aspects of our service:
* we rate information quality, not just popularity
* we evaluate news and opinions based on core principles of journalism
* our rating criteria include fairness, evidence, context and importance
* we track ratings for each publication in our source reputation database
* we feature stories from our most trusted sources in our daily listings

To discourage gaming, we offer these preventive measures:
* reviewers are identified by their real names
* we rate our reviewers based on the quality of their work
* our reviewers' ratings are weighted based on their own member level
* member levels are based on activity, experience, ratings and transparency
http://www.newstrust.net/slides/graphics/trustnetwork.jpg

Our research studies suggest that:
* citizens using our tools can assess news quality as well as professionals
* our multiple-rating evaluations appear more reliable than single ratings

Again, we're still in early stage and we certainly don't claim to have all the answers. But some the methods described above have already proven invaluable in assessing information quality, which is why I thought I'd point them out here.

But don't take my word for it. Try it out for yourself:
http://www.newstrust.net/

You can also read more about us here:
http://beta.newstrust.net/about/

And be sure to check the testimonials from members of our community:
http://beta.newstrust.net/about/testimonials.htm

Enjoy!

Fabrice Florin
Executive Director, NewsTrust