Clay Shirky, discovering the net since 1993

A social networking theorist takes aim at crowdsourcing

Manikant (Mani) Narayanan interviews Clay Shirky via telephone May 30, 2007.

Clay Shirky is an adjunct professor at New York University's graduate Interactive Telecommunications Program. He is also a writer, consultant and speaker. His work has appeared in Business 2.0, The New York Times, The Wall Street Journal, the Harvard Business Review and Wired.

Shirky rides against the waves. He has spoken about the wrong side of "tagging," the "semantic Web," and other heavy-duty concepts.

Manikant (Mani) Narayanan: How do you define crowdsourcing?

Clay Shirky: It's one of those words where the definition is a little complicated. If I were to offer a formal definition, I would say it's an analogy with outsourcing, in which work previously done by company employees is offered to and performed by a group of people on the Internet.

Q: You have watched lots of sites transition in terms of participation — including eBay and Amazon's Mechanical Turk. How do you think the participation has changed?

A: The big change in crowdsourcing has been the introduction of money. In "Cathedral and Bazaar," (Eric S.) Raymond makes the point that Linus Torvalds was the first person working on free software who realized that he could recruit developers from all over the world using the Internet. So, since at least 1991, we had this model of distributed, nonmanagerial production, where groups of people come together and don't have the same boss and their salary is not their motivation for creating. That model is going on 20 years — in part because of financial lacking and in part because people were most adept at finding tasks they were interested in rather than what they were paid for.

What's new with crowdsourcing is the introduction of financial motivation on the part of people doing the work. The big question for crowdsourcing is whether this is a series of special cases or is this a general business infrastructure?

Q: Do you see this becoming more and more adopted as a business model?

A: It's a business model now. The question is how broadly adopted can it be. A lot of people have focused on the risk to their own intellectual property, i.e., the supply-side risk. I am more focused on the demand-side risk, which is to say, we have these historical examples like AOL back in the late '90s, where the guides on AOL, the unpaid volunteers, essentially made it such a welcoming environment. Suddenly they realized that billions of dollars were on the back of their work, and they did not feel adequately appreciated by AOL. They and sued and they won. So, AOL had a crowdsourcing model, though we didn't call it crowdsourcing back in mid-'90s. And at the end of day, the very fact that crowdsourcing was so effective for AOL caused the crowds being sourced to sour on it.

So, the big question for the business model is: Is it possible to pay people a fraction of a salary and still get significantly good work done. Because if the answer is no, then crowdsourcing doesn't have any good advantages over outsourcing. But if the answer is yes, then it's a revolution.

Q: Do you see crowdsourcing drastically changing society?

A: No, I see peer production changing things drastically in society — which is to say, all forms of production, whether the participants are paid or not. ... Crowdsourcing is a way of doing it. There are several other ways to do it. For example, Wikipedia is not crowdsourced because no one gets paid to work on Wikipedia.

So, if it's going to be revolution, it's going to be confined to the business world, but the revolution will then be in businesses, which sticks with a traditional sourcing model of higher employees and management. They are going to be in trouble because they are going to face competitors who can amass a lot of work at a fraction of their cost.

Q: Do you see the end of competition?

A: If it works, there will surely be a lot more competition.

The real question is if this going to work. Right now, we have a bunch of evocative-use cases. What we don't yet know is: Are we looking at a series of special cases or are we looking at general-purpose case?

Q: Do you see ingroups/outgroups forming out of disagreements and negating the advantages of crowdsourcing?

A: No, I don't think there is a crowdsourcing movement. I don't think that people participating in crowdsourcing think of themselves as part of a moral force in society, as people in the open source movement did. So I don't think there will be any general-purpose failure, because we already have the existence proofs.

I don't think that there is a coherent whole here, so much as a set of techniques for paying people for their work.

Q: Do silent watchers belong to the crowdsourcing community? There are several people in the open source community who post a "please fix this" and vanish. Do you think they belong to the community?

A: That is a really, really interesting question, and, in a way, the hallmark of all of these large-scale tier production efforts. There is a very small community, a kind of core group, that manages the project and a huge number of people contribute only from time to time.

So the average commons number of check-ins for a Linux kernel per developer is one. That is also the commons number of contribution by a Wikipedia contributor.

The community is a much smaller group that takes in all of the people who participate at the lowest possible way and integrate them, and I am sure crowdsourcing will go the same way.

Q: Isn't favoritism bad for crowdsourcing? There are several communities to choose from where they may want to contribute?

A: I don't think anyone would start with a goal of participating in any crowds operation. I think people are going to do things like contribute music or contribute to software.

I don't think a Linux kernel and crowdsourcing are analogous, because Linux is a project while crowdsourcing is a technique.

So the real question for any given problem:

a) Can you use methods of peer production to try to solve it?
b) Can you use crowdsourcing techniques to encourage that peer production?

I think, right now, crowdsourcing is fairly square.

The more methods that are available to be tried, the better chances for crowdsourcing to be a general-purpose technique.

Q: In your recent Corante article, you write that "mature systems have more controls than immature ones." Did you mean established companies or systems that have learned from mistakes?

A: I meant more mature open systems. ... Wikipedia now has many layers of control and internal rules than it had in 2002. In 2002, it was genuinely open. There were no lost pages, no admin controls, etc. Now Wikipedia has a fairly considerable bureaucratic apparatus behind it.

When eBay launched, there was no reputation market, and now there is.

As these systems grow big, they need to add the control to handle the problems they have. Obviously, this will happen in crowdsourcing as it succeeds; they will have to add controls to manage the problems, just like Wikipedia did.

Q: You mentioned once that the "group is its own enemy." Do you still see it being relevant in today's world?

A: Yes, because that's not technology; it has more to do with psychology. Rather about crowdsourcing, the question is: Are you putting together a group, which is crowdsourcing, working as a team or working as individuals or, in some cases, in direct competition from each other. To the degree that crowdsourcing becomes about putting together teams, it will absolutely be the case that the difficulty faced by collaborative groups will be faced by those groups.

Q: It seems like the American society has been more connected after the advent of the Internet. The human connection has been very relevant in day-to-day life of other cultures. For example, in India, farming communities form groups to accomplish a goal. In Bangladesh, there has been Grameen Bank. Your thoughts?

A: This also happened when peer2peer came about. Suddenly everyone wanted to say everything was now peer2peer. In the American context, what you are talking about is collaborative communal work for social reasons/economic reasons, and Robert Putnam's book "Bowling Alone" is a great treatment of that.

But that's not crowdsourcing; crowdsourcing is specifically related to outsourcing. It doesn't mean every place people work together. It means a business not using standard managerial techniques to produce something and instead relying on the crowd to do it. So I would be very careful of extending the idea of crowdsourcing to include any kind of collaborative production, simply because the phrase becomes much blander than I think, what Jeff (Howe) intended by it. I don't know of any model of crowdsourcing that assumes that the people being sourced to or act as a single collaborative team. I don't think that there is any model of work in a single collaborative team anywhere in the world that should be called crowdsourcing, because the comparison with outsourcing goes away.

Q: Assignment Zero experimented with open source religion. Would you consider that an example of crowdsourcing? There are several other projects, like open source car projects and open source MP3 players. Do you still see those as not being crowdsourced, because some big company is not involved?

A: It's not about the question about a big company. Certainly, people are going to adopt techniques irrespective of whether the company has existed for a long time or not. The question is really: What's the relationship of the group doing the work to the ownership?

The open source model has one model, in which the motivation of people are mainly noneconomical and are not managed by a single boss.

For instance, compare open source software to the Mechanical Turk; I have explicit managerial control over the work that gets produced. Say I need 5,000 pictures of this neighborhood in Brooklyn over the next three months and I will pay for those. I am not saying that the photographers go out and take any picture and upload it. That's Flickr. So it's not a big business or small; it's whether there is a managerial decision about what kind of work to invite and whether payment is usually the motivation. So anybody saying, "I have got a kind of job that you would have to ordinarily hire someone to do, but instead, I am going to put an open call on the Internet and offer to pay for it, without making these people my employees" — that's a crowdsourcing model.

So I think, in as much as the work Jay (Rosen) is doing, involves both of the decision as to what to cover (bringing people on board) and also involves payments to the work, it seems to me a crowdsourcing model. When either of those isn't true, it is not crowdsourcing.

Q: So you don't see crowdsourcing taking us into a blend of socialistic and democratic culture?

A: I am not sure how crowdsourcing relates to democracy. This is an economic model. You can do it in democracy, you can do it outside of democracy. Look at Amazon's Mechanical Turk. It is specifically nondemocratic. It's pure managerial culture. Here is my job, here is what I am paying, do it if you can. The workers don't go underneath the production. The workers don't own the surplus of labor value.

Q: Outside crowdsourcing, has anything surprised you over the years as you've watched the Internet and general behavior?

A: The biggest surprise has been that open systems create their own problems, and systems, typically at launch, have the highest degree of openness they can tolerate and then close down to some degree.

The big surprise has been watching open systems slowly add, for various reasons, control over production, consumption, features, etc., because the problems they suffer are actually problems that come from openness, not from closedness.

Q: Do you study online behavior by active participation?

A: Some, yes. Some, no. For instance, I am married and I have a child, so my participation in Myspace is always going to be as an outsider. I setup a MySpace account so I can look around, though I can never really dive into the MySpace community, because I am too old and too subtle. I certainly participate where I think the participation will be useful, because the service is for people like me. On the other hand, I don't kid myself going to teenage communities, other than observing as an outsider.

Q: What do you teach at NYU? Do you see your new students already trained in crowdsourcing?

A: I teach social networking — the overlap of social and human networks. I don't make that distinction; this is what I have been trying to say about the definition. I don't believe that crowdsourcing is principally about teamwork. Most of the examples of crowdsourcing are finding individuals to contribute either in aggregate or solo. Crowdsourcing isn't a replacement for teamwork.

I don't see any students coming in and saying, "I want my work to be appropriated by someone paying me remotely." They all come in at tunes to social networking, to the possibility of virtual collaboration. They all come in and do team work. I think that doesn't incline them to crowdsourcing as way of making a living.

In as much as they are founding projects that are meant to engage other people, they are doing it as open sourced projects, where there is no money involved, as opposed to crowdsourced projects.

Q: When is your next book coming out?

A: The book's coming out February 2008. It’s called "Here Comes Everybody," about social networks.

(Edited by David Cohn)