What Exists Beyond the Seen Crowd

Taking note: the wisdom of invisible crowds

Becky Carroll interviews Jack Jia, founder and CEO of Baynote Inc, in person on May 16, 2007.

Jack is a founder and CEO of Baynote, Inc. For eight years, he was SVP & CTO of Interwoven Inc. with executive responsibilities in engineering, products, marketing, strategy and vision. Prior to Interwoven, he was a founder and CEO of V-max America. Jack led operating systems and applications development at SGI, Sun Microsystems, Stratus and NASA for over a decade. He is a frequent speaker at major conferences and has appeared on television programs in several countries. He is a contributing author in "XML Handbook, the 4th Edition," "Online! The Book," "Content Management Bible," and writes regularly about key technology issues and trends. He is a board advisor for Santa Clara University, and the president of HYSTA, a premier non-profit organization for promoting entrepreneurship.

Becky talked with Jack about crowdsourcing information on websites. Customers are often reluctant to give feedback on the usefulness of website information, thus creating a challenge for web teams. Customers are even less likely to “rank” or “tag” useful items. This interview tackles this subject as well as using the “wisdom of invisible crowds^TM” to grasp the long tail.

Becky Carroll: What do you think is new about crowdsourcing and where do you see it going?

Jack Jia: I think crowdsourcing, up to this point, you can define it as the first generation of Web 2.0, as being very explicit crowdsourcing, which is really forums, blogs, and people coming to write things. That has been very useful in connecting a lot of people in the world together. At least from our end goal, the biggest missing piece is what we call "the invisible crowd." These are the people behind the website, the ones who don’t have time to write blogs, who don’t have time to write comments.

Q: The lurkers?

A: Yes, the lurkers. I like that phrase! They are 80-90% of the population on the web. They know a lot, and if we can have them participate, with or without their permission, that would be the biggest thing for the future. I think a lot of the leading thinkers are starting to talk about this; even people such as Tim O’Reilly are starting to talk about it. It’s not about blogs, it’s not about forums, and although he didn’t use the term “invisible crowd,” he did say it is about everyone who is using your site.

Q: How do you get these people to start to be part of the conversation and participate?

A: That is the hardest part. If you study psychology, humans are self-motivated. There has to be something in it for us. What that is becomes very critical. Unless you can line up their incentive with your site’s incentive, whatever outcome you are expecting is not going to be very ideal. There are several ways to do it. One is to make them part of the game. There are sites that use the wisdom of crowds to predict anything from stock prices to sports games. The crowds actually participate with monetary incentives. They can win if they predict correctly. There are tons of examples where a non-incentivized web prediction site does OK, maybe not so well. If you put money behind it for these people, however, you can predict elections, economic growth, stock prices, everything. The crowds seem to really, really know.

Q: But you say this is more the case when there is money behind it. Why is that?

A: Money is one way to incentivize people. This is not always the case, but some of these prediction sites are using experimental ways of testing out how well incentives really work. Most people won’t participate in those kinds of game-like sites. For the real business world website, there are other incentives you can find. For an e-commerce and marketing sites, when a person comes to your site, they have a clear goal. If someone comes to your site, and says they are shopping for a refrigerator, the incentive for them is to find the best refrigerator and buy it. You can watch their behavior and figure out where they are having success and failure on the site, without having them actually answer questions (they don’t have time to answer questions).

Q: It is more of an irritant to have to answer questions.

A: That’s right. There are well-known studies by psychologists that show when people know they are being watched, they behave differently. They will say things they wouldn’t really do because there is no consequence for them to say it. They want to look good, they want to pretend. They try to guess what other people might do, which is not always correct, so you have this extreme survey bias. Survey bias can be so severe that if you take this as your feedback result, it can sometimes give you worse feedback than if you didn’t have any at all. Really understanding the true intent of the community and sourcing them through their action, then bringing that wisdom back to the site can truly to help an online effort to succeed.

Q: If we take say, a wiki, why do you think some people are willing to participate in this for free rather than getting paid?

A: Wikis are examples of where there are enough people in the world who have enough time or alternative incentives for them to spend time writing something. There are people who have time and would like to write, there are people who want to correct something written by someone else! Negative emotions can sometimes be more powerful than positive, so if someone is dissatisfied, they will do something about it. There are also enough altruistic people out there who just want a better world, and that is incentive enough for them. In the isolated individual business site, you simply don’t have enough of those people to give you the wiki effect.

Q: The people who are more likely to respond to a site are those who maybe have extra time on their hands or who are very opinionated?

A: Yes, I think it does. It works beautifully, even in the wiki context where the incentive is somewhat fuzzy. In a wiki, you know the writing of each article may not be fully correct, but you have to read enough different sources on the web so you can create the wisdom of crowds effect. If I read ten wiki articles, and ten people are saying something I can trust in the aggregate of those ten articles.

Q: Tell me about the wisdom of "invisible crowds" and how that is different.

A: The wisdom of invisible crowds speaks to the notion that there are people behind every website. You have already built this community; they are just invisible to each other. It doesn’t matter if your site has hundreds of people who come to read it every day or every month or if there are millions. That is a big, big group of people.

Today, they are doing something which is very unhuman. They are visiting the site, exploring it on their own, and they don’t see a single soul, don’t see any other human traces. Whatever they do, whether successful or unsuccessful, the moment they leave that site that information or knowledge is gone. Human society would not be here if we had to work that way in the rest of our lives. We have to learn from our past mistakes and successes and pass that knowledge on from generation to generation. We can do that with the invisible crowd who possesses the knowledge about that particular business site. If you can tap into that, then you can effectively crowdsource everybody. That is your customer, typically. It is the most valuable asset you have: your customers, prospects, or employees if it is an intranet site.

Q: Tell me more about how that works, how you can take the wisdom of the "invisible crowd" and use it.

A: Basically, for any website, there are a large amount of people on the site. Just recognizing there are a large number of people doesn’t really help much. You have to watch their behaviors from their mouse actions and navigations and searches they do; this gives you a clear sense of whether they are happy or not happy. You just have to know where to look. To create the wisdom of crowds effect, you have to find like-minded peer groups. Individually, they can all be random. At any given moment, we look it could be random. We can try to measure the time spent on a given page and say, hey more time is good, but it could be they have just gone to get a cup of coffee. Is that good or bad? We don’t know. But, through the wisdom of crowds, with like-minded peers having similar interests, the signal of something good for that group of people will emerge (called emergent behavior by the psychologists). That is a powerful thing.

You can find these unique like-minded peer groups; all sites have their communities, their sub-communities, their sub-sub-communities. There are literally thousands of self-segments within a site, and they all have people behind them. Some of the segments have a small number of people (five, seven, ten people); the smallest segment we get wisdom from is seven. The largest crowds can be thousands of people. They start to create these like-minded peer recommendations. Effectively, they start to recommend the long-tail products and site content, which is Chris Anderson’s theory and that’s how you make your business more profitable and how you differentiate it from your competitors. You can actually have products and content that people love, instead of products or content they are “just OK” with.

Q: You’ve been doing this project for awhile now. I’d like to find out what surprised you most about digging into the wisdom of invisible crowds.

A: The crowds really, really know! I have been trained to trust the “experts,” and I am changing even the way we practice our own business. Peer learning becomes actually far more important for anything we do, whether it is marketing, sales, engineering, or operations. We do a lot of peer learning. We found that through our customer site as well. We actually purposefully picked a few sites we knew fairly well initially. We were “the experts” for this company as former employees. We knew some of the domains very, very well. The crowds completely surprised us. Not only in finding the best content, but also in finding the diamond in the rough. They were somehow able to surface the diamond. And they ranked the diamonds in their value quality, the most pure diamond on the top, the next with a little yellow in it further down the list, etc. I can’t even do that using experts and committees to help direct the “best content.” Once I see it, I can certainly see that it makes perfect sense, but I never would have dreamed of being able to get it done. It was ranked well on all dimensions of information, not just on one dimension. You can put a lot of money in and get one thing right, hopefully, through a lot of interaction with your customers. In this case, you can just do it. You don’t even need a focus group or any of that, because my entire community is the focus group. There is no sample error, either. That is truly the surprise.

They also connect things, to a certain extent, that we would not have guessed they would have known. They connect words that have normally a very distant connection. For example, on the Interwoven site, people do a search and type in “web content management.” The result would show the product TeamSite. How would they know the web content management is TeamSite? They actually created a new set of vocabulary words. Some people will misspell the product name, “Team Site,” with a space in the middle. There are several products with that problem, for example, OpenDeploy is often typed in as “Open Deploy.” The crowd just connects the dots. “Oh, Open Deploy is OpenDeploy.”

Q: They don’t really care about the branding, do they?

A: Right. It’s kind of just understanding the local slang and watching how new vocabularies emerge. It’s like watching how people talk around water coolers, making notes and saying, “Oh, that’s what they mean!” Experts have certain notions of how we define certain products, how we label it, certain jargon, but the community may or may not know that or respect that. Turn that around and reflect what the community is saying, and it will basically increase the conversion rate for any online businesses.

Q: Give me the 30-second view, the high-level view of how Baynote does that.

A: Baynote does that by watching every single visitor’s behavior on a site or a set of sites. You put five lines of html tag into your site template page; we call it the Observer, that’s our sensor, our eyes and ears on a particular site. Drop that into a template page header or footer and then basically, every single page has a watchdog to silently watch every single movement of a person. We don’t know that person’s name, we don’t know their private information, and we don’t care about their private information. All we know is this person has found a certain set of content for certain reasons and in context. We compare it with other people who have similar fingerprints, as we call it, or similar signatures. We group them into like-minded peer groups, also called visitor clubs. Based on that, we project content back, in real-time, for the site’s search and navigation. We will make recommendations, such as people who liked this content also read these content pages. Or, next step, if you liked this, what would you do next? People who bought this have also considered these products or these accessories. People who read this (support) article found these other articles useful in solving their problem.

Q: By watching the crowd, by watching where they are going and the path they are taking to get there, both backwards and forwards, you can figure out, for someone who has similar behaviors…

A: Effectively, people are voting it or tagging it through their actions, which is really far more accurate than trusting in what they are saying. Another metaphor we use quite a lot is thinking about your website as a mountain. Before anyone steps foot on the mountain, it looks pretty even everywhere, there are no trails yet, no signs anywhere. When communities start climbing the mountain, by traversing the site, and people start to discover where the peak is, where to find the watering hole, where to find the treasure…

Q: An easier way up?

A: Yes, short cuts. You may tell them to “go here,” but the community finds the short cuts. They will reinforce it, the trail starts to emerge, and then we will put up the road signs based on the community (here for the watering hole, here for the peak), and the trail emerges by itself.

Read More about Baynote and the wisdom of invisible crowds

The Baynote Blog
Social Search: Can Baynote Hit A Chord With B2B Marketers?, Forrester's Marketing Blog.
Why understanding your right brain will help you succeed., by Jack Jia.

(Edited by David Cohn)