Crowdsourcing – MarcioSaito.com

The Emergence of Collective Intelligence

The whole is above and beyond the sum of its parts. ~Aristotle

(this article originally written for Ledface Blog)

When we observe large schools of fish swimming, we might wonder who is choreographing that complex and sophisticated dance, in which thousands of individuals move in harmony as if they knew exactly what to do to produce the collective spectacle.

So, what is “Emergence”?

School of fishes dancing is an example of “emergence”, a process where new properties, behaviors, or complex patterns results of relatively simple rules and interactions.

One can see emergence as some magic phenomena or just as a surprising result caused by the current inability of our reductionist mind to understand complex patterns. Whichever way we think, examples of emerging behaviors are abundant in nature, science, and society and are just a fact of life.

Humans can do it too

We humans have even built artificial environments that allow for collective intelligence to express itself. For example, in global financial markets, we are able to track and analyze millions of variables (e.g. weather patterns in India, political tensions in the middle-east) to price thousands of commodities and more complex financial instruments in real-time.

Each and every actor in the financial markets has no significant control over or awareness of its inputs. They are just trying to follow a very simple set of rules: buy low, sell high, maximize profits. Regulators try to enforce the rules to keep the game fair, but there is no central coordination in the computation of prices. Commodity prices results from the interaction among investors, not directly from their individual knowledge.

As any economist will explain, the financial market works best when there are enough participants so that no individual has meaningful influence over the aggregate result, everyone plays by simple transparent rules, there is low friction in the interactions, and each participant is autonomous and responsible for its participation.

Can we transpose it to other domains?

When it comes to other areas of human knowledge, because spoken or written language has not been a tool as perfect as buy/sell in an exchange or the intuitive reactions of a fish observing its school mates, we have resorted to structure to create productive patterns (as in a group of soldiers marching at the beat of a drum).

Digital Media is still far from frictionless, but it getting close to allow millions of people to interact more or less freely. Environments like Wikipedia and primitive Q&A sites are the initial samples of that.

Nobody can single-handedly create “collective intelligence”. But Ledface is building an environment that empowers individual knowledge and aggregates it (like Wikipedia), but also provides the mechanisms for interactive co-creation (differently from other Q&A sites) so that, maybe, the conditions are right for the emergence of true collective intelligence.

Too remote of a possibility? If fishes and economists can do it…

Crowdsourcing, Social Media

Crowdsourcing: The Next Step in Enterprise Collaboration

[This post originally written for the Ledface Blog]

Enterprise Collaboration Tools can improve the productivity of organizations by reducing the friction in the flow of information. This post argues that the application of Crowdsourcing techniques in that context is the next step in blending the best of traditional functional segmentation with the benefits of open collaboration.

The Organizational Intelligence Problem

Most companies today are structured as a hierarchy segmented in business units, then in specialized functional areas (sales, operations, engineering, marketing etc), then into recursively smaller groups responsible for a portion of the big problem. For every single issue that emerges, there is an unequivocal “expert” in the organization, an individual who is accountable and (supposedly) knowledgeable to solve it.

Here are two common challenges:

Analysis imperfection. Real-life problems often cannot be broken perfectly along designed functional lines. They require multiple people to work together and inter-dependently to solve them, which goes against the personal accountability paradigm.
Expert fallibility. While it is generally good to have a person assigned as the primary responsible for issues in a specific area, nobody can be expected to know everything, even if it is in a very narrow domain of knowledge.

Cross-functional project management, collaboration tools, external consultants have been applied to address those challenges. But there is more help on the way.

Social Collaboration Tools and their limitations

Enterprise social collaboration tools employ some of the same mechanisms brought to us by consumer social network platforms and have been successfully introduced in large commercial enterprises to bring a collaboration overlay to current organization.

Beyond the file sharing systems of the past, they let interaction happen directly, in real-time, across functional lines. Information flows with less friction and interactions are more spontaneous.

But they also create two new potential problems:

Low signal/noise ratio – In an extreme structured environment, you are afraid people don’t have the information they need to do the best for the organization. In a totally social environment, you worry about people being overwhelmed with too much irrelevant data.
Ego – If my value to the organization is determined exclusively by my expertise on a narrow subject, I might be reluctant to seek help from non-experts who might have the answer to a specific problem. We need to create environments where that informal collaboration can happen without threatening the position of the domain expert.

Crowdsourcing: The Next Step in Enterprise Collaboration

Trying to use social communication tools to solve the entire problem of cross-functional collaboration is akin to trying to se email as the single technology to implement structured business processes.

Crowdsourcing is usually thought of as a way for companies to get some work sent outside its boundaries. But the same set of technologies enabling it can be applied internally to an organization to greatly improve the signal/noise ratio in social media by intelligently routing information from and to the right people. It also lets each person filter what they want to see, focusing research of the collective intelligence.

Crowdsourcing in the context of a hierarchical organization can also reduce the threat the domain expert feels when he/she shares problems and seeks help from the internal or external community. It can do that by facilitating collaboration among peers without specifically assigning individual credit for the co-created solution.

Technology providers such as Ledface, are creating environments where social collaboration generates less noise and co-creation happens without ego, promising a brighter future for enterprise collaboration.

Crowdsourcing, Social Media

Gamification for Kids, Collaboration for Grown-Ups

[This post originally written for publication in the Ledface Blog]

Next time you are at a cocktail party, take a step back and observe. What do people talk about? How conversations evolve?

When people meet for the first time, they exchange facts. They look for coincidences, they try to establish connections. “Did you grow up around here?” “How did you meet this group?”

Then, they engage in social gaming, trying to position themselves within the group, competing for attention and admiration. “Last summer, we went to Hawaii.” “I developed this software at work that…”

Once Trust is in place, people share effortlessly

You then see groups breaking into smaller ones as people find their circles and conversation deepens. “I should introduce to this person, I think she will be able to help you on this.” “Can I ask you about your trip? We are going to Hawaii next month.”

Social relationships in the real world are built on identification and trust. Once relationships are in place, we are wired to share information, collaborate, and help each other.

Do we help others expecting reciprocity?

It is easy to rationalize that we help others to get help from others through reciprocity, that living in society is a game of giving and receiving. But that is only a typical analytical inversion of cause-effect.

It might be true that reciprocity is at the very core of life in society, but we don’t trust, share and collaborate to get something back. Living in society is more efficient and effective if we collaborate, so we evolved (in the Darwinian sense) that way over generations.

Social Media let natural collaboration instincts flourish

Social Media is “social” simply because it can better support some of the social behaviors we display in real life. Classical media (print, broadcast, etc) force us to be analytical, linear, introspective, independent.

Being more open and symmetrical, social media let people establish relationships and trust. As the medium evolves, we should expect that people exercise the natural pleasure in sharing knowledge and helping others as they do in real life.

Ask anyone about their passions and they will share knowledge, dispense advice, and talk about it for hours for the pleasure of an audience, without the expectation of tangible reward.

Look at Facebook or Wikipedia. Why do people share information with others? There are no extrinsic incentives to do so. The same way we search for connections, talk about ourselves, share knowledge offline, people are social in Facebook because computer networks created an environment that emulates the social environment.

Ledface and the essence of Crowdsourcing

When I talk to people about Ledface or other Crowdsourcing projects, a common question is “Why would people want to help others? What is the reward? Is there a gamification mechanism?”.

Our analytical mind believes people only move with a tangible rewards or some sort of ego game playing. But the essence of Crowdsourcing is exactly to set conditions that free people from the industrial revolution training (analyze, segment, create independence and accountability, earn money doing something you dislike so you can spend time doing what you like) and express the natural instincts of collaboration.

Gamification is important, Collaboration is the name of the game

Should crowdsourcing tools and environments use gamification and consider the use of extrinsic rewards? Sure. At a cocktail party, before trust and collaboration develops, you must create a favorable setting and let people find each other. We all do play social games in society, there is no reason it wouldn’t be useful in virtual places.

But that is just how things start. Once relationships grow up, collaboration is the name of the game.

—

Marcio Saito’s (@Marcio_Saito) interest in Collaboration and Co-Creation originates in his early involvement with the Open Source Software community in the early 90′s. He writes about Social Media and Collective Intelligence and is a co-founder and advisor to Ledface, a startup using Crowdsourcing to create a new kind of Intelligence.

Crowdsourcing, Social Media, Uncategorized

Crowdsourcing, Freedom, and Anonymity

[This post originally written for publication in the Ledface Blog]

The Wisdom of the Crowds

When James Surowiecki published “The Wisdom of Crowds” in 2004, he presented several experiments and anecdotes to note that a diverse collection of independently-deciding individuals is likely to make certain types of decisions and predictions better than individuals or even experts.

It is not difficult to miss the importance of the word “independently” in the sentence above. This post reflects on the need of trust and freedom to create the conditions where individuals can independently contribute to a co-creation process.

Collective behavior is not Collective Knowledge

I have a lot of interest in politics, influence, social media, behavioral economics and other areas of study on how people can influence each other through organizational structures and create patterns of collective behavior. But “herd behavior” is not necessarily a good expression of individual or collective choices.

We buy and sell stocks at the wrong times based on headlines. We spend the week working on things we don’t enjoy so we can earn the weekend to do what we love. We adopt certain behaviors because that is just the proper thing to do. What we decide, buy, do, or choose is influenced by our social context and that is just a fact, neither good or bad.

Governments, companies, universities, armies and other modern organizations have become very good at creating environments that uses social factors, ego and extrinsic rewards to express organizational patterns, where successful leaders can use power/financial/reputation structures to get groups of people acting in a coordinated fashion.

Freedom and decentralization is requisite for Crowdsourcing

Crowdsourcing is not simply breaking work in pieces for efficiency. It is not organizing and managing people to achieve a stated goal.

Crowdsourcing is about harnessing the human potential in every individual so that we can aggregate and express knowledge that is larger than what any individual can express.

To achieve that, we need to create a trusted environment where people can express their independent opinions. We need to remove the social, organizational, financial pressures to create true freedom of expression for every individual.

The Ledface Challenge

In an ideal co-creation environment, anonymity is one way of creating that freedom of expression. But it also comes with a challenge: anonymity removes ego from the co-creation process (which is a typical reward for social behavior), requiring the system to offer some other intrinsic rewards for its participants.

This is the challenge the startup Ledface is tackling to create a crowdsource-based personal assistant.

Final Thoughts

Ego (need for personal recognition) and extrinsic rewards (do something you don’t like to earn freedom elsewhere) are two proven driving forces for action, but relying on them will earn you “survival of the fittest” results, not collective intelligence.

While we recognize the value of those systems, we believe that, to achieve true crowdsourcing results, we need to build an environment where co-creation happens with individual freedom, independence, and is driven entirely by intrinsic rewards.

—

Marcio Saito’s (@Marcio_Saito) interest in Collaboration and Co-Creation originates in his early involvement with the Open Source Software community in the early 90′s. He writes about Social Media and Collective Intelligence and is a co-founder and advisor toLedface, a startup using Crowdsourcing to create a new kind of Intelligence.

Crowdsourcing, Social Media, Uncategorized

Co-Creation: When the Crowd goes Beyond the Experts

[This post originally written for and published at Ledface’s Blog]

Historically, most organizations in our society (governments, armies, companies) structure themselves in a hierarchy, with knowledge and power concentrated at the top and segmented visibility of smaller decomposed tasks distributed along its base.

Crowdsourcing technology is enabling less structured and specialized organizations to solve complex knowledge problems competitively. It is not a cheap way to outsource work to volunteers. When done right, it is an entire new form of knowledge production with the potential to revolutionize our approach to solving simple and complex problems.

Knowledge has a long tail

The Open Source Software Community and the volunteers of Wikipedia show that in at least some knowledge domains, unstructured crowds of regular people produce results that are competitive with hierarchical organization of professional experts (professional software teams, the Phd’s of Encyclopedia Britannica).

To understand how that happens, let’s look at how knowledge distributes among a population (picture above). “Long Tail” refers to a statistical property of a distribution where its “tail” is larger than its “head”.

The group at the head of the curve (“experts”) accumulates personal knowledge that is individually higher than the average person. But the total knowledge held by the experts is still relatively small compared to the knowledge held by the broader population.

Our current organizations are very efficient expressing the knowledge of experts at the top of the pyramid or at the head of segmented functional groups, but it fails to capture the knowledge at the base. There are good reasons why they evolved that way…

The Medium is the Message

The history of human civilization is tightly connected with our ability to use language to organize groups to tackle complex problems that are beyond the abilities of a single person.

The most effective way to exchange ideas with other people is through rich direct interaction face-to-face. But because it is difficult to have that kind of interaction with a large number of people, most large organizations have relied primarily on the printed word (books, documents, memos, email messages) to collaborate.

The print medium, being unidirectional, is particularly good at transferring knowledge from someone who knows more (“expert”) to a number of people who know less. The predominance of print led to the evolution of highly specialized and hierarchical organizations, where experts accumulate knowledge in narrow functional fields and lead a large group of less knowledgeable people.

In this environment, collective intelligence does not have channels of expression. The voice of the expert is the voice of knowledge.

Crowdsourcing Technology

Digital technologies, the Internet and Social Media are starting to provide a communication medium that emulates some of the characteristics of rich, direct interaction. It is real-time (not linear), it links peer-t0-peer (not hierarchically), and it is interactive.

So, if that is true, why don’t we see the effects of co-creation emerge in the most popular social (Twitter, Facebook, Google+) and Q&A websites (like Quora or Yahoo Answers)?

The reason is that we are still using the new medium as we used the old one. Quora is still dominated by the individual desire of owning the truth and position as an expert. True co-creation requires collaboration, not ego, as the driving force from the process of expressing collective knowledge.

Also, we are already using medium interactivity, but not yet created the mechanisms behind the interaction to allow for true co-creation and collective intelligence to emerge and be expressed. That back-end infrastructure is still emerging in the form of new Crowdsourcing Technology.

Conclusion

In domains being affected earlier by the digital medium, (e.g. software development and publishing), the long tail of knowledge held by the unstructured crowd is now able to express itself in ways that are competitive with hierarchical organizations.

As Crowdsourcing technologies evolve and spread over other domains, more and more complex problems can be tackled using a new form of human organization.

We will be able to solve problems, not by analytically decomposing big problems into smaller ones, but by presenting the complex problem to the collective intelligence and let it holistically express the solution.

—

Marcio Saito’s (@Marcio_Saito) interest in Collaboration and Co-Creation originates in his early involvement with the Open Source Software community in the early 90’s. He writes about Social Media and Collective Intelligence and is an advisor to Ledface, a startup using Crowdsourcing to create a new kind of Intelligence.

Crowdsourcing, Social Media, Uncategorized

Crowdsourcing: Knowledge Has a Long Tail

Why the Crowds will beat the Experts

The history of human civilization is tightly connected with our ability to use language to organize groups of people to tackle complex problems and projects that are beyond the abilities of a single person.

Today, most organizations in our society (governments, armies, companies) structure themselves in a Hierarchical Pyramid, with knowledge and power concentrated at the top and segmented visibility of smaller decomposed tasks distributed along its base.

In this post, we argue this will change, that we will start effectively using less structured and specialized organizations (let’s call it Unstructured Crowd) to solve complex knowledge problems through Crowdsourcing Technology.

Crowdsourcing is not a cheap way to outsource work to volunteers. When done right, it is an entire new form of knowledge production with the potential to revolutionize our approach to solving simple and complex problems.

The Long Tail of Knowledge

The anonymous Open Source Software Community (producing the software that runs the Internet) and the volunteers of Wikipedia (have you consulted the Encyclopedia Britannica lately?) show that in at least some knowledge domains, Unstructured Crowds of regular people can effectively compete with Hierarchical Pyramids of professional experts.

How does that happen?

To understand, let’s look at how knowledge distributes among a population. The picture above shows a Long Tail distribution. “Long Tail” refers to a statistical property of a distribution where its “tail” is larger than in its “head”. This concept was made popular by Chris Anderson in a Wired Magazine article that applied it to the retail business.

If we agree that knowledge exits in a long tail distribution, the group at the head of the curve (“experts”) accumulates personal knowledge that is individually higher than the average person. But the total knowledge held by the experts is still relatively small compared to the knowledge held by the broader population.

The Medium is the Message

Differently from rich face-to-face interaction, the printed word imposes an uni-directional form of communication, where someone who knows something communicates information to someone who knows less, with little opportunity for real-time interaction.

Because we have used primarily books, documents, e-mail messages to accumulate and transfer knowledge, the voice of the expert became the voice of knowledge. We have built a segmented-knowledge society where each of us is specialized on a narrow domain (be it tightening a bolt, writing software, doing tax returns, or defining strategy). In this environment, collective intelligence does not have channels of expression.

But if there were technologies that lower the friction and cost of collaboration and co-creation, there is a point where the long tail of knowledge is tapped to produce concrete results. Digital technologies, the Internet and Social Media are starting to do just that.

The emerging Internet-based social medium emulates some of the characteristics of rich direct interaction. It is real-time (not linear), it links peer-t0-peer (not hierarchically), and it is interactive. It is creating the low-friction conditions for true co-creation to emerge.

So, if that is true, why don’t we see the effects of co-creation emerge in the most popular social (Twitter, Facebook, Google+) and Q&A websites (like Quora)? It is a new medium, but it takes some time for people to fully utilize it and the old model based on individual ego and segmented knowledge to fade.

Conclusion

In domains being affected earlier by he digital medium, (e.g. software development and publishing), the long tail of knowledge held by the Unstructured Crowd is now able to express itself in ways that are competitive with Hierarchical Pyramids.

As Crowdsourcing technologies evolves and the adoption and application of those technologies spread over other domains, more and more complex problems can be tackled using a new form of human organization.

We will be able to solve problems not by analytically decomposing big problems into smaller ones, but by presenting complex problem to the collective intelligence and let it holistically express the solution.

—

Marcio Saito’s (@Marcio_Saito) interest in Collaboration and Co-Creation originates in his early involvement with the Open Source Software community in the early 90’s. He writes about Social Media and Collective Intelligence and is a co-founder and advisor to Ledface, a startup using Crowdsourcing to create a new kind of Intelligence.