Popular keywords: Industry, IKEA effect, Team Dynamics, Project Outcome
About the Project
This longitudinal project from Harvard University tracked common cognitive decision-making shortcuts and factors that supported versus inhibited progress on 300+ professional team-based innovative projects. The full dataset is comprised of interviews and observations of over 150 creative professionals. It includes projects from 176 companies, 23 industries, 8 countries, and 5 continents. This site makes 232 of these projects explorable.
Creative Collaboration Study
These professional teams hail from the most innovative companies in the world. Half of their projects succeeded and half of them failed. What do you think differentiates the top and bottom performing teams? Dive in to see what challenges they faced, and what you can learn about how they dealt with them.
sample cognitive decision-making heuristics tags
There are over 180 known cognitive shortcuts (decision-making biases and heuristics). We narrowed that list down to 86 biases that are most relevant to team based creative projects. Of these, only 65 of these showed up in our data with at least one tag.
sample of Projects
About The Process
Projects were categorized as successful or unsuccessful, based on the innovativeness of the idea. Successfully innovative projects were defined similarly to a patent definition, in that they: a) solve a problem, and b) do so in an original way. Success was judged from the perspective of the people who worked on the project, as well as multiple expert raters.
Metadata tags make it possible to compare teams with similar characteristics and see what patterns of biases and common pitfalls occur most frequently. We identified 25 metadata categories. Example metadata tags include team size, industry, time pressure, budget pressure, team dynamics, number of ideas generated, etc. Metadata categories can be used to compare tags associated with projects in the same industry or what types of tags show up when budget was an issue on a team.
Tags were created using academic research on team-based innovation and creativity. These include individual traits and team-level factors that have been previously linked to innovation success and failure, as well as cognitive decision-making biases that the principal investigator hypothesized would affect the process and outcome of innovation projects.
Data collected from interviews and observations were transcribed, translated, and cleaned in preparation for ethnographic coding (similar to tagging). Tagging qualitative data for meaning is a time-consuming process. All of the tagging was done by human raters using Dovetail App software that is designed to help discover patterns in qualitative data. When tagging qualitative interview data, each transcript was read in its entirety twice by each reviewer. In the first round, the reviewer identified anything they found obviously matched a tag and marked other text to revisit later that seemed important but not yet clear. In the second tagging round, each reviewer resolved any question mark tags and checked for any errors in the original tags. After each rater completed their tagging, we used inter-rater reliability to test the degree to which independent raters agreed on the meaning of the text. The raters showed very high (80%) tagging agreement, which gives us confidence that human tagging was objective enough for our results to be valid.
There are over 180 known cognitive cognitive (decision-making biases and heuristics). We narrowed that list down to 86 biases to track as highly relevant to team based creative projects. Only 65 of these showed up with at least one tag.
Example Metadata Filters for Team Characteristics
- General mood within team: The filter allows you see what tags appear for the different moods within the teams. Team mood refers to the team at the agency (not the client team) and attempts to capture the overall mood throughout the project rather than a particular incident.
- Leadership signals (team): The filter allows you see what tags appear for different types of leadership signals on the team on the creative team side (how did the interviewee interpret the internal leadership). You are also able to filter by the number of design team decision-makers.
- Leadership signals (client): The filter allows you see what tags appear for different types of leadership signals on the client side (e.g., how did the interviewee interpret the the client leadership?). You are also able to filter by the number of client decision-makers.
- External influences: This filter allows you to explore whether and how external people influenced the project, like friends and family, celebrity examples, investors, or suppliers.
- Product motivation (client): This filter is for the core motivation for creating the product. This can be one of the following: 1) Technology: Client has a technology and want it to take off in a market (google glass is cool tech… let’s make people want it!), 2) User: we see a real need from our data that is unmet… let’s design a product to meet the need (similar to human centered design), 3) Artistic: they want to explore a concept or make a point or have creative fun (I love Alexa… what would Alexa be like in an old rotary phone… I want to make that because I just want to… maybe people will like it), 4) Ideological: I want to make a point to the world. (Fashion should use more sustainable materials and be less wasteful. Other people will definitely care and buy my high price item that is green and has good social impact)
- Personal motivation (client): This filter is for the core personal motivation for the project. It can be: 1) Intrinsic: I love making this even in my free time without pay or acknowledgement (a lot of pro-bono work is here), 2) Extrinsic: This project better get awards and money and fame or I wouldn’t be here, or 3) Collaborative: I’m here because I love working with this team. We could stop making robots and start making purple toilet paper and I might still stay because my colleagues are so great.”
- Product motivation (design team): This filter has the same options as the client version.
- Personal motivation (team): This filter has the same options as the client version.
- Client Risk Tolerance: Risk tolerance refers to the level of risk the client was comfortable with during the project.
- Number of decision makers (client): This filter allows you look at teams with varying number of decision makers on the client side.
- Client size :This filter allows you to see what tags appear with different client sizes.
- Client type :This filter allows you to see what tags appear with different client types.
- Was time an issue:This filter allows you to see how time issues impact a project. Time in this context means the project timeline.
- Was budget an issue:This filter allows you to see how budget issues impact a project.
- Number of decision makers (team) :This filter allows you look at teams with varying number of decision makers on the team.
- Product industry: This filter allows you look at projects in different industry areas.
- Team size: This filter allows you to sort by different team sizes.
- Personal relationships within the team: Use this filter filter allows you to see what tags appear with various types of personal relationships on teams.
- Gender :This filter allows you to explore how the tags change for different gender’s of the interviewees.
- Team dynamics:This filter allows you to explore the different tags that show up based on how the interviewee described the team dynamics.
- Project outcome :This filter shows the three different types of outcomes the interviewees discussed. Outcomes refer to the degree to which the team finished something that qualified as both novel and useful. Outcomes include: successful, unsuccessful, and control (where the outcome was unknown at the time of data collection).
- Additional project outcomes: Products are only one of several project outcomes. Others include: process learning, reputation, improved relationships between those involved, domain knowledge, mentor relationships, and personal motivation.
- Innovation team norms filter: This filter summarizes common norms for innovative teamwork like risk taking, psychological safety, mutual openness to ideas, trust, shared commitment, active management of uncertainty, and functional diversity (diversely skilled team members).
Example Researcher-Assigned Tags
Core Creative Collaboration Skills
- Creative Skills
- More and Less Effective Conflict Skills
- Resilience Skills
- More and Less Supportive Leadership Skills
Common Team Challenges:
- Absence of Diverse Skills
- Alignment Problems
- Communication Issues
- Cultural Differences
- Finding Out Too Late Their Idea is Not New
- Indecisive Leadership
- Insufficient Feedback
- Internal Personnel Changes
- Irreconcilable Differences
- Lack of Challenging Work
- Lack of a Genuine Innovation Mandate
- Lack of Organizational Support
- Lack of Resources
- Lack of Trust
- Language Barriers
- Scope Creep
- Selfish Motivations
- Unbalanced Workload
- Vague Roles
- Vague Goals
- Unresolved Personal Conflicts
Decision Shortcut Tendencies
- Implicit stereotypes : Unconscious attribution of particular qualities to a member of a certain social group. Implicit stereotypes are influenced by experience, and are based on learned associations between various qualities and social categories, including race or gender.
- Peak-end rule : Psychological heuristic in which people judge an experience largely based on how they felt at its peak (i.e., its most intense point) and at its end, rather than based on the total sum or average of every moment of the experience.
- Anchoring: Relies too heavily on one trait or piece of information when making decisions
- Confirmation bias: Interprets new evidence as confirmation of one’s existing beliefs or theories
- Conservatism: The tendency to revise one’s belief insufficiently when presented with new evidence.
- Empathy gap : The tendency to underestimate the influence or strength of feelings, in either oneself or others.
- Focusing effect: Occurs when people place too much importance on one aspect of an event, causing an error in accurately predicting the utility of a future outcome.
- Humor effect: Humorous material tends to be recalled at higher rates than non-humorous material
- Negativity bias : Your brain is simply built with a greater sensitivity to unpleasant news.
- Ostrich effect: The Ostrich Effect is the tendency to ignore a dangerous or risky situation. This bias takes its name from the widely held, though completely incorrect, belief that an ostrich will bury its head in the sand when faced with danger.
- Self-relevance effect: The tendency for individuals to have better memory for information that relates to oneself in comparison to material that has less personal relevance.
- Subjective validation : A person will consider a statement or another piece of information to be correct if it has any personal meaning or significance to them.
- Better than average: Overestimating one’s desirable qualities, and underestimating undesirable qualities, relative to other people
- Ideas speak for themselves : Overestimate the extent to which one’s opinions, beliefs, preferences, values, and habits are more normal and typical of those of others, aka false consensus bias
- Inexperience: Failure to effectively understand context, problem
- Perfectionism: Sets excessively high performance standards, accompanied by overly critical self-evaluations and concerns regarding others’ evaluations
- Premature idea evaluation: Reactionary negative response that focuses on flaws of idea
- Romanticized Notion of team: Overestimates team productivity and efficiency, belief in egalitarian team roles, belief that everyone needs to be friends (and thus avoid conflict)
- Anecdotal fallacy : Using personal experience or an isolated example instead of a valid argument, especially to dismiss statistics.
- Authority bias : The tendency to attribute greater accuracy to the opinion of an authority figure (unrelated to its content) and be more influenced by that opinion. The Milgram experiment in 1961 was the classic experiment that established its existence.
- Declinism: The belief that a society or institution is tending towards decline. Particularly, it is the predisposition, due to cognitive bias, particularly rosy retrospection, to view the past favourably and future negatively.
- Group attribution error: The biased belief that the characteristics of an individual group member are reflective of the group as a whole or the tendency to assume that group decision outcomes reflect the preferences of group members, even when information is available that clearly suggests otherwise.
- Halo effect : The tendency for a person’s positive or negative traits to “spill over” from one personality area to another in others’ perceptions of them
- Hindsight bias : The inclination, after an event has occurred, to see the event as having been predictable, despite there having been little or no objective basis for predicting it.
- In-group bias : The tendency to favor one’s own group.
Pessimism bias : Pessimism bias is an effect in which people exaggerate the likelihood that negative things will happen to them. It contrasts with optimism bias.
- Planning fallacy: Underestimates the time needed to complete a project or task
- Pro-innovation bias: The belief that an innovation should be adopted by whole society without the need of its alteration.
- Reactive devaluation : A cognitive bias that occurs when a proposal is devalued if it appears to originate from an antagonist.
- Rosy retrospection : the psychological phenomenon of people sometimes judging the past disproportionately more positively than they judge the present.
- Appeal to novelty : A fallacy in which one prematurely claims that an idea or proposal is correct or superior, exclusively because it is new and modern.
- Effort justification : People’s tendency to attribute a greater value (greater than the objective value) to an outcome they had to put effort into acquiring or achieving. (an idea and paradigm in social psychology stemming from Leon Festinger’s theory of cognitive dissonance)
- False consensus effect: An attributional type of cognitive bias whereby people tend to overestimate the extent to which their opinions, beliefs, preferences, values, and habits are normal and typical of those of others (i.e., that others also think the same way that they do).
- Identifiable victim effect : The tendency of individuals to offer greater aid when a specific, identifiable person (“victim”) is observed under hardship, as compared to a large, vaguely defined group with the same need. The effect is also observed when subjects administer punishment rather than reward.
- Ikea effect: The tendency for people to place a disproportionately high value on objects that they partially assembled themselves, such as furniture from IKEA, regardless of the quality of the end result.
- Illusion of control : The tendency for people to overestimate their ability to control events; for example, it occurs when someone feels a sense of control over outcomes that they demonstrably do not influence. It is thought to influence gambling behavior and belief in the paranormal.
- Illusory superiority : A cognitive bias whereby a person overestimates his or her own qualities and abilities, in relation to the same qualities and abilities of other persons.
- Irrational escalation : A situation in which people can make irrational decisions based upon rational decisions in the past or to justify actions already taken.
- Law of the instrument: An over-reliance on a familiar tool or methods, ignoring or undervaluing alternative approaches. “If all you have is a hammer, everything looks like a nail.”
- Loss aversion : More motivated to avoid bad self-definitions or consequences than to pursue good ones
Overconfidence bias: Person’s subjective confidence in his or her judgments is reliably greater than the objective accuracy of those judgments
Reactance : A motivational reaction to offers, persons, rules, or regulations that threaten or eliminate specific behavioral freedoms.
- Reactance occurs when a person feels that someone or something is taking away his or her choices or limiting the range of alternatives.
- Risk compensation : A theory which suggests that people typically adjust their behavior in response to the perceived level of risk, becoming more careful where they sense greater risk and less careful if they feel more protected.
- Status quo bias : An emotional bias; a preference for the current state of affairs. The current baseline (or status quo) is taken as a reference point, and any change from that baseline is perceived as a loss.
- Sunk cost fallacy : Reasoning that further investment is warranted on the fact that the resources already invested will be lost otherwise, not taking into consideration the overall losses involved in the further investment.
- Zero-risk bias : Zero-risk bias is a tendency to prefer the complete elimination of a risk even when alternative options produce a greater reduction in risk (overall).
Who We Are
Dr. Beth Altringer, Harvard University (earlier stage of research) and now Brown University & Rhode Island School of Design
Role: Principal Investigator, Study Design, Data Collection, Quantitative Data Analysis, Qualitative Data Analysis, Web Development
Role: Collaborator, Data Visualization Graphic
Laurie Delaney, Harvard University
Role: Research Assistant, Qualitative Data Tagging and Cleaning, Qualitative Data Analysis
Jared Meyers, Harvard University
Role: Research Assistant, Qualitative Data Tagging
The ultimate planning solution for busy women who want to reach their personal goals.
Apply your learning with real-world projects and test your knowledge with tailor-made quizzes.
From building websites to analyzing data, not sure where to start?
No matter your experience level, you’ll writing real, working code in minutes.
Your code is tested as soon as you submit it know if you’re on the right track.
Learn everything you need to take your career to the next level.
Because learning to code might be the easiest way to change your career.
Form our Blog
Apply your learning with real-world projects and test your knowledge with tailor-made quizzes.