Complex adaptive systems and the Cynefin framework

From: jimruttshow8596

Introduction to Dave Snowden

Dave Snowden is the founder and chief scientific officer of Cognitive Edge, with international work covering government and industry related to strategy, organizational design, and decision-making [00:00:20]. He has pioneered a science-based approach, drawing on anthropology, neuroscience, and complex adaptive systems theory [00:00:34]. Snowden is known for his pragmatic cynicism and classic style [00:00:48]. He is also affiliated with the University of Pretoria, Hong Kong Polytechnic University, and the University of Warwick [00:01:05].

Snowden is best known as the inventor of the Cynefin framework, pronounced “can-ev-in” (Welsh) [00:01:16]. His early work at IBM in knowledge management led him to understand that decision support is more complex than simply codifying information into databases [00:02:10]. This led to his work on narrative and complexity theory, and involvement in DARPA programs focusing on weak signal detection and decision-making in complex policy environments [00:02:27].

The Cynefin Framework

The Cynefin framework is based on a fundamental division into three main types of systems: ordered, complex, and chaotic, with phase shifts between them rather than mere gradations [00:08:02]. A central “disorder” domain represents the state of not knowing which system type one is in [00:11:51]. The essence of Cynefin is that context is key [00:12:19], challenging the idea of universal management fads [00:12:25]. It helps decide what context you’re in before choosing a method [00:12:38].

Naïve Newtonianism and Causality

A common mistake in business and government is “naïve Newtonianism,” which assumes that with enough data, the future can be predicted, ignoring concepts like deterministic chaos [00:03:22]. Many still take a linear approach to causality, assuming that if inputs are right, outputs can be defined or forecast [00:03:46]. Complexity theory, distinct from deterministic chaos, is described as the “science for uncertainty” [00:04:09]. It allows understanding the present and mapping coherent pathways from it, but not defining a future outcome [00:04:16].

The Cynefin framework acknowledges that highly predictable Newtonian systems can be created, like traffic management or operating theaters, but they are not universal [00:04:53].

Ordered Systems

Ordered systems are characterized by a very high level of constraint, making everything predictable [00:08:27]. Humans use constraints to create predictability [00:08:37].

1. Obvious (or Simple): In this domain, the relationship between cause and effect is self-evident and understood by everyone [00:08:44]. This is the domain of best practice, where there is a single right way of doing things. The process is to sense, categorize, and respond, operating under rigid constraints [00:08:56].
2. Complicated: Here, cause and effect may be obvious to experts but not to the decision-maker [00:09:05]. It requires investigation and expertise, following a sense, analyze, respond approach [00:09:13]. There is a right answer that can be discovered, possibly within a range of possibilities, leading to “good practice” rather than “best practice” [00:09:22]. An example is a medical practitioner having flexibility in patient decisions [00:09:34].

Over-constraining an ordered system can cause it to break or fragment into chaos [00:10:04].

Complex Systems

A complex system has “enabling constraints,” where everything is interconnected, but the connections are not fully known [00:10:40]. A “dark constraint” concept, analogous to dark energy, describes seeing the impact of something without knowing its source [00:10:51].

In a complex adaptive system, understanding requires probing and experimenting [00:11:04]. Critically, experiments must be run in parallel, especially when evidence supports conflicting hypotheses of action that cannot be resolved within the decision timeframe [00:11:11]. In Cynefin, instead of trying to resolve conflicts, “safe-to-fail micro-experiments” are constructed around each coherent hypothesis and run in parallel, allowing solutions to emerge as dynamics change [00:11:34].

Chaotic Systems

Chaos is defined in social science as the absence of constraints [00:33:05]. While in physics this is a low-energy gradient, in human systems it’s a high-energy gradient because human systems are open and constraints form quickly and naturally [00:33:09]. Chaos is always temporary; one doesn’t want to accidentally fall into it but may deliberately enter it [00:33:01]. To deal with a truly chaotic system, the guidance is to create constraints rapidly [00:34:05].

Disorder

This is the central domain in Cynefin, representing the state of not knowing which of the other system types one is in [00:11:51]. Entering it can be accidental or deliberate, but it is a type of inauthenticity [00:12:02]. For example, a bureaucratic tendency might impose order inappropriately, while a tendency towards complexity might fail to impose order when appropriate [00:12:05].

Complicated vs. Complex Systems

A key distinction is:

• A complicated system is the sum of its parts; problems can be solved by breaking things down and solving them separately [00:13:30]. Complicated systems tend to be engineered and their components are not “antagonistically adaptive” [00:14:12].
• In a complex system, the properties of the whole result from interaction between the parts, their linkages, and constraints [00:13:36]. How things connect is more important than what they are, meaning emergent patterns cannot be decomposed to their original parts [00:13:47]. In a complex adaptive system, what was beneficial one day can change (e.g., symbiosis evolving from parasitism) [00:14:46].

Complicated systems can be embedded in complex systems, for instance, a factory (complicated) operating within a marketplace of suppliers, customers, and competitors (complex) [00:15:51]. Conversely, complex systems can be embedded in complicated ones, as seen with populist political movements creating micro-complexity within an overarching, easily managed political framework [00:16:47].

Applications and Related Concepts

Apex Predator Theory and Populism

This theory states that when something becomes commoditized, frequency variety is lost, the system becomes perverse, and something new can emerge [00:17:27]. For example, when hardware and then software became commodities, new entities (Microsoft, Apple/Google) dominated [00:18:15]. Politically, neoliberalism homogenized the left and right, causing people to feel they lacked choice, which lowered the energy cost for extremists to gain influence [00:18:18]. The concern is that new “predators” (like populism) can stabilize a new ecosystem that is difficult to disrupt for a significant period [00:18:40].

The rise of populism, exacerbated by social media’s positive feedback loops, can create perverse situations [00:19:57]. Solutions involve introducing human agency that is horizontally and bottom-up mediated into social media to increase empathy and human interaction [00:20:45]. This means focusing on low-grade, low-level human interaction and systems that support it, rather than expecting grand schemes [00:21:09].

Agent-Based Modeling and Simulation

Agent-based models and simulations are useful if there is single agency and clear rules [00:27:25]. However, most human systems deal with multiple identities and patterns beyond rule-based decisions [00:27:28]. The danger is confusing simulation with prediction, similar to confusing correlation with causation [00:27:39]. Murray Gell-Mann famously stated that the only valid model of a human system is the system itself [00:28:18]. While agent-based modeling can offer insight and clues, it doesn’t provide the predictive element often sought [00:28:24].

In social systems, one should not take a single trajectory from an agent-based model as a prediction [00:28:46]. However, the statistics on the ensemble of trajectories can be very interesting, revealing whether a system is in a Gaussian or “fat-tailed” (extremistan) space [00:29:02]. Managing in these different spaces requires different approaches [00:29:20]. In Pareto distributions (fat-tailed), the best approach is to “trigger” human beings to a heightened state of alert to look at something, rather than predicting specific outcomes [00:29:37].

Human Agency and Cognitive Diversity

Human beings are good at chaos, having evolved for it, particularly for collective decision-making [00:36:39]. There is a large gap between what AIs can do and what humans can do [00:36:47]. The risk is that current AI work may reduce human intelligence and capability rather than augmenting it, by forcing humans into rigid processes and structured approaches [00:37:06].

A key recommendation for designing systems is to increase human agency [00:54:26]. This involves empowering people to assess situations, which creates training datasets that can then be used by AI systems, ensuring traceability and understanding of the mechanism for decision-makers [00:54:40].

Ethics and Aesthetics in AI Design

For better AI and human systems, three areas are crucial:

1. Ethics: Engineers should have basic training in ethics from a young age [00:39:41].
2. Aesthetics: Understanding aesthetics, which is about abstractions, is important for effective decision-making [00:40:00]. Music and language evolve from abstractions, allowing for rapid “exaptive thinking” (repurposing things) [00:40:22]. Abstraction also fosters higher empathy than material things, as parables offer better moral guidance than values or principles [00:40:39].
3. Epigenetics: Understanding the mechanism for cultural inheritance within a single generation is crucial for designing tools that augment human intelligence and capability, rather than reducing it [00:42:27].

Narrative and SenseMaker®

Narrative is central to Snowden’s work. The idea is that “we always know more than we can say” and “we can always say more than we can write” [00:58:31]. To understand what’s truly happening, one needs “watercooler stories” – day-to-day micro-narratives [00:58:43].

SenseMaker® is a software platform designed to gather these day-to-day micro-narratives (or “micro observations”) and allow individuals to self-interpret them, rather than relying on interpretation by text search, algorithms, or experts [00:59:02]. This allows for scaling to very high volumes rapidly [00:59:13]. Narrative carries ambiguity, sitting between explicit data (like a map) and tacit knowledge (like a taxi driver’s intuition) [00:59:28].

SenseMaker® replaces traditional surveys (e.g., employee satisfaction) by asking non-hypothesis questions (e.g., “What story would you tell your best friend if they were offered a job in your workplace?“) [01:02:00]. Individuals then self-interpret their stories by positioning them on a series of “triangles” (triads) or other graphical forms [01:02:15]. This process adds metadata to the original narrative, which is then analyzed to draw “fitness landscapes” showing patterns, including dominant and outlier views [01:02:56]. The original narrative is carried with the statistical data to explain what the patterns mean [01:03:00]. This self-scoring allows for instant results [01:05:29].

SenseMaker® can be used to:

• Map organizational culture for mergers and acquisitions [00:25:35].
• Provide distributed decision support by presenting ambiguous situations to thousands of employees for interpretation [00:25:44].
• Identify outlier views in an organization, as these individuals think differently about problems [00:26:05].
• Measure attitudes (e.g., to cybersecurity, ethics, customer purchasing behavior) as early indicators [00:46:56].
• Facilitate “site casting” – understanding what is possible in the present before risking the future [00:47:23].
• Encourage coherent heterogeneity (diversity that can come together in different ways) and identify useful dissent by statistically mapping narratives and testing for coherence [00:49:06].
• Act as a replacement for political polling to discover shared values [01:03:56].
• Empower individuals (e.g., social work clients, 16-year-olds as ethnographers) to tell and interpret their own stories [01:04:42].

Management Implications (Complexity Science in Organizational Context)

• Multifaceted Leadership: There isn’t one universal leadership style; different styles (e.g., servant leadership, draconian leadership) work in different contexts [00:45:50]. Leadership should be distributed [00:45:55].
• Attitudes as Early Indicators: Attitudes are more important than compliance because they are early indicators of dispositional states [00:46:56].
• Contextual Appropriateness: Understanding the domain one is in is crucial, as there is no single answer to every problem (no free lunch theorem) [01:13:31]. Management fads often lack a scientific base and claim universal solutions that don’t exist [01:12:22].
• Embracing Diversity and Dissent: Organizations should shift from homogeneity to “coherent heterogeneity” to maintain resilience [00:48:16]. This means maintaining diversity without being overwhelmed by “crankery” [00:49:27]. By running safe-to-fail experiments based on clustered outlier views, one can determine which ideas are coherent and worth exploring [00:50:02]. There is an optimal level of dissent to maintain within an organization for effectiveness [00:52:15].
• Knowing When to Switch: In an evolutionary context, stable times call for exploitation, while unstable times require more exploration [00:52:50]. AI can be used to “trigger” when to switch between these two modes before it’s too late [00:53:08]. This is achieved by building human-mediated training datasets that executive buy-in ensures trustworthiness [00:53:31].
• Downward Causality: In complex systems, behavior at lower levels can be highly constrained by emergent higher-level structures, illustrating a form of “downward causality” [00:55:20]. However, one cannot extrapolate from chemical reactions to human systems due to multiple levels of emergence and greater uncertainty [00:56:06].
• Red Teams and Ritual Dissent: While red teams are valuable if truly independent, a technique called “ritual dissent” fragments into multiple small “micro red teams” to examine decision-making [00:57:24].

Current and Future Work

Snowden’s work is increasingly focused on democracy, education, and creating a more humane society under conditions of control and restraint [01:07:20]. This involves taking a natural science approach to social systems, using natural science to constrain what is possible in social systems [01:11:01]. This contrasts with case-based inductive social science, which is often self-referential, politically influenced, and focused on prediction where it is not possible [01:10:36].