# Technical Report 1420 ## A Conceptual Model of Army Leader Systems Thinking Michelle Wisecarver Chelsey Byrd Personnel Decisions Research Institutes Ava Loer U.S. Army Research Institute Lia Engelsted Cory Adis Michelle Kaplan Personnel Decisions Research Institutes William S. Weyhrauch U.S. Army Research Institute ![](_page_1_figure_1.jpeg) *Figure (p. 1):* The image is a logo for the Army Research Institute. It features a shield with a profile of a head and a checkered pattern, framed by a circle. Below the shield is a banner with the text "ARMY RESEARCH INSTITUTE". December 2022 United States Army Research Institute for the Behavioral and Social Sciences ## U.S. Army Research Institute for the Behavioral and Social Sciences Department of the Army Deputy Chief of Staff, G1 Authorized and approved: ## MICHELLE L. ZBYLUT, Ph.D. Director Research accomplished under contract for the Department of the Army by Personnel Decisions Research Institutes Technical Review by Glen Downing, U.S. Army Command and General Staff College Cary Stothart, U.S. Army Research Institute ## DISPOSITION This Technical Report has been submitted to the Defense Information Technical Center (DTIC). # REPORT DOCUMENTATION PAGE Form Approved OMB No. 0704-0188 1. REPORT DATE (dd-mm-yy): December 2022 2. REPORT TYPE: Final 3. DATES COVERED (from...to): September 2019–March 2021 4. TITLE AND SUBTITLE: A Conceptual Model of Army Leader Systems Thinking 5a. CONTRACT OR GRANT NUMBER: W911NF-19-C-0052 5b. PROGRAM ELEMENT NUMBER: 622785 6. AUTHOR(S): Wisecarver, Michelle; Byrd, Chelsey; Loer, Ava; Engelsted, Lia; Adis, Cory; Kaplan, Michelle; Weyhrauch, William S. 5c. PROJECT NUMBER: A790 5d. TASK NUMBER: 0901 5e. WORK UNIT NUMBER: 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES): Personnel Decisions Research Institutes 1911 N. Fort Myer Drive, Suite 410 Arlington, VA 22209 8. PERFORMING ORGANIZATION REPORT NUMBER: 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES): U.S. Army Research Institute for the Behavioral and Social Sciences 6000 6th Street (Bldg. 1464 / Mail Stop: 5610) Fort Belvoir, Virginia 22060-5610 10. MONITOR ACRONYM: ARI 11. MONITOR REPORT NUMBER: Technical Report 1420 12. DISTRIBUTION/AVAILABILITY STATEMENT: Approved for public release; distribution is unlimited. 13. SUPPLEMENTARY NOTES: ARI Research POC: Dr. Ava Loer, Fort Leavenworth Research Unit 14. ABSTRACT (Maximum 200 words) This research identified systems thinking requirements for Army leaders and developed a conceptual model that describes behaviors needed for successful systems thinking. Literature on systems thinking was reviewed, and a series of individual and group sessions were held with active-duty officers, and with military and civilian instructors, to identify dimensions and behaviors related to leader systems thinking in the Army. A model of systems thinking was proposed that has five dimensions: Identifying Elements, Understanding Dynamic Relationships, Shifting Perspectives, Identifying Holistic Patterns, and Responding to Change. Each dimension is described further with specific behavioral examples. The conceptual model can serve as the foundation needed for developing tools to assess systems thinking in Army leaders. 15. SUBJECT TERMS Systems Thinking, Officer Assessment, Leadership
SECURITY CLASSIFICATION OF19. LIMITATION OF ABSTRACT Unlimited Unclassified20. NUMBER OF PAGES 7021. RESPONSIBLE PERSON Dr. Rhett Graves (913) 547-6811
16. REPORT Unclassified17. ABSTRACT Unclassified18. THIS PAGE Unclassified
# Technical Report 1420 # A Conceptual Model of Army Leader Systems Thinking Michelle Wisecarver Chelsey Byrd Personnel Decisions Research Institutes Ava Loer U.S. Army Research Institute Lia Engelsted Cory Adis Michelle Kaplan Personnel Decisions Research Institutes William S. Weyhrauch U.S. Army Research Institute Fort Leavenworth Research Unit Rhett Graves, Chief December 2022 Approved for public release; distribution is unlimited. ## ACKNOWLEDGMENT We would like to thank the numerous Soldiers who participated in the data collections for this project and the instructors and staff at the U.S. Army War College, Command and General Staff College, School for Advanced Military Studies, University of Foreign Military and Cultural Studies, and School for Command Preparation who met with us and reviewed materials that we developed. We are grateful to MG John Kem and COL Michael Hosie for making a significant part of our data collection possible. We would also like to thank Alex Ryan, Daniel Kim, Derek Cabrera, and Laura Cabrera for their input and feedback on the systems thinking model, and Glen Downing and Cary Stothart for their suggestions and improvements on an earlier version of this manuscript. # A CONCEPTUAL MODEL OF ARMY LEADER SYSTEMS THINKING ## EXECUTIVE SUMMARY ## Research Requirement: Previous research by the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) identified systems thinking as one of six strategic thinking competencies (Sackett et al., 2016). Developing a more comprehensive understanding of systems thinking in the Army includes understanding the behaviors that are required for successful systems thinking. This knowledge can define systems thinking development implications for various Army leaders. This research identified the systems thinking requirements for Army leaders and developed a conceptual model that describes behaviors needed for successful systems thinking. ## Approach: The systems thinking literature was reviewed to determine an appropriate definition for leader systems thinking and to identify likely components of systems thinking. To understand the requirements for leader systems thinking in the Army, individual and group sessions were conducted with 58 active-duty officers in operational assignments and military and civilian instructors who were assigned either to Army educational institutions or to unit training programs. Results were analyzed qualitatively to describe respondents’ perceptions and experiences regarding systems thinking job requirements and training for Army officers, and to develop a conceptual model of leader systems thinking. Interviews were conducted with four systems thinking academic subject matter experts (SMEs) who provided further literature resources and expert opinions on developing a theoretical approach to systems thinking that would be useful for the Army. Once the conceptual model was developed, behaviors representing each dimension were developed and reviewed by active-duty officers and by 15 Professional Military Education (PME) instructors. ## Findings: Based on the literature, leader systems thinking was defined as: A cognitive approach that applies a holistic perspective to identify and understand interrelationships and emergent properties among elements. Results from the individual and group sessions found that Army leaders must be effective systems thinkers to operate and maintain existing systems and to create plans, make decisions, and solve problems that involve a variety of interrelated elements. Most personnel who participated in individual and group sessions viewed systems thinking as important at all officer levels, with systems thinking developing over time through experience, age, and training. Even though most respondents were familiar with the concept of systems thinking, many did not have an organized understanding of it and were unsure how to define or describe it. In discussing the need for systems thinking, respondents identified two fundamental concepts of “systems” in the Army: (a) formal systems, which are established and documented, and (b) informal systems, which are not necessarily documented and need to be uncovered. Both types of systems are relevant and important. The formal systems in the Army, such as various personnel, communication, and planning systems, tend to be clearly defined and may be readily identified or described by documents. The informal or emerging systems, such as social systems, informal coordination systems, and sociocultural systems in operational environments, tend to be more ambiguous, changing, and harder to identify. The majority of PME personnel consulted for this research indicated that systems thinking was required for success in their course or program, although only one-fifth indicated that systems thinking was needed for a specific evaluation. Based on information gained from the review of the literature and individual/group sessions, an initial model of leader systems thinking was proposed with five dimensions: (a) Identifying Elements, (b) Understanding Dynamic Relationships, (c) Shifting Perspectives, (d) Identifying Holistic Patterns, and (e) Responding to Change. A set of focus groups and interviews with 29 active-duty officers produced 42 draft behaviors that described actions relevant to the five dimensions. These 42 behaviors were reviewed by a group of 15 institutional instructors and resulted in a final version of the leader systems thinking model with 25 behaviors across the five dimensions. ## Utilization and Dissemination of Findings: The conceptual model provides the foundation needed to develop systems thinking assessment and training tools for Army leaders. The dimensions and associated behaviors were subsequently used to form the content of a multirater assessment tool (see Loer et al., in preparation), and will also be used as the foundation for an interactive scenario-based tool to assess leader systems thinking in specific situations or environments. # A CONCEPTUAL MODEL OF ARMY LEADER SYSTEMS THINKING ## CONTENTS
Page
BACKGROUND1
Defining Systems Thinking.1
Systems Thinking for Army Leaders...3
Systems Thinking for Leaders/Managers6
Empirical Studies of Systems Thinking..7
Summary.
REQUIREMENTS REVIEW
Method
Results...10 1965
Discussion25
MODEL DEVELOPMENT.26
Academic SME Input.26
The Conceptual Model.27
Identifying Systems Thinking Behaviors...28
Subject Matter Expert Reviews.32
LIMITATIONS AND FUTURE RESEARCH...37
CONCLUSIONS..38
REFERENCES40
APPENDICES
APPENDIX A: SOCIOCULTURAL COMPETENCY MODEL FROM WISECARVER ET AL. (2013)A-1
APPENDIX B: SYSTEMS THINKING REQUIREMENTS DEMOGRAPHIC QUESTIONS.B-1
APPENDIX C: SYSTEMS THINKING REQUIREMENTS DISCUSSION QUESTIONS.....C-1
APPENDIX D: DRAFT SYSTEMS THINKING COMPETENCY MODEL DISCUSSION QUESTIONS..D-1
APPENDIX E: SUBJECT MATTER EXPERT DEMOGRAPHIC AND RATING QUESTIONS..E-1
APPENDIX F: SYSTEMS THINKING DIMENSION BEHAVIOR QUESTIONSF-1
## LIST OF TABLES
TABLE 1. EXAMPLE SYSTEMS THINKING DEFINITIONS
TABLE 2. TOTAL NUMBER OF RESPONDENTS BY RANK 11
TABLE 3. DISTRIBUTION OF RESPONDENTS BY RANK ACROSS COMBAT ARMS, COMBAT SUPPORT, AND COMBAT SERVICE SUPPORT. 11
TABLE 4. RESPONDENTS' TIME IN MILITARY OR AS ARMY CIVILIAN BY RANK AND POSITION...... 12
TABLE 5. HIGHEST DEGREE OR LEVEL OF EDUCATION OBTAINED BY EACH RANK OR POSITION. 12
TABLE 6. SYSTEMS THINKING TERMS COMMONLY MENTIONED DURING INDIVIDUAL/GROUP SESSIONS.. 14
TABLE 7. RESPONDENT DESCRIPTIONS OF NOVICE, INTERMEDIATE, AND EXPERT SYSTEMS THINKERS. 17
TABLE 8. OPERATIONAL PARTICIPANT EXAMPLES OF SYSTEMS THINKING IN THEIR CURRENT JOBS. 19
TABLE 9. CHALLENGES TO SYSTEMS THINKING IN THE ARMY IDENTIFIED BY OPERATIONAL PARTICIPANTS 22
TABLE 10. SPECIFIC COURSES/TOPICS THAT HAVE SYSTEMS THINKING TRAINING AS INDICATED BY INSTITUTIONAL SMES.. 23
TABLE 11. OPERATIONAL PARTICIPANT RESPONSES FOR PROFESSIONAL MILITARY OR OFFICER EDUCATION THAT TEACHES SYSTEM THINKING 24
TABLE 12. INITIAL SYSTEMS THINKING MODEL 27
TABLE 13. IDENTIFYING ELEMENTS - FINAL DESCRIPTION. 33
TABLE 14. UNDERSTANDING DYNAMIC RELATIONSHIPS - FINAL DESCRIPTION.. 34
TABLE 15. SHIFTING PERSPECTIVES - FINAL DESCRIPTION. 35
TABLE 16. IDENTIFYING HOLISTIC PATTERNS - FINAL DESCRIPTION 36
TABLE 17. RESPONDING TO CHANGE - FINAL DESCRIPTION 37
# A CONCEPTUAL MODEL OF ARMY LEADER SYSTEMS THINKING Previous research by the U.S. Army Research Institute for the Behavioral and Social Sciences (ARI) identified systems thinking as one of six strategic thinking competencies (Sackett et al., 2016). Systems thinking was described by Sackett et al. as involving the identification and comprehension of complex and dynamic interdependencies between entities, and as a holistic perspective, synthesizing interdependencies into a concept of a comprehensive whole. Building a more comprehensive understanding of the various requirements for systems thinking in the Army requires understanding the individual behaviors needed for successful systems thinking. Understanding these systems thinking behaviors and requirements will enable us to successfully develop systems thinking capabilities in Army leaders. The objective of this research was to identify the systems thinking requirements for Army leaders and to develop a conceptual model that describes behaviors needed for successful systems thinking. This model can then be used to guide the development of assessment and training tools for systems thinking. ## Background ## Defining Systems Thinking To examine systems thinking requirements in the Army, it is useful to first define systems thinking. Systems thinking has been examined conceptually in the literature from a variety of different disciplines, perspectives, and approaches, from systems engineering and cybernetics to soft systems methodology and social networking (e.g., Jackson, 2003; Midgely, 2003; Ryan, 2014). Example definitions or descriptions of systems thinking from across multiple decades can be seen in Table 1. ## Table 1 ## Example Systems Thinking Definitions
CitationDefinition/Description
Arnold & Wade (2015)“Systems thinking is a set of synergistic analytic skills used to improve the capability of identifying and understanding systems, predicting their behaviors, and devising modifications to them in order to produce desired effects. These skills work together as a system” (p. 675).
Behl & Ferreira (2014)“Systems thinking provides ways to think about a problem and its solutions using whole system perspectives... It builds upon the idea that a system is greater than the sum of its parts and therefore should be studied holistically. The use of it allows us to better understand and develop successful complex systems and is integral to systems engineering” (p. 105).
## Table 1 (continued)
Cabrera et al. (2008)“Systems thinking is not necessarily a matter of drawing an entirely new skill set out of the intellectual ether; rather, it is a unique perspective that transforms the approach taken to evaluate any program, policy, or initiative… systems thinking is not something one does, but something one gets as a result of applying simple rules based on patterns of thinking” (p. 300).
Dedmond (2014)“The definition for Systems Thinking should include the study of interrelated parts (structures, institutions, relationships, interactions, etc.) that connect and affect the behavior of other parts within the system… [thus removing] discriminatory demarcations that restrict “systems” to a specific categorical domain” (p. 8).
Kim (1999)“A school of thought that focuses on recognizing the interconnections between the parts of a system and synthesizing them into a unified view of the whole” (p. 19).
Macaulay (2011)A “practice of thinking that allows strategic leaders to holistically appreciate and interpret the chaos and complexity of a given environment by recognizing the cause-and-effect relationships of its respective actors” (p. 2).
Senge (1985)“Systemic thinking is integrative, synthesizing diverse viewpoints in order to understand the organization as a whole. It is structural thinking, focusing on the structure of interrelationships… that determine organizational success” (p. 3).
Valerdi & Rouse (2010)“Systems thinking is a process of identifying, estimating or inferring how local policies, actions, or changes influences the state of the neighboring universe” (p. 185).
Although there seems to be agreement that systems thinking is important across various disciplines and schools of thought (Arnold & Wade, 2015; Behl & Ferreira, 2014; Macaulay, 2011; Senge, 1985; Valerdi & Rouse, 2010), there is not agreement regarding what constitutes system thinking (e.g., Cabrera et al., 2008). Also, there is not a universally accepted definition of systems thinking. The various definitions reveal factors or aspects of systems thinking that differ. For example, some authors describe systems thinking as a process of synthesizing viewpoints or perspectives (e.g., Reynolds, 2011; Senge, 1985), while others describe systems thinking as understanding the interrelationships of system components (e.g., Gumbert, 1998; Kim, 1999). Many writers describe systems thinking as an approach (e.g., Behl & Ferreira, 2014; Dedmond, 2014; Henning et al., 2012), while some writers describe systems thinking as a set of skills or abilities (e.g., Arnold & Wade, 2015; Gumbert, 1998). Cabrera et al. (2020c) describe systems thinking as a process or emergent property that results from engaging in thinking about distinctions, organizing systems, articulating relationships, and taking many perspectives and results in a mental model that approximates reality. Despite these differences, several themes emerge across the descriptions. First, systems thinking is cognitive in nature and therefore requires a set of cognitive skills related to thinking and understanding. Second, systems thinking involves an attempt to understand the whole of a situation rather than focusing solely on any facet or portion of the situation. Third, systems thinking involves understanding how various elements connect and interrelate. Based on these themes, the following broad and inclusive definition for systems thinking was developed for this project: ## Systems thinking: A cognitive approach that applies a holistic perspective to identify and understand interrelationships and emergent properties among elements. In this project we are particularly interested in (a) understanding the ways in which systems thinking is important for Army leaders and (b) developing a conceptual model that identifies specific behaviors relevant to systems thinking for Army leaders. We will first briefly examine literature that considers the role of systems thinking specifically in the military and as it relates generally to leaders or managers. ## Systems Thinking for Army Leaders Limited literature exists specifically considering systems thinking in the context of the military. This literature largely consists of theses or articles produced at the U.S. Army War College (e.g., Allen et al., 2009; Eifler, 2011; Macaulay, 2011; Williams, 2007) or the School for Advanced Military Studies (SAMS; e.g., Dennis, 2010; Dixon, 2006; Wheeler, 1999). However, there are notable exceptions, such as a chapter that describes military applications of complex systems (Ryan, 2011), and articles describing the application of the systems thinking concepts of Derek and Laura Cabrera that are being taught at the U.S. Military Academy (Schwandt, 2018; Schwandt & Ryan, 2018). All the articles endorse the importance of systems thinking for Army leaders. Several of the articles emphasize the importance of recognizing that the systems that Army leaders deal with are open systems (e.g., Allen et al., 2009; Dennis, 2010; Dixon, 2006). For example, Allen et al. (2009) discusses the importance of the distinction between systems thinking in engineering science, which operates with closed cycles, and systems thinking for Army leaders, who operate in open, dynamic, and constantly shifting situations, particularly at the strategic level. Military contexts include open systems with porous boundaries that are part of a larger system and therefore cannot be understood in isolation (Dennis, 2010). Army leaders also encounter many systems that fall in between these two extremes and are neither completely closed, such as a mechanical system, nor are wide-open dynamic systems, such as sociocultural systems. One approach that has been recommended for use in a military setting that deals with varying degrees of open systems is the soft systems methodology (SSM; Checkland, 1981), which originated from computer science and systems engineering approaches. The SSM approach is an action-oriented approach that helps users gather information about “messy situations of all kinds” and take action to improve them. Dennis (2010) argues that the key activities of SSM, which are finding out, model building, discussion/debate, and defining/taking action, offer a flexible but organized approach to integrating systems thinking into the existing military operations process. Another concept that has been applied conceptually to the military is the idea of the Army as a learning organization (Gumbert, 1998; Hill & Gerras, 2016; Williams, 2007). The learning organization is a concept defined by Senge (2006) that describes characteristics important for organizations to adapt, survive, and grow within a changing environment and to maximize the contributions of all organizational members. The five prerequisites of a learning organization, referred to as “disciplines,” are: personal mastery, mental models, team learning, shared vision, and systems thinking. Systems thinking within this concept is needed to build organizational members’ understanding of the interdependencies within the organization and deal more effectively as a whole when encountering changing situations. Both Gumbert (1998) and Williams (2007) argued that the Army should become a learning organization. Both Gumbert and Williams identified systems thinking as an area in which Army leaders needed improved development and training for the Army to become a learning organization. In a learning organization, all organizational members need to understand the interdependencies within the organization to deal effectively and efficiently with changing situations. Therefore, the concept of the Army as a learning organization suggests that systems thinking is important at all echelons in the Army. Leaders at all echelons within the Army must understand their roles in the various systems, their impact on the systems, and the potential impact of changes on those systems. To be effective, all leaders need to look at various relationships and perspectives that apply across individuals, groups, and echelons in the Army through a systems thinking lens (e.g., Cabrera et al., 2015; Cabrera et al., 2020a; Schwandt & Ryan, 2019). Wheeler (1999) argued that systems thinking should be implemented even at the team level to develop a highly flexible and relevant fighting force. Complex organizations such as the Army are comprised of a series of subsystems. Thus, understanding teams as systems provides the foundation needed to understand organizations as a system of subsystems. Understanding teams as systems can accelerate team development and the team lifecycle, ultimately leading to a more adaptive fighting force (Wheeler, 1999). This link between systems thinking and adaptability places value on systems thinking at all levels within an organization. Other literature on systems thinking in the Army has emphasized the importance of systems thinking specifically for leaders at the strategic level (e.g., Allen et al., 2009; Macaulay, 2011; Van Riper, 2012). Strategic thinking is a capability needed in organizations to anticipate needed change (e.g., Sackett et al., 2016). Sackett et al. (2016) described six strategic thinking competencies: comprehensive information gathering, learning, critical thinking, innovative thinking, thinking in time, and systems thinking. Systems thinking is critical to enable strategic thinkers to apply a holistic perspective to the dynamic and complex environment, identify key relationships, and integrate multiple factors into a comprehensive whole. Strategic thinking becomes increasingly important across a Soldier’s career. However, the skills such as systems thinking that are required for strategic thinking need to be developed and applied earlier in Soldiers’ careers (Sackett et al., 2016). While systems thinking may become increasingly important at strategic levels of leadership, applying systems thinking at lower echelons encourages a more adaptive and effective organization, and prepares leaders for more complex and strategic thinking required at higher echelons. One systems thinking domain that has received particular attention in the Army is sociocultural systems thinking. Using a systems thinking approach to work within complex sociopolitical environments can provide a clearer picture of the operational environment (Dedmond, 2014). A report edited by Strong et al. (2013) provides a series of conceptual articles that describe various aspects of the sociocultural systems encountered during military operations in cross-cultural contexts. The primary emphasis in the report was to understand how to operate within the sociocultural systems that are inherent to the Army as well as understand how they can be evaluated and influenced (Strong et al., 2013). The articles in the report, however, focus primarily on the sociocultural systems themselves as opposed to systems thinking specifically or how systems thinking can be applied to the sociocultural systems. Another research effort that focused more specifically on systems thinking in sociocultural situations offered the following definition for sociocultural systems thinking: “an individual’s capability to continuously build and analyze a mental model of the sociocultural relationships and dynamics that exist in an environment, and leverage this model when planning, preparing, executing, and assessing missions” (Adis et al., 2013). Based on a conceptual review of related literature, Adis et al. (2013) proposed six dimensions of sociocultural systems thinking: awareness of system elements, identifying system relationships, evaluation and revision of model, understanding system dynamics, operational questioning, and application to mission. These dimensions are conceptualized as part of a dynamic and cyclical process that develops a mental model of the sociocultural system. The process then continuously evaluates and revises the model as an individual gains new information. Also identified in this effort was a series of knowledge, skills, abilities, and other characteristics (KSAOs) that were relevant to sociocultural systems thinking. The KSAOs were organized into five areas: sociocultural knowledge, systems thinking skills, systems thinking knowledge, social/cognitive skills, and personal attributes. In addition to identifying key dimensions and KSAOs, a draft competency model for sociocultural systems thinking was proposed and can be seen in Appendix A (Wisecarver et al., 2013). The competency model provides an understanding of the behaviors associated with the overall construct and a framework for defining the skill and knowledge requirements of individuals that need to engage in sociocultural systems thinking. The draft competency model also delineates developmental proficiency levels for the construct, which can guide future training and provide a foundation for developing assessment tools and training aids. While the Adis et al. (2013) and Wisecarver et al. (2013) efforts focused specifically on sociocultural systems thinking, the dimensions and behaviors that were identified as part of those efforts also provided ideas for a more general systems thinking model. The literature on systems thinking in the military supports the relevance and importance of systems thinking for Army leaders. Although systems thinking may be particularly important for strategic leaders or leaders working within complex sociocultural settings, several the conceptual reports suggest that having all Army leaders engage in systems thinking is potentially useful (Allen et al., 2009; Dennis, 2010; Eifler, 2011; Macaulay, 2011; Wheeler, 1999). Having leaders engage in and build systems thinking in their organizations can support the development of organizations that are more agile and able to succeed in the dynamic and changing environment in which the Army exists. The research conducted to define sociocultural systems thinking and identify relevant KSAOs can serve as a model and a source of information to define a broader systems thinking competency and the associated relevant behaviors. ## Systems Thinking for Leaders/Managers Some of the same or similar themes that emerged in the military literature are also seen in the broader literature. These themes include the importance of systems thinking for leader and organizational success, the role of leaders in developing learning organizations and collaborative systems thinking, and the importance of systems thinking for strategic planning. The literature suggests that systems thinking is generally important for leaders to enable them to engage in effective planning, problem-solving, and decision-making, particularly in complex environments, and to build adaptive organizations that engage in systems thinking. One critical function of systems thinking for leaders is to facilitate problem-solving and decision-making. As such, numerous articles highlight the importance of systems thinking to various planning, decision-making, and problem-solving activities (e.g., Arnold & Wade, 2015; Cabrera et al., 2008; Jackson, 2003; Maani & Majaraj, 2004; Stroh & Zurcher, 2012). Though not specific to leadership, Maani and Majaraj (2004) explored the role of systems thinking in complex decision-making, confirming the importance of systems thinking to complex decision- making and identifying specific systems thinking approaches that were consistently followed by better performers. Savage and Sales (2008) described systems thinking as the primary competency that differentiated successful organizational leaders from less successful ones. Savage and Sales referred to these leaders as “anticipatory,” describing how the leaders were able to galvanize action across all organizational levels through their ability to clearly articulate and inform people around them of strategic decisions while engaging those people throughout the decision-making process. Systems thinking is identified as particularly important for addressing complex problems (e.g., Arnold & Wade, 2015; Conti et al., 2009; Jackson, 2003; Merali & Allen, 2011) and when engaging in strategic thinking and decisions (Savage & Sales, 2008; Zahn, 1999). Using a systems approach to solve a problem can overcome pitfalls encountered by other approaches such as linear or ordered thinking (e.g., Cabrera & Cabrera, 2018; Stroh & Zurcher, 2012). In linear thinking, individuals look at direct connections between different events and actions (Allen et al., 2009; Stroh & Zurcher, 2012). Linear thinking assumes that each event that occurs, each action that is taken, is independent of the others (Stroh & Zurcher, 2012; Valerdi & Rouse, 2010). However, this form of thinking makes individuals incapable of understanding the full scope of a situation, by not allowing for the possibility of other relationships such as indirect causality; interdependency of individuals, actions, and people; and the possibility that fixing some portion of a problem will not necessarily aid in resolving the overall issue (Allen et al., 2009; Bentley & Davy, 2009). Linear thinking can lead to solutions developed with good intentions that ultimately fail or even make the problem worse. An example is a situation where nongovernmental organizations providing food aid to countries results in undermining local agriculture and leads to increased starvation rather than less starvation (e.g., Stroh & Zurcher, 2012). Another critical aspect of systems thinking requirements for leaders is building and encouraging systems thinking within their organization. As noted previously when discussing military systems thinking research, Senge (2006) emphasized the importance of having leaders create learning organizations by building five relevant disciplines within the organization, with one of the disciplines being systems thinking. This approach enables individuals and the entire organization to operate and adjust more effectively in dynamic and changing environments. Systems thinking is viewed as the discipline that integrates the other four elements of strategic thinking into a coherent whole. Leaders play an essential role in fostering systems thinking within their organizations. Leaders can introduce systems ideas into their organization or inhibit the adoption of these ideas (Dibble, 2011; Martens, 2011; Morgan, 2005). When leaders develop systems thinking in their organization, individuals gain a better perspective of the overall interconnectedness of what members of the organization are doing, thereby allowing improved understanding across the organization and allowing greater integration of new learning and new ideas (Martens, 2011; Morgan, 2005; Tideman et al., 2013). Other authors have pointed to systems thinking as the component that integrates important functions within the organization as well. For instance, Cabrera et al. (2020b) offered a theory of systems leadership that encompassed four universal functions of organizations: vision (goal state), mission (work needed to accomplish goal), capacity (system of systems that make it possible to do the mission), and learning (efforts to seek and use feedback to improve the organization). According to Lamb and Rhodes (2009), the importance of building systems thinking capabilities also applies at the lower levels within organizations, with leaders supporting collaborative systems thinking. Lamb and Rhodes defined collaborative systems thinking as, “an emergent behavior of teams resulting from the interactions of team members and utilizing a variety of thinking styles, design processes, tools and communication media to consider the system, its components, interrelationships, context, and dynamics” (p. 4). Collaborative systems thinking teams are valuable to the organization in that the teams foster an environment that values systems thinking and knowledge sharing, allowing individuals to draw from a broader range of experience and expertise when problem-solving within the organization. ## Empirical Studies of Systems Thinking While there is considerable literature that describes and discusses systems thinking from theoretical and conceptual perspectives, there are very few empirical research efforts that have examined the dimensions, behaviors, and KSAOs relevant to systems thinking. Also, little has been demonstrated empirically regarding whether these dimensions, behaviors, and KSAOs are antecedents, correlates, or outcomes of systems thinking. Beyond the theoretical frameworks that the literature helps to create, there are no predominant empirical measures of systems thinking nor demonstrations regarding its consequences and outcomes. The research stream from the Cabrera Research Lab identified four universal rules that were proposed as foundational to thought in general and systems thinking specifically: distinctions, systems, relationships, and perspectives (Cabrera et al., 2015). Cabrera et al. (2020a) provided a detailed description of empirical research that supported the importance and relevance of these four elemental rules. The rules served as the foundation for their Systems Thinking Metacognitive Inventory (STMITM ), which was described on the website of Cabrera Research Lab (2021) as a personal inventory of skills in metacognition and systems thinking. Empirical data examining construct or criterion related validity using the STMITM was not identified on the website. Another research stream that examined empirical data for systems thinking research was that of Skaržauskienė and her colleagues (Kvedaravičius et al., 2009; Palaima & Skaržauskienė, 2010; Skaržauskienė, 2008; 2009; 2010; Skaržauskienė, & Jonušauskas, 2013). In this research stream, systems thinking was conceptualized as a ‘cognitive intelligence competency’ and was measured through the university version of the Emotional and Social Competency Inventory (ESCI, Boyatzis et al., 2000). This measure used self-report to assess emotional intelligence, social intelligence, and cognitive intelligence competencies. Results indicated that the combination of six systems thinking competencies (understanding of mental models, continuous learning, process orientation, systems logic, interactivity, and dynamic thinking) measured by the ESCI accounted for 32% of the variance in organizational performance, as measured by a self- report organizational performance measure (Skaržauskienė, 2010). The six competency dimensions were created based on loosely supportive evidence from an exploratory factor analysis. While each competency dimension had a significant relationship with organizational performance, most of the dimensions had small regression weights, except for process orientation. In a separate analysis (Kvedaravičius et al., 2009), the researchers split the sample into two industry groups: retail trade and manufacturing. The authors found that systems thinking as measured by the ESCI was more important for leaders in manufacturing jobs (accounting for 53% of the variance in performance) than in retail trade jobs (accounting for 27% of the variance in performance). Palaima and Skaržauskienė (2010) examined the importance of systems thinking factors for different types of leadership. Palaima and Skaržauskienė found that more systems thinking factors were important for organizational/strategic leadership performance than systems thinking factors were important for personal or relationship leadership performance. Specifically, dynamic thinking ( β = 0.19, p < 0.001), interactivity ( β = 0.16, p < 0.001), systems logic ( β = 0.16, p < 0.001), and process orientation ( β = 0.22, p < 0.001) were important for organizational/strategic leadership, but only process orientation ( β = 0.37, p < 0.001) was important for leadership performance and relationship leadership performance. In summary, in this continuing line of research by Skaržauskienė and colleagues, they were able to find a relationship between systems thinking and organizational performance (Skaržauskienė, 2010). Systems thinking also showed a strong positive correlation with leadership performance as well as with social and emotional intelligence competencies (Kvedaravičius et al., 2009; Palaima & Skaržauskienė, 2010; Skaržauskienė, 2009; Skaržauskienė, & Jonušauskas, 2013). Furthermore, the principles underlying systems thinking helped leaders to achieve higher quality organizational management (Skaržauskienė, 2009; Skaržauskienė, 2010; Skaržauskienė, & Jonušauskas, 2013). All the above empirical work, however, appeared to have been done using a singular dataset and is thus limited in terms of the conclusions that can be drawn. In addition, only 18.4% of the leaders sampled were from organizations with 250 or more people, suggesting limited generalizability to large organizational contexts. Additional limitations stem from the fact that the items used to measure systems thinking were not initially intended to measure systems thinking, so the items might not be comprehensive in measuring any given conceptualization of systems thinking. Common method bias was also a limitation of this research, since leaders rated their own leader performance, organizational performance, and systems thinking using self- report Likert-type scales. ## Summary Systems thinking is recognized as important in the military in general and the Army in particular. While there is little in the way of empirical research on systems thinking, there is a growing number of papers written by Army leaders reflecting on uses of systems thinking in the Army. Viewing Army processes as open systems, transforming the Army into a learning organization, and the importance of engaging with sociocultural systems are all common themes of the Army-related literature. The systems thinking literature, more broadly, often focuses on leaders because they are in boundary spanning roles that require the decision-making latitude to adopt systems thinking approaches. The management literature provides many conceptual reflections on systems thinking. In these contexts, systems thinking is described as a problem-solving tool that facilitates solving existing and future problems. Systems thinking is also seen as a critical tool at the organizational level, where leaders take on the role of encouraging systems thinking from others in the organization. Empirical research is particularly sparse, with a lack of consensus on what defines systems thinking and a lack of a framework to measure it. The review of existing literature provided insights regarding the requirements for systems thinking in Army leaders and regarding some behaviors relevant to systems thinking. To gain an understanding of the systems thinking experiences and perspectives of current Army officers and instructors, a series of focus groups and interviews were conducted with active-duty officers in operational assignments. In addition, consultations were conducted with military and civilian instructors who were assigned either to Army educational institutions or to unit training programs. ## Requirements Review ## Method ## Participants Thirty-five Army officers participated in 26 individual interviews or focus group Sessions. These officers will be referred to as the “Operational” sample or participants. In addition, 23 officer or civilian instructors or administrators assigned to unit training or leader development programs were consulted as subject matter experts (SMEs) during 20 individual or group sessions. These SMEs will be referred to as the “Institutional” SMEs. Collectively, Operational participants and Institutional SMEs will be referred to as respondents. Individual sessions were primarily conducted with Institutional SMEs and colonels in the Operational sample, whereas group sessions were primarily conducted with captains, majors, and lieutenant colonels in the Operational sample. All respondents within a group session were of the same or similar rank and none were in the same chain of command. For the remainder of this report, the term “response data” is used to refer to both individual sessions and group sessions with respondents. ## Procedure Respondents completed a demographic questionnaire (see Appendix B). A semistructured discussion was then conducted for 30 to 60 minutes per session. The discussion questions captured respondents’ experiences regarding systems thinking job requirements and training for Army officers. Respondents were asked to describe systems thinking, leadership levels, or positions for which systems thinking was important for Army officers, military education that teaches systems thinking, and descriptions of novice, intermediate, or expert systems thinkers. Operational participants were also asked whether systems thinking was used in their current and previous Army jobs, and Institutional SMEs were asked whether systems thinking was needed to succeed in the course or program they taught. The list of discussion questions can be seen in Appendix C. ## Data Analysis Demographic data were compiled and summarized in descriptive tables. The response data were analyzed qualitatively using an in vivo coding process, which identifies words or phrases used by the respondents in the data summaries (see Miles et al., 2014). Two raters with master’s degrees in Industrial/Organizational Psychology independently coded the response data gathered for each of the questions. After coding responses to the individual or group session questions, the raters met to compare codes and engaged in consensus discussions to resolve any areas of disagreement. Interrater agreement between coders was calculated as the number of sessions on which rater codes agreed for a given question/topic divided by the total number of sessions. For the initial coding of the Operational participant responses, interrater agreement ranged from 75% to 100%, with an average of 85% across all questions. For the Institutional SMEs, the initial interrater agreement ranged from 72% to 100%, with an average of 91% across all questions. Coded responses were then tabulated and summarized in table format. Responses from the Operational participants and Institutional SMEs were coded and summarized separately. In addition to the coded data, a thematic analysis was conducted on the response data to identify any additional themes not captured in the coded responses. ## Results ## Demographic Data The ranks of the respondents are shown in Table 2. Respondents who were former military members provided the highest military rank they achieved in service. There were four civilian instructors listed as Institutional in Table 2 who did not list any prior military experience. ## Table 2 ## Total Number of Respondents by Rank
RankOperationalInstitutionalTotal
Colonel19*10
Lieutenant colonel10212
Major11415
Captain13417
Civilian instructor-44
Total352358
*Among the Institutional SMEs who were ranked as colonels (O-6), five of them were retired as follows: three were retired from the Army, one from the Air Force, and one from the Navy. Respondents were from 15 military occupational specialty (MOS) series, which were categorized into combat arms (11, 12, 13, 14, 15, and 19 series), combat support (25, 35, and 46 series), and combat service support (36, 68, 92, and 94 series). Table 3 shows the distribution of respondents across the MOS categories. Respondents were predominantly from MOSs associated with combat arms. ## Table 3 ## Distribution of Respondents by Rank Across Combat Arms, Combat Support, and Combat Service Support
RankCombat armsCombat supportCombat service supportNone/ unknownTotal
Colonel6013*10
Lieutenant colonel821112
Major923115
Captain923317
Civilian instructor00134
Total32691158
*Two colonels did not select Army branches because they were retired Air Force and Navy members. The total time respondents spent in service ranged from a low of four years for one of the captains to a high of 30 years for a retired Army service member who is now an instructor. Table 4 shows the average time in service for the respondents based on their rank at the time of their individual or group sessions with the researchers. ## Table 4 ## Respondents Time in Military or as Army Civilian by Rank and Position
Current rankAverage total time in military (years) (n=54)Average time as Army civilian (years) (n=4)
Colonel27-
Lieutenant colonel21-
Major17-
Captain12-
Civilian instructor269
Respondents were also asked about their highest level of education. Most of the respondents (66%) had completed either a master’s or a doctorate degree, with 29% completing a bachelor’s degree. Differences in degree completed across the rank categories can be seen in Table 5. ## Table 5 ## Highest Degree or Level of Education Obtained by Each Rank or Position
RankProfessional degree n (%)Bachelor’s degree n (%)Master’s or doctorate degree n (%)
Colonel3 (30)0 (0)7 (70)
Lieutenant colonel0 (0)1 (8)11 (92)
Major0 (0)3 (20)12 (80)
Captain0 (0)13 (76)4 (24)
Civilian instructor0 (0)0 (0)4 (100)
Total3 (5)17 (29)38 (66)
## Individual Session/Group Session Responses Results are presented in three key topic areas: Understanding of Systems Thinking, Using Systems Thinking on the Job, and Systems Thinking Education and Training. Except for responses to one question (whether respondents were familiar with the term systems thinking), information collected within group sessions was not identified with specific respondents, only with the session. Thus, the percentages provided in this section indicate the percent of the 26 Operational sessions or of the 20 Institutional sessions in which a given response was discussed, rather than the percent of individuals who gave a specific response. Understanding of Systems Thinking. Most of the individual Operational participants (67%) and Institutional SMEs (82%) indicated that they were familiar with systems thinking as a general concept. However, many respondents explained that they were only aware of systems thinking in a general sense and were not familiar with specific terms or concepts. When asked how they would describe systems thinking or what they knew about it, Operational participants described systems thinking using a variety of terms such as “systems engineering” (23%) or mentioning specific types of systems such as social systems, biological systems, and economic systems with which they were familiar (23%). Other concepts participants in Operational sessions mentioned were military processes such as Military Decision-Making Process (MDMP) and Army Design Methodology (19%), the interrelationships of processes (15%), and systems and change management (15%). Participants in a few of the Operational sessions described concepts such as system failure, feedback loops, and forecasting/prediction. Operational participants familiar with specific systems thinking terminology said they had learned about systems thinking in the Systems Engineering program at the U.S. Military Academy (USMA) or from training in an information technology field. While a greater percentage of Institutional SMEs (82%) indicated they were familiar with the term systems thinking, many of them still were not sure how to describe it. The most common response (31%) was to describe systems thinking by describing related types of thinking (e.g., divergent, critical, strategic, nonlinear thinking). Another common response was to describe systems thinking as a certain “perspective” (15%) or as the interrelationship of processes (12%). SMEs in some Institutional sessions provided examples of specific types of systems (12%), mentioned specific authors who wrote about system concepts (12%), or relayed specific terms or phrases related to systems thinking such as “whole is greater than the sum of the parts” (12%), information flow (12%), and forecasting (8%). To gain an understanding of the extent to which words related to systems thinking were used during the individual or group sessions with the researchers, an automated search was conducted for 111 whole words (e.g., “forecasting”) or word stems (e.g., “holis” as a stem for “holistic,” or “holistically,” etc.) related to systems thinking. The words were selected by first reviewing Operational and Institutional data transcripts and identifying terms related to systems thinking and various systems theories. At the end of this review, 253 phrases were identified representing 111 unique systems-related terms and concepts. Since the list of unique terms was not available until after the manual review, a second, computer-based search of the transcripts was conducted to count the stemmed search terms. This search yielded 513 mentions of key terms overall. Results indicated that the most commonly occurring term was “perspective” (48 times), followed by “design” (45 times), “complexity” (20 times), “social system” (20 times), and “strategic thinker” (17 times). The 28 terms that were mentioned five or more times are displayed in Table 6. ## Table 6 ## Systems Thinking Terms Commonly Mentioned During the Individual/Group Sessions
CPTMAJLTCCOLCivilianTotal
perspective3120101448
design216152145
complexity0274720
social system10730020
feedback loop7181017
strategic thinking10012215
critical thinking0327113
strategic leadership0009312
big picture6221011
holis*2002711
information flow4610011
order effects3322010
creative*2141210
information system702009
micro*612009
systems engineer214209
systems perspective002079
bigger picture503008
mental model016007
whole system331007
complex adaptive system000516
forecasting021126
interconnected202116
analyt*112105
assumptions201115
formal system002305
macro*230005
specialist230005
*Indicates that the term was searched as a word stem rather than a whole word (e.g., “holis” as a stem for “holistic,” or “holistically”) The two terms that were most used were “perspective” and “design.” In many cases, the word “perspective” was mentioned in the context of perspective taking or learning the different perspectives in a problem situation. The word “design” was often used in the context of campaign design and the Army design course that teaches it. Both terms were mentioned more frequently among colonels and lieutenant colonels or civilian instructors. There are other common uses of these terms, however, such as “by design” or “from my perspective,” which might make these terms more likely to be mentioned in a non-systems thinking context. “Complexity” and “social system” were two terms that were repeated often as well. Complexity was mentioned more by respondents of higher ranks. Colonels discussed “complex adaptive systems” more than other ranks. “Social system” on the other hand, was mentioned more at lower ranks, with 85% of the mentions coming from sessions with captains and majors. The “big picture” or “bigger picture” was mentioned numerous times, mainly by lower ranking Operational participants. Civilian instructors mentioned “holism” or “holistic” perspectives more than Operational participants or other Institutional SMEs. Another concept that was mentioned numerous times was the feedback loop. Many respondents were aware of feedback loops and recognized their importance in systems thinking. However, many references to feedback loops were superficial in nature, and it is difficult to determine if respondents were talking about complex, self-stabilizing feedback loops, or simply referring to a simple bipolar feedback loop where actions are taken, and feedback is received. When respondents were asked if they viewed systems thinking as something that developed over time, most participants in Operational sessions (62%) and SMEs in Institutional sessions (70%) responded “yes,” describing systems thinking as something that developed over time through a combination of factors such as experience, age, and training. The remaining respondents (38% of Operational sessions and 30% of Institutional sessions), indicated that whether systems thinking developed over time depended on the individual; while an individual could develop over time, they must be willing and take the time to be introspective and develop an understanding of systems. Because of this argument, these respondents indicated, some individuals were more likely than others to become expert systems thinkers. Eleven respondents were able to provide a description of the differences between novice, intermediate, and expert systems thinkers. These descriptions are summarized in Table 7 using four categories: system-specific knowledge, system-general knowledge, depth of processing, and communicating with and mentoring others. Many of the characteristics that respondents used to describe levels of systems thinkers revolved around the extent of their system-specific and system-general knowledge. There was a heavy emphasis on learning specific systems, with many respondents seeing systems thinking developing in parallel with knowledge about the systems in their own problem-solving domain. As leaders develop a better understanding of the systems with which they are interacting, the leaders will have more opportunities to take a higher-level perspective or to consider multiple interacting elements. System-general knowledge was also identified as developing across the three levels. The levels included knowledge of systems thinking methodologies, system approaches, and systems theories. Respondents saw general knowledge about systems expanding with experience and that the knowledge was required in high levels for individuals to develop into expert systems thinkers. Some respondents indicated that they found it easier to describe the novice and expert levels of systems thinking than the intermediate level. This outcome may be because leaders differ in their pattern and rates of development across the four categories of characteristics, so there is greater diversity in the capabilities of leaders in the intermediate level. Respondents also discussed the depth of processing or amount of thinking leaders do as changing across novice, intermediate, and expert levels. Novice systems thinkers were described as dealing with issues at a surface level and accomplishing narrow tasks with little consideration for side effects. Intermediate level systems thinkers were described as willing to accept new ideas and to engage in critical thinking, but (at least at times) not assessing the impact of solutions. Again, intermediate level systems thinkers are in the process of fully developing their systems thinking knowledge, so a broader range of behaviors are expected within this echelon, making it hard to describe specific behaviors that apply across all intermediate level systems thinkers. Experts were described as updating their understanding of a system with new information and being comfortable with unresolved ambiguity. Experts do not just deal with the problems they are given but look ahead and anticipate additional problems that will arise. This phenomenon is often described as “understanding second- and third-order effects,” as listed in the expert column of Depth of Processing in Table 7. While one respondent described understanding or uncovering “fifth- or sixth-order effects,” this understanding may be possible with bounded systems such as a machine but would be unrealistic when working with complex adaptive systems. One other theme in respondents’ distinctions among novice, intermediate, and expert systems thinkers was how the leader communicates with others to obtain or disseminate information and mentor more junior leaders. This dimension hinged upon sharing information, communicating different perspectives, and asking questions to get to the root of an issue. Expert systems thinkers must work with and mentor staff to build a shared understanding of a system or system of systems in the problem domain. Expert systems thinkers must also understand the Army social system well enough to work effectively within existing structures and through individuals in the system. # Table 7 ## Respondent Descriptions of Novice, Intermediate, and Expert Systems Thinker
NoviceIntermediateExpert
System specific knowledge• Becoming aware of system/s and how systems operate (e.g., becoming creative with resourcing, rather than relying of one point of contact) • Understands simple systems, though not complex adaptive systems • Views system as sum of parts, and own role as outside influence • Makes links among factors in the environment • Knows the core competencies of the job, but not all the other jobs in the space around them• Is learning effective ways to interact/ communication within and across systems • Considers how events or actions in one part of the system, affect other parts of the system or mission objectives • Beginning to lead organizations and participate in Joint planning with reduced guidance • Understands the big picture and knows how to apply it but does not have the foresight to predict• Has a complete understanding of the pieces of the system, how any two pieces are related, and how to communicate the system to others • Able to consistently and effectively act or make decisions on intuition about the system • Consistently thinks about the interrelatedness of the system; considers indirect effects and implications of actions • Observes events outside the branch and successfully predicts their impact and the need for resources
System general knowledge• Is capable of understanding that they should think about systems; recognizes when they are doing systems thinking • Has basic knowledge of the nature of systems and language to describe system properties • Able to describe something as a system and recognize interrelationships within that system or between systems • Is learning to recognize when to take a holistic view• Understands different approaches to systems thinking and can describe different theories • Able to fully examine a system, communicating relationships and tensions between links and nodes, and identifying critical nodes • Applies understanding of general systems to one's own discipline • Applies fundamental knowledge of theory to gain practice with applying systems thinking on the job• Understands/classifies the different systems approaches; uses various approaches based on the problem • Examines data, including big data, to identify correlations in the environment • Able to examine the system from basic to very big-picture perspectives • Sees systems throughout the environment, views those systems from multiple perspectives • Leverages experience to quickly take holistic view while also discerning relationships among elements
## Table 8 (continued)
NoviceIntermediateExpert
Depth of processing• Does not fully understand the system or question the task • The 'just get it done' leader • Does not think through all potential outcomes • Does not understand how their cog fits into the larger wheel • Is only able to see the face of the problem, not the whole of the problem• Beginning to understand how tasks fit into the larger system and question the procedures in a constructive way • Understands the 'why' behind the tasks to be accomplished • Is willing to accept new ideas, adapt processes that are not working, and beginning to understand impacts of decisions on neighboring or superordinate systems • Engages in critical thinking but may not do assessments on impact of solutions • Only capable of seeing 2nd or even 3rd order effects• Uses position power to make or modify the system • Quickly accepts and integrates new information to build an understanding of how the system will be impacted • Displays patience to effectively deal with uncertainty and ambiguity • Uses critical thinking to anticipate and overcome roadblocks or issues • Recognizes 2nd and 3rd order effects quickly; understands or uncovers 5th or 6th order effects
Communicating with and mentoring others• Does not have sufficient understanding of individual systems to effectively mentor/manage lower- level leaders to interact with or improve the system • Treats people like they are expendable; burns bridges• Can work with S1 (personnel officer) and mentor leaders to make improvements to systems • Understands the systems around them and their position in an interconnected environment • Learning to recognize when expert opinions are needed on the problem • Gaining skills in extracting expert opinions• Anticipates issues and prepares all staff sections so they can respond in a timely fashion • Can explain elements of the system and the impacts those elements might have on getting tasks done • Has a vision for the end state and works through the processes and interactions involved to get there
Using Systems Thinking on the Job. Operational participants were asked to describe ways in which they used systems thinking on the job. Responses were tabulated separately with respect to their current and previous jobs. When responding about systems thinking in their current job, leaders offered examples such as operating and maintaining systems (19% of Operational sessions); working with specific systems such as social systems (e.g., relationships within a battalion, onboarding new Soldiers to a battalion ), personnel systems (e.g., personnel databases, the promotion system), and training exercises (19%); planning and decision-making (15%), using feedback loops (15%), Mission Command/battle rhythms (15%), and knowledge management/gathering information (15%), among others. A summary and examples of systems thinking requirements that were described by respondents can be seen in Table 8. For example, respondents in one session (two captains) indicated that systems thinking was required when they planned training to ensure that timing and resources were coordinated. Respondents in another session (two majors) indicated that systems thinking was needed to work out time and resource factors among the various companies in the battalion. In yet another session, respondents (two majors) described Mission Command as a system of people working together to support their boss in the decision-making process. These respondents also described the targeting process in artillery units as requiring systems thinking. A respondent from the Adjutant General (AG) branch described numerous AG systems that existed with various information and databases, which required systems thinking to understand how they operated and fit together to get the information needed out of them. In another session a respondent from Public Affairs described needing systems thinking to deal effectively with communications crises or incidents that arose and to understand how various individuals, community groups, or other stakeholders were affected by various actions. When examining responses, it was noted that two categories of examples emerged: one category that involved thinking about systems that were well established and documented, and another category that involved identifying and thinking about systems that were not necessarily clearly defined or already identified as systems. So, for example, leaders who mentioned operating and maintaining systems described systems that already existed. On the other hand, leaders who mentioned working with specific systems described some systems that were well- defined, such as personnel systems, and other systems that were not well-defined, such as certain social systems. ## Table 8 ## Operational Participant Examples of Systems Thinking in Their Current Jobs
Response categoryPercentageExample(s)
Operating and maintaining systems19%•Working out time and resource factors among the various companies in the battalion.
## Table 8 (continued)
Response categoryPercentageExample(s)
Working with specific systems19%• Social systems, personnel systems, and training exercises. • Engaging in the targeting process in artillery units.
Planning and decision- making15%• Planning training to ensure that timing and resources are coordinated.
Using feedback loops15%• Dealing effectively with communications crises or incidents that arise and to understand how various individuals, community groups, or other stakeholders were affected by various actions.
Mission command/ battle rhythms15%• Mission Command is a system of people working together to support their boss in the decision-making process.
Knowledge management/ gathering information15%• Numerous Adjutant General (AG) systems exist with various information and databases which require systems thinking to understand how they operate and fit together to get the information needed out of them.
Note. N = 26 Operational focus group or interview sessions; percentages indicate the percent of the Operational sessions, out of 26 sessions, in which that example was provided. Participants in 12% of the Operational sessions indicated that there were times when existing systems had experienced complete system failure, and almost all Operational participants indicated that problems with existing systems often arose that they needed to address. Participants in 16% of Operational sessions indicated that they had used system thinking or some component of systems thinking to help determine fixes to problems, potential second- and third-order effects of problems, or patterns to understand the bigger picture of what was going on and what needed to be done. When asked about systems thinking in their previous jobs, responses were similar, although some officers used systems thinking in previous roles with more minor applications since they were previously at a lower rank with less responsibility and fewer systems to manage. For previous jobs, leaders in Operational sessions highlighted using systems thinking in areas such as sociocultural systems (19%), process management (19%), communication (15%), planning and decision-making (12%), military focused systems (12%), knowledge management (12%), systems operations (8%), feedback loops (8%), existing systems usage (MDMP/intelligence preparation of the battlefield; 8%), damage control (8%), prioritization (8%), forecasting effects (8%), and miscellaneous other areas (12%). When asked the level at which systems thinking was important for Army officers, respondents in most Operational and Institutional sessions (62% and 60% of the sessions, respectively) indicated that systems thinking was important at all levels. One reason that some respondents said that all levels needed systems thinking was because information flow among commanders and staff or leaders and subordinates was a system that could be improved. To leverage communication systems, leaders should start learning about systems early. Participants in 62% of the Operational sessions said that lieutenants should be starting to get familiar with systems terms and getting familiar with relevant systems. The sentiment that all levels are required to understand systems thinking was not universal, however. Some participants argued that lieutenants and captains should be focused on doing their jobs effectively and on accomplishing what they were tasked to accomplish, and that it was not until the rank of major or higher when officers began to have the purview and perspective to leverage systems thinking. Of the respondents who did not think all rank levels needed systems thinking, some (19% and 15% of Operational and Institutional sessions, respectively) said it was important for captains and above, and some (8% and 10%, respectively) said it was important for majors and above. SMEs in a few Institutional sessions (15%) said systems thinking was important for only lieutenant colonels and above, participants in one Operational session (4%) said systems thinking was important for only colonels and above, and participants in two Operational sessions (8%) said systems thinking was important for “higher ranks.” While respondents in most Institutional sessions (70%) and some Operational sessions (35%) did not specify any position for which systems thinking was particularly important, respondents in some sessions identified certain positions. For Operational participants, the most common position mentioned during the sessions was “staff positions” in general (77% of the sessions), with participants in some sessions specifically mentioning executive officer (XO; 38%) and/or operations officer (S-3; 31%). Other positions mentioned by participants in Operational sessions were battalion commander (15%) and specific branches/MOS (30%) such as aviation, intelligence, logistics, maintenance, and fire support. SMEs in Institutional sessions specifically mentioned staff positions (20% of the sessions), battalion commander (15%), and specific branches/MOS (30%) including cyber, intelligence, logistics, and “soft skill MOS” (e.g., MOS that required significant cognitive and/or interpersonal skills in addition to specific technical and physical skills needed for their MOS). This nuance suggests a complex relationship between organizational level, MOS, and systems thinking requirements. Below battalion level, requirements depend on one’s role and the type of unit the leader is in. However, regardless of MOS, systems thinking appears to be important at the highest levels and one respondent commented that the leaders who made it further in their careers as leaders were the ones who were good at systems thinking. Participants in Operational sessions associated several benefits with leaders engaging in systems thinking, such as improved information flow in the unit (12%), organizational adaptability/agility (8%), learning/open mindedness (8%), innovation (4%), building relationships/opening doors (4%), process improvements (4%), empowering people (4%), feedback loops (4%), and fresh perspectives (4%). Challenges associated with leaders who did not engage in systems thinking included poor leadership and decision-making (23%), failure to track the status of systems (23%), information overload (15%), having tunnel vision (15%), and communication problems (15%). Additional details can be seen in Table 9. ## Table 9 ## Challenges to Systems Thinking in the Army Identified by Operational Participants
Challenges to systems thinkingCount (%)
Bad/poor leadership or decision-making (e.g., bad apples or groupthink)6 (23)
Not understanding the system/tracking the statuses6 (23)
Information overload4 (15)
Tunnel vision/not seeing the full picture/focusing on minor processes4 (15)
Communication issues/information not being disseminated4(15)
Close-mindedness/conservative decision-making4 (15)
Close-Army does not set officers up for systems thinking3 (12)
System failure3 (12)
Bureaucracy/large entity that is not agile3 (12)
Not anticipating issues/forecasting2 (8)
Lack of education/experience2 (8)
Bad timing/delayed responses2 (8)
Budget/lack of funding1 (4)
Lack of consequences for not engaging in systems thinking1 (4)
Other2 (8)
Note. N = 26 Operational focus group or interview sessions; percentages indicate the percent of Operational sessions, out of 26 sessions, in which that challenge was mentioned. Systems Thinking Education and Training. Institutional SMEs were asked whether systems thinking was required for success in their course/program and whether there were specific topics that provided training on systems thinking. SMEs in 60% of the Institutional sessions said that officers in their program needed systems thinking to succeed; SMEs in 30% of the sessions stated they did not need systems thinking to succeed, and those in 10% of the sessions stated that they only needed systems thinking indirectly to succeed, as it was not directly assessed. SMEs in 20% of the Institutional sessions indicated that students needed systems thinking to pass specific assessments in the course. Some of those who indicated that systems thinking was not needed to succeed in the course/program specified that, although systems thinking was not required to pass the course, having systems thinking was helpful to understand and succeed in the course. When asked whether there were specific topics in their course/program that provided training on systems thinking, SMEs in 18 of the 20 Institutional sessions (90%) answered “yes.” When asked to list specific programs or topics that provided some type of instruction on systems thinking, SMEs in 55% of the Institutional sessions mentioned the U.S. Army War College (AWC) as having systems thinking instruction, and those in 30% of the sessions specifically mentioned the AWC Strategic Leadership Course. Various courses or topics that were specifically listed can be seen in Table 10. Other programs that were mentioned included the School for Advanced Military Studies (SAMS; 35%), Command and General Staff College (CGSC; 20%), and Red Team (15%). A few SMEs stated that systems thinking was not really trained or taught at all, or that if it was in the curriculum, it was not being taught effectively. ## Table 10 ## Specific Courses/Topics That Have Systems Thinking Training as Indicated by Institutional SMEs
Training topicsCount (%)
Army War College (AWC)11 (55)
Strategic Leadership course6 (30)
Other AWC course5 (25)
School of Advanced Military Studies (SAMS)7 (35)
Design thinking course4 (20)
Other SAMS course2 (10)
Command and General Staff College (CGSC) core curriculum4 (20)
Other CGSC lessons2 (10)
Design course1 (5)
Red Team3 (15)
National Training Center (NTC) Leader Training Program (LTP)1 (5)
NTC After Action Reviews (AARs)1 (5)
NTC Mentor/tutoring/one-on-one discussion1 (5)
School for Command Prep: Brigade course on decision-making1 (5)
Joint, Interagency, and Multinational Operations (JIMO): Joint operational planning process/Army Design methodology1 (5)
Note. N = 20 Institutional individual or group sessions; the percentages indicate the percent of Institutional sessions, out of 20 total Institutional sessions, that mentioned a given training program/topic. SMEs in 50% of Institutional sessions stated that students in their course/program received feedback on systems thinking, and those in 20% of the sessions said that students did not receive such feedback. SMEs in 30% of Institutional sessions stated that students received feedback indirectly; that is, students received feedback in areas related to systems thinking, even though the area was not identified as system thinking to the students. The systems thinking feedback was most described as provided by the instructor or observer/controller (OC) (70% of sessions), with SMEs in 25% of sessions indicating that officers received feedback from peers. Other descriptions of systems thinking feedback included feedback that was provided in evaluations during the course (15% of sessions), information from after action reviews (AARs; 10%), through self-awareness (10%), and during oral exams (5%). A few comments indicated that whether individuals received systems thinking feedback depended on their particular instructor/OC. When asked about courses they had taken that taught aspects of systems thinking, participants in most Operational sessions (58%) mentioned CGSC. Other courses participants mentioned included the Captains Career Course (CCC; 38%), West Point (15%), and Basic Officer Leader Course (BOLC; 12%; see Table 11). Participants in about one third of Operational sessions (31%) indicated that they learned about systems thinking on the job, and Operational participants in several sessions emphasized the importance of mentors, experience, and on-the-job training over specific courses or other educational processes. No Operational participants who were commissioned through the Reserve Officer Training Corps (ROTC) described having formal training in systems thinking, though some of those commissioned through West Point (15% of sessions) indicated that they had formal training in systems thinking, typically through the Systems Engineering program. ## Table 9 ## Operational Participant Responses About Professional Military or Officer Education Courses That Taught System Thinking
Professional military or officer educationCount (%)
Command and General Staff College (CGSC) or ILE15 (58)
Captains Career Course (CCC)10 (38)
On-the-job training8 (31)
U.S. Military Academy at West Point4 (15)
Basic Officer Leader Course (BOLC)3 (12)
Other (Canadian CGSC, Marines, business school, etc.)3 (12)
Pre-Command Course (PCC)2 (8)
Professional Military Education (PME)/Leader Professional Development2 (8)
Advanced Leader Course (ALC)1 (4)
SAMS - design course1 (4)
Note. N = 26 Operational focus group or interview sessions; percentages indicate the percent of Operational sessions, out of 26 sessions, total that mentioned a given program. Institutional SMEs were asked whether having a tool to assess systems thinking would be valuable and SMEs in 75% of the 20 Institutional sessions stated that a tool would be valuable. These SMEs described potential benefits of such a tool as: providing progress or developmental feedback, such as before and after scores (23% of sessions), having a diagnostic tool or something to see which systems were more easily understood (8%), evaluating performance or potential (8%), standardizing and providing a common language around systems thinking (8%), providing officers with self-awareness around systems thinking (8%), being able to incorporate systems thinking feedback into an existing tool (8%), exposing officers to systems thinking (4%), and other positive outcomes (e.g., drive conversations, determine working groups, etc.; 8%). Institutional SMEs also identified several challenges with having a tool to assess systems thinking, such as ensuring that instructors understood the concept themselves and were properly trained to discuss it (15% of sessions). Other challenges included ensuring that systems thinking was relevant/tied to the learning objectives and was memorable (12%), did not produce an additional burden on instructors (8%), was not overly complex (8%), was practical/pragmatic (8%), and could be customized rather than generic (4%). ## Discussion Army leaders must be effective systems thinkers to operate and maintain existing systems and to create plans, make decisions, and solve problems that involve a variety of interrelated elements. Most personnel who participated in the individual or group sessions viewed systems thinking as important to all officer levels, with systems thinking developing over time through experience, age, and training. Even though most respondents were familiar with the concept of systems thinking, many of the respondents did not have an organized understanding of the concept and were unsure how to define or describe it. An analysis of the words that respondents used in the individual and group sessions, however, revealed many terms in their responses that were commonly associated with systems thinking, such as perspective, design (i.e., Army Design), complexity, social system, feedback loop, and big picture. Notably, however, although respondents made references to aspects of systems thinking such as feedback loops, these references were superficial in nature, making it difficult to determine whether the terms were being used to specify a technical or a common meaning. In discussing the need for systems thinking, respondents identified two fundamental concepts of “systems” in the Army: formal systems, which are established and documented, and informal systems, which are not necessarily documented. Both types of systems are highly relevant and important. The formal systems in the Army, such as various personnel, communication, and planning systems, will tend to be clearly defined and may be readily identified or described by documents. While these systems may operate smoothly on their own for a time and do not need leader intervention, when problems arise with one or more elements of the system, the leader must then understand the elements and relationships in the system to fix the system and get it operating again. The informal or emerging systems such as social systems, informal coordination systems, and sociocultural systems in operational environments, tend to be more ambiguous, changing, and harder to identify. Operating with and within these systems requires proactive systems thinking to identify and understand the systems and understand their impact on decisions and problems. Respondents indicated they viewed systems thinking as something that developed over time, although several respondents emphasized that it did not just “happen”—the individual must be motivated and introspective to develop systems thinking over time and across their experiences. This perspective suggests that providing some degree of guidance and direction is likely to be useful in facilitating leaders’ development over time. As leaders move from novice to intermediate to expert levels of systems thinking, factors that distinguish higher levels of systems thinking include having greater knowledge about systems in general, greater knowledge of domain-specific systems, using deeper processing to plan or solve a problem, and skill at leveraging others when engaging in systemic action. Importantly, both general systems thinking skills and knowledge of specific systems need to develop in parallel. The majority of instructors consulted indicated that systems thinking was required for success in their course or program, although only one fifth of the instructors indicated that systems thinking was needed for specific evaluations. Nearly all instructors indicated that there were topics in their courses that provided some instruction relevant to systems thinking, particularly the AWC Strategic Leadership Course, SAMS, CGSC, and the Red Team curriculum. Half of the instructors indicated that their students received feedback on systems thinking. Although there is some degree of systems thinking being taught in various courses, there were some instructors who suggested there could be improvements to how systems thinking was taught across the various leadership courses. Most instructors thought a systems thinking assessment tool would be valuable if it was used as a diagnostic tool, such as providing pretest and posttest scores, and if instructors understood it and it did not create an undue burden on the instructors. In the following section we will describe the initial development of a conceptual model of leader systems thinking that is based on the information gained from the literature and the Army leaders. After the initial conceptual model is described, the results from additional interviews with active-duty officers are described that identified example behaviors for the model. Finally, institutional SMEs provided a final review of the full model with behavioral examples, and the SMEs provided feedback and suggestions. ## Model Development ## Academic SME Input After conducting the requirements review, four academic systems thinking SMEs were interviewed about the background information that had been obtained and the results of the requirements review. These SMEs hold PhDs in various mathematics, engineering, or other scientific fields, have each published multiple articles, books, or book chapters on systems thinking, work as consultants to the military and industry on systems thinking, and engage in teaching others about systems thinking through university faculty positions and as consultants. The SMEs concurred that a useful breadth of the information had been identified regarding systems thinking. They provided supplemental resources in certain areas, which were incorporated into the background section of this report. The SMEs indicated that the information learned from the Army leaders during the individual and group sessions concurred with their experiences working with and studying systems thinking in organizations. Two SMEs discussed the key elements of the Distinctions, Systems, Relationships, and Perspectives (DSRP) theory (e.g., see Cabrera & Cabrera, 2018), a theory that emphasizes balance between holism and reductionism in systems thinking. From their work with the Army, two SMEs emphasized that the Army environment was volatile, uncertain, complex, and ambiguous, and that a large proportion of the systems that military personnel dealt with were, therefore, complex and adaptive systems. Other SMEs echoed this view concerning the complexity of the current military environment and emphasized the importance of considering a leader’s “timespan of responsibility” when determining required systems thinking skills and abilities. Timespan of responsibility refers to how far forward (e.g., a few months, a few years, a few decades) a leader is expected to be planning. Timespan of responsibility generally, though not always, corresponds to the leader’s rank and time in service. Additionally, one SME explained that leaders sometimes must reduce and linearize this complexity to enable them to act. Another tendency noted was that some leaders apply systems thinking at the start of their planning processes, when seeking to understand a situation, but then abandon systems thinking once engaged in action. This SME indicated it would be helpful to enable leaders to incorporate systems thinking in both the planning phase and action phase of operations. Overall, the feedback from the academic SMEs confirmed the view that systems thinking is highly relevant to Army leaders and provided additional ideas and literature to guide the development of the conceptual model of systems thinking. ## The Conceptual Model Previous ARI research conceptualized Army leader systems thinking as a component of strategic thinking (Sackett et al., 2016). Therefore, our systems thinking model focuses on leader systems thinking as a component of strategic thinking. Our goal in developing the model was to use the information gathered from the systems thinking literature review and individual/group sessions with respondents to identify and define dimensions of leader systems thinking within the broader framework of strategic thinking. As identified previously, leader systems thinking was defined as a cognitive approach that applies a holistic perspective to identify and understand interrelationships and emergent properties among elements. Through an iterative process of identifying and reviewing systems thinking components found in the literature and considering these components vis-à-vis the construct definition and the individual/group session notes, five systems thinking themes emerged: Identifying Elements, Understanding Dynamic Relationships, Shifting Perspectives, Identifying Holistic Patterns, and Responding to Change. The conceptual definitions for each of the dimensions can be seen in Table 12. ## Table 12 ## Initial Systems Thinking Model Conceptual Definition: A cognitive approach that applies a holistic perspective to identify and understand interrelationships and emergent properties among elements.
DimensionDefinition
Identifying ElementsIdentifies people, objects, locations, or concepts in an environment or situation that together serve a purpose as a whole or nested whole
Understanding
DynamicConceptualizes nonlinear, bidirectional, and changing relationships among
Relationshipsa set of elements that form a whole or nested whole
ShiftingRecognizes how subordinate and superordinate levels relate to one another
Perspectivesin a system, switching perspectives from one level to the other as needed
Identifying Holistic PatternsUnderstands the larger picture of a system; understands common systemic patterns
Responding to ChangeRemains open to new information and takes multiple points of view when examining how a set of elements are related and interdependent; maintains a fluid and flexible concept of the boundaries that link elements
In the initial version of the model in Table 12, the dimension referred to as Shifting Perspectives was initially labeled Hierarchy, and the dimension referred to as Responding to Change was initially labeled Openness. These two dimension labels were subsequently changed following SME input. Once the conceptual dimensions were identified and defined, the next step was to identify a series of specific behaviors that would serve as exemplars for each dimension. These behaviors will provide a concrete foundation for understanding the essence of the dimension and offer observable actions that can serve as markers for assessing an individual’s capability in the five areas. Our objective was to have active-duty SMEs help us to identify behaviors associated with each of the five dimensions that were specific enough that the dimensions could be observed by others, but broad enough that the dimensions would apply to all or nearly all leaders. Interviews or focus groups were held remotely with active-duty officers to identify possible behaviors for each dimension. ## Identifying Systems Thinking Behaviors ## Method Participants. Participants were 29 active duty officers from three CONUS Army installations who participated in the interviews and focus groups via phone. The participants’ ranks included one colonel, five lieutenant colonels, 10 majors, and 13 captains, and their time in service ranged from four to 31 years. Twenty of the participants were categorized as being in a Combat Arms branch, eight participants were from a Combat Service Support branch, and one participant was from a Combat Support branch. Fifteen participants held a bachelor’s degree and 14 participants held a master’s, doctoral, or professional degree. Procedure. A semistructured interview was conducted for 30 to 60 minutes per session. Participants were asked to review the model and to identify example behaviors that were associated with each competency. The list of discussion questions can be seen in Appendix D. ## Results An initial draft list of specific behaviors was identified for each dimension based on notes from the interviews and focus groups. Researchers met as a group to discuss the set of behaviors for each dimension and review (a) relevance to the dimension definition, (b) potential overlap with and distinction from other dimensions, (c) optimal wording or phrasing, and (d) the appropriateness of the behavior’s specificity. The goal with regard to specificity of the behavior was to ensure the behavior was broad enough to apply across jobs but specific enough that it was something that raters could observe and use to distinguish between high and low performers. Researchers engaged in an iterative process of identifying potential behaviors, discussing the behaviors vis-à-vis the criteria, and making modifications. The behaviors selected for each dimension are described in the following sections. Identifying Elements. Officers discussed identifying elements for many different types of systems. It was important to extrapolate from the specific behaviors to create a more generally applicable statement. One officer discussed leading an effort to convert a Stryker Infantry brigade to an Armor brigade, and the challenge of trying to determine the various stakeholders who needed to be involved in the conversion process, as well as the various elements that were needed to make the conversion happen. Another officer talked about being on a deployment when problems arose with their mail system. This officer needed to identify the elements of the system to specify the source of the problem, then contacted individuals at each of the key organizations and started to sketch the system out on a white board. In this process, the officer started with putting “known” elements on the board and then worked to identify various “unknown” or missing elements. Based on a variety of examples, such as the examples that were provided by the officers as well as the definition of the dimension, the following behaviors were identified as descriptive of the dimension: • identifies stakeholders relevant to a situation, • identifies participants who have the perspectives needed for a meeting or decision, • identifies units and equipment needed for a mission, • recognizes relevant aspects of a problem, • understands the elements in a system that are critical to a mission, • determines the components of a complex or dynamic situation, and • identifies the boundaries that separate elements within the system from those external to the system. Understanding Dynamic Relationships. Several examples that were provided centered on understanding relationships among various social, political, and foreign military elements while on deployments. One officer described needing to understand complex political and social dynamics in the European theatre to work effectively on training missions with partners. Another officer detailed challenges of understanding local partnerships and loyalties when deploying to a new area of operations. Other examples involved understanding relationships involved in various planning or problem-solving activities. One officer described needing to understand the relationships in the system involved in making decisions in a Fires battalion, and the impact on the system of changing from using human scouts for information gathering to using unmanned aerial systems (UAS). Changing this one element of the system had a huge impact, by creating an enormous increase in the amount of information the officer received and needed to process in order to make decisions. Another officer discussed the importance of understanding the system involved in neutralizing targets on a mission and needing to understand the relationships in the targeting cycle across a period of time, including the maneuver planning process, battle rhythm events and meetings, and the interface with the Air Force and their systems and timelines. The following behaviors were developed as descriptive of this dimension, based on the examples provided by the officers: • understands complex connections among multiple elements in the relevant environment when developing a plan, • recognizes hidden, unusual, or nonlinear relationships among elements in the relevant environment when working through a problem, • visualizes the interaction of elements in the relevant environment using appropriate timeframes and scales of magnitude, • recognizes the interaction of elements in an overarching system during analysis of a course of action, • recognizes when decisions and actions are likely to have second- and third-order effects that affect the initial situation, • considers second- and third-order effects that may affect the unit or mission, • determines how the introduction of new factors in the environment will change existing relationships in a system, • identifies when certain elements of a system produce nonlinear or disproportionate effects, • understands how a changing situation can have complex effects on resourcing needs, and • uses feedback loops to learn about effects of decisions on different levels of the organization. Shifting Perspectives. Shifting perspectives focuses on understanding and switching perspectives from one level of a system to another. Officer examples described the importance of being able to shift one’s perspective from closely examining various elements and relationships when planning and making decisions to closely examining the impact of decisions and approaches on the different levels or different units. The resourcing systems were often mentioned as ones for which it was important to “zoom in” and understand the resource needs or impacts at the unit level, but also to “zoom out” and understand the resource needs or impacts on the operation as a whole or on the broader sustainment enterprise. The following behaviors were developed as possible examples to describe this dimension: • uses both big-picture information and details as necessary when briefing a plan, • ensures subordinate unit plans integrate into the larger mission, • understands effects of decisions from a higher level on the lower-level units, • understands how actions of a subordinate unit affect the mission of the larger organization, • coordinates actions effectively with both subordinate units and headquarters, • understands a specific subsystem in a mission when needed while also maintaining an understanding of the impact of that subsystem on the entire mission, • understands resourcing needs of subordinate units as well as the impact of resourcing decisions on other parts of the organization, • determines which problems need to be addressed at a higher level and which problems should be addressed at a lower level, • identifies relationships between microlevel factors that produce macrolevel effects • recognizes the emergence of higher-level events or phenomena from lower-level patterns, and • understands the organizational levels of other groups working in a system (e.g., joint, coalition, local government). Identifying Holistic Patterns. This dimension focuses specifically on understanding systemic patterns and the big picture of a system, which is essentially the “zoom out” perspective from the Shifting Perspectives dimension. For that reason, the discussion of behaviors for Shifting Perspectives and this dimension often overlapped. Behaviors for the two dimensions were considered carefully to ensure there were sufficient distinctions between the behaviors for the two dimensions. While behaviors for the Shifting Perspectives dimension focused on the action of switching from one perspective to another, behaviors associated with Identifying Holistic Patterns focused specifically on understanding the broad view and overall patterns. Because this dimension was very specific and unidimensional in its focus, only a handful of behaviors were identified to describe the dimension: • builds a graphic or model of a problem to better understand the problem, • understands a mission plan from a big-picture view, rather than just details of one or a few units, • understands how subordinate units integrate to accomplish a mission, • identifies patterns that emerge in how various elements within a mission interact, and • considers a situation or problem as a whole. Responding to Change. Responding to Change focuses on leaders continually remaining open to new information or perspectives that may impact their understanding of a system. One officer assigned to a combat training center talked about the system for planning and executing rotations, and how the current pandemic introduced new elements into the system that had multiple impacts. An officer who had been deployed to an area where they were working with many different agencies described the importance of remaining open to the different emerging information and perspectives from the different agencies. Several examples were provided from the Field Artillery perspective. One officer described how the fire support plan was based on the maneuver plan, so any change in maneuver plans necessitated a review of the fire support plan. Another officer emphasized the importance of continually remaining open to new intelligence that affected targeting. A third officer described how the introduction of using precision guided munitions meant that weather was no longer an element of the planning system, but that it took some time for leaders to adjust to the fact that weather was no longer being relevant. The following behaviors were developed as examples to describe this dimension: • identifies emerging information that should be included in the current understanding of a situation or problem, • remains open to changing the framework or model of a situation when necessary, • understands multiple points of view that expand the understanding of a situation or problem, • considers different points of view when examining the interrelationships among the elements in a plan, • remains open to changing plans due to changing circumstances, • recognizes the need to review resource allocation when circumstances change, • remains open to other perspectives, • remains open to the changing stakeholders involved in solving a problem or making a decision as the situation changes, and • reconsiders the critical elements in the environment as new information emerges. ## Subject Matter Expert Reviews Once the initial set of behavioral examples was developed, institutional instructors were asked to review the behaviors in the model with respect to each dimension and provide feedback and suggestions for changes. ## Method Participants. Fifteen institutional SMEs were consulted individually to obtain feedback on the draft behaviors. One SME was an active duty officer and the other 14 SMEs were Army civilians. Twelve SMEs had earned a graduate degree and three SMEs did not report degree information. The SMEs’ ranks or former ranks were as follows: 10 colonels, one lieutenant colonel, one major, and three SMEs did not report any rank information. All SMEs were instructors in leader development programs. Many of the SMEs had also participated in the first round of individual and group consultations and were therefore somewhat familiar with the research project. Procedure. At the beginning of the session, SMEs answered the demographic questions and reviewed a list of behaviors that may be indicative of each systems thinking dimension (see Appendix E). A consultation session was then conducted for 30 to 60 minutes per session. SMEs were asked to provide feedback on the model in general, the utility of the competencies, the applicability of the behaviors across jobs, and the accessibility and conciseness of the wording used throughout the model. Feedback regarding behaviors that were particularly important (or, conversely, particularly unimportant) was solicited in an effort to determine which behaviors could be dropped or be combined and which behaviors needed to be kept. The model was reviewed in detail as SMEs provided this feedback for each dimension in turn. The list of discussion questions can be seen in Appendix F. ## Results and Final Behaviors Qualitative data were combined across the SMEs and summarized for each dimension. Three types of feedback were provided for each dimesion: general positive feedback, general constructive criticism, and changes or suggestions for specific behaviors. In addition, some SMEs provided feedback on the model as a whole, on the dimensions, and on the dimension definitions. General Feedback. The overall model received positive comments. SMEs indicated that the model was useful and that the dimensions and behaviors described in the model were important for success as an Army officer. SMEs also indicated that systems thinking was important to engage in across all levels of leadership. However, many SMEs also indicated that it became even more important to understand and engage in systems thinking as Soldiers moved up in rank. Most SMEs indicated that all five dimensions were important and should be retained. Many SMEs suggested that the definitions and behaviors were too wordy and complex, and therefore should be simplified wherever possible. SMEs also pointed out areas where more consistency was needed, such as either using the term “whole” or “nested whole” but not using these terms interchangeably throughout the model. One SME pointed out that some of the verbs in the behavioral statements referred to internal processes that might be more difficult and subjective to rate; for example, “understands” or “remains open to.” The suggestion was made to change the verbs to something more observable, such as “discuss.” Based on this feedback, the format of the behavioral items for each dimension was changed to focus on whether Army leaders “describe or discuss” the various systems thinking issues related to each dimension. Finally, several SMEs noted that some of the behavioral items (e.g., “Identifies units and equipment needed for a mission”) were relevant to certain leader positions but would not be relevant to other positions. The research team determined that this situation was not necessarily a problem because the behaviors were intended to be examples of relevant behavior, and therefore did not need to be relevant to every leader position. However, item wording was reviewed to ensure that the behaviors were as inclusive as possible. Identifying Elements. SMEs generally provided positive feedback for this dimension and they indicated that the behavioral statements seemed to be comprehensive and appropriate. SMEs noted that some of the items, such as the first and the second behavioral items, were redundant and therefore could be combined or dropped. One SME suggested adding another behavior that captured whether an officer paid attention to elements that could become relevant in the future: “Identifies elements that have relevance to the system as well as elements that don’t have direct relevance that may become relevant.” This behavior was not added, however, because the suggested behavior was similar to a behavior that was already included in the Responding to Change dimension: “Reconsiders the critical elements in the environment as new information emerges.” Based on the feedback from the Institutional SMEs, the second and sixth items were dropped and the third, fifth, and seventh items were modified. The final definition and behaviors for Identifying Elements can be seen in Table 13. ## Table 13 ## Identifying Elements - Final Description
DefinitionIdentifies people, objects, locations, events, or concepts in an environment or situation that function together as an integrated whole
Behaviors1. Stakeholders relevant to a situation
This dimension involves cognitive activities that are demonstrated by an Army leader describing or discussing:2. Resources needed for a mission 3. Relevant aspects of a problem 4. Critical elements (people, objects, location, events, concepts) that are important to the operational environment 5. Distinguishing critical elements within the system from those that are less critical
Understanding Dynamic Relationships. Although in general SMEs agreed that Understanding Dynamic Relationships was an important dimension of systems thinking, SMEs provided a number of constructive suggestions for changes to this construct. Some SMEs indicated that the definition for this dimension was too complicated and needed to be simplified. The term “nonlinear relationships” was particularly confusing to some, though not to others. One SME indicated that one relevant relationship concept that seemed to be missing from the behaviors was identifying causality in relationships and distinguishing causality from correlation. SMEs also indicated that another concept that could be added was the concept of time horizon in the behaviors; that is, that cause-and-effect relationships in a complex system may take place over a period of time rather than immediately. Finally, one SME suggested removing the contextualized modifiers from the behaviors, such as “when developing a plan,” in order to make the behaviors more broadly applicable across jobs. Based on the feedback from the Institutional SMEs, item numbers 3, 4, 8, and 9 were eliminated, numbers 5 and 6 were combined and modified, and the wording of the other items was modified. The final five behaviors and the dimension definition can be seen in Table 14. ## Table 14 ## Understanding Dynamic Relationships - Final Description
DefinitionUnderstands complex causal and correlational relationships among a set of elements that form an integrated whole
Behaviors This dimension1. Complex connections among multiple elements in the environment when developing a plan
involves cognitive activities that are2. Hidden, unusual, or nonlinear relationships among elements in the environment
demonstrated by an Army leader describing or discussing:3. Second- and third-order effects that may affect the unit or mission 4. How the introduction of new factors in the environment will change existing relationships in a system 5. How to use feedback loops to determine the impact of decisions and actions
Shifting Perspectives. SMEs indicated that Shifting Perspectives was a useful dimension that was important and highly relevant to systems thinking in the Army. General feedback on this dimension focused on two aspects. First, at the time of the data collection this dimension was called “Hierarchy,” and a number of SMEs pointed out that the term hierarchy implied specifically a subordinate and superordinate relationship, excluding taking different perspectives from peer-to-peer relationships such as between units. Based on this feedback, the name of the dimension was changed from Hierarchy to Shifting Perspectives to ensure the dimension included switching between perspectives other than just subordinate and superordinate relationships. Second, multiple SMEs indicated that this dimension seemed so closely related to Identifying Holistic Patterns that perhaps the two dimensions could be combined. These SMEs suggested that Identifying Holistic Patterns was inherently nested within the concept of Shifting Perspectives. The research team considered this feedback and the conceptual overlap but decided to keep both as separate dimensions because the essence of the two dimensions was distinct. Shifting Perspectives specifically focuses on the behavior of switching from one perspective to another. Shifting perspectives may at times require shifting from a micro view to a holistic view, but may at other times require shifting from the perspective of one unit to the perspective of another unit. Identifying Holistic Patterns focuses on behaviors associated specifically with understanding the big picture. The feedback from the Institutional SMEs was used to eliminate and modify items to a final list of six, as shown in Table 15. Four of the original items were removed: 2, 7, 8, and 10. Items 3 and 4 were combined and the wording was modified. The remaining items’ wording were modified to make the items more inclusive of multiple types of units or situations. ## Table 15 ## Shifting Perspectives - Final Description
DefinitionTakes perspectives from different subsystems or levels of a system and switches perspectives as needed (e.g., higher level, lower level, lateral levels, different subsystems)
Behaviors This dimension involves cognitive activities that are demonstrated by an Army leader describing or discussing:1. Both big-picture information and details as necessary when briefing a plan 2. How decisions at one level or unit affect other levels or units 3. How actions of a specific group affect the mission of the larger organization 4. How to effectively coordinate actions between units and headquarters 5. The roles and impacts of multiple groups working in a system (e.g., joint, coalition, local government) 6. How actions of a specific group affect the mission of other groups or stakeholders
Identifying Holistic Patterns. Most SMEs indicated that they found Holism to be an important aspect of systems thinking, with seven SMEs indicating that this dimension was an especially important dimension for Army officers. SMEs found the definition and behaviors to be understandable and concise. However, as mentioned previously, some SMEs indicated that Identifying Holistic Patterns was somewhat redundant with Shifting Perspectives and the two dimensions could be combined. SMEs indicated that the behaviors used to describe this dimension were important and understandable. Based on the feedback from the Institutional SMEs, only one item (#3) was dropped, and the wording for the other four items was modified. The final four behaviors and the dimension definition can be seen in Table 16. ## Table 16 ## Identifying Holistic Patterns - Final Description
DefinitionUnderstands the larger picture of a system and common systemic patterns
Behaviors1. A situation or problem as a whole
This dimension involves cognitive activities that2. A mission plan from a big-picture view, rather than just from the perspective of one element
are demonstrated by an Army leader describing3. Patterns that emerge between elements within an operational environment
or discussing:4. The development of a graphic or model to better understand and communicate the problem
Responding to Change. The Responding to Change dimension received mixed feedback from SMEs. While all SMEs indicated that the behaviors described in this dimension were important to success as a leader and widely used, some SMEs were concerned that the behaviors were internal processes and difficult to observe. This situation would mean that in the future when the behaviors are used to develop a rating form, raters would have trouble making objective ratings on the dimension. For example, one phrase used multiple times in the items was “Remains open to…” which some SMEs suggested would be difficult to observe. These SMEs also indicated that it would be important to seek out opposing opinions from others rather than just passively remaining open to those opinions. Another issue that surfaced was confusion from the name of the dimension. In the version reviewed by SMEs, this dimension was titled “Openness.” A number of the SMEs were familiar with the personality construct labeled “Openness,” and pointed out that it could create confusion to use “Openness” as a dimension title because raters might make assumptions about the meaning of the dimension if they were familiar with the personality construct. Based on the feedback provided by the Institutional SMEs, the original nine items were reduced to five. Items 2 and 6 were removed, and items 3, 4, and 7 were combined and modified to form a single item. The wording of the other items was modified based on the feedback. In addition, the name of the dimension was changed to Responding to Change. The final five behaviors and the dimension definition can be seen in Table 17. ## Table 17 ## Responding to Change - Final Description
DefinitionRemains open to emerging information and updates understanding of system as needed; maintains a flexible concept of the system boundaries
Behaviors This dimension involves cognitive activities that are demonstrated by an Army leader describing or discussing:1. Emerging information that should be considered to understand a situation or problem 2. Taking multiple points of view to expand their understanding of a situation or problem 3. Changes to plans or approaches that are needed due to changing circumstances 4. The need to change the stakeholders involved in solving a problem or making a decision when the situation changes 5. Redefining the critical elements in the environment as new information emerges
## Limitations and Future Research There are several noteable challenges and limitations in this research. One is that systems thinking is a cognitive activity, and thus involves numerous thinking behaviors that take place within an individual’s mind that are not observable to others. This limitation presents a particular challenge when developing a model and an assessment that are grounded in observable behaviors. The solution the researchers used to address this challenge was to focus on whether leaders had “described” or “discussed” various factors, which would then reveal the internal cognitive activity. It is possible, however, that leaders would engage in the cognitive behaviors of systems thinking without describing or discussing those cognitive behaviors of systems thinking to or with others, leading to an incorrect assessment. Given the interactive nature of Army planning and mission execution, the researchers determined that there would be a low likelihood of leaders not needing to share their thoughts and understanding with others. Another challenge was developing a systems thinking model and assessment that were appropriate for the wide breadth of jobs and levels that Army leaders hold. The behaviors identified had to be general enough to apply across all of the jobs and levels, yet specific enough that the behaviors could be recognized and observed by raters. The SMEs who reviewed the behaviors and indicated they were relevant had backgrounds in various Army branches, which provided some degree of confirmation that the behaviors were relevant across the breadth of jobs and levels. Optimally, however, job incumbents from across the entire range of applicable jobs and levels would rate the importance of the behaviors to their specific job. This approach would validate the direct relevance of the model and assessment content for that job. This type of extensive validation was not possible, however, due in part to limited availability of personnel due to COVID-19-related shutdowns, as well as due to the massive cost and effort it would require to collect ratings from across the numerous Army jobs and levels. Another way in which the project needed to be scoped to fit the available time and budget was to limit the the population of participants. Although it would be useful to examine systems thinking requirements for NCOs and Warrant Officers, this project focused only on commissioned officers as the population of interest. This decision was made for practical reasons, as it would not have been feasable to examine all three groups due to time and resource constraints. Officers were chosen as the target population because they are more likely to engage in systems thinking due to the nature of the decision-making and problem-solving requirements in their jobs. NCOs, especially at higher levels, are also likely to engage in systems thinking, and this would be a potential area for future research. While our research effort focused solely on systems thinking in leaders themselves, leader decision-making and problem-solving occurs within the context of a teams, or more accurately a team of teams. Another area for future research would be to examine systems thinking in the context of teams. Two questions of particular interest are: (1) How do leaders affect systems thinking processes within their teams? (2) To what extent can systems thinking within a team rely on a single individual, or is there a need for more than one individual to have systems thinking capabilities (e.g., see Sorrells et al., 2005)? This line of research is related to the concept of thought diversity within teams and the research would likely be highly dependent on the team’s context and goals. Finally, much of the discussion of systems thinking in this research has focused on systems thinking involving systems within the Army. At a broader level, systems thinking in the Army involves decision-making and problem-solving in which the enemy represents one or more elements of the system; that is, the enemy is an integral part of the system, and one that the leader does not control. From this perspective, Army leaders engaging in systems thinking need to consider “enemy focused” perspectives of systems thinking, specifically considering how to incorporate adversaries into their systems concepts. This type of systems thinking might be something that members of certain MOS (e.g., special operations) do more than others. Examining systems thinking from the perspective of planning and problem-solving with enemy forces elements could also be a productive topic for future research. ## Conclusions This research sought to develop a conceptual model that describes the behaviors demonstrated by leaders who engage in successful systems thinking. An evaluation of the requirements for systems thinking in the Army found that many leader planning, decision- making, and problem-solving situations require some degree of systems thinking. Some of these situations involve working with established and documented systems, presenting only a minimal or moderate challenge for leaders as the elements and relationships in the system are well documented and described. At the other extreme, there are situations that involve systems that are undefined, open, and dynamic, requiring that the leader define and establish the elements and relationships in the system and proactively determine the impact of constantly changing systems on their plans and decisions. Leaders at field grade and higher levels tend to encounter the dynamic and ambiguous systems more often than company grade leaders, but systems thinking requirements will differ by branch and position, and leaders at all levels may need to work within complex dynamic systems, such as the sociocultural systems they encounter on deployments. Given the variety of requirements, the leader systems thinking model needed to be general enough to apply to a wide range of systems with different characteristics and different types of elements and relationships. The conceptual model that was developed identifies and describes five distinguishing dimensions of systems thinking: (a) identifying the elements that comprise the system, (b) understanding dynamic relationships among them, (c) switching among multiple perspectives to understand the system, (d) taking a holistic view of the system, and (e) watching for new and emerging information that modifies or further defines the system. The specific behaviors identified for each of the dimensions were written to enable the model to apply to a variety of different types of systems and situations. Having a conceptual model of leader systems thinking provides the foundation needed to develop systems thinking assessment and training tools for Army leaders. The dimensions and associated behaviors were subsequently used to form the content of a multirater assessment tool (see Loer et al., in preparation), and will also be used as the foundation for an interactive scenario-based tool to assess leader systems thinking in specific situations or environments. These assessment tools will enable leaders to identify their development needs and specific behaviors on which to focus as the leaders build their systems thinking capabilities. Institutional personnel consulted during the requirements analysis viewed systems thinking as important to all officer levels, with systems thinking developing over time through experience, age, and training. SMEs indicated that courses at AWC, CGSC, SAMS, and USMA provided some instruction relevant to systems thinking, although the instruction may be indirect and systems thinking was not usually captured in student evaluations. Students are also not likely to have an opportunity to receive feedback on their systems thinking. The assessments tools associated with the leader systems thinking model can provide insight and feedback regarding system thinking and facilitate the development of systems thinking over time. Finally, it is worth noting that, as discussed in the background section of this report, there are many different definitions and conceptualizations of systems thinking in the literature. This research considered those conceptualizations and developed a specific definition and approach determined to be useful for the purposes of developing content for a multirater assessment tool appropriate to assess systems thinking across all Army leaders. While our definition and competency model drew heavily from previous academic and military literature, the definition also incorporated ideas generated from Army leaders in order to produce a model that captures the application of systems thinking in the Army. Our result provides a useful and practical approach to thinking about and measuring systems thinking in Army leaders, but our approach does not purport to be the only or the single best approach to defining and assessing leader systems thinking. ## References Adis, C., Wisecarver, M., Key-Roberts, M., Hope, T., Pritchett, C. S., & Henderson, J. (2013). Building sociocultural system thinking capabilities (Project Report). Personnel Decisions Research Institutes. Allen, C. D., Cunningham, G. K., & Klinger, J. (2009). Systems thinking for strategic leaders. Army War College. https://apps.dtic.mil/sti/pdfs/ADA592793.pdf Arnold, R. D., & Wade, J. P. (2015). A definition of systems thinking: A systems approach. Procedia Computer Science, 44, 669–678. https://doi.org/10.1016/j.procs.2015.03.050 Behl, D. V., & Ferreira, S. (2014). Systems thinking: An analysis of key factors and relationships. Procedia Computer Science, 36, 104–109. Bentley, B., & Davy, S. M. (2009). Military decision-making and soft systems methodology. In E. P. Greener & J. Stouffer (Eds.), Decision-making: International perspectives (pp. 669– 679). Canadian Defense Academy Press. http://madgic.library.carleton.ca/deposit/govt/ca_fed/DND_decisionmaking_2010.pdf Boyatzis, R. E., Goleman, D., & Rhee, K. (2000). Clustering competence in emotional intelligence: Insights from the Emotional Competence Inventory (ECI). In R. Bar-On and J.D.A. Parker (Eds.), Handbook of emotional intelligence (pp. 343–362). Jossey-Bass. https://weatherhead.case.edu/departments/organizational- behavior/workingPapers/WP%2099-6.pdf Cabrera, D. & Cabrera, L. (2018). Systems thinking: The crux of organizational learning. Plectica Publications. https://www.plectica.com/publications/crux-of-organizational- learning Cabrera, D., Cabrera, L., & Cabrera, E. (2020a). A literature review of the universal patterns and atomic elements of complex cognition. Journal of Applied Systems Thinking, 20(6), 1– 48. https://doi.org/10.13140/RG.2.2.22452.78727 Cabrera, D., Cabrera, L., & Gibson, H. (2020b). Leapfrog leaders: How lowlighting content and highlighting cognitive structure and dynamics can leapfrog leaders to the next level. Journal of Applied Systems Thinking, 20(5), 1–17. https://help.cabreraresearch.org/leapfrog-leaders Cabrera, D., Cabrera, L. & Powers E. (2015) A unifying theory of systems thinking with psychosocial applications. Systems Research and Behavioral Science, 32(5), 534–545. Cabrera, D., Cabrera, L., & Troxell, G. (2020c). The future of systems X? If you want a better systems X discipline or program, upgrade 2 your systems thinking to version 4.0. Journal of Applied Systems Thinking, 20(5), 1–13. https://help.cabreraresearch.org/the-future-of- systems-x Cabrera, D., Colosi, L., & Lobdell, C. (2008). Systems thinking. Evaluation and Program Planning, 31(3), 299–310. https://doi.org/10.1016/j.evalprogplan.2007.12.001 Cabrera Research Lab (2021). Systems thinking personal inventory. https://store.cabreraresearch.org/products/systems-thinking-personal-inventory Checkland, P. (1981). System thinking, systems practice. Wiley. Conti, T., Fitness, O. S., & Saudino, V. (2009). Systems thinking: The new frontier in quality management. In Proceedings of the 53rd Congress of European Organization for Quality. Curran Associates, Inc. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.462.5794&rep=rep1&type=pdf Dedmond, W. K. (2014). Systems thinking: A way to understand sociopolitical environments. Army Command and General Staff College. https://apps.dtic.mil/sti/pdfs/ADA614191.pdf Dennis, M. B. (2010). Systems thinking and design: Making learning organizations a reality in the U.S. Army. Army Command and General Staff College, School of Advanced Military Studies. https://apps.dtic.mil/sti/pdfs/ADA536647.pdf Dibble, D. (2011). Systems thinking is leadership. The Systems Thinker. https://thesystemsthinker.com/%EF%BB%BFsystems-thinking-is-leadership Dixon, R. G. (2006). Systems thinking for integrated operations: Introducing a systemic approach to operational art for disaster relief. Army Command and General Staff College, School of Advanced Military Studies. https://apps.dtic.mil/sti/pdfs/ADA449979.pdf Eifler, B. S. (2011). Developing strategic thinking leaders in the U.S. Army. Army War College. https://apps.dtic.mil/sti/pdfs/ADA560994.pdf Gumbert, J. (1998). Leadership in the digitized force. Military Review, 78(1), 13–23. Henning, P. B., Wilmhurst, J., & Yearworth, M. (2012). Understanding systems thinking: an agenda for applied research in industry [Paper presentation]. 56th Annual Meeting of the ISSS-2012, San Jose, CA, United States. https://journals.isss.org/index.php/proceedings56th/article/view/1909/612 Hill, A., & Gerras, S. (2016). Systems of denial: Strategic resistance to military innovation. Naval War College Review, 69(1). Jackson, M. C. (2003). Systems thinking: Creative holism for managers. Wiley. http://dspace.vnbrims.org:13000/jspui/bitstream/123456789/1166/1/Systems%20Thinkin g.pdf Kim, D. H. (1999). Introduction to systems thinking. Pegasus Communications. https://thesystemsthinker.com/wp-content/uploads/2016/03/Introduction-to-Systems- Thinking-IMS013Epk.pdf Kvedaravičius, J. E., Skaržauskienė, A., & Palaima, T. (2009). Systems thinking as intelligence competence and its relationship to leadership performance. Problems and Perspectives in Management, 7(2), 75–85. Lamb, C. T., & Rhodes, D. H. (2009, March 23–26). Collaborative systems thinking: Uncovering the rules of team-level systems thinking [Paper presentation]. 3rd Annual Institute of Electrical and Electronics Engineers International Systems Conference, Vancouver, Canada. https://ieeexplore.ieee.org/abstract/document/4815837/footnotes#footnotes Loer, A., Byrd, C., Wisecarver, M., Adis, C., Engelsted, L., & Weyhrauch, W. (in preparation). Multirater assessment for systems thinking (MAST) (Research Product). U.S. Army Research Institute for the Behavioral and Social Sciences. Maani, K. E., & Maharaj, V. (2004). Links between systems thinking and complex decision- making. System Dynamics Review: The Journal of the System Dynamics Society, 20(1), 21–48. https://doi.org/10.1002/sdr.281 Macaulay, D. (2011). Systems thinking: A foundational art for strategic leaders. Army War College. https://apps.dtic.mil/sti/pdfs/ADA547424.pdf Martens, B. R. (2011). The impact of leadership in applying systems thinking to organizations [Paper presentation]. 55th Annual Meeting of the International Society for the Systems Sciences, Hull, United Kingdom. https://journals.isss.org/index.php/proceedings55th/article/view/1648/581 Merali, Y., & Allen, P. (2011). Complexity and systems thinking. In P. Allen, S. Maguire, & B. McKelvy (Eds.), The SAGE handbook of complexity and management (pp. 31–52). SAGE Publications. Midgley, G. (2003). Systems thinking (1st Ed.). Sage Publications. Miles, M. B., Huberman, A. M., & Saldaña, J. (2014). Qualitative data analysis (3rd ed.). SAGE Publications. Morgan, P. (2005). The idea and practice of systems thinking and their relevance for capacity development. European Centre for Development Policy Management. https://ecdpm.org/wp-content/uploads/2005-Idea-Practice-Systems-Thinking-Relevance- Capacity-Development.pdf Palaima, T., & Skaržauskienė, A. (2010). Systems thinking as a platform for leadership performance in a complex world. Baltic Journal of Management, 5(3), 330–355. https://doi.org/10.1108/17465261011079749 Reynolds, M. (2011). Critical thinking and systems thinking: Towards a critical literacy for systems thinking in practice. In C. P. Horvath & J. M. Forte (Eds.), Critical thinking (pp.37–68). Nova Science Publishers. https://oro.open.ac.uk/30464/1/Reynolds- CTchapterFINAL.pdf Ryan, A. J. (2011). Military applications of complex systems. In D. M. Gabbay, P. Thagard, & J. Woods (Eds.), Philosophy of complex systems (pp. 723–780). North Holland. https://doi.org/10.1016/B978-0-444-52076-0.50024-9 Ryan, A. J. (2014). A framework of systemic design. Form Akademisk, 7(4), 1–14. Schwandt, J. (2018, July 5). The nature of logistics: Mimic ant colonies to reshape battlefield supply. Association of the United States Army. https://www.ausa.org/articles/nature- logistics-mimic-ant-colonies-reshape-battlefield-supply Schwandt, J. & Ryan, T. (2018, December 18). Thinking about thinking: Soldiers have a better way of solving problems. Association of the United States Army. https://www.ausa.org/articles/%E2%80%98thinking-about-thinking%E2%80%99- soldiers-have-better-way-solve-problems Sackett, A. L., Karrasch, A. I., Weyhrauch, W. S., & Goldman, E. F. (2016). Enhancing the strategic capability of the Army: An investigation of strategic thinking tasks, skills, and development (Research Report 1995). U.S. Army Research Institute for the Behavioral and Social Sciences. https://apps.dtic.mil/sti/pdfs/AD1006147.pdf Savage, A., & Sales, M. (2008). The anticipatory leader: Futurist, strategist and integrator. Strategy & Leadership, 36(6), 28–35. https://doi.org/10.1108/10878570810918331 Senge, P. M. (1985). Systems principles for leadership. Massachusetts Institute of Technology. http://dspace.mit.edu/bitstream/handle/1721.1/2137/SWP-1758-14371964.pdf Senge, P. M. (2006). The fifth discipline: The art and practice of the learning organization (2nd ed.). Random House Business. Skaržauskienė, A. (2010). Managing complexity: Systems thinking as a catalyst of the organization performance. Measuring Business Excellence, 14(4), 49–64. https://doi.org/10.1108/13683041011093758 Skaržauskienė, A. (2009). Systems thinking as a competence in the leadership paradigm. Management Theory and Studies for Rural Business and Infrastructure Development, 16(1), 97–105. Skaržauskienė, A. (2008). Theoretical insights to leadership based on systems thinking principles. Organizacijų Vadyba: Sisteminiai Tyrimai, 48, 105–120. https://www.ceeol.com/search/article-detail?id=135652 Skaržauskienė, A., & Jonušauskas, S. (2013). Defining innovative competencies of leading: Systems intelligence. International Journal of Innovation and Regional Development, 5(2), 132–148. https://www.researchgate.net/profile/Aelita- Skarzauskiene/publication/264821184_Defining innovative_competencies of leading_s ystems_intelligence/links/547f6e880cf250f1edbdc621/Defining-innovative- competencies-of-leading-systems-intelligence.pdf Sorrells, W., Downing, G., Blakesley, P., Pendall, D., Walk, J., & Wallwork, R. (2005). Systemic operational design: An introduction. School of Advanced Military Studies, U.S. Army Command and General Staff College. Stroh, D. P., & Zurcher, K. (2012). Acting and thinking systematically. The Systems Thinker. https://thesystemsthinker.com/acting-and-thinking-systemically Strong, B. E., Babin, L. B., Zbylut, M. R., & Roan, L. (Eds.). (2013). Sociocultural systems: The next step in Army cultural capability (Research Product 2013-02). U.S. Army Research Institute for the Behavioral and Social Sciences. https://apps.dtic.mil/sti/pdfs/ADA591027.pdf Tideman, S. G., Arts, M. C., & Zandee, D. P. (2013). Sustainable leadership: Towards a workable definition. Journal of Corporate Citizenship, 49, 17–33. Valerdi, R., & Rouse, W. B. (2010). When systems thinking is not a natural act (Technical Report). Institute of Electrical and Electronics Engineers. https://dspace.mit.edu/bitstream/handle/1721.1/84042/CP_100405_Valerdi_Rouse_IEEE. pdf?sequence=1&isAllowed=y Van Riper, P. (2012). The foundation of strategic thinking. Infinity Journal, 2(3), 4–10. https://www.militarystrategymagazine.com/article/the-foundation-of-strategic-thinking Wheeler, T. A. (1999). U.S. Army rapid deployment task force dynamics - Rapid teaming. Army Command and General Staff College, School of Advanced Military Studies. https://apps.dtic.mil/sti/pdfs/ADA383677.pdf Williams, J. D. (2007). Is the U.S. Army a learning organization? Army War College. https://apps.dtic.mil/sti/pdfs/ADA469631.pdf Wisecarver, M., Adis, C., Key-Roberts, M., Hope, T., Pritchett, S., & Henderson, J. (2013). Sociocultural systems thinking: A development roadmap. Personnel Decisions Research Institutes. Zahn, E. (1999). Strategizing needs systems thinking [Plenary]. 17th International Conference of the System Dynamics Society and Australian New Zealand Systems Conference, Wellington, New Zealand. https://proceedings.systemdynamics.org/1999/PAPERS/PLEN11.PDF ## Appendix A ## Sociocultural Competency Model From Wisecarver et al. (2013)
Sociocultural Systems Thinking
DefinitionContinuously builds and analyzes a mental model of the sociocultural relationships and dynamics that exist in an environment, and leverages this model when planning, preparing for, and conducting missions.
Actions• Understands the key organizations/groups in an area/region and each of their roles. • Demonstrates knowledge and understanding of the actors (e.g., political, military, cultural) within a region as well as the key operational relationships. • Is familiar with different organizational power structures, communication styles, and technologies, and understands their potential impact on the sociocultural systems, planning, and decision-making. • Continuously captures new information regarding individuals, organizations, and groups and revises his/her mental model of the local sociocultural structures and relationships. • Asks questions of staff and units to determine the potential effects of mission decisions on system elements. • Identifies the possible impact of system changes on the mission by asking targeted questions of supporting staff and units. • Comprehends the interdependencies between systems, decisions, and organizations and the tools that support their management. • Considers multiple facets of a situation or problem, how they relate to one another, and the perspectives and needs that the key players contribute. • Considers the impact of the regional interaction of local government, opposition parties, and other groups, on mission planning and execution. • Leverages knowledge of formal and informal leadership, systems, and organizational dynamics in the local area to accomplish the unit’s mission. • Can describe the relationships among the various political, military, economic, and cultural organizations and other players involved in operations and considers their role when planning or executing missions.
Development StageYears in ServiceIllustrative Actions at Each Level
Sustain17+ years• Builds and diagrams a mental model of the sociocultural systems (political, military, economic, cultural) and relationships in a vast and highly complex area of operations. • Analyzes and evaluates the impact of the regional interaction of local government, opposition parties, and other groups, on mission planning and execution. • Leverages knowledge of formal and informal leadership, systems, and organizational dynamics in the local area to accomplish the unit’s mission. • Analyzes and evaluates the relationships among the various political, military, economic, and cultural organizations in the area of operations and considers their role when planning or executing missions. • Continuously evaluates his/her existing sociocultural mental model, analyzes new information regarding individuals, organizations, and groups, and revises his/her mental model of the local sociocultural structures and relationships. • Evaluates sociocultural information from staff and units to determine the potential effects of mission decisions on system elements, and bridges information gaps. • Identifies the possible impact of system changes on the mission by asking targeted questions of supporting staff and units.
Development StageYears in ServiceIllustrative Actions at Each Level
Expand8–16 years• Builds and diagrams a mental model of the sociocultural systems (political, military, economic, cultural) and relationships in an identified area of operations. • Understands the key organizations/groups in a region of the host country, and each of their roles. • Analyzes the interdependencies between systems, decisions, and organizations and the tools that support their management. • Considers multiple facets of a situation or problem, how they relate to one another, and the perspectives and needs that the key players contribute. • Considers the impact of the regional interaction of local government, opposition parties, and other groups, on mission planning and execution. • Identifies formal and informal leadership, systems, and organizational dynamics in the local area that can be leveraged to accomplish the unit’s mission. • Analyzes the relationships among the various political, military, economic, and cultural organizations and other players involved in the area of operations and considers their role when planning or executing missions. • Continuously captures new information regarding individuals, organizations, and groups and revises his/her mental model of the local sociocultural structures and relationships. • Asks questions of staff and units to determine the potential effects of mission decisions on system elements. • Identifies the possible impact of system changes on the mission by asking targeted questions of supporting staff and units.
Development StageYears in ServiceIllustrative Actions at Each Level
UnderstandInitial Military Training (IMT) - 7 years• Understands the key organizations/groups in a village/local area and each of their roles. • Demonstrates an understanding of the actors (e.g., political, military, cultural) within a village/local area as well as the key operational relationships. • Understands the potential impact of different organizational power structures, communication styles, and technologies on goal focus, information sharing, planning, and decision-making. • Engages in operational questioning and forecasting to build his/her understanding of sociocultural elements in the local village. • Understands the importance of continuously collecting new information to evaluate and revise his/her mental model of the local sociocultural systems. • Learns about local pop culture and social media in the village/local area and analyzes the role they play in the culture.
LearnRecruit - IMT• Aware of the systems in a village/local area (key organizations/groups). • Demonstrates basic knowledge of the key people (e.g., political, military, cultural) within the batttlespace as well as the key operational relationships. • Is familiar with different organizational power structures, communication styles, and technologies. • Recognizes that there are different organizations and groups that are interdependent and will impact a mission. • Identifies key relationships in the region. • Aware of the need to collect continuous information and evaluate and revise one’s model.
## Appendix B # Systems Thinking Requirements Demographic Questions 1. Please list your job title: 2. Are you currently either in active status or a U.S. Army civilian federal employee? [ ] Yes (if you selected this option, please move on to Question 3) [ ] No (if you selected this option, please stop here) 3. What is or was your rank? (mark all that apply) [ ] Current U.S. Army civilian federal employee [ ] CPT [ ] MAJ [ ] LTC [ ] COL [ ] Other (describe: ) 4. For current and former Army officers only: a. Including Active and Reserve, how many years did you serve in the military? b. What is/was your branch? 5. For current Army civilian employees only: How many years total have you served as an Army civilian employee? 6. What is the highest degree or level of school you have completed? [ ] Some high school, no diploma or GED [ ] High school graduate, diploma, or GED [ ] Trade/technical/vocational training [ ] Associate degree [ ] Bachelor's degree [ ] Master's or doctoral degree [ ] Professional degree # Appendix C # Systems Thinking Requirements Discussion Questions ## General Questions (for all Respondents) 1. Is systems thinking a term that you are familiar with? • Where have you heard the term used? • What does systems thinking mean to you? • Possible similar terms: big-picture thinking, holistic thinking, model thinking, testing 2. At what level(s) is systems thinking (big-picture thinking, holistic thinking, thinking across time, etc.) important for Army officers? • Are there certain positions where it is particularly important? • How important is systems thinking compared to other competencies needed for the job? ## Operational Job Questions (for Officers in Operational Positions) 3. In what ways do you use systems thinking in your current job? • Probe possible different kinds of systems (social, mechanical, open systems) • Probe possible types of ST components Ideas they mentioned in response to Question 1 о Holistic thinking, seeing emerging systems Identifying structure Model testing о Continuum thinking (seeing degrees and continuum vs discrete factors) о Nonlinear thinking, feedback loops Dynamic, thinking over time 4. What about in your previous jobs—in what ways have you used systems thinking in previous jobs? • Probe possible different kinds of systems (see #3) Probe possible type of ST components (see #3) • 5. Think about a time when you have worked with an officer in a current or previous position where that officer performed poorly at systems thinking (thinking over time, seeing the emerging system, etc.). What were his or her key challenges? • Probe for any role of systems thinking 6. If you think about the courses you have attended in the Officer Education System, can you think of topics or exercises that addressed systems thinking? If yes, how have you used the information or skills on the job? Probe possible types of ST components о Ideas they mentioned in response to Question 1 Holistic thinking, seeing emerging systems 。 Identifying structure о Model testing ○ Continuum thinking (seeing degrees and continuum vs discrete factors) о Nonlinear thinking, feedback loops Dynamic, thinking over time • Probe various OES courses - o ROTC, OCS, USMA - o Basic Officer Leader Course - o Captains Career Course - o Command and General Staff College - o Army War College • Are there aspects of systems thinking that were not taught in certain courses that, looking back, you think would have been useful to learn in those classes? 7. Do you view systems thinking as something that develops over time? How would you describe someone who is a novice, intermediate, or expert level systems thinker? ## Education and Training Job Questions (for Instructors and Trainers) 8. Do the officers that attend this course or training exercise need to use systems thinking to succeed here (big-picture thinking, holistic thinking, thinking across time, etc.)? - Do you think officers that use systems thinking perform better here? • Probe possible types of ST components ○ Ideas they mentioned in response to Question 1 - о Holistic thinking, seeing emerging systems - Identifying structure - о Model testing - о Continuum thinking (seeing degrees and continuum vs discrete factors) ○ Nonlinear thinking, feedback loops ○ Dynamic, thinking over time 9. Are there specific topics in this course or training exercise that provide training or development on systems thinking? - Probe possible types of ST components ○ Ideas they mentioned in response to Question 1 ○ Holistic thinking, seeing emerging systems - о Identifying structure ○ Model testing ○ Continuum thinking (seeing degrees and continuum vs discrete factors) - о Nonlinear thinking, feedback loops - о Dynamic, thinking over time • Please describe the topics and exercises 10. Do the officers in this course or training exercise receive any feedback on systems thinking (big- picture thinking, holistic thinking, thinking across time, etc.)? • Please describe the type of feedback 11. If you had a tool to assess systems thinking (big-picture thinking, holistic thinking, thinking across time, etc.) would it be valuable for this course or training exercise? • Please describe why or why not. • Would it be valuable to officers for their long-term career development? • What types of challenges would you anticipate? 12. Tell us about other training or development opportunities you are aware of that Army officers have to develop their systems thinking skills? Probe possible types of ST components о Ideas they mentioned in response to Question 1 ○ Holistic thinking, seeing emerging systems Identifying structure Model testing 000 Continuum thinking (seeing degrees and continuum vs discrete factors) Nonlinear thinking, feedback loops ○ Dynamic, thinking over time 13. Do you view systems thinking as something that develops over time? How would you describe someone who is a novice, intermediate, or expert level systems thinker? Ask the individual or group if there are any questions or final comments. # Appendix D ## Draft Systems Thinking Competency Model Discussion Questions 1. We have a list of topics that are relevant to systems thinking and we’re trying to identify examples from your everyday work that are related to these. Can you tell us about a time when you or someone you were working with needed to…. ## Elements • Figure out what elements were actually part of a system, situation, or problem ## Dynamic Relationships • Understand or identify nonlinear relationships among elements (e.g., a small increase in one element leads to a huge increase in the other) • Understand or identify feedback loops/bidirectional relationships (e.g., an increase in one causes a decrease in the other, which causes an increase in the first) • Understand or identify nested relationships (e.g., smaller systems within larger systems) • Understand how elements were interdependent or interrelated ## Hierarchy • Switch between looking at something from a higher and lower perspective; zoom in and out ## Holism • Specifically look at a situation from a broader or forest view ## Openness • Take multiple perspectives to look at a situation • Stay open to new information that might change how you look at a system • Understand system boundaries that were shifting/changing 2. How would these behaviors differ for leaders at different levels of expertise (beginner, intermediate, and advanced)? a. Discuss specific behavioral examples at different levels to the extent time allows. [Ask the individual or group if there are any questions or final comments.] [Thank them for their time.] ## Appendix E # Subject Matter Expert Demographic and Rating Questions 1. Are you currently either in active status or a U.S. Army civilian federal employee? [ ] No (please stop here) [ ] Yes - Active Duty officer (please continue) [ ] Yes - U.S. Army Civilian (please continue) 2. For current and former Army officers: a. Including Active and Reserve, how many years have you served in the military? b. What is/was your Army branch? ## 3. For current Army civilian employees only: a. If you are prior military, what was your rank upon leaving military service? [ ] N/A - never military [ ] CPT [ ] MAJ [ ] LTC [ ] COL [ ] Other (describe: ) b. How many years total have you served as an Army civilian employee? 4. What is the highest degree or level of school you have completed? [ ] Some high school, no diploma or GED [ ] High school graduate, diploma or GED [ ] Trade/technical/vocational training [ ] Associate degree [ ] Bachelor's degree [ ] Master's or doctoral degree [ ] Professional degree When you answer the following questions, please respond with respect to your FORMER JOB. 5. Please indicate your former job title: [ ] Co CDR [ ] Bn S-1 [ ] Bn S-5 [ ] Bde S-1 [ ] Bde S-5 [ ] Bn CDR [ ] Bn S-2 [ ] Bn S-6 [ ] Bde S-2 [ ] Bde S-6 [ ] Bn XO [ ] Bn S-3 [ ] Bde CDR [ ] Bde S-3 [ ] Other (describe: ) [ ] Bn S-4 [ ] Bde XO [ ] Bde S-4 6. What was your rank when you finished your former job? [ ] CPT [ ] LTC [ ] Other (describe: ) [ ] MAJ [ ] COL Please rate each action listed in the tables below. You will rate each action twice: on (a) Importance and (b) Frequency. First, rate how important the action was to perform your former job well. Use the following scale:
Not at all importantSlightly importantSomewhatImportantVery important
(1)(2)important (3)(4)(5)
Next, rate how frequently the action was required in your former job. Use the following scale:
Not PerformedEvery Few MonthsMonthlyWeeklyDaily
(1)(2)(3)(4)(5)
ActionImportance Rating 1=Not at all important 2=Slightly important 3=Somewhat important 4=ImportantFrequency Rating 1=Not performed 2=Every few months 3=Monthly 4=Weekly
5=Very important 5=Daily Elements 1. Identifies stakeholders relevant to a situation.
2. Identifies participants who have the perspectives needed for a meeting or decision.
3. Identifies units and equipment needed for a mission.
4. Recognizes relevant aspects of a problem.
5. Understands the elements in a system that are critical to a mission.
6. Determines the components of a complex or dynamic situation.
7. Identifies the boundaries that separate elements within the system from those external to the system.
Dynamic Relationships
ActionImportance Rating 1=Not at all important 2=Slightly important 3=Somewhat important 4=Important 5=Very importantFrequency Rating 1=Not performed 2=Every few months 3=Monthly 4=Weekly 5=Daily
8. Understands complex connections among multiple elements in the relevant environment when developing a plan.
9. Recognizes hidden, unusual, or nonlinear relationships among elements in the relevant environment when working through a problem.
10. Visualizes the interaction of elements in the relevant environment using appropriate timeframes and scales of magnitude.
11. Recognizes the interaction of elements in an overarching system during analysis of a course of action.
12. Recognizes when decisions and actions are likely to have second- and third-order effects that affect the initial situation.
13. Considers second- and third-order effects that may affect the unit or mission.
14. Determines how the introduction of new factors in the environment will change existing relationships in a system.
15. Identifies when certain elements of a system produce a nonlinear or disproportionate effects.
16. Understands how a changing situation can have complex effects on resourcing needs.
17. Uses feedback loops to learn about the effects of decisions on different
levels of the organization. Hierarchy
18. Uses both big-picture information and details as necessary when briefing a plan.
19. Ensures subordinate unit plans integrate into the larger mission.
ActionImportance Rating 1=Not at all important 2=Slightly important 3=Somewhat important 4=Important 5=Very importantFrequency Rating 1=Not performed 2=Every few months 3=Monthly 4=Weekly 5=Daily
20. Understands effects of decisions from a higher level on the lower-level units.
21. Understands how actions of a subordinate unit affect the mission of the larger organization.
22. Coordinates actions effectively with both subordinate units and headquarters.
23. Understands a specific subsystem in a mission when needed while also maintaining an understanding of the impact of that subsystem on the entire mission.
24. Understands resourcing needs of subordinate units as well as the impact of resourcing decisions on other parts of the organization.
25. Determines which problems need to be addressed at a higher level and which problems should be addressed at a lower level.
26. Identifies relationships between microlevel factors that produce macrolevel effects.
27. Recognizes the emergence of higher-level events or phenomena from lower-level patterns.
28. Understands the organizational levels of other groups working in a system
(e.g., joint, coalition, local government). Holism 29. Builds a graphic or model of a problem in order to better understand the
problem.
30. Understands a mission plan from a big-picture view, rather than just details of one or a few units.
31. Understands how subordinate units integrate to accomplish a mission.
ActionImportance Rating 1=Not at all important 2=Slightly important 3=Somewhat important 4=Important 5=Very importantFrequency Rating 1=Not performed 2=Every few months 3=Monthly 4=Weekly 5=Daily
32. Identifies patterns that emerge in how various elements within a mission interact.
33. Considers a situation or problem as a whole. Openness 34. Identifies emerging information that should be included in the current
understanding of a situation or problem.
35. Remains open to changing the framework or model of a situation when necessary.
36. Understands multiple points of view that expand the understanding of a situation or problem.
37. Considers different points of view when examining the interrelationships among the elements in a plan.
38. Remains open to changing plans due to changing circumstances.
39. Recognizes the need to review resource allocation when circumstances change.
40. Remains open to other perspectives.
41. Remains open to the changing stakeholders involved in solving a problem or making a decision as the situation changes
42. Reconsiders the critical elements in the environment as new information emerges.
## Appendix F ## Systems Thinking Dimension Behavior Questions 1. In what ways did it seem difficult or easy to make these ratings? a. Which aspects or actions seemed difficult? (Importance vs Frequency?) b. Were there any ratings you weren’t sure how to answer? Why? 2. Did the actions on the rating form generally seem important to your current job? Previous job? a. When you think about different jobs that officers have, do you think these behaviors would be more important for some jobs than others? More important at some ranks than others? 3. How would these behaviors differ for leaders at different levels of expertise (beginner, intermediate, and advanced)? a. Discuss specific behavioral examples at different levels to the extent time allows. 4. Are there any wording or phrasing changes that you would suggest for the behaviors listed on the form? a. In #9: What does the phrase “nonlinear relationships” mean to you? b. In #17: What does the phase “feedback loops” mean to you? c. Questions 18–28 ask about understanding systems at different levels. Can you think of examples of when you need to do this that involve something other than different levels of Army command? 5. In your current (or previous) job, think of someone who is very good at systems thinking—which of the behaviors on the rating sheet do they engage in most? a. Reminder of systems thinking definition, if needed: “understanding how various elements and domains in a complex and dynamic environment are interrelated and form a coherent whole.” b. Are there any systems thinking related behaviors that are not listed on the rating sheet that they engage in? [Ask the individual or group if there are any questions or final comments.] [Thank them for their time.]