Insights into Agile Development Methodologies and Traditional Environments

Theoretical Reflections on Agile Development Methodologies [1] & Can We Manage Agile in Traditional Project Environments?[2] Chandrika Seenappa 23rdFebruary 2015 Professor: Hossein Saiedian

Agenda [1] Introduction Strategic Thinking Design Metaphor Conceptualization of Software Development Holographic Organization Conclusions 2

Agenda [2] Introduction Agile - Unique Project Management Concept Principles of Agile Agile vs Conventional Project Management Research Results Conclusions 3

Theoretical Reflections on Agile Development Methodologies 4

Introduction In the past decade, there has been much debate among software developers over contradictory perspectives of the traditional plan-driven approaches vs. the agile philosophy of software development Identify theoretical and conceptual support for the emerging epistemology (a branch of philosophy that investigates the origin, nature, methods, and limits of human knowledge) Enriches our understanding of the agile phenomenon by examining its theoretical roots Software development is a complex undertaking beset with many problems, called wickedproblems termed by Horst Rittel, an urban planner who pioneered the concept of issue-based information systems to facilitate the formulation and clarification of complex administrative decisions 5

Wicked Problem Problem are unique and difficult to formulate Solutions evolve continually as the designer gains a greater appreciation of what must be solved Need to satisfy multiple conflicting viewpoints makes it difficult to devise a test to determine the effectiveness of solutions Argumentation critical for problem solving 6

Wicked Problem Characteristics There is no definitive formulation of a wicked problem Wicked problems have no stopping rule Solutions to wicked problems are not true-or-false, but good or bad There is no immediate and no ultimate test of a solution to a wicked problem Every solution to a wicked problem is a "one-shot operation"; because there is no opportunity to learn by trial and error, every attempt counts significantly Wicked problems do not have an enumerable set of potential solutions, nor is there a well-described set of permissible operations that may be incorporated into the plan Every wicked problem is essentially unique Every wicked problem can be considered to be a symptom of another problem The existence of a discrepancy representing a wicked problem can be explained in numerous ways. The choice of explanation determines the nature of the problem's resolution The social planner has no right to be wrong 7

Strategies to Tackle Wicked Problems Cannot be tackled by the traditional approach in which problems are defined, analyzed and solved in sequential steps No clear problem definition of wicked problems In a paper published in 2000, Roberts identifies the following strategies to cope with wicked problems Authoritative : Tame wicked problems by vesting the responsibility for solving the problems in the hands of a few people Competitive: Attempt to solve wicked problems by pitting opposing points of view against each other, requiring parties to come up with their preferred solutions. Advantage is that different solutions can be weighed up against each other and the best one chosen. Collaborative: Aim to engage all stakeholders in order to find the best possible solution for all stakeholders 8

Design Concepts in Architecture Chris Alexander, a professor of architecture at the University of California, Berkeley, pioneered in the 1970s work on patterns in architectural design Inspired development of software design patterns Design patterns are proven design experiences and best practices for solving problems across domains Early model conceived design as an instrumental process dictated by technical rationality Aimed to identify the optimal means to a predictable end by employing scientific methods used in problem solving in the engineering disciplines Reduced design to a linear sequence of well-defined steps, with analysis preceding synthesis Key activity in analysis- systematic problem decomposition, hierarchical top-down functional decomposition 9

Design Concepts Development As early as 1963, key aspects of design are Continuous feedback among phases, communication, and iterative cycles of analysis and synthesis Argumentation involving stakeholders became an essential part of the design process inspired by wicked problem Need to integrate multiple viewpoints did not explicitly include the concept of learning Model of design process viewed design as a learning process in which the solution evolves through repetitive cycles of problem formulation, solution evaluation, and documentation Designers use their own personal expanding knowledge to continually reframe the problem and devise an appropriate solution 10

Insights of Nature of Design Complexity Problems are recognized as wicked, or not easily tractable Emergence Design conceived as a repetitive emergent process interlacing argumentation, introspective reflection, learning, and continual problem and solution refinement Integrative Integration of multiple perspectives is critical to problem resolution 11

Strategic Thinking The dominant paradigm for strategy formulation is a formal process to identify the best fit between an organization and its environment Predicated on the assumption of a foreseeable and unchanging world Mintzberg, conceptualizes strategy formulation as a craft, likening strategists to craftspeople whose minds work in concert with their hands as they learn and improvise in shaping an artifact Effective strategies evolve through a learning process involving skills, experience, and insights gained through the dynamic interplay among formulation, implementation, and critical reflection Strategy is an emergent process in which strategy formulation and implementation are inseparable 12

Design Metaphor Design connectedness of knowing and doing (thought and action), the interwoven nature of means and ends, and the need to reconcile multiple worldviews. Approach is substantiallyrational facilitating learning and adaptation through the constant questioning of assumptions Constantly reexamining assumptions and evaluating discrepancies against existing norms and values fosters double-loop learning , which is generative by increasing both learning and the ability to innovate and use change to one s advantage The traditional school of design is more in the spirit of single-loop learning or adaptive learning , in which routine problem solving parameters and preset rules, norms, and contrasts determine goal-seeking behavior metaphor incorporates learning and acknowledges the 13

Evolutionary Shift in Design Thinking 14

Conceptualization of Software Development Trend in management thinking, moving from a deterministic/mechanistic view of problem solving to a dynamic process, characterized by iterative cycles and the active involvement of all stakeholders, is reflected in software development Agile methods are people-centric, recognizing the value competent people and their relationships bring to software development Focuses on providing high customer satisfaction through three principles: quick delivery of quality software; active participation of concerned stakeholders; and creating and leveraging change methods Important features of this approach include evolutionary delivery through short iterative cycles of planning, action, reflection intense collaboration, self-organizing teams, and a high degree of developer discretion 15

Traditional and Emerging Perspectives of Design 16

Holographic Organization Holographic organization draws its inspiration from the fact that every fragment or piece of a broken holographic film contains the information required to completely construct the image represented in the whole film Can be accessed through any of its parts, as each one is a reflection of the whole Agile philosophy facilitates formation of holistic teams through a culture that encourages the interchangeability of roles or jobs based on autonomy Builds both specialized and generalized skills among members so they become multifaceted and exhibit overlapping knowledge so they can self- organize in response to emergent requirements 17

Holographic Organization (Cont.) Agile practices include Continuous code integration Refactoring to improve design and code Reflection workshops and stand-up meetings to determine what worked and what didn t Instant feedback from participating stakeholders Agile practices facilitate double-loop learning, supporting the principle of learning to learn 18

Alignment of Agile Practices with Holographic Principles 19

Conclusions Agile methods depart from the traditional orthodoxy of software development Metaphor of design and the theory of holographic organization offer a strong theoretical basis for the conceptual foundation of agile methods Efforts to understand the theoretical roots of software development practices by examining the evolution of design ideas in architecture, strategic management, and other disciplines are more relevant as system domains extend beyond simpler to the complex social aspects of software applications 20

Can We Manage Agile in Traditional Project Environments? 21

Introduction Agile project management is the execution of projects in a highly flexible and interactive manner, in opposite to the waterfall model Agilehas gained much attention and controversy as a contemporary project management approach Great benefits and advantages over traditional execution methods Others experienced disappointments or outright failures Results of a three-year field study into the practices of agileproject management at 37 technology-intensive companies 22

Introduction (Cont.) Agile principles are applicable to most projects which are Independent of their nature Size or IT-orientation Improving resource effectiveness Project execution time Overall project success Large and highly complex projects Carefully modified to fit the organizational processes and cultures of the enterprise 23

How did Agile Project Management Philosophy Emerge? Up to 1980s the waterfall model was the prevailing method for project execution Dramatic changes in the business environment Required project management to be reengineered Deal effectively with new challenges Balance efficiency, speed, and quality New technologies, in computers and communications Created enormous opportunities and challenges Transformed global economy into a hypercompetitive enterprise system 24

How did Agile Project Management Philosophy Emerge? (Cont.) To survive and prosper project leaders had to Deal effectively with time to-market pressures Innovation, cost, and risks in an increasingly fast-changing global business environment Iterative, incremental project management gained momentum in the 1990s Norm for developing and introducing new products, systems, and services gradually replacing the traditional waterfall model Agile plus its derivatives, such as Extreme, Lean and Guerilla project management were introduced in 2001 Iterative lifecycle or adaptive project life cycle or change-driven methods Facilitate change and require a high degree of ongoing stakeholder involvement 25

Agile - A Unique Project Management Concept Agileis an extension of the multi-phased approach to project management Promotes concurrent execution of tasks segments Requires more intense and effective interaction among overlapping task teams Need for strong cross-functional cooperation, integration and team involvement Creates both managerial benefits and challenges the operational and strategic values 26

Agile - A Unique Project Management Concept (Cont.) Wide spectrum of benefits to the enterprise Validating functionality of work-in-progress Better communication and information sharing among all stakeholders to total project lifecycle thinking and strategic alignment Effective cross-functional collaboration, and full integration of the project management process with the total enterprise and its supply chain Systematic approach to integrated project execution Emphasizes parallel, integrated execution of project phases Replaces the traditional linear process of serial engineering and expensive design-build-rollout rework 27

Principles of Agile Customer satisfaction by rapid delivery of useful software Welcome changing requirements, even late in development Working software is delivered frequently Working software is the principal measure of progress Sustainable development, able to maintain a constant pace Close, daily cooperation between business people and developers Face-to-face conversation is the best form of communication (co-location) Projects are built around motivated individuals, who should be trusted Continuous attention to technical excellence and good design Simplicity the art of maximizing the amount of work not done is essential Facilitate self-organizing teams Regular adaptation to changing circumstances 28

Agile vs Conventional Project Management Agile project management relies on the iterative, incremental project execution Involvement and collaboration of all project stakeholder groups Top management, suppliers, partners, users and customers Deal with changes and uncertainty by aiming for quick, short-term results, called releases New product or service system developed in short activity cycles called sprints Shared power and responsibility concept between product owner and scrum master 29

Typical Stages of Agile Project Life Cycle 30

Action Research Summarizes the last three years (2010-2013) of this investigation with focus on Agile management practices and experiences of 35 project management teams in 17 high-technology enterprises dealing with major product developments Qualitative methods for data retrieval Participant observation, in-depth retrospective interviewing, questionnaires The questionnaires were personally introduced to 230 team members and 35 team leaders, yielding an overall return of 75% The focus was on four interrelated sets of variables Risk Team Team leader Project environment 31

Summary of Field Sample Projects were on information system, financial services, automotive, airplane, computer and pharmaceutical products Three project sizes Small medium-sized projects with 50 team members or less Large projects with over 50 team members Mega projects with over 1000 team members Data evaluated via content analysis and standard statistical methods 32

Results Findings of study organized into three sections First, a simple summary of Agile usage across project categories is presented Second, the reaction of managers to the 12 Principles of Agile is summarized and discussed with focus on benefits, challenges, limitations and performance impact of managing agile Third, the lessons learned from this study are summarized based on effective agile management 33

Intensity of Agile Application on Project Type One size does not fit all project 34

Experiences from Agile with Focus on twelve Agile Principles 35

Experiences from Agile with Focus on twelve Agile Principles (Cont.) 36

Agile Principles in Large Projects To be truly agile, management process must be Iterative, that is take several cycles to complete Incremental, delivering the product and/or service in workable pieces Rely on self-organizing teams Evolve regarding its work structures and processes as needed during the project lifecycle Difficult to realize in large projects Require more execution formality and discipline to deal with specific complexities, contractual requirements and project interfaces Airplane developments, advertisement campaigns, Olympic Games or aerospace missions, cannot be delivered incrementally Hybrid approaches: Simulations, emulations, focus grouping, user- centered design and computer modeling achieve objective of early validation of deliverables and flexibility in changing requirements 37

Agile Challenges in Projects with Long Development Cycles Major challenge to agile methodology is rapidly (or immediately) delivering functionality Difficult to achieve without compromising long-range optimization of overall results 38

Integrating Agile Approaches into Traditional Project Systems Featured benefits of agile Flexibility Responsiveness to changing environments Active user involvement Iterative requirements verification Difficult to realize in traditional or large project environments with established client relations and specific contractual procedures One of the biggest challenges relates to process execution conflict 39

Challenges to Established Standards, Norms and Processes of Professional Project Management Agile often is unworkable in projects that need to be executed to meet specific results within given time and resource constraints Difficult in large projects and special categories of projects, such as government contracts, where the overall requirements and project scope must be established up-front and becomes the basis for performance measurements Despite these challenges, some organizations are able to use agile at least in part or in principle, even for very large and complex projects. Agile is not a do as you want process Hard work and skillful leadership required to introduce a general process template to a specific work environment 40

Not all Project Activities fit Agile Documentation, training or other logistics support not part of agile, but require major adjustment Agile can often be adapted As stated by a project manager responsible for implementation of large training program: I hear people saying all the time that Agile is not meant for this kind of project. The reality is that no method will guarantee you success. You have to fine-tune it to your specific situation. I select the pieces that are appropriate for my projects. Agile gives me a framework for interacting with my user community, testing out deliverables before we go to a major training session. 41

Not all Challenges are Limitations Challenges and barriers to agile are perceptions, but not necessarily true limitations. The culture or process is fixed and not under the control of the project manager The process of agile is not a rigid template, but a guideline that must be adapted to a specific project situation, especially if the situation is outside the framework for which agile was originally designed for As explained by a senior manager of a large transport system development: Just because you're following the process doesn't mean you're doing it right. You really need to think creatively and have talented people who understand the objectives of the agile principles to adapt the ideas to work in our environment, and to do it right. If you re not involving your team, and constantly evaluating your practices what works, what doesn't and hat needs to be fine-tuned, you're likely to fail. 42

Lessons Learnt Introduce Agile to your organization incrementally. Start with agile practices that already support existing processes and organizational cultures Examine the various components of your project management process for opportunities for applying Agile principles without compromising existing management processes and controls; identify compatible and incompatible assumptions, and encourage synergism between agile and traditional methods 43

Lessons Learnt (Cont.) Examine customer interfaces and contracting practices, redesign procedures to support agile execution Examine progress measurement, reviews and payment practices for realignment with agile execution Win the hearts and minds and support of senior management, convincing them that Agile is a more effective way of executing projects leading to higher success rates and customer satisfaction pay attention to people issues, such as empowerment, collaboration, commitment and ownership which are at the backbone of Agile and crucial to its successful practice 44

Conclusions Agile has great potential for improving project management effectiveness in our ultra-competitive, fast-changing business environment In spite of the challenges, the principles and basic philosophy of agile can be integrated into traditional project management processes In order to be effective, change leaders must go far beyond issuing new procedures and guidelines Each component of agile, including underlying assumptions, must be carefully examined to determine the degree of adaptability, redesigned and fine-tuned if necessary to fit into the existing management system Companies are combining, integrating and synchronizing agile and traditional processes to create a unique agile hybrid system which fits specific project situations, together with their people, organizational cultures and the total enterprise business environment 45

References Nerur, S., & Balijepally, V. (2007). Theoretical reflections on agile development methodologies. Communications of the ACM, Vol. 50(3), pp.79-83. Thamhain, H.J. (2014). Can We Manage Agile in Traditional Project Environments?. 2014 Proceeding of PICMET (Portland International Conference on Management of Infrastructure and Service Integration, pp. 2497-2505. Engineering and Technology): 46