1

Surveying the Role of the Agile Development Methodology in 21st Century Business Strategy Marc-Philip Brenninkmeijer
Northeastern University

I. Introduction Over the last few decades, businesses in a variety of markets have struggled to keep up in a globalized economy. The changing nature of business in the late twentieth and early twentyfirst century has forced businesses to reconceptualize strategies for long-term success. (For a good overview of these broad issues, see Teece [7]). There are many reasons why businesses have faced these problems, but two stand out above all others. First, thanks to the dissemination of the personal computer and to the growth of internet access around the globe, businesses no longer have to compete against other businesses within their communities; they have to compete with businesses around the globe. The expansion of the internet has not been wholly negative for business, of course, because it also allows businesses to reach a wider audience they could not have without the internet. But it has certainly presented a unique set of challenges, forcing businesses to take a more holistic and globalized view of the markets they compete in. Second, the rapidly increasing rate of technological innovation over the last two or three decades has provided its own constant stream of challenges, even driving many old forms of business out of the market (e.g. bookstores in the age of digital media). Businesses that rely on technology must now struggle to keep their products and services up-to-date and relevant in a much more dynamic, fast-paced environment than they would have faced two decades ago. These unique challenges have impacted businesses in many ways, but among the most salient effects is that businesses have been forced to pay more attention to the role their information technology (IT) infrastructure plays in their overall business strategy. A few decades

2 ago, information technology in a business context mostly referred to communication services (email) and computer and network troubleshooting within a company. But now, IT has taken on a much wider role and importance in the functioning of most businesses. Most information technology departments do much more than provide routine communication and network services; they are often responsible now for developing and constructing the very technological platform that makes the functioning of the rest of the business possible, especially because of their oversight of software development. Because of this expansion of the role of IT in business has come hand-in-hand with the unique set of challenges described above, businesses have had to reconceptualize, in particular, how to organize and leverage their IT departments to meet the demands of a globalized twenty-first century marketplace. In short, businesses have had to reconsider the methodologies they use to structure IT business practices. One of the most striking ways in which this has manifested itself is the development of a significant body of research literature surrounding such development methodologies, and, in particular, the innovation of a new kind of development mindset called agile. The agile methodology was developed in response to the unique set of challenges described above that all businesses must face as they head into an even more globalized and fast-paced economy. Its main purpose was and remains to help businesses execute more agility in their IT and software development practices, which in turn should help them to better confront the rapidly changing needs of a dynamic, unusually competitive marketplace. The purpose of this paper is to help illuminate the role this special agile development methodology plays in twenty-first century business strategy by surveying the literature surrounding the rise of the agile development methodology and its impact on businesses. Several intriguing questions motivate such an analysis. Have agile methodologies truly helped businesses

3 to better deliver products and services more efficiently than their more traditional counterparts? Have they been shown to lead to better competitive advantage? How widely have they been accepted? More generally, what methodologies are available to businesses beyond the agile way of thinking, what factors determine which methodology they should implement, and how have they developed historically? On the whole, the literature displays some surprising answers to these questions. The consensus by and large seems to be that even though agile approaches were developed to respond to the unique challenges of businesses today, and even though they have helped businesses to better compete in certain respects, they have not come without their own share of difficulties and problems. Perhaps this result is not altogether surprising, since we should not expect any methodology to be perfect. More interesting may be the fact that the vast majority of published literature has evaluated the role of agile methodologies in fairly narrow and limited contexts, often focusing exclusively on the needs of the IT department instead of thinking more generally about the place of development methodologies in overall business strategy. The structure of the paper is as follows. First, the paper will briefly survey literature on the history of development methodologies and the rise of the agile mindset. Then the paper will turn to literature that evaluates the agile methodology by comparing and contrasting it to others, focusing on research that explores its comparative advantage over other kinds of methodologies. The paper ends with a brief conclusion summarizing the state of research on agile methods in business today. II. The History of Development Methodologies and the Rise of Agile The literature surrounding the history of development methodologies and the rise of agile approaches is extensive, suggesting just how urgent and hot a topic development methodologies

4 have become in business in recent years. Three papers that are especially clear, concise, and helpful are Avison [2], Zhang [10], and Highsmith [4]. Avisons short five-page review gives an excellent picture of the fact that development methodologies even have a history, which is often taken for granted in the kind of fast-paced technology-driven world businesses inhabit today. The four major sections of his paper track the origins of thinking about development methodologies, moving from the Pre-Methodology Era through the Early Methodology Era into the full-blown Methodology Era and finally into the Post-Methodology Era. To suggest just how different todays environment is from the one businesses faced only a few decades ago, it is worth quoting the opening sentences of Avisons description of the Pre-Methodology Era: The early computer applications of the 1960s and 1970s were developed and implemented without explicit or formalized development methodologies. The emphasis was on programming and solving technical problems, particular those resulting from the rather limited hardware of the time. Developers were trained in computer technology but rarely understood the business or organizational contexts in which their systems were implemented. (Avison 79) Avison explains how development methodologies were conceptualized and implemented in order to respond to these structural weaknesses in businesses of the time. The main contrast drawn is between those methodologies in the Early Methodology Era (1970s and early 1980s) according to which one phase of a project had to be completed before the next one could begin (hence the term waterfall) and the more flexible methodologies developed in the Methodology Era to respond the problems created by the waterfall arrangement (Avison 79-80). Among the problems generated by the early waterfall methodologies were: failure to meet the real needs of the business, overly conservative systems design, instability (due to the modeling of processes that are unstable due to changing businesses and markets), and inflexibility (Avison 79-80). This complex set of issues is what set the stage for the rise of the methodology era that

5 Avison skillfully summarizes. Although Avisons paper gives an excellent compact summary of the history and basic framework of development methodologies, it is slightly surprising the paper never mentions agile methods in particular. Perhaps this is because Avison was more interested in sketching the general background rather than discussing specifics, providing way to introduce and orient researchers to the field as a whole. Zhangs paper makes up for this gap and therefore serves as a natural second step from Avison. The paper takes up where Avisons history leaves off and goes into much more detail about specific methodology names and philosophies. Zhang opens the paper by mentioning the broad relevance of development methodologies, which might otherwise seem on first glance a relatively specialized or technical aspect of contemporary businesses having little to do with globalizing market forces. Zhang writes, As software is becoming critical to almost every organization, software development, the set of activities that produce software, has become an important topic to software development educators, students, practitioners, and researchers (Zhang 1749, emphasis added). Zhang then divides contemporary methodologies into three broad groups: analysis-coding methodologies, life cycle methodologies, and agile methodologies. Zhang gives many details about how these methodologies were conceptualized and their overriding philosophies. The key contribution Zhangs paper makes to the literature, however, is the way it integrates this broad classification of methodologies with an analysis of overall trends and implications. The most important trend is summarized as follows: Agile software development methods are gaining ground on life cycle models. Compared to life cycle models, agile methods offer several advantages. Agile methods can deliver working software faster. Agile methods are better at dealing with changing user requirements. And finally, agile methods promote better working relationships among all stakeholders, including business analysts, developers, testers, end users, and project managers. (Zhang 1750-1751)

This expands on the more general analysis Avison gives by suggesting some of the reasons that agile methods, in particular, have begun to find wide acceptance. Moreover, Zhangs paper (published in 2010) also makes it clear that because agile methods have only started to be implemented over the last decade, empirical research about the success of agile methods in responding to contemporary business challenges is not widely available and still needs to be completed more systematically. For example, Zhang writes that as software development organizations adopt agile methods (Trend 1), empirical studies are needed to clarify the impact of using those methods (Zhang 1751). The paper goes on to list helpful research questions that such empirical studies could address, which are worth listing here because they anticipate the various dimensions of the research literature this paper will survey in section III: (1) Does the use of agile methods lead to a better quality product? (2) Does the use of agile methods lead to higher job satisfaction for the development team members? (3) Does the use of agile methods lead to improved working relationships among team members? (4) Does the use of agile methods lead to improved adherence to project budgets and schedules? (5) Does the use of agile methods lead to a higher level of project success? A better understanding of these questions and their answers would help any software development organization make critical decisions about adopting agile methods. (Zhang 1751) These questions help to situate the rise of agile methodologies within a larger framework, dealing not just with their history (as Avison does) but also with how they can be evaluated for their effectiveness as part of an overall business strategy. Zhangs paper helps to widen the scope by integrating both history and business theory. On the other hand, Zhangs paper still does not discuss agile methods as exhaustively as they could be. For that, Highsmith [4] serves as an excellent source, giving a broad overview of specific types of agile methodologies and the philosophical background for how they are supposed to confront the challenges facing businesses in general and software development

7 teams in particular. The basic problem agile methods are supposed to solve, Highsmith explains, is that Expectations have grown over the years. The market demands and expects innovative, high-quality software that meets its needsand soon (Highsmith 120). Highsmiths paper clearly lays out how agile methods are supposed to be different from their more traditional counterparts: Traditional approaches assumed that if we just tried hard enough, we could anticipate the complete set of requirements early and reduce cost by eliminating change. Today, eliminating change early means being unresponsive to business conditionsin other words, business failure (Highsmith 120). Highsmiths paper goes in to summarize, in detail, the various facets, principles, and practices that agile methods developed in order to respond to these challenges, rounding out the basic framework described by Avison and Zhang. But it does not suggest why or how agile methods might not be successful, or how their performance should be evaluated, or even whether there are any examples showing the success of agile methods in helping businesses better adapt to their twenty-first century environment. III. Evaluating the Spread and Performance of Agile Methodologies Based on the general picture that emerges from the three papers discussed above, it is natural to ask just how effective agile methods can be, whether they truly have contributed to better business performance in globalized fast-paced markets, and whether they are worth implementing. Indeed, as we saw above in Zhang [10], some papers even lay out a basic framework for the kind of empirical research that needs to be done to evaluate the impact of agile methods. It is surprising, then, that relatively little research has been completed on these issues. The only in-depth, on-point investigation of all the various dimensions of the role of agile methods in contemporary business thinking seems to be Rico [5], a book-length study. The peerreviewed journal research that has been completed, on the other hand, is often relatively

8 specialized or narrow in scope. For example, Aarnink [1] does explore the Contributions of Agile Software Development Methods to Business-IT Alignment, but only in non-profit organizations, and investigating the role of agile methods in business-IT alignment is not quite the same thing as evaluating its impact on business strategy as a whole. Similarly, Boehm [3] explores a relatively limited dimension of the effectiveness of agile methods by investigating what management challenges prevent organizations from implementing agile methods. Obviously such a study cannot say much about the effectiveness of agile methods that are implemented, although it is still of some value since it shows what kinds of considerations and tradeoffs businesses must face when adopting agile methods. One dimension of analysis that seems to have a fair amount of representation in the literature is whether it is possible for agile methods to exist alongside more traditional methods within the same organization. The two studies Seyam [6] and Vinekar [9] both explore this question and both give evidence that it is not uncommon for agile methods to be supplemented by more traditional methods. This work is important because it shows that even though agility has been an important component of the revolution in software development over the past few decades, it might not be the only important component when it comes to adopting an appropriate methodology. Finally, one study that does directly discuss some of the questions posed by Zhang above (namely questions 2 and 3, but perhaps also 5) is Williams [8], which describes the results of a study based on the following questions: How well do the Agile Manifesto and its 12 principles still capture what is valued by practicing software engineers in industry and by teams that have adopted agile methodologies as their own practices have matured and evolved? How do agile teams regard the principles today? (Williams 72) In short, Williamss survey finds that the answer to the second question is overwhelmingly

9 well. In his words, the authors of the Agile Manifesto and the original 12 principles spelled out the essence of the agile trend that has transformed the software industry over more than a dozen years. That is, they nailed it (Williams 76). The survey evidence is useful because it is empirically based, and it does help to shed light on the place of agile methods in business today, but it is unfortunate that Williams did not use the opportunity to ask more penetrating questions (including all of Zhangs, and more) that could have formed the basis of a much richer, more informative analysis. IV. Conclusion The importance of the rise of agile development methodologies seems well-documented, now an established part of business theory and practice. A large amount of literature has been devoted to understanding the basic framework in which to make sense of the place of development methodologies in contemporary business practice, and even to understanding how agile methods are supposed to handle the unique challenges of a globalized fast-paced economy. But very little research has attempted to tackle head-on important questions about evaluation and performance of agile methods. On the one hand, this is probably because agile methods are still relatively young. But on the other hand, much more work needs to be done to better understand both the value of agile methods and the factors that businesses should consider when deciding to adopt a specific methodology. This work should be both theoretical (deepening our understanding of what role methodologies play in business and how they can be evaluated) and empirical (giving case studies about how agile methodologies have performed in specific organizations). Once that research has been completed, businesses will be in a better position to understand how to confront the challenges that await them as globalization and technology progress in the twenty-first century.

10 References 1. Aarnink, Arjan and Kruithof, Gijsbert. Contribution of Agile Software Development Methods to Business-IT Alignment in Non-Profit Organizations. Communicatins of the IIMA 12.2 (2012). Available at http://www.iima.org/CIIMA/CIIMA.html 2. Avison, David E. and Fitzgerald, Guy. Where Now for Development Methodologies? Communications of the ACM 46.1 (January 2003). Available at http://cacm.acm.org/magazines/2003/1/6909-where-now-for-development-methodologies/. 3. Boehm, Barry and Turner, Richard. Management Challenges to Implementing Agile Processes in Traditional Development Organizations. IEEE Software (September/October 2005). Available at http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=1504661 4. Highsmith, Jim and Cockburn, Alistair. Agile Software Development: The Business of Innovation. Computer (September 2001). Available at 5. Rico, David F., Sayani, Hasan H. and Sone, Saya. The Business Value of Agile Software Methods: Maximizing ROI with Just-in-Time Processes and Documentation. Ft. Lauderdale: J. Ross Publishing, 2009. 6. Seyam, Mohammed S. and Galal-Edeen, Galal H. Traditional versus Agile: The Tragile Framework for Information Systems Development. International Journal of Software Engineering 4.1 (January 2011). Available at http://www.ijse.org.eg/2011_Vol4_No1.asp?. 7. Teece, David J. Business Models, Business Strategy and Innovation. Long-Rage Planning 43 (2010). Available at http://www.sciencedirect.com.ezproxy.neu.edu/science/article/pii/S002463010900051X 8. Williams, Laurie. What Agile Teams Think of Agile Principles. Communications of the ACM 55.4 (April 2012). Available at http://cacm.acm.org.ezproxy.neu.edu/magazines/2012/4/147352/fulltext 9. Vinekar, Vishnu, Slinkman, Craig W., and Nerur, Shridhar. Can Agile and Traditional Systems Development Approaches Coexist? An Ambidextrous View. Information Systems Management (Summer 2006). Available at http://www.tandfonline.com/toc/uism20/current#.Un1cMWTwLrV 10. Zhang, Xihui, Hu, Tao, Dai, Hua, and Li, Xiang. Software Development Methodologies, Trends and Implications: A Testing Centric View. Information Technology Journal 9.8 (2010). Available at http://scialert.net/jindex.php?issn=1812-5638