Software Development Methodologies

eXtreme Programming

Connie Murphy

Sound methodologies prevent teams from getting frustrated with having to spend more time than necessary to create processes rather than to solve problems. Methodologies provide project direction with the right tools, templates and guidelines to facilitate the delivery of results as effectively as possible. The methodology I have chosen to write about is the Agile methodology, and more selectively, the Extreme Programming method.

Several methodologies fit under the agile category. They share many characteristics, but there are also important differences. I have narrowed my topic to XP (Extreme Programming). The others in this category include:

· Cockburn's Crystal Family

· Open Source

· Highsmith's Adaptive Software Development

· Scrum

· Feature Driven Development (FDD)

· DSDM (Dynamic System Development Method)

· Rational Unified Process (RUP)

The Agile Manifesto

The materialization of Agile methodologies led to the development of a paper called the Agile Manifesto. There were several principles behind the Agile Manifesto and they are:

1. “The highest priority is to satisfy the customer through early and continuous delivery of valuable software. “

“Customers don't care about documents, UML diagrams or legacy integration. Customers care about whether or not you're delivering working software to them every development cycle—some piece of business functionality that proves to them that the evolving software application serves their business needs." (SDMagazine)

Implementing a ‘customer value’ attitude is easier said than done. Traditional project management assumes that project success equals demonstrated customer value. The instability associated with today's projects requires that customer value be reevaluated frequently, and project plans may change radically to ensure a project's ultimate success.

2. “Welcome changing requirements, even late in development. “

Agile processes control modifications to the customer's benefit. The agile approach accommodates change rather than struggling against it, all the while watching out for the consequences of these changes. Although it is agreed that feedback is necessary, this fact is ignored because the consequence of accepting feedback is change. Agile methodologies accept that the need to make changes easier is a more effective venue than trying to prevent it.

This point is so much opposite the values at Microsoft. They are constantly working against ‘feature creep,’ which occurs towards the end of a development cycle. It refers to the wishes of the development team to add features or enhance them beyond the scope of the original specifications. It tends to be the test team which provides these brakes, frequently pointing to the specification. My belief is that there are some features which do not become evident until well into the development cycle, and there are those features that should be included for the benefit of the customer.

3. “Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter time scale.”

Process specialists have been telling everyone for some time to use an incremental or iterative style of software development, with multiple deliveries of ever-growing functionality. It is still not predominant; however, but it is essential for agile projects. It is important to point out that you must remember that delivery is not the same as release. Some projects don’t achieve release status for a year or more. But there are frequent internal deliveries that allow everyone to assess the emerging product.

This is actually done with most software releases by Microsoft. There are daily builds that are seen internally, as well as beta releases for the testing public to evaluate.

4. “Business people and developers work together daily throughout the project. “

There is no detailed set of requirements at the beginning of the project; rather, a high-level view of requirements that is subject to frequent change. This view does not have enough specifications to design and code, so there must be frequent interaction between the developers and those they are developing for. The frequency of this contact should occur daily, or at least in regular intervals to stress the software customer's commitment to actively participate in, and accept joint responsibility for, the finished product. (Martin-Fowler.com)

5. “Build projects around motivated individuals, give them the environment and support they need and trust them to get the job done.”

Trust is the hardest thing to give. Decisions must be made by those who know the most about the situation. This means that managers must trust their staff to make the decisions about the things they're paid to know about. Even with all the tools at the developer’s disposal, without the knowledge of the internal workings and needs of the company, progress will not be satisfactory to the project without this interaction of developer and customer.

6. “The most efficient and effective method of conveying information with and within a development team is face-to-face conversation. “

Inevitably, when discussing agile methodologies, the topic of documentation arises. The issue is not documentation; it is understanding how the product works. Physical documentation is necessary, but the real measure of success is if it provides the crucial support to make the software function properly, providing the best service to the customer. Writing is a difficult and inefficient communication medium. It is used because it provides a necessary link, but most project teams can and should use more direct communication techniques.

“So the distinction between agile and document-centric methodologies is not one of extensive documentation versus no documentation; rather a differing concept of the blend of documentation and conversation required to elicit understanding.” (SDMagazine.com)

7. “Working software is the primary measure of progress. “

Too often, project teams don't realize they are not on schedule until shortly before delivery. Requirements were met on schedule, the design was on schedule, maybe even the code on schedule, but testing and integration took much longer than was planned for. Iterative development provides milestones that can't be slipped, which gives an accurate measure of the progress and a deeper understanding of the risks involved in any given project. As Chet Hendrickson stated, "If a project is going to fail, I'd rather know that after one month than after 15." (Hendrickson),

"Working software is the measure of progress because there's no other way of capturing the subtleties of the requirements: Documents and diagrams are too abstract to let the user 'kick the tires,'" says Dave Thomas (Thomas).

8. “Agile processes promote sustainable development. “

The sponsors, developers and users should be able to maintain a stable pace for as long as necessary. The software development industry is characterized by long nights and weekends, which in many cases does not actually lead to greater productivity.

Software creation using the agile methodology relies upon alert, creative people who can maintain that vigilance and vision for the entire software development project. Sustainable development means finding a working pace (40 or so hours a week) that the team can continue over time and remain healthy.

9. “Continuous attention to technical excellence and good design enhances agility. “

Agile development is similar to RAD in terms of speed and flexibility, but there's a big difference when it comes to technical cleanliness. “Agile approaches emphasize quality of design, because design quality is necessary to maintaining its own agility.” (SDMagazine.com)

Agile processes encourage the modification of requirements while the code is being written. Therefore, design cannot be an activity to be completed before construction. Instead, design is a continuous activity that's performed throughout the project. Every iteration has design work.

“The different agile processes emphasize different design styles. FDD has an explicit step at the beginning of each iteration in which design is executed, usually graphically with the UML. XP places great emphasis on refactoring to allow the design to evolve as development proceeds. But all of these processes borrow from each other: FDD uses refactoring as developers revisit earlier design decisions, and XP encourages short design sessions before coding tasks. In all cases, the project's design is enhanced continually throughout the project.” (SDMagazine.com)

10. “Simplicity—the art of maximizing the amount of work not done—is essential. “

Any software development task can be approached with a host of methods. In an agile project, ease of change makes it important to use simple approaches. As with any recipe, it is easier to add something to a process that is too simple, than it is to take something away from a process that is too complicated. Include only what everybody needs rather than what anybody needs, to make it easier for teams to add something that addresses their own particular needs. (SDMagazine.com)

"Simple, clear purpose and principles give rise to complex, intelligent behavior," says Dee Hock, former CEO of Visa International. "Complex rules and regulations give rise to simple, stupid behavior." No methodology can ever deal with all the intricacies of a modern software project. Set the rules and encourage creativity. This will produce better results than imposing intricate, unyielding regulations that cannot possibly be followed.

11. “The best architectures, requirements and designs emerge from self-organizing teams. “

Form doesn't always follow function: Form frequently follows failure. "The form of made things is always subject to change in response to their real or perceived shortcomings, their failures to function properly," writes Henry Petroski, civil engineering professor. (Petroski). Petroski's views are similar to one of the two key points of this principle—that the best designs (architectures, requirements) emerge from iterative development and use rather than from early plans. The second point of the principle is that emergent properties (emergence, a key property of complex systems, roughly translates to innovation and creativity in human organizations) are best generated from self-organizing teams in which the interactions are high and the process rules are few. (SDMagazine.com)

12. “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly. “

Any agile team must refine and reflect as it goes along, continuously improving its practices under its current circumstances.

What Really Matters?

In Kent Beck’s own words:

“Coding. At the end of the day, if the program doesn't run and make money for the client, you haven't done anything.

Testing. You have to know when you're done. The tests tell you this. If you're smart, you'll write them first so you'll know the instant you're done. Otherwise, you're stuck thinking you maybe might be done, but knowing you're probably not, but you're not sure how close you are.

Listening. You have to learn what the problem is in the first place, then you have to learn what numbers to put in the tests. You probably won't know this yourself, so you have to get good at listening to clients - users, managers, and business people.

Designing. You have to take what your program tells you about how it wants to be structured and feed it back into the program. Otherwise, you'll sink under the weight of your own guesses. (c2.com)

Listening, Testing, Coding, Designing. That's all there is to software. Anyone who tells you different is selling something.” (Beck).

XP (Extreme Programming) – How it started

Of all the agile methodologies, this is the one that has received the most attention. There are significantly more pieces of information on the web and books written about it, than any other methodology in this category. It would not be fair to start a discussion on XP without mentioning the name of Kent Beck, because of his ability to draw people to XP, and to lead the way. The popularity of XP has become a problem. though, as the other methodologies have not received the same attention for their valuable ideas.

The roots of XP lay in the Smalltalk community, through the close collaboration of Kent Beck and Ward Cunningham in the late 1980's. Both of them worked on frequent projects during the early 90's, using their ideas of a software development methodology that was both adaptive and people-oriented.

In the spring of 1996, the move from informal practice to a methodology occurred. Kent was asked to review the progress of the C3 payroll project for Chrysler. The project, developed in Smalltalk by a contracting company, was having major difficulties. Kent recommended throwing out the entire code base and starting from scratch, since the quality of the coding was very poor. The project then restarted under his leadership and has since become model for XP.

The first phase of C3 was implemented in early 1997. The project continued but ran into difficulties later, and further development was cancelled in 1999, although it still pays the original 10,000 salaried employees. (Martin-Fowler.com)

What is XP?

Extreme Programming (XP) was conceived and developed to address the specific needs of software development conducted by small teams in the face of vague and changing requirements. This new lightweight methodology challenges conventional thought, including the assumption that to change a piece of software becomes more costly over time. XP recognizes that project teams have to work to achieve this reduction in cost.

XP begins with four values:

· Communication

· Feedback

· Simplicity

· Courage

It then encompasses several practices which XP projects should follow. Many of these practices are old techniques that are being re-visited and woven into a pattern where each is reinforced by the others.

There is a strong emphasis on testing. While all processes mention testing, it is not a priority. However XP has every programmer writing tests while they write their production code. The tests are integrated into a continuous build process which becomes a platform for development.

On this platform, XP relies on changing the design with every release or iteration. Design is centered around the current iteration with no design thoughts for future needs. The result is a design process that makes it the most well developed of all the adaptive methodologies. (Martin-Fowler.com)

Kent Beck wrote Extreme Programming Explained the key manifesto of XP, which explains the reasoning behind the methodology and enough of an explanation of it to determine if one is interested in looking into it further.

Fundamentals of XP

XP has several tenets that distinguish it from other methodologies. Among them are the following:

· Distinguishing between the decisions to be made by business interests and those to be made by project stakeholders.

· Writing unit tests before programming and keeping all of the tests running at all times.

· Integrating and testing the whole system--several times a day.

· Producing all software in pairs, two programmers at one screen.

· Starting projects with a simple design that constantly evolves to add needed flexibility and remove unneeded complexity.

· Putting a minimal system into production quickly and growing it in whatever directions prove most valuable.

Why is XP so controversial? The following are examples of why some companies and developers tend to back away from XP:

· Doesn't force team members to specialize and become analysts, architects, programmers, testers, and integrators--every XP developer participates in all of these critical activities every day.

· Doesn't conduct complete up-front analysis and design--an XP project starts with a quick analysis of the entire system, and XP programmers continue to make analysis and design decisions throughout development.

· Develops infrastructure and frameworks as you develop your application, not up-front--delivering business value is the heartbeat that drives XP projects.

· Doesn't write and maintain implementation documentation--communication in XP projects. This occurs face-to-face, or through efficient tests and carefully written code.

XP is set up for small groups of programmers--between 2 and 12, though larger projects of 30 have reported success. Programmers can be average, with no additional education. to use XP. XP cannot be used on a project with a large staff. On projects with dynamic requirements or high risk, a small team of XP programmers will be more effective than a large team anyway.

XP requires an extended development team. The XP team includes not only the developers, but the managers and customers as well, all working together elbow to elbow. Asking questions, negotiating scope and schedules, and creating functional tests require more than just the developers be involved in producing the software.

Another requirement is testability. Developers must create automated unit and functional tests prior to and during the development stages. The system design may need to change to be easier to test. Just remember, where there is a will there is always a way to test.

The last thing is productivity. XP projects always report greater programmer productivity when compared to other projects within the same corporate environment. This was never a goal of the XP methodology. The real goal has always been to deliver the software that is needed when it is needed. If this is what is important to the project it may be time to try XP. (extremeprogramming.org)

References

http://c2.com/cgi/wiki?ExtremeProgramming

http://users.vnet.net/wwake/xp/

http://www.extremeprogramming.org/when.html

http://www.martinfowler.com/articles/newMethodology.html

http://www.xprogramming.com/

http://www.sdmagazine.com/documents/s=844/sdm0108a/0108a.htm

Beck, K. (1999). Extreme Programming Explained: Embrace Change. Boston: Addison-Wesley.

Hendrickson, C. et. al (2000). Extreme Programming Installed. Boston: Addison-Wesley.

Petroski, H. (1994). The Evolution of Useful Things . Vintage Books.

Thomas, D. et. al (1999). The Pragmatic Programmer. Boston: Addison-Wesley.