How Can Agile IT Transform the IT Industry?
For many, the waterfall model of software development has been the standard for decades, but this is now being replaced by the much more flexible Agile methodology.
The Agile methodology for software development can positively impact the IT industry. The results of Agile methodology adoption can be measured in a number of ways. Quicker turnaround of software change requests, fewer bugs, quantitative measurement of team performance and bottlenecks are all reflections of a successful implementation of Agile. To successfully measure the impact of Agile, an organization needs to compare various metrics related to the pre-Agile and post-Agile development. The real impact of Agile cannot be measured just by the increase in revenue or by the increased number of bugs fixed. Several internal parameters need to be considered to understand the real impact. (For more on Agile development, see Agile Software Development 101.)
Why Agile IT?
The IT industry has been leaning towards Agile practices mainly because of the constraints of the waterfall model of software development. Generally, it has been observed that IT companies are unable to respond to changing customer demands or market situations or reduce costs with the waterfall model of software development. Even if we counterbalance this overwhelming tilt towards Agile methodology and consider some of the excitement to just be hype, there is a lot of empirical feedback against the waterfall model.
Simply put, the waterfall model is a software development model where work is done in a sequential manner — one phase after another. There are five phases of this model: requirements, design, implementation, verification and maintenance. Usually, after one phase has been completed, it is difficult, if not impossible, to make changes to an earlier phase. So, the assumption is that the requirements are pretty much fixed. The main difference with the Agile model is in the assumption that there will be no change in requirements. Agile assumes that business situations will change and so will requirements. So, software is delivered in smaller chunks over sprints, whereas in the waterfall model, the first delivery or release is made after a long time. (For more on development, see How Apache Spark Helps Rapid Application Development.)
The most notable criticism against the waterfall model has been its assumption that there will be no change in requirements. The very assumption is flawed and unrealistic. For example, in 2001, a study on 1,027 IT projects in the U.K. showed that this assumption was the single biggest reason for the failure of IT projects.
In another example, Craig Larman, the author of the book "Agile and Iterative Development: A Manager’s Guide," has pointed to how a number of projects executed by the Department of Defense (DoD) using the waterfall model in the U.S. have failed to achieve their objectives. Throughout the 1980s and 1990s, the DoD was required to use the waterfall model for its software development projects as per the standards published in DoD STD 2167. Shocking statistics revealed that 75% of these software projects were never used. Following this report, a task force was launched under Dr. Frederick Brooks, a well-known software engineering expert. The task force came out with a report that commented “DoD STD 2167 likewise needs a radical overhaul to reflect modern best practice. Evolutionary development is best technically, it saves time and money.”
The following four assumptions of the waterfall model had failed in real-world scenarios:
- The requirements given are reasonably well defined and so, will not change significantly.
- Even if the requirements change during the development phase, they will be small enough to be accommodated within the development cycle.
- System integration, which happens after software delivery, will go as per plan.
- Software innovation and the effort required to innovate will go according to a planned and predictable schedule.
How Does Agile Methodology Address the Problems of the Waterfall Model?
The Agile methodology is fundamentally different from the waterfall model and that is clear from its assumptions:
- No one, not even the customer, can fully know or understand the requirements. There is no guarantee that requirements will not change.
- Requirement changes might not be small and manageable. In fact, they will come in various sizes and will keep coming. So, the software will be delivered in small increments to keep track of the changes.
How Has Agile Impacted the IT Industry?
Agile is being adopted in a lot of places, while a lot of companies are making plans to adopt Agile. Though Agile has definitely made fundamental changes to the IT industry, facts and figures are still a little difficult to obtain. But the impact of Agile can be understood with the case study of British Telecom (BT) given below.
Reasons BT Shifted to Agile Practices
BT started to experience a number of problems with its software development practices back in 2004. BT developed a number of software projects, both simple and complex. Many software projects were unable to develop quality output within the agreed time frame. BT found that the problems owed their roots to the waterfall model. So, reinforcing the waterfall model was not going to help. The main causes of the problems are given below:
Poor Management of Requirements
- Too many requirements were given to be fulfilled within too limited a time.
- Many requirements were irrelevant to the business needs.
- Almost all, if not all requirements were assigned high-priority status.
- The requirements represented current business needs with no eye on the future scenarios. That left open a possibility of future software changes.
Poor Software Design
- Given the huge number of requirements, designers expended too much time just trying to figure out what the requirements meant. Little time was left for actual design.
- The requirement analysts moved to other assignments, taking with them unwritten, tacit knowledge.
- The above two factors resulted in poor design. Designers had to still deliver according to the original timeline.
Coding was hasty and of poor quality because of flawed software design. Developers would realize that a poor software design would result in poor code, but nevertheless had to deliver by the agreed timeline. A lot of bugs would be reported during integration because unit tests and regression tests were not run.
By the time the software is deployed, the customer notes that the requirements have already changed and so has the business scenario. The software also has a lot of problems. Effectively, the entire effort of software development is now considered wastage.
What Did BT Do to Address the Above Problems?
BT realized that reinforcing the waterfall model was not the answer to the problems. It needed a new approach. So, it decided to implement the Agile approach. Under the new approach, the following things were decided:
- Instead of the 12-month development cycle, BT would now deliver workable pieces of software in a 90-day cycle. The idea was to focus on one or two business problems and target to deliver a software solution within 90 days. The beginning of this cycle was marked by an intense discussion between different stakeholders so that the requirements were clearly identified, analyzed and prioritized.
- The target was to deliver clear, tangible business values. The internal customer was under pressure to provide clear requirements. So, instead of vague goals, user stories were given with clear acceptance criteria.
- The software that would be delivered would be fully tested and documented. The software would go through frequent integration tests to identify problems beforehand.
With the Agile approach, BT could adapt to changing requirements or business situations more easily. The code quality improved and last-minute basic bugs were addressed.
Agile, for all its advantages and the hype around it, is still at a stage where its potential is not fully realized. This is because a lot of organizations are customizing the approach to the extent of modifying its original principles. As a result, the waterfall model is making a comeback in some cases. While customization will continue, it is important to retain Agile's basic principles. A lot of organizations claim to be fully Agile, but they still have some way to go in order to become a truly Agile company.