"Agile Software Development Methodology" for Laymen

Everybody is talking this “Agile Software Development” thing. Following is to explain the basics.

Skyscrapers and waterfalls

First, let’s see what non-agile is supposed to be. Skyscrapers have to be well planned and designed before start building. This is because, it is big trouble to change design-plans midway. It is not very cost effective to change the base holding-structure after building five floors. This pattern of designing before building is true for many types of projects; like building bridges or building cars. These projects go through some activities like plan, design, prototype, validate, get customer approval… and then start building. In the software development world, this way of doing projects in sequential stages (like analyze, plan, design, build, test, deploy etc.) is known as the “waterfall method”; as activities flow one after the other. Traditionally, when software developers got a problem to solve; they analyzed the problem, designed a solution, built it, and then delivered the promised product to the customer (some named it waterfall methodology).

Typewriters and waterfalls

Twenty years ago… typing documents using type-writers were a well-planned activity too. Documents needed to be well planned on paper before actual typing started, because the tool did not allow us to shift paragraphs or insert new sections midway.

Modern word-processing software goes flexible

Today’s current word processors allow us to start typing a document without any plans. We know that the software allows us to change anything with ease. So… current word processors allow us to be very “agile” (or nimble) in the process of creating documents. We can start typing, show a first draft to the boss, incrementally improve the document based on feedback etc. We do not complain that the documents are difficult to change after typing or not having needed skills operate the tool.

Software development tools go flexible too

“Software Development Tools” also started to get matured like the “Word Processing Software”. Process of implementing changes (or enhancements) started becoming easier. In other words, allowed us to be more “Agile” in building software.

Gist of the above explanation is that… at the heart of “Agile” is the ability of modern-software-development-tools to implement changes with little effort. And the developers itself be well trained in the process implementing changes; using the tool at hand.

So this idea that “software development tools allow us to implement changes at will” caught on… that the tools now allow us to embed designing and planning into actual building.

Now we can just start building… show output to customer… get feedback… implement changes… refine the output… add new pieces. Keep on showing the incremental results over and over again to the customer till the customer is happy. Then… why don’t we bring the customer into the development team too; to direct the development-process-outcome itself as it is being built (no major upfront planning needed, let’s change as see fit). Then they thought… if the customer is sitting with the development team, maybe we don’t have to create unnecessary intermediate design documents to get customer approval too; as we can show the solution result itself easily (instead of first designing on paper, let’s show the actual working product; then change based on feedback). This could make the Software-development-process more efficient too.

The idea that the most important deliverable in software development is the solution; and all other effort is to support this outcome.

In other words…. today’s software development tools now allow us to start building from day one (at minimal cost to fix mistakes or change direction), solution visibility from day one, customer feedback early; on delivery direction and value. Feedback based enhancements… thoughts went on.

Then… some smart people got together in 2001 at a ski resort in Utah… and formalized this “Agile” idea; and called it “Agile Manifesto” (http://agilemanifesto.org).

Today… there are many formal methodologies that are claiming to be agile oriented (Extreme Programming, Scrum etc.). “Agile Development” has become an umbrella-term to cover many methodologies that are based on ideas like “Non waterfall”, “build in small cycles” , “iterative output”, “embrace change”, “let owners see results and request changes”, “let owners add enhancements during build” etc.

What we should not forget is that for agile to work well; the basics are…

1.  Tools should allow for easy rapid changes.
   Keep in mind that not all software development tools are created equal…. Pick tools that are better suited for non-planned changes. 
   Today’s software tools are good, but not magical enough to allow any mid-cycle change with no effort.
2. People should have needed skills to… use the tools to accommodate rapid changes.
   If not, you will start hearing statements like… “this is a big change”, “that change will take a lot of time”, “I have to re-engineer more than a few existing modules to do this change”.
3. Even if the “agile” model thrives on “change”… lesser the changes needed the better it is.
   A bit of planning, designing or thinking-ahead at the right time will go a long way in pointing in the right direction.
   straight path to the solution is more cost effective than zigzagging without a plan. 
   Plan ahead a bit, as effort needed for some changes could be more than you bargained for.
   Plan ahead a bit, as some drastic mid-cycle changes could cause more pain than value.
4. Common sense and sound experience beats it all…
   Agile methodology may work better for certain projects. Agile methodology may work better in certain stages of a given project etc.
   Incorporate heavier upfront analysis or design when the need is obvious.

One day far in the future… when skyscrapers are built using easily re-configurable tiny cubes… skyscraper engineers too will be arguing about agile-scraping. 

Some Agile Development Methodology related myths

1. Agile only means no unnecessary documentation needed.a) Need documents to handover to other teams.b) Need documents for Training new employees. c) Initial developers may move on or forget eventually.d) Regulatory or Approval requirement may call for documents.
   e) Most importantly, when something goes wrong after two years… and your boss wants to show the design to other bosses, if you do not have pretty design documents, you are in trouble.
2. Agile does not need architecture or design. Planning with experience will set the direction to minimize changes; and improve solution value.
3. Agile is only for small projects.
   There are many examples of successful large agile projects in the real world.
   
4.  In Agile… effort needed, time needed and the outcome itself… is unknown; as the end is not planned for in the beginning.
   This totally depends on your approach, how much to plan upfront.
   In a regular project… upfront planning will help you identify big buckets and estimates. Then you will be able to identify and adjust for scope-creep or unexpected.

Question

What if there is a project that is to be delivered in 6 months; and final solution qualities are well defined and understood.

Will design and build the solution in a “waterfall method” be better than using “Agile Method”?

[Argument is that… why bother with iterative, multi-cycle development… when we can do it in one cycle; as the needed exact outcome is known upfront]