Building Software the New Flexible Way

I like the way the industry of software development has progressed in the last ten years or so in many ways, but one significant area with which I am particularly impressed is the project lifecycle. In the eighties and nineties, massive, expensive and ambitious projects rose and fell everywhere. A software project was like a juggernaught that began with an idea, grew into a grand proposal, proceeded into a giant list of requirements, and from there was expected to progress steadily and admirably through the phases of high level design, low level design, build, test and deliver all through the miraculous coordination of huge teams of people and tremendous complexity. Oh the naivety, the broken hearts, the stress, the shattered budgets and the sour relationships!

Over time many bold and smart figures in the industry have turned the activity of developing software into a much more 'human' enterprise. We have recognized that computer software is so much more malleable than the products born out of centuries of engineering like bridges and buildings. We have had the courage to allow the undecided and the unknown into the process in a way that feels much more natural to us and much less frightening overall. And I believe today's software is better for it.

The modern software process still begins with an idea of course, one that is the seed of the project. That can be anything and, in the business world at least, usually arises from need for a business refinement or new opportunity. This idea broadly sounds like "we would a like a computer system to do [such and such]". Sometimes, a totally brand new computer system small or large is to be purchased and customized and sometimes the requirement is for an existing system to be modified.  In any case, running with the idea leads to a proposal being formulated (sometimes only mentally) stating the overall aims of the project. These parts of the project remain unchanged from the past. The main changes to the project cycle begin next.

The diagram below tries to summarize what happens from here. Notice a few important things about the project flow: (1) At no point do we try to nail down all possible details of the target system before developing. We agree from the outset that the project will be completed over time through a collaborative and iterative process. And (2) Each phase of the project is more similar to the old style of project. The phase has a certain set of requirements that are met by a process of design, deliver, quality control and refined according to feedback. Unlike the past, however - and this is important - quality is built into this process in the form of test-driven development over the low level design. This benefits both the phase and the whole project by guaranteeing quality in the deliverable while facilitating ongoing reqression testing as the product grows. (3) The end of each phase signals a continue-or-not point in the project. Invoices, salaries, etc are ideally paid up until the end of an agreed and delivered phase, but the sponsor always has a) something for their money and b) control over future spending. If phases are well designed, there is also a reasonable chance of useful deliverables coming out of even an aborted project.

Diagram 1:  A very high level summary of new project structure

DRY

One key principle behind Ruby on Rails is the one call DRY, an acronym for Don't Repeat Yourself.  Expanded, it means that you should never write the same complex piece of code twice, but instead use function definitions and calls.  The main objective behind it is to ease maintenance, by restricting future changes to code to one place instead of multiple as things need to be modified or fixed.

The principle is by no means a new one to software development and has been touted without the fancy acronym outside of Ruby on Rails for many years.   Subroutines in the original assembly languages had this value in mind as did macro definitions C and function and method definitions in many languages.  Class definitions and even dynamic linking could arguably be considered to have some basis in the same idea.

The visible effects of this principle on Ruby code in Rails are multiple.  For a start, method bodies tend to be small and application structure granular.  Reasons for this include the fact that every method that doesn't implement the most primitive of tasks calls at least one other method.  And Ruby's expressiveness and dynamic nature mean that predefined logic can be reused in all sorts of grammatical contexts, such as overridden operators, enumerators, or as overridden basic classes.

Another effect of DRY on Ruby programs related to the above is that method definitions are numerous.  Following the "no repetition allowed" principle in coding means that nearly every non-trivial method's definition relies on an spawns several other method definitions.   A knock-on effect of this for development is that stack traces from Ruby code like that in Ruby on Rails are long and sometimes challenging to follow due to the deep nesting of method calls.

The DRY principle does overall benefit both the application and the programmer in a number of ways.  It forces the programmer to think in a modular fashion, breaking down large problems domain into key atomic ideas.  It results in logic fragments and concepts that are nicely packaged, easily reusable and usually easy to understand.  And it decreases the number of duplicate code changes that need to be made to modify code thereby reducing coding/copying errors.

Most modern languages employ the DRY principle in some form as code reuse mechanisms.  In coding views, fragments of HTML are instinctively packaged into includable fragments such as Rails "Partials".   Most libraries and languages including Javascript and jQuery rely heavily on function definitions.  CSS supports runtime merging of files (and, using SASS, as many Rails developers do, the reusability of styles through "mixins"). In fact, nearly every language or tool implements some mechanism for defining reusable code fragments, not only within an application, but amongst more that one.  The grammatical contexts and semantics for re-use in software development tend to follow a core set of known patterns, but are on these there are many variations.

In general, the DRY principle is every programmer's friend.  If you or someone else may need to design or do something more than once, turn it into a method, class, module, or even plugin/engine.  It might add future value to even portions of your present output, or at least save on headache medication later.