Changing Requirements in Software Development

During my 16 years of experience architecting and writing software, I’ve never seen a project where software requirements remain untouched until the end of the project. Typically system requirements start evolving to address new business needs, usability study results, changing assumptions. As a result, the project scope may extend for quite a number of reasons. The modern world is changing very fast, and today fixed project requirements are a thing of the past.

Agile methodologies of software development cover the management part of the process, allowing project owners to keep track of changes and maintain the pipeline uninterrupted, protecting the overall project value. But what about the technical side? Do engineers need to apply certain software design and development techniques to the system code itself to support this idea of evolving requirements and project demands?

We can divide all requirements changes into two big categories:

• a need to update some code that already exists
• a need for some new functionality on top what we currently have

This actually describes what system flexibility and extensibility are. Of course, there might be changes that require both.

Let’s deal with flexibility first. From my experience, many developers don’t pay much attention to this part of system design because modern OOP and functional languages provide decent tools for basic separation of concerns. I’m talking about such things as functionality encapsulation and singularity of purpose (single responsibility principle). Developers widely use these tools, sometimes even subconsciously.

In addition, clients often ask to not overthink, to keep things simple and make the first version faster, this way collecting some “technical debt” from the very beginning. And this is ok if this debt is closed (by system refactoring) in the release number two and postponed by the time the system becomes unmaintainable (which we will address in a future post). So with basic separation and consequently basic flexibility everything is more or less good by default.

The areas that request special system design and development efforts are “horizontal” and “vertical” system separation. Horizontal separation means splitting and defining APIs between logical layers of functionally that fulfill the same role within the application:

• presentation layer
• business logic layer
• data access layer
• domain model
• integration layer and so on

Vertical separation means dividing the system by modules or subsystems on a feature-related basis. Usually, there is no API between features but they can be utilized one inside another and depend on one another. And here is the key moment: features should be very well-isolated and testable separately from the rest of the system. This means there must be zero hardcoded dependencies inside any given module. All dependencies should be injected on the fly. This requires applying advanced inversion of control techniques and this is where most of the initial technical debt is collected.

Developers often write modules and just use them one inside another, creating hard dependencies. As a result, when they need to change something small in one module, they actually affect the entire dependency chain. The more dependencies – the less flexible and maintainable the system.While the idea of the IOT is quite simple.

The implementation of this idea is quite tricky – if you want to know more about the inversion of control techniques I recommend the book by Marc Seeman. 500+ pages of inversion wisdom 🙂

In order to build a system that would tolerate significant requirements changes, system architect should take care of three things:

1. No system component should share its responsibilities with another one or include unrelated responsibilities.
2. Clear APIs between system logic layers
3. Gain control over dependencies applying IoC practices

Speaking of extensibility, first we need to distinguish the so-called white-box and black-box approaches. White-box extensibility means that software system can be extended by modifying/extending the existing source code. This looks very much like flexibility and it is truly so.

Actually, it is the logical sequel which stems from the same low coupling and modularity principles. The source code that deeply embodies those principles is easy to maintain, change and extend. In my career, this statement was proved in practice many times. Converse is also true, I checked – don’t go there 🙂

The main challenge here is that object-oriented programming (OOP) and interface-based architecture by nature are tightly-coupled. Meanwhile, system decoupling demands:

• a profound understanding of the issue
• the use of best platform-specific decoupling practice
• the whole development team has to embrace this idea and keep dependencies under control on a daily basis

Last but not least, let’s take a closer look at “black-box” extensibility. The black-box approach means that any new module has zero knowledge about the system it is integrating with. It just implements the interface specifications provided by the system.

There are a variety of ways to build APIs, but recently the data-centric approach is becoming more and more popular. This is for a good reason. It turns out that the best API for decoupling system components and system-of-systems is data itself.

For example, let’s take REST – simple 4 methods for manipulating data:

• POST – Create
• GET – Read
• PUT – Update
• DELETE – speaks for itself

These 4 methods are actually enough for transferring any data between system components. There is no need to invent new wheels and bikes and create complex APIs with hundreds of methods like GetUserName(id), SetUserName(id, name, magicParameter), createNewGroup(type) and so on.

Instead, we just compose a data structure and use the appropriate method from these four to transfer the data. Another alternative gaining popularity these days is “data publishing”. Take a closer look at the Firebase or PubNub solutions – this is how true black-box extensibility looks like. This is how the Internet of Things will look like. However, this is a topic for another article. Stay tuned!