Complexity and Architecture

It has been too long since I had a post... I have a number of scribbled notes of blog ideas, but little time. I'm in the process of starting to write the spring 3 book, which may take me off grid more than normal. I will make an effort to post a couple of times a month (more if there is something cool to document). Currently my mind has been on 3 subjects; Spring, Architecture and languages. This blog is regarding one of the often forgotten concerns of architecture... complexity.

Defining Software Architecture
I have been in a large number of conference BOFs this year. When the audience is slow in starting a conversation, the question we gravitate to is: "What is a Software Architect?" or its keen "What is Software Architecture?". I guess I shouldn't be surprised, but it is amazing how diverse the answers to these questions are. It is a reflection of at least two things... 1) the immaturity of our industry and /or 2) the lack of standardization in this space. The conversation spans from design decisions to application architecture to enterprise architecture.

Martin Fowler defines architecture as "things that people per-ceive as hard to change". This is a really responisble definition.

I like to define it as "making decisions on all the non-funcational requirements" and "managing all the things you can't control"

The area of non-functional requirements would include:

• User Experience
• Performance / Response Time
• Scalability
• Security
• Frameworks
• Complexity
  

In the category of things you don't control:

• On public web: Number of Requests
• Number of Threads / Connections
• Number of returns from a user generated search
• Items in a batch

Complexity in Architecture
The subject that is rarely vocalized and debated in architecture is that of complexity. Neal Ford, in a couple of his talks with NFJS, talks about principles that were documented by anciet philosophers... that of essential and accidental properties. He carries this forward into software architecture in a way that we as an industry really need to talk about. That of software architecture decisions moving the enterprise forward in a way that has essential complexity or in a way that increases the accidental complexity. Neal further clarifies the point with the following comments... We have a hard problem (Essential complexity) and we have made our problem hard (Accidental complexity). Well done Neal! Very easy to envision.

Is complexity one of the non-functional requirements which your architecture team is looking at? If not, It seems likely that what is left is accidental.

What is a concrete example of this?
Let's say that your enterprise has just purchased a shiny new ESB. Suppose there wasn't a project need for it... it was just the stated SOA direction of your organization. When down from on high the edict is brought to your project that you must use the ESB in order to justify the $1M expense. Perhaps it is a fit... but perhaps you are about to fit the square peg in the round hole... in other words accidental complexity.

Where does Accidental Complexity come from?
Neal would say that it comes from vendors who are the pushers of accidental complexity... however there is something deeper here. The reason they are referred to as the pushers is that they are incented to sell you a solution. The solution may not fit your needs exactly, but hey, if they can show you a vision of how it could help, then there is a framework to following and life should be more simplified. So essentially we are saying that any choice which doesn't provide the best solution brings complexity.
We could also say that the more stuff / more solutions we have brings complexity. So by adding more frameworks and options we have added complexity. Conversely, many organizations fall into the trap of standardizing certain aspects of development... like everything is an EJB or we don't use store procedures here. In this situation what happens is that the architect and developers will have to come up with a more complex solution in order to be compliant with the corporate standards.

I'm open to suggestions and challenges... but I think we can conclude that much of our accident complexity comes from not applying the best solution to the problem. I would expect that most teams would not do this deliberately. So the team is either 1) ignorant of the best option (which can be fixed through experience and training) or 2) it is forced upon them.

I will close with the following challenges and questions:
Next time you are looking at a software component... Ask yourself the following
1) Does it simplify the problem?
2) Is there a more simplified solution?
3) Is it testable and verifiable?
4) Does it add complexity upstream or downstream of the problem?

When evaluating solutions... try to put a value on the level of complexity. Here are some ideas:
1) If it moves code or configuration into a proprietary solution then increase its complexity score.
2) If it allows you to write less code for the same functionality then decrease its complexity score.
3) If it enhances testability, decrease its complexity score.
4) How easy is it to monitor and maintain?
5) How easy is it to integrate with and is it standards compliant?

Good luck and Happy Holidays!