/ Methodology

The "Silver Bullet" Obsession in Software Development

The Silver Bullet

The Silver Bullet
Image by Nailenchez via Dribble

The term "silver bullet" originally comes from fairy tales and folklore, where they are used to shoot and kill mythical beasts[1]. Colloquially we use the term to mean something that can be used to fix complex problems in a quick and easy way, compared to a more difficult traditional solution[2].

In this article I will discuss the latter definition, and how it relates to software development. For some reason there is an obsession to find a "silver bullet" within several aspects of the software industry, which typically is presented to be the new technology, framework, methodology or tool to end them all. As long as you adopt whatever it is your problems will be solved.

I take inspiration from Fred Brook's famous article[3] on essential and accidental complexity in software engineering, from Hadi Hariri's talk[4] on the syndrome of seeking "the silver bullet" in the software industry, as well as other sources that touch upon this topic. But mostly I base it on my own experience as a developer and technical architect, where I constantly come across this way of thinking. It's a very common idea, and I admit to be guilty of this myself from time to time.

The sin of repeating yourself

Don't Repeat Yourself
Image by Ryan Corradini

The DRY (Don't Repeat Yourself) principle[5] seems to be one of the fundamental rules that all software developers has an almost religious relationship with. It is hammered into our heads by all mentors we have as wannabe developers, and we are thought that breaking it is a cardinal sin.

It does make a lot of sense on the face of it, because we want to advocate reuse, as well as reduce the cost of refactoring and maintenance. But it is so ingrained in our methodology that we do it even when it breaks architectural boundaries of our systems. This point builds up to the main message of this article. My clear opinion is that the aspiration to achieve clean architecture[6] should trump the rule of reuse, if there is a conflict between the two principles. The maintenance penalty of developing more or less the same component more than once is much less profound than creating an architecture that is messy and where boundaries and separation of concerns are broken. Of course if the same component is duplicated a number of times then there is something wrong. If you encounter this you might need to have a look at your overall design, because it might be a sign of bad architecture. The rule of three[7] is still an important principle, also in the technical architecture.

Yes this is fine, but how does this relate to a "silver bullet"? My point is that once we have a component that we can reuse in our architecture, there is a tendency to wanting to use this component everywhere where it can utilized, regardless if it breaks good architecture, because it is so important to be DRY. This hinges onto the next section, about selecting the right solution for the given use case.

The right tool for the job

The right tool
Image

As born and bred techies we software developers do like our tools and gadgets. By this I mean all different kinds of technologies that we can use to develop a system or an application, be it development tooling, collaboration tooling, software components, software frameworks, programming languages and development methodologies. When starting a new project, privately or professionally, we often start thinking about which technologies to use, even before we have a full understanding of the business domain or any use cases for that project.

Often the choices of technologies are already made by tech leads, bid managers and project managers before a project is even started. When working for larger software development companies this is also pushed upon us by software vendors, that obviously seek to get as much turnover as they can for their products and licenses. They are more than eager to cooperate on bids and projects in order to push their solution on the customer.

The flip side of this is that for many software developers they are bound to work in a preset and rigid regime, where there is little room for new ideas and alternative technologies. Even software architects are often quite hamstrung when trying to design the best possible solution for a business case. The result is that architects tries to shoehorn different technologies into use cases where they often are a bad fit for the solution. But the customer is paying large sums for that piece of technology, so you are forced to use it regardless.

A classic example is databases. The enterprise grade relational database is still the mainstay of the data model for most of the larger software projects today. But relational databases are built to optimize leveraging of tabular data, and especially tabular data that have a limited dependency graph, meaning that there isn't very deep relations between the data entities. But the customer is paying a huge amount of money to buy and maintain this database, so it should be used for everything. There are other types of databases that have been optimized for different types of data, like document-, temporal-, graph-, key/value- and distributed databases. I have seldom seen any of these types in use, but rather the different data models have been shoehorned into the big relational database.

The same goes for other parts of our architecture. There is such a (somewhat misguided) eagerness to want to draw major synergies throughout the architecture. If a use case is solved in a certain way in one part of the architecture then you often find that people want to solve all other similar use cases in the same way. Intuitively this seem like a very sensible thing to do, but like with the DRY principle if it will break the principle of clean architecture then architects should refraining from using the approach, and instead seek other solutions.

I also see a tendency that architects and developers choose what they know well to solve a use case. Again this seems sensible, but I advocate very much to take a step back and make an analysis of possible solutions before making the choice. Active use of rapid prototyping and fast architectural analysis/decisions are techniques to find the best solution, but are often frowned upon by both architects and product owners. It is not very effective in rigid enterprise projects and in projects using a waterfall based approach, where much of the technologies have already been agreed upon. But even though you don't get to use them in any project in the foreseeable future, it is essential for a software professional to keep up to date with new technologies through schooling, courses, conferences and self study.

Obviously there is a danger that the architecture will become too fractured if a new approach or technology is used to solve every different use case in a system. I don't mean that we should go bananas with choosing all different solutions. The important message here is that we are trying to create a clean architecture that uses the best tools for the job in the different parts of the solution. Fractured architecture is mostly an anti pattern in traditional layered and monolithic architectures, but is almost sought after in principal SOA and microservice architectures, which I will discuss more in a subsequent section.

The new shiny

The newest gadgets
Image by James Bareham via The Verge

As mentioned earlier it is not strange that people who work with IT are interested in the latest technologies. New technology has transformed our day to day life, both professionally and personally, and continue to do so. In software development there are new libraries, frameworks, tools and platforms released every day, so keeping up with the latest technologies and trends can be hard. But what is for certain is that almost as soon as you have adopted some new technology there is an even newer and better alternative available. And they all promise to be a game changer for the way we develop software.

The funny thing is also how we present these new technologies to each other. We also often present them as the only way forward, even though it is something that we didn't knew existed a couple of days ago. We haven't even tried it out in a proper business case, and often nobody has, because it is brand spanking new. -If you are not using technology X then you are old school. Well then maybe being old school once in a while is not such a bad thing. At least be a little sceptical before you jump on the technology X bandwagon.

The frontend community has taken this to a whole new level. They are pumping out new JavaScript libraries and frameworks in an extreme pace. I have almost given up trying to keep up with what is happening in this community, because that would be all I would have time to be doing. I guess that is why we in latter years have seen a much bigger divide between being a frontend developer and being a backend developer. You typically see developers being one or the other. But what I feel is important to grasp is some of the same points I made about databases, and indeed what this article is about, there is no technology to rule them all.

When the SPA (Single Page Application) frontend frameworks[8] started to gain adoption in software projects it soon became that same old tune; -We have to use this everywhere. Again software developers focused on the technology rather than the use case. As the title of the previous section said, use the right tool for the job. JavaScript rich pages are not needed for static web pages, so don't over engineer it. Likewise, server side rendering is not useful for developing modern responsive web pages.

The backend community also have their new shiny, and it is called microservices[9]. If you are not basing your architecture on microservices you are basically a stone age person. -Let's use microservices everywhere. -Why? -Because we can have small self sustained services that can be called in any order we like. -We can also maintain and deploy them individually. -But isn't that the same as SOA[10]? -No, no, SOA is so old school, nobody is using that anymore. Well by just changing the name of a architecture pattern, and tweaking some of the principles, we now have something that promises to be a total game changer. I feel it's the same old story.

One thing you typically get when you choose software from an enterprise grade software vendor is proper attention to crosscutting concerns. Security, monitoring, patching, integration and other features are typically supplied out of the box. If you choose to build a lightweight architecture like that based on microservices you need to have much of the same features, because that is what the customer expects. Often you will need to develop these features as well, which is not a trivial thing to do. You need to weigh the pros and cons of doing this, and taking the requirements of the system in to account when deciding the approach.

Where do we go from here?

Going forward
Image

By all means, we should embrace the new technologies that are coming. They might indeed be a total game changer, like many technologies have already become. All I ask is a little bit of restraint. Don't choose a technology just because company X is using it, and have been very successful at it. The same is true when thinking that we should use the same approach that we used in a previous project, because it worked to well there. The requirements are typically very different from project to project and company to company, and so is the architecture required.

Let's try so sum up the important points...

  • Repeating yourself is not the end of the world.
  • Trying to achieve a clean architecture is important.
  • Proper understanding of the business case before choosing technologies.
  • It's hard to achieve a good architecture in a rigid project structure.
  • Major software vendors will push their products, often creating a rigid project structure.
  • Try to ensure that you are using the right technology for the given use case.
  • Don't be mesmerized by the new shiny technology, instead be a little cautious.
  • Lightweight architecture is nice, but consider cross cutting concerns.
  • Always be curious and learn new things, but get the whole picture.

And to spare you the suspense...
There is no "silver bullet"!




  1. Wikipedia says: In folklore, a bullet cast from silver is often the only weapon that is effective against werewolves, witches, or other monsters. - https://en.wikipedia.org/wiki/Silver_bullet ↩︎

  2. Dictionaries says: Something that acts as a "magic weapon", in particular something that easily solves a difficult problem. ↩︎

  3. "No Silver Bullet - Essence and Accidents of Software Engineering" - The well known article by Fred Brook from 1986 on reducing the complexity of software and its methods. ↩︎

  4. "The Silver Bullet Syndrome" - The great talk by Hadi Hariri from the 2015 Devoxx conference in Belgium about inappropriate use of "silver bullets" in software development. - https://youtu.be/3wyd6J3yjcs ↩︎

  5. Don't repeat yourself - Wikipedia article - https://en.wikipedia.org/wiki/Don't_repeat_yourself ↩︎

  6. "Clean Architecture" - The book by Robert "Uncle Bob" Martin - https://amzn.com/0134494164 ↩︎

  7. Rule of three - Wikipedia article - https://en.wikipedia.org/wiki/Rule_of_three_(computer_programming) ↩︎

  8. SPA - Wikipedia article - https://en.wikipedia.org/wiki/Single-page_application ↩︎

  9. Microservices - Wikipedia article - https://en.wikipedia.org/wiki/Microservices ↩︎

  10. SOA - Wikipedia article - https://en.wikipedia.org/wiki/Service-oriented_architecture ↩︎