Given that there are limits to our cognitive abilities and our understanding of complex systems, Corstian Boerman argues that software development should become boring. It suggests removing infrastructure so it doesn’t burden the day-to-day development process and focusing on delivering business value in a predictable and repeatable way.
Corstian Boerman gave a talk on how complex software affects our cognitive abilities at NDC Oslo 2023.
Boeman mentioned that because our memory is limited and fallible, there is a limit to our understanding of complex systems. This is important for two reasons, he explained:
First, it’s expensive to train programmers to remember details. This is not only evident during initial training, but also during continuous development where low speed is also a symptom of this.
Additionally, the limit of complexity is reached faster with an irrationally complex system than with a naturally complex system, so I would argue that we should take more inspiration from nature to develop our systems.
There is a clear difference between natural complex systems and man-made complex systems, Boerman said. Through physics, natural phenomena can be explained by a series of equations, while for man-made systems we have to take the context into account, he added. This context – arguably irrational – cannot necessarily be generalized to the same extent as is possible with natural systems.
Boerman argued that software development should become boring. He made a distinction between boring software and exciting software:
Boring software in that categorization resembles all software that has been built countless times, and will be like that a billion more times. In this context, I’m specifically referring to back-end systems, although this applies to front-end systems as well. Exciting software are all projects that require creativity to build. Think dedicated algorithms, automation, AI integrations and the like.
Making software development boring again means putting the main focus on delivering business value and making the delivery of these aspects predictable and repeatable, Boerman argues. This requires moving the infrastructure out of the way in such a way that it is still there, but does not burden the day-to-day development process:
While infrastructure takes up most of the development time, it technically delivers the least amount of business value, which can be found in the data and operations performed on it.
New exciting experiments can be fast and unstable, while the boring core needs to be and remain of high quality so that it can withstand external disturbances, Boerman concluded.
InfoQ interviewed Corstian Boerman about building complex software systems.
InfoQ: What does the relationship between software complexity and cognitive abilities look like?
Corstian Boerman: Cognitive ability includes three primary things: thoughts, experiences, and senses. Together we use them to understand everything that happens around us.
Our thoughts as such can include both the conscious, slow rational system, and the unconscious, fast emotional system. Experiences can uniquely identify a human being among his peers, and we use our senses to perceive the world around us. Taken together, they define our cognitive abilities as well as our deficits.
Because cognition itself is multifaceted, extraordinary abilities and disabling disabilities are not mutually exclusive and can coexist within the same person, Boerman said, whether it’s through the senses (a missing limb), experiences (traumatic experiences), or our thoughts ( mental illness).
Complexity and cognition are not directly related to each other. While complexity is a certain characteristic of our environment, cognition refers to the way we interact with our environment. Our cognitive abilities then enable us to understand complex systems as well as create complex systems.
InfoQ: How should we deal with boring and exciting software?
Peasant: Most boring systems follow the same paradigm: we have some data and want to run some operation against that data. When working with systems like these, we spend a lot of time setting up the proper infrastructure. Database queries, statement records, system integration and more.
As opposed to boring software, there is also what I consider to be exciting software, which is just as important to keep boring software boring. These are all projects that require creativity to build.
In this symbiosis, the boring part contains the business processes most important to the company without any overlap. This core functionality is then enhanced with more exciting external software. When these exciting bits interact with the processes defined by the boring core, it becomes possible to automate manual work or manually take over automation. This gives freedom in the way the system can be developed.