Time for a classic rant again; yeah it’s always good to express thoughts and hear about the feelings of others – a good way to learn.
Lots of people think the most difficult aspects of software development revolve around engineering themes like:
- Writing elegant pieces of code that are a joy to extend and scale beautifully
- Crafting brilliant algorithms that can land rockets on small floating platforms (yup, SpaceX, I see you…)
- Inventing new cryptographic systems (please don’t attempt this at home…)
- Building and maintaining massively parallel computation systems
Agreed, these are extremely challenging and sometimes it is difficult to find a perfect solution. However, since most of these deal with code and systems, the required skills can be learned and there are usually workarounds available.
There is more to software development than engineering and these other facets can spawn tougher (or even impossible-to-solve) challenges.
Here are three of those difficult areas:
1. Exponential Chaos
The combinatorial complexity of code grows exponentially. It’s well nigh impossible and also futile trying to exercise all possible execution paths of a program. Equivalence class partitioning helps a lot to cut down the test space but invariably, we still miss out on a few.
A single if statement with one condition has two paths – the condition is either true or false. Let’s assign the simple one-condition if check code above a theoretical complexity value of 1. If that if statement is nested in another if statement, the number of paths explode to 4; ditto for two conditions in the if condition check. Re-using our complexity model, this comes to a value of 2 or so.
Most codebases have loads of conditional branching, loops, multi-threading, various components and what have you. So we can safely our complexity values for such code bases in in the high millions or thereabout. Scary? Not yet.
Now imagine what happens when there are hundreds of developers working in that same codebase and making a few hundred check-ins daily? Maybe the complexity value should sky-rocket to the high billions? Trillions?
Given the rapid rate of change and inherent complexity, how do you ensure that quality is maintained? How do you enforce consistency across large groups? A very difficult problem to solve – there are approaches to mitigate the risk but I do not know of any foolproof method that works all the time. If you know of a way, do let me know.
2. I’ll know what I want when I see it
We all think we know what we want – alas, we typically don’t until we see the finished product. Let’s take the following series of interactions between Ade who wants a new dress and his tailor.
Ade: I want some beautiful dress that fits me, is wearable all year round and casual. I want it in 2 weeks.
Tailor: Aha, so you want something casual that fits you, is wearable all year round and need it in 2 weeks.
Ade: Yup right on point
2 weeks later
Tailor: Here it is! (Beaming proudly)
Ade: (Not impressed); I like the fabric and design. But I don’t like the colour, the sleeve length and it doesn’t fit me quite right. Can you change it to black?
Tailor: here, it is in black
Ade: On second thoughts, black would be too hot, could you change it to brown?
Tailor: here it is in brown
Ade: Great! Could the sleeves be shortened by 2cm?
Ade: Hhmm, could you revert the sleeves to their original length? I think I now like the earlier design.
Tailor: Done!! (getting annoyed probably)
Ade: Great! This is becoming awesome, could you please reduce the width of the dress? It’s too wide…
Most people usually don’t have physical products tailor-made to their desires. We go to the store (to meet a car dealer, a tailor or an architect) and choose one of the several options there. We can take a car out for a ride, walk through a building or try on a new dress. This helps a lot as we know if we want that product or not.
In software development, it’s a different story – we want something tailored but we cannot express that need accurately until we see the product. Invariably, our descriptions do not match what we desire. To restate: it’s highly probable that you wouldn’t like a dress you described in its entirety to a tailor when you try it on.
Figuring out what users actually want is a difficult problem – probably why agile methodologies are popular. Less difficult way? Do the minimum possible thing and allow users to play with it. For example, the tailor could have given Ade a paper dress to evaluate all the styles and all that.
Let’s play a simple game: when you start your next project, make sure you document all user requests, also record every update as you go along. I am pretty sure the new requests will significantly differ from the original one. The end product might be totally different from the initial ask even.
3. No laws – it’s the wild wild west out there
If I release my grip on an apple, I expect it to fall down – why? Gravity of course. Most interactions in the physical world are bound by these models. Imagine that a car manufacturer has to design a new super car for some super-rich guy. Mr-rich-guy asks for the following:
- Must be drive-able by adults, teenagers and infants
- Must work on Earth, Venus and Mars
- Can run perfectly on gas, water or coal
The manufacturer can tell him it’s impossible since the current physical models make it extremely difficult to achieve the three impossible orthogonal requirements; maybe if he wants a movie though…
Let’s go to the world of software; consider the typical AAA game, to capture the largest possible market share, products have to be usable on:
- Multiple consoles (XBox, PlayStation, Nintendo etc)
- Other architectures (e.g. 32-bit and 64-bit PCs)
- Operating systems – Windows, Linux
- Various frame rates
There are also limitations in software (hardware limits, processors, memory etc) but we often have to build ‘cars’ that can be driven well by people in various age groups living in multiple planets!
The support matrix explodes all the time and generating optimal experiences is an extremely difficult task. In fact, most times, the workaround is to have a limited set of supported platforms.
Alas, it’s the realm of 0s and 1s, so software engineers have to conjure all sort of tricks and contortions to make things work. Maybe some day, cars would run everywhere too…
So apart from the technical challenges, there are a few other just-as-challenging (or even more challenging areas) in software development. These include
- Ensuring your requirements match hidden customer desires
- Working to meet various regulations and ensuring proper support across platforms
- Managing technical debt and reducing risk in heavily changed code bases