Efficiency vs Effectiveness
In IT we are not developing with commodity products. It might seem to be that way, but that is an incorrect perception. Software is engineered, not produced. Therefore, in order to improve our business performance, our focus should be on being effective instead of efficient. Even more so, when our business revolves around products based on software. For example, software applications or SaaS products. Effectiveness is a key trait necessary to continuously succeed in a market where swapping one solution for another is becoming increasingly trivial. Open standards and the democratisation of IT resources allow customers to treat the risk of vendor lock-in as something benign. This change in attitude requires an organisation that is able to adapt to the wishes and needs of its market. It has nothing to do with being able to efficiently produce software, instead it has everything to do with being effective at meeting customers demands.
In order not to perish in today’s world, effectiveness comes before efficiency. We will need to be efficient at being effective.
Some time ago, I found myself in some discussions with a colleague of mine, a fellow IT architect. We quite often find ourselves in very similar situations where we are asked to not only define an architecture, but to also coach teams to transition from a traditional organisation to one that is agile.
At some point I was asked by this colleague if I could co-review a report written by one of his clients. The report described the plan for the transformation from a legacy waterfall organised project to an agile organised one. What struck me, and fortunately my colleague concurred, was that the main motivation for this transition was to reduce cost and become a more efficient organisation. Which in fact is an ill-chosen motive.
Agile is about value creation and not cost reduction.
Let us back-up a bit and consider two similar words that are fundamentally different in meaning: Efficient vs Effective.
Efficiency vs. Effectiveness
In process engineering we are striving to become more efficient. The motivation is that by becoming more efficient, we can produce our products at a lower cost per product. It is a process improvement adagio that has been around for a long time. It is also a motive for improvement that leads to specialisation, to specialised silos. Furthermore, we see that this also leads to resource utilisation optimisation, by means of idle time reduction. Specialist resources are ‘kept busy’ as much as possible to make the best use of their valuable time.
Which, by the way, is a fallacy when it comes to increasing efficiency. Lean theory as well as Critical Chain Theory argue that reduction of idle time often has a negative impact on throughput.
When it comes to efficiency improvements, organising ourselves into specialised silos is very beneficiary when it comes to cost control. A concentration of expertise will allow for better planning and therefore better efficiency. Setting the idle time reduction fallacy aside and embracing Lean principles, you will find that your product process will become more efficient and the cost of product delivery will go down in most if not all cases.
And this is where we see that all falls into shambles. Software development is not a matter of producing software, of producing code. The creation of software is not a production process, it is an engineering effort. It is a matter of continuously solving (new) problems. Formerly, the production aspect of software-based products was in the creation of media containing the result of the engineering effort. Producing the discs with the application on it, not the production of the application itself. Currently, when the product is being delivered through digital means, the production aspect for software-based products is reduced to almost nothing at all. Here we see why an efficiency-focused approach is wrong when it comes to software development, since software development is a matter of engineering and not of producing. Most of the cost in software product development is related to engineering, costs that cannot be reduced through improved efficiency.
Understanding that in a context of software products being effective is more important than being efficient, is to understand the difference between producing and engineering.
A common motivation for becoming more agile besides the ill-advised cost-reduction, is to be able to better respond to changes. This can be an offensive trait like the cheetah chasing the warthog, or a defensive trait like the warthog running from the cheetah. Thus, in order to be agile, you need to be able to turn on a dime at a moment’s notice. To catch your prey, or to prevent being caught. Which means that whatever you do, you need to be very effective when you move. Efficiency is less of a concern.
This means that we need to reduce the time to required to respond, i.e. reduce the time needed to transform thought into action. In an agile world this means that we want to get reduce the number of silos. Silos introduce bureaucratic overhead in order to increase efficiency.
Clear and formalised communication between silos, i.e. the bureaucratic information flow, makes for a well-oiled process. Bureaucracy is a means to streamline lengthy information flows. Bureaucracy improves your ability to plan.
When agility is becoming a necessity, further streamlining of the process will not do. In fact, long term planning is less of a concern as with every response to a change we need to reevaluate our course of action and plan again. Information flow improvements are realised through shortening the communication lines. In effect, overhead reduction leads to an improvement of effectiveness.
Typically, communication lines between dependent parties in a process are shortened by introducing multi-disciplinary teams.
The takeaway from this is, that Efficiency focuses on minimising cost by spending as little as possible on the creation of a product on a per product basis. By doing so, the cost of the product is reduced and as a consequence the margin per product increases. Production processes benefit from cost reduction. Typically, this is achieved by leveraging specific capacity for specialised tasks.
Effectiveness on the other hand focuses on reducing the time required to generate revenue by only doing what is needed, i.e. reducing overhead. By doing so, the cost of the product increases, but so does the relevance of the product for the consumer. The product’s value increases more than the cost to develop it, which obviously impact’s the organisation’s bottom-line positively. Engineering processes benefit from cycle-time reduction, which is the consequence of improving effectiveness.
The difference between efficiency and effectiveness as illustrated above is a key component in the business of software products.
Efficiency builds on the concept of increasing margins through reduction of cost. Since software is not produced according to a repetitive task, but is an engineering effort instead, the delivery process of software-based products cannot be streamlined in the same sense as, say the production of a car. Improving efficiency will not lead to better margins.
Although the abstract software engineering process can be described as a repeatable process, the actual execution of the process will be different for every new product, feature and even enhancement.
It makes sense to focus on efficiency when you need to produce large quantities of the same product. For example, when you produce nuts and matching bolts. Efficiency is for growing your market share with a commodity product. When your product is anything but a commodity, striving for more efficiency is close to impossible.
It takes this understanding to see why agility is a key aspect of being able to survive in today’s digital world. A world where every product is produced only once but continuously engineered. Compared to a world where a product used to be engineered only once and then continuously produced, this is a major paradigm shift.
It is exactly this paradigm shift that requires companies to no longer focus on efficiency but rather spend all effort on becoming more effective instead. A paradigm shift that engineers are likely to make. It is no coincidence that agile transformations typically start within the (software) engineering departments. Those that struggle the most with this paradigm shift can usually be found at the leadership level. Managers are accountable for an organisation’s bottom-line. Either by reducing cost or by selling more products. The latter being the least controllable option, managers are conditioned to focus on cost reduction.
Cost reduction can mean doing the same at a lower cost or doing more at the same cost. A subtle difference with a very significant business impact. Doing the same at a lower cost effectively means maintaining the status quo. On the other hand, doing more at the same cost means a focus on business growth.
At the leadership level, shifting paradigms is a matter of understanding that cost control, albeit of significant importance, is not a viable strategy. Not for organisations that deliver software products. It requires managers to understand that improved efficiency is not going to contribute to the bottom-line, since it will not impact the engineering effort required to create a product. Improved effectiveness during the engineering process will have the required impact.
As you have found out by now, the development of software-based products relies more on being able to master the engineering process instead of the production process. Production, in this digital age, is a non-issue since distribution is merely a matter of providing a link to your product.
Users are accustomed to continuously upgraded products, that meet ever changing requirements. Organisations that are not able to address these requirements in a timely manner, will proof to be unsustainable. Those that can effectively fulfil the needs of their users will gain a competitive edge over those that cannot.
It all boils down to the quote from Peter Drucker: “Efficiency is doing things right, effectiveness is doing the right things”. In the digital age there is not a lot to gain from doing things right, unless the right things are done. It is the difference between perish or flourish.
In my experience, the transformation from waterfall to agile software development projects is often the spark that ignites the transformation from a siloed organisation to one that is product oriented. With this change in transformation scope I see the discussion regarding efficiency vs. effectiveness become relevant. Relevant, because it determines the chance for success regarding the envisioned transformation and more so because it determines the chance for future successes.
The text very explicitly communicates my own personal views, experiences and practices. Any similarities with the views, experiences and practices of any of my previous or current clients, customers or employers are strictly coincidental. This post is therefore my own, and I am the sole author of it and am the sole copyright holder of it.
Image source: http://bit.ly/Car-Assembly-Line-Image
Special thanks to Sytse, who took the time and made the effort to review my article. I am confident that the article’s quality was significantly improved with his feedback.