Self-assessment and systems thinking: are we doing things right or doing the right thing?

(John Seddon, Vanguard Consulting, April 1996)

"All models are flawed. Some are useful."

The British (and European) Quality Award framework is intended to help managers take stock of where their organisation is today and help them determine what to do next to further the application of quality principles. It would appear to be quite straightforward. However, our experience of organisations conducting self-assessments has identified weaknesses in the process.

The most common method for self-assessment is a facilitated workshop. The top management team have a two-day meeting facilitated by an expert (someone who has been trained in using the model for assessment). The features peculiar to this situation are usually as follows: the expert, (the facilitator), knows the model but does not know the business: the managers do not know the model and even more importantly, the managers do not know their business in a way in which they would need to use the model effectively. In other words they don't think of their organisation as a system. This is hardly surprising. They have been conditioned by their years of experience in managing the organisation through 'command and control' assumptions and practices.

Typically, the assessment shows gaps and plans are made for improvement. Is the planning based on a sound analysis? It depends on how well this method achieves an understanding of the causes of current performance - how well the participants have understood the organisation as a system. Will the plans result in genuine changes? It depends on the extent to which the actions result in changes to the system.

In one case, the managers had chosen fifty areas for improvement, prioritised them and set up projects. What happened next? Project activity waned, it was treated as 'extra work'. Consequently the projects attracted more management attention. The notion was that more management attention to the projects would ensure that they happened (a 'command and control' assumption). Over time, as no real change occurred, managers became more disenchanted and going to further 'self assessment' sessions produced cynicism.

In another organisation, which had been using the Baldrige model for five years, cynicism was rife. It was an opportunity lost. The managers had been doing projects within a traditional, 'command and control' system. The object of change is to change the system, projects were not going to achieve it.

There is an important difference between doing things right and doing the right thing. The focus of many programmes of change has been to do things right. For example, the focus of registration to ISO 9000 is often 'what do we need to do to achieve registration?', regardless of whether these are the 'right things' to do for the business. This is why there are so many companies registered to ISO 9000 whose performance has worsened. Similarly, the decision to have everybody trained in customer care must be 'right', it has plausibility. Yet people found themselves returning to a system which hadn't changed and which, effectively, would not let them 'do it'.

The Baldrige NQA... "provides companies with a comprehensive framework for assessing their progress toward the new paradigm of management and such commonly acknowledged goals as customer satisfaction and employee involvement."

David Garvin, Harvard Business Review, Nov-Dec '91.

The secret of success with this and all other quality frameworks is how one thinks about running organisations - what Garvin refers to as the new paradigm.

"Quality is not just a strategy. It must be a new style of working, even a new style of thinking."

Introduction to the Baldrige NQA.

Beyond this exhortation, there is little guidance as to the new thinking. It is no surprise, therefore, that many organisations approach the framework with a 'traditional paradigm'. Where have we seen this before? What we have witnessed in the last 20 years is a series of programmes of change, all pursuing the goals of customer satisfaction and employee involvement, failing to achieve their intended outcomes. Customer Care, ISO 9000, TQM, ABC, BPR. All the research shows that the latest panacea does no better than its predecessors.

There is a single, common cause. The cause of failure is that these interventions, by their very nature, are non-systemic. To understand what that means we have to understand what it is to view an organisation as a system and understand the implications of that view for what it means to manage.

Although we may not have learned to treat them as such, organisations are systems. A system is a whole made up of parts. Each part can affect the way other parts work and the way all parts work together will determine how well the system works. This is a fundamental challenge to traditional management thinking. Traditionally we have learned to manage an organisation by managing its separate pieces (sales, marketing, production, logistics, service etc.). Managing in this way always causes sub-optimisation. To take an example: managers who have relied on measures of activity to (mistakenly) manage productivity will give them up when they understand just how such measures damage productivity. Moreover, they will give them up with confidence when they know which measures to use instead (measures which relate to purpose) and they know how best to use such measures (to learn from variation).

Much of the effort in programmes of change is given to doing things right, though there is not much questioning whether these are the right things to do. The quality award models represent a clearer exposition that the organisation should be managed as a system than any of the previous programmes of change. A systems view of the organisation leads to different measures used in a different way, it means designing work according to different principles.

The principles and practices of traditional, hierarchical, functional management which today constitute the accepted norms of organisational life are antithetical to quality principles and practices. For example, a systems view of organisations shows the fallacy of conceptualising service and quality problems as people problems ('if only they would do it'). The failures in co-operation, poor morale and conflicts in our organisations are symptoms. Their causes lie in the system. This is not just a matter of attitude and belief, the everyday practical matters which managers work with are different in a quality organisation in very real ways.

Change for improved performance means changing the system. When features of a traditional management system are left in place, they undermine (or, at the very least, compete with quality principles and practices). If change doesn't change the system, the system doesn't change.

Failure to make this distinction results in the sort of fudges which ultimately lead to no real improvement. Two examples: managers claim extra points for leadership in their self-assessment because they review quality projects in a monthly committee meeting; managers claim extra points for process management because their processes have been defined. But there are no measures of current capability and the historical, functional measures are still in use as though they were appropriate.

The components of the quality models have plausibility. Take, for example, employee involvement. It is certainly relevant to success, but success is determined by how organisations go about it. Empowerment is a pre-occupation of traditional, 'command and control' management. Command and control management produces systems which dis-empower people. Sending people on empowerment training does not solve the problem. Only changing the system solves the problem. We have seen significant resources wasted on such 'good' ideas because they have been tackled without first thinking differently about the causes of the current problems. The same is true for management development (where it is assumed that leadership training will affect performance) and education in quality methods (assuming that these can be effective when treated as 'additional' tasks).

In the worst cases it is almost as though the model's criteria are treated as though they are 'quality wisdom', a list of things to do within the present system of management. Whereas it is the system of management itself which needs to change - this, in fact, is the foundation for developing knowledge. This is virtually impossible to do with projects. Projects sit alongside or within the current system and get subsumed by it. The priority should be to change the system and the best way to do that is to take direct action on the system. Projects which do not share the current system's underlying assumptions will inevitably be resisted and/or rejected.

Demonstrating the inherent conflict between 'business as usual' and running an organisation as a system is the best 'way in' to helping managers get the best from the quality award models. The foundation to good decision-making is consciousness-raising. In the hands of top management who understand how to manage their organisation as a system, self-assessment can play a useful role in the company's success.

Changing the system means taking out things which have been limiting or damaging current performance. For example, removing activity measures, arbitrary targets and ceasing to manage performance through budgets. Managers will only take such radical action if and when they appreciate that their traditional means of control in fact mean less control; managing costs causes costs. When the organisation is viewed as a system, the inappropriateness of such practices becomes stark. It is a major source of motivation for change. Good diagnostics are important for making the right decisions about action.

The ultimate test of quality is not an award - it is found in satisfied, loyal customers and efficient and continually improving business processes. These are the two central concerns - service and efficiency. All of the matters covered in the frameworks bear directly on those concerns or are subordinate to them. Concentration on service and efficiency provides a central management 'perspective' for successful application of any formal framework.

To provide leaders with the overall perspective from which they will learn how to lead quality improvement, we ask them in an analysis to concentrate on three questions:

How well do we serve our customers? (And how do we know?)

How well do our processes work? (And how do we know?)

What system conditions encourage or discourage quality improvement? (And how do we know?)

This approach is simple but not simplistic. It is not simplistic because it focuses on the priorities and gives the correct perspective from which improvement can be determined. It is a simple focus that is the foundation for learning about the organisation as a system.

The supplementary question (And how do we know?) recognises each time that measurement and knowledge are essential for improvement. Each question poses others as the leaders pursue their enquiry. For example, they begin to appreciate the differences in method (and thinking) between customer data which results in action and customer data which results in reports only, with no action. Learning to ask better questions is at the heart of developing leadership for improvement.

The third question (what system conditions encourage or discourage quality improvement?) deliberately makes 'conditions' subordinate to purposes. Conditions are such things as the use of measures; structure; process and job design; information; management attitudes and behaviour; empowerment, and so on. Each condition should be viewed from the perspective of how it impacts on performance and not as a goal or priority in its own right.

Developing the management's perspective through these three questions in the analysis provides a practical means to start thinking about the quality award frameworks in a more integrated way.

From this perspective, employee involvement, education in quality and other such 'commonly acknowledged' goals are directly driven by needs to improve service to customers and process efficiency. They are not viewed as independent issues - each is viewed both from its root in the task (the way work is done) and from its interdependence with other categories.

Taking action means 'doing the right things', putting in place the right 'system conditions' to ensure that performance is managed from a strong base of understanding. These actions are now taken on the basis of prediction, because the current cause and effect relationships have been understood. Knowledge is the pre-requisite of prediction and the application of knowledge is the foundation for becoming a learning organisation.

It is easy to have sympathy with the manager who lamented that 'internal advisors were coming to do a friendly audit against the UK quality model'. The origin of his lament is his failure to have any belief that the experience will help him improve the way he runs his business. He will only be convinced of the model's potential value if he can see the relationship between the 'system' and 'performance'.