Benchmarking and performance measurement in public sectors Towards learning for agency effectiveness

Learning for agency effectiveness 121

Alexander Kouzmin

University of Western Sydney, Nepean, Australia

Elke LoÈffler and Helmut Klages

Post-Graduate School of Administrative Sciences, Speyer, Germany, and

Nada Korac-Kakabadse

Cranfield School of Management, Cranfield, UK Keywords Benchmarking, Information technology, Learning, Performance assessment, Quality assurance, Self-assessment Abstract Given the prevailing emphasis on agency performance, customer focus, stakeholder's interests and other methods of assessment under new public administration and prevailing managerialism in many public sectors around the world, administrative practitioners have taken to benchmarking as an instrument for assessing organizational performance and for facilitating management transfer and learning from other benchmarked organizations. The introduction of benchmarking into the public sector is still in its early stages. Technical problems, scepticism about usefulness and the appropriateness of transferring putative private sector competencies into public administration and the resistance in accepting organizational change as a necessary consequence of benchmarking exercises in the public sector, prevent the widespread acceptance and use of benchmarking in public sectors, arguably ``punch-drunk'' with systemic change. Nevertheless, there are some encouraging examples of benchmarking within the public sector. This paper critically analyzes these examples in order to establish the vulnerability points of such measurement instruments which, possibly, need more research in order to establish the specific learning dimensions to benchmarking and to illustrate the importance of such benchmarking and learning within the highly risky, information technology (IT)-driven experiences of systems development and failure.

Trends in performance measurement in the public sector in the 1990s Performance is a key word permeating all discussion about ``new public management'' (OECD, 1993, p. 7). Part of its attraction is that performance is a broad concept: it has various meanings, for different audiences, in different contexts (Carter, 1991). This makes the design of performance indicators (PIs) in both the private and the public sectors very difficult. Besides the technical problem of operationalizing an abstract concept, the same set of PIs may need to be used to answer questions about the different dimensions of performance. Whereas performance measurement in the private sector is, in general, seen as something normal ± the assumption being that the private sector is imbued

The International Journal of Public Sector Management, Vol. 12 No. 2, 1999, pp. 121-144. # MCB University Press, 0951-3558

IJPSM 12,2

122

with a performance-based culture ± conventional wisdom suggests that there are special characteristics of the public sector which make performance measurement inappropriate or, at least, very difficult. Two explanations are commonly used to explain the differences in public/private performance measurement (Carter, 1991). The first assumes that because private firms putatively adhere to bottom-line profit requirements, performance measurement is a straightforward and contestable technical procedure. The second argument focuses on the particular social and political pressures on public sector agencies. Public services operate with a fixed budget and consumer groups are in competition with each other for scarce resources. The market solution to this situation is to introduce the user pays principle in selected public services so that users of a service are the actual people paying for that service. But the imperative of the welfare state precludes this market option from economic textbooks. This problem of scarce resources implies for performance measurement in the public sector that a certain degree of insensitivity to consumer demands is positively desirable in order to protect the interests of those vulnerable consumers, least satisfied with services delivered and with the least resources for either ``exit'' or ``voice'' modes of protest (Klein, 1984). In other words, consumer satisfaction cannot be the only, or dominating, dimension in performance measurement in the public sector and has to be handled with considerable caution (Swiss, 1992). There are also different dimensions to performance measurement. Whereas in the 1980s the focus was on the ``three Es'', economy, efficiency and effectiveness, in the 1990s attention has shifted to quality and consumer satisfaction. One reason for this can be seen in a value change to the phenomenon of the ``difficult citizen'' (Klages, 1994); one who wants to be the ``subject of his actions''. This value change involves the ``difficult citizen'' not passively accepting the state per se, in the sense of the ``older'' theories about the functions of the corporatist state, but taking a consumerist and ``instrumental view'' (Klages, 1993) of the state in the sense of public services delivered. Three major trends in performance measurement, in OECD countries in the 1990s at least, can be identified (Australia, MAB-MIAC, 1993): (1) the development of measurement systems which enable comparisons of similar activities across a number of areas (benchmarking instruments, such as citizens charters and quality awards); (2) efforts at measuring customer satisfaction (citizen surveys; output as indicators, such as the number of complaints; and throughput measures such as indirect proxies for measuring direct impact of programs on clients); and (3) some lessening in the focus on the long-term impact of programs, particularly in evaluating such programs.

While these trends help to establish a performance-based culture in the public sector there are problems related to the lessening of attention to the long term and with the dominance of the consumerist orientation in performance measurement. In the ``new public management'' paradigm, the basic question which drives performance measurement in the public sector is not ``how much'' government one has, but ``what kind'' of government (Osborne and Gaebler, 1992). By concentrating performance and evaluation exercises on the means instead of the ends of public agencies, one risks measuring ``peanuts'' in public management. Major public sector performance measurement concepts Benchmarking Benchmarking can be seen as an important management tool of total quality management (TQM). It was first developed by Xerox Corporation in 1979, when severe quality and costs problems became visible in the face of the extremely low price of Canon copier machines (Horvath and Herter, 1992). Today, this instrument is used by a large number of US companies, such as Motorola, Ford, GTE, IBM, AT&T, Honeywell and Alcoa. Benchmarking is a term which was originally used by land surveyors to compare elevations. Today, however, benchmarking has a narrower meaning in the management lexicon since the benchmark is industry best-practice and is not in any sense a standard. Camp (1989, p. 10) defines benchmarking as ``the continuous process of measuring products, services and practices against the toughest competitors or those companies recognized as industry leaders, (that is) ... the search for industry best practices that will lead to superior performance.'' The aim is to identify competitive targets which render the weak points of the benchmarking organization visible and to establish means of improvement. In other words, the basic idea behind benchmarking is not to find out ``by how much others are doing better but, rather, how they make it to do better in certain areas'' (Horvath and Herter, 1992, p. 5). Given the fact that industry best practice for a given product, service or process may never be found, because of high transactions costs, only relative or local optimums are found as benchmarks. In the real world ``ideal-type'' definitions of benchmarking need, however, to be modified. Consequently, benchmarking is a continuous, systematic process of measuring products, services and practices against organizations regarded to be superior with the aim of rectifying any performance ``gaps''. With regard to the organization being compared, the object and the targets to be improved, different forms of benchmarking can be distinguished (see Table I). Benchmarking is a more comprehensive exercise than ``reverse'' product engineering which only focuses on the analysis of specific components and functions of the products of competitors (von der Oelsnitz, 1994). Nevertheless,

Learning for agency effectiveness 123

IJPSM 12,2

124

in industry, reverse product engineering is often the starting point of a company's benchmarking exercise. Companies often concentrate on the comparison of costs, as was the case with Xerox (Tucker et al., 1987). The extension from the comparison of products to the analysis of methods and processes implies that comparisons can be made with companies of other sectors that ``excel'' in their methods or processes. With the focus on methods and processes being compared, new targets such as time, quality and customer satisfaction emerge. The practical advantage of comparing with noncompetitors is that information can be obtained much easier since competing organizations naturally have a common reluctance in sharing commercially sensitive information. Benchmarking against competitors may also uncover practices that are unworthy of emulation. While competitive benchmarking may help a company to unravel the competitor's performance, it is unlikely to reveal the required practices needed to surpass that competitor's performance (Camp, 1989). The fact that most companies develop their benchmarking analysis in subsequent stages shows the complexity of this ``strategic-competition/ analytical planning instrument'' (von der Oelsnitz, 1994). What is important for the introduction of benchmarking is that this instrument needs to be permanently used by staff (Walleck et al., 1991). One major issue is the need to decide on appropriate indicators to be used in the benchmarking process. Financial indicators rarely exist within public agencies. Furthermore, the search for the ``best of the class'' is the most difficult part of the benchmarking exercise. A systematic search involves high cost and since, in most cases, only secondary information sources are available, comprehensive searches do not necessarily lead to comparable outcomes. Another problem is to get the data needed for the analytical part of the benchmarking process. This is easier when comparisons are made ``across-theborder'' since companies more readily release information they would not, normally, give to direct competitors. The success of benchmarking depends on employees understanding the results (and the consequences) of the benchmarking exercise and that they will need to participate in determining and implementing necessary organizational change. Possible new performance targets have to be set and actions plans made. In markets that are changing Parameter

Table I. Forms of benchmarking

Object of benchmarking Target of benchmarking Reference of comparisons

Examples Products

Methods

Processes

Costs

Quality

Intradepartmental competition

Constituencies and clients

Customer satisfaction Same agency or sub-unit

Source: Horvath and Herter (1992, p. 7)

Time Different agency or sub-unit

fast, today's ``best of the class'' may not be tomorrow's as one has subsequently learned from the In Search of Excellence (Peters and Waterman, 1982) example. Benchmarking, therefore, has to be a permanent exercise, even tracking corporate and agency failures, in order to help establish objects, indicators and target companies to be benchmarked. Needless to say, such indicators and selected companies and agencies need to be constantly validated. The search for the ``best of the class'', the definition of ``good'' indicators and data collection turn out to be critical elements of the benchmarking process. It is striking that only large companies can afford to develop their own benchmarking. Other companies rely on previously developed programs (for example by participating in quality awards) or, at least, investing in external counselling (by participating in benchmarking training workshops). Benchmarking in the public sector is analogous to that in the private sector but the motivational forces and obstacles are somewhat different. Benchmarking introduces cooperation in the private sector that, subsequently, motivates competition for market share. In the public sector, according to the paradigm of ``new public management'', benchmarking is supposed to introduce competition into a state apparatus context that is characterized by the cooperation of public sector agencies for the ``collective'' public good. Meaningful competition is only possible between producers of the same products and services. If benchmarking is supposed to introduce competition into the public sector it has to be done, it is argued, between public agencies with very similar goals and other organizational characteristics so that actors actually perceive differences or qualitative improvements in delivering similar services to constituencies. A health agency, therefore, finds it difficult to be compared with a local municipality so that ``across-the-border'' benchmarking in the public sector is not as useful an exercise from the point of view of competition and its putative benefits on actor performance (Dixon and Kouzmin, 1994; Dixon et al., 1996). There are also methodological considerations against ``across-the-border'' benchmarking in the public sector. Whereas a state health agency may learn innovative practices from a local municipality, it may not have the degrees of freedom to implement consequential structural or procedural improvements within its own sphere of jurisdiction. Functional benchmarking leads to frustration because information obtained from non-related public sector organizations cannot be used. This may be the explanation for the empirical finding of a survey of the Western Australian public sector that public organizations tend to compare related government agencies more than any other sources (Frost and Pringle, 1993). As far as the availability of information is concerned, there is no need for ``across-the-border'' benchmarking since public agencies have no competitive drawbacks to fear from passing information on to peer organizations. It is difficult for public sector organizations to identify the ``best of the class'' experience themselves. Since simple financial indicators such as revenue per employee and inventory returns are missing, it is very difficult to establish

Learning for agency effectiveness 125

IJPSM 12,2

126

which organization can be labelled ``successful'' and be used as a benchmark. With respect to this problem, Quality Awards and Citizens' Charters fulfil a very important function by delivering a benchmark for public sector organizations. The benchmarking instruments serve to identify superior performance and to make their superior practices visible to other organizations. Quality standards such as DIN ISO 9000-9004 can be interpreted as a precondition to benchmarking because it helps private and public organizations alike to define their own quality assurance system. DIN ISO 9000-9004, citizens' charters and quality awards are measurement instruments setting benchmarks for different performance levels. Figure 1 visualises the context of the instrument of the benchmark family. The quality assurance system DIN ISO 9000-9004 The DIN ISO 9000-9004 is an internationally recognized benchmark for quality management. It gives indications to companies as to how to develop quality management and a quality assurance system and gives a standard to external and internal audits to assess the degree of quality management of companies. The implementation of DIN ISO 9000-9004 in companies is an important step to TQM and such standardization is not as contradictory to a TQM approach as the literature often suggests (Antilla, 1993). Underlying the mysterious formula, DIN ISO 9000-9004, is a synopsis of norms which needs to be understood as a guideline, as a benchmark for companies to improve their individual quality management but also being suitable to their needs and organizational characteristics. It is by no means a ready, of-the-shelf solution or a regulatory system in a negative sense. With regard to the content of DIN ISO 9000-9004, it can be divided into three sections (BlaÈsing, 1992, p. 27): (1) instruction to use, selection criteria (9000); (2) guidelines for the development of quality management and one's own responsibility (9004); and Level of Performance

Figure 1. Benchmark instruments and performance level targets

ISO 9000-9004 Citizens’ charters Source: Löffler (1996, p. 147)

Quality Awards

Benchmarking Instruments

(3) proposals for the selection of elements in the framework of contractbased negotiations (9001, 9002, 9003). It is also apparent that the ISO-system differentiates between the contracting and non-contracting situation. The non-contract situation is the starting point for the development of TQM for which DIN ISO 9000-9004 delivers the mode of operation. A quality assurance system consists of 23 elements, out of which a selection is proposed for the formulation of contracts. For trading in the internal European Union market, for example, this is an important marketing advantage. This also explains the increasing importance of certification for trading companies (Meier, 1993). Does DIN ISO 9000-9004 also make sense for public sector organizations? To date, the application of the ISO-system to the public sector is still at an experimental stage. The Finnish Association of Local Authorities has done pioneering work by developing, together with five municipalities, an ISO Handbook for municipal authorities. Such a handbook, written in a language that fits the administrative culture in Europe, could be a starting point providing a minimum standard of quality in public services in the European Union. Nevertheless, the ISO-system focuses very strongly on quality and omits all other dimensions of performance measurement. For industry, in a buyer's market, quality is an important competitive advantage; in government, however, the incentive structure is such that political success is more important than quality-oriented management (Sensenbrenner, 1991). Quality in the public sector is always desirable, in the sense of competence in the routine activities of the organization. It may not always be desirable for various stakeholders in the sense of quality being considered as a luxury. In poor constituencies, for example, taxpayers may prefer to have a poorer quality of drinking water and, consequently, pay lower water rates. Citizens' charters Charters are official frameworks for assessing and awarding quality in the public sector. Whereas DIN ISO 9000-9004 measures meso-quality as an ``external quality concept which applies to producer/consumer relationships'' (Bouckaert, 1992, pp. 4-5), charters address macro-quality as a ``generic system concept which applies to the public service/citizen relationship'' (Bouckaert, 1992, p. 5). The essential idea behind charters is to increase the quality of life in society and to pay more attention to the needs of citizens. The ultimate purpose is to renew citizen trust not only in public services but also in the State. The method is to renew the Rousseau-like ``social contract'' (Broome, 1963) between the rulers and the ruled, by developing charters. Belgium, France, the UK and Portugal have now considerable experience with designing and developing distinctive and operational charters. Charters sound very convincing at first sight but there are two in-built contradictions in the basic concept. The first inconsistency of citizens' charter

Learning for agency effectiveness 127

IJPSM 12,2

128

is that it makes allusion to the English Magna Charta Liberatatum of 1215, which was a letter of privileges for noblemen rather than a catalogue of liberal rights for all citizens (LoÈw, 1986). Nevertheless, it was an important milestone in the history of human and basic rights. At the centre of this early codification of liberal rights is the citizen; a citizen being defined as a concentration of rights and duties within a constitutional state, within the rule of law, and a hierarchy of laws and regulations (Bouckaert, 1995). The ``client'' is a much more limited concept since the citizen is part of the social contract, whereas the client is part of the market contract. When charters are used in a new ideological context, such as with ``new public management'' (Kouzmin et al., 1997), the meaning of charters changes significantly. It is no longer a catalogue of rights and duties of the ruler and the ruled, but a ``quality'' checklist for clients of public services. This is especially true in the British Citizens' Charter which builds on the userservice provider relationship instead of citizen-state relationships (Bouckaert, 1995). Second, charters are based very much on older social contract theories (Rousseau, Hobbes, Locke) according to which citizens believe, unconditionally, in the legitimacy of the state as long as the state provides security (Hobbes, 1966). This is no longer true today where the market contract becomes more dominant than the classical social contract. The Belgian French, Portuguese and British Charters are clearly influenced by contextual macro-variables such as administrative culture, the politicaleconomic system and corporate interests (Bouckaert, 1995). The British Citizen Charter basically adds to the 3Es of economy, efficiency and effectiveness, a quality dimension through competition (UK Prime Minister, 1991). The French and Belgian reasoning, however, bases very much on the idea of accountability of civil servants to the citizen. The Portuguese Public Service Quality Charter also focuses on the idea of accountability of the state to the citizen but also on value for money. Thus, even though the charter concept is basically the same, the operationalization of citizens' charters is very much different in the various OECD countries. Citizens' charters are consistent with their public sector environments and are not just an ideological overlay on developed and mature administrative systems. There is one particularity of the British Citizen Charter which is the Charter Mark Award. Applicants ± all kinds of public sector organizational units ± have to prove to the Prime Minister's Citizen's Charter Advisory Panel that they meet the Citizen's Charter principles for delivering quality in public services (UK Prime Minister and the Chancellor of the Duchy of Lancaster, 1992). The Charter Mark winners then use the Charter Mark on their products to show that their achievements have been recognized. The Charter Mark Award is a special kind of quality award lying at the minimum end of the spectrum between DIN ISO 9000-9004 and quality awards, emphasizing optimal levels of agency performance.

Quality awards Before analysing quality award competitions, one has to make clear what this label means; there is a whole variety of quality award competitions in the private as well as in the public sector. A public quality competition award is defined as a performance measurement instrument which fosters innovation and quality in the public sector by the identification of excellent public organizations by independent panels and with active participation of public agencies. These processes render the success factors of excellent administrative practices transparent. This definition excludes awards that are given to organizations for outstanding past achievements without using performance indicators and without submitting to a competitive selection process. For example, the grant of the Distinguished Service Cross to public organizations in Germany does not fall into the quality award category because the selection is not by a competitive process. Nevertheless, the above definition is broad enough to include innovation awards in the public sector. As Borins (1995) points out, the difference between specific and general quality awards is gradual rather than fundamental. Given the elusive nature of innovation in the public sector, judges of specific innovation awards will need to include effectiveness in overall judgements. Quality award competitions come from the private sector and have been transferred to the public sector in the process of the paradigm shift taking place in ``new public management'' of many Anglo-American and European economies. The basic assumption, again, involves the putative benefits of competition; why not introduce surrogates of competition into public sectors which lack the ``discipline'' of market competition? The competition among the participants of an award program is supposed to motivate other actors of public agencies. However, due to specific characteristics of the public sector, quality competition awards have another important function. Whereas in the case of private goods there is a price, which is a complex measure of all the quality characteristics economic actors attribute to such goods, there is no such indicator in the case of public goods. As a result, there is at least a quantification problem in the assessment of quality in the public sector. Quality award competitions can be instruments measuring quality by using complex multi-dimensional indicators. This technical function is closely related to the third function of public quality award programs, which is to help public agencies improve their quality assurance systems by learning from each other. By quality competition awards, excellent public organizations are identified and their success factors are made visible to other agencies (Haubner, 1993). There is also a cooperative element in quality competition awards, which is perhaps the most important function of such awards if they claim to be an instrument in fostering innovation and quality improvement in the public sector. It is obvious that there is a tension between the competitive and cooperative elements of public quality award competitions. On the one hand, participants of award programs want to know how good they are compared to other

Learning for agency effectiveness 129

IJPSM 12,2

130

organizations and to conduct, subsequently, SWOT analyses. On the other hand, nobody wants to ``lose''. Organizers of quality awards have to stress the cooperative element of awards; namely that every participant ``wins'' by learning from others and that winning the award is not terribly important. This trade-off has to be made by organizers of award programs. If the learning aspect is central, it might be more functional for the program to avoid the term ``competition'' and to look for another term. On the other hand, if the aim is to introduce competition, winning an award becomes the crucial element of the program and ``losing'' has to be part of the outcome. Like the Citizen Charter Award, quality awards only consider ``good'' organizations. ``Bad'' organizations are not considered. In this sense, the term ``quality award competition'' is somewhat misleading because quality awards are not a substitute for market competition since, officially, there are no losers. As Halachmi (1995) has pointed out, quality awards do not automatically have a motivational function for the ``winning''' organizations. For organizations which do not achieve an award, such competitions can be highly dysfunctional. Quality awards tend to be self-selecting in that organizations that do not even dare to participate in an award program are ignored. Does the award program have indirect motivational effects by giving such organizations an example? It becomes clear that the capability of such organizations to become ``learning organizations'' (Senge, 1990a; Argyris, 1992) is the precondition for benchmarking. Quality awards do not reach, and can have no impact on, those organizations which lack elements of a learning culture. Pessimistically, this means that quality awards are more likely to be a means of making relatively ``good'' organizations ``better'' and that they cannot ``mobilize'' the mass of public organizations and agencies. Nevertheless, quality awards have the important function of raising the awareness of quality in the public sector and for facilitating research on innovation. The basic concept of benchmarking, however, raises further questions. First, does benchmarking stimulate innovation? It might be more convenient to look to what others are doing rather than think about original solutions. Benchmarking helps to spread innovation but its link to simulating innovation is an unanswered question. Second, which benchmark instrument best fulfils the performance measurement needs of public organizations? Attention needs to be given to the fact that benchmarking is a complex measurement instrument with high transaction costs. Thus, it needs to be ensured that measurement costs do not exceed organizational ``slack'' that is being eliminated in the name of economic efficiency (Kouzmin et al., 1996). Finally, the ultimate goal of benchmarking is to make organizations ``lean and mean''. If, however, organizational slack is a precondition for innovation and crisis response capabilities (Rosenthal and Kouzmin, 1996) and benchmarking aims at reducing slack, benchmarking could be counter-productive, especially in industries, agencies and firms for whom innovation is a constituent element of competitive advantage.

From benchmarking to learning strategies Recent European experience with benchmarking indicates that the benchmarking community is cultivating colourful differentiation; so much so that benchmarking can be now viewed as a terminological ``umbrella'' covering a wider range of processes than the core-elements conventionally viewed. Recent forms of benchmarking now include comparisons of organizations on the basis of self-assessment, but using identical indicators; benchmarking cooperatives ± groupings of organizations or sub-units that get together in order to learn from one another; and, finally, benchmarking has broadened to incorporate empirical studies of the different paths by which social modernization processes take place. The development of instruments for selfassessment of public organizations and measurement of the degrees and profiles of specific modernization and reform strategies is also proceeding. The creation of institutions called ``innovation-rings'', in which change agents in several branches of the German public sector, for example, get together in order to learn from each other, is also a recent benchmarking innovation in Europe. Finally, some detailed work has emerged on the comparison of the performance of public sector organizations by means of employee, citizen or client surveys, the results of which are retrieved and relayed to informants by means of data-feedback techniques. A preliminary attempt at drawing some general conclusions about expanding German and other European benchmarking experiences involves nine emergent propositions: P1: The different forms of benchmarking have many common effects, although they differ with respect to certain other effects. As a consequence, one has been able to successfully practice combinations of different forms of benchmarking which have been successfully used with more differentiated functional profiles. This has been done without having a ``clear-cut'' theory at hand. P2: Benchmarking can, on the whole, be seen as a learning strategy; as a widely-shared conviction and aspiration within the benchmarking community itself that having experimented with benchmarking, the learning potentialities and outcomes of benchmarking can be considerable and real. Empirical evidence supports this learning conviction and aspiration. For example, to a considerable degree, by merely sending public agencies award-questionnaires, many such organizations can be influenced by being forced to evaluate previous accomplishments and on-going weaknesses. P3: A full utilization of the ``learning'' potential of benchmarking seems to be only possible if such forms of benchmarking are chosen or developed so as to combine elements of competition and cooperation. This proposition appears to be paradoxical in view of the traditional arguments that these two social mechanisms are antagonistic. One might call this the ``dialectical'' dimension to modern complexity. The

Learning for agency effectiveness 131

IJPSM 12,2

132

character of social and organizational learning itself might be understood better if conceptualized as a ``dialectical'' combination of both competition and cooperation. P4: Recent developments in the field of benchmarking indicate that this ``dialectical'' combination of competition and cooperation is already being widely practised. This is quite apparent wherever ``benchmarking cooperatives'' have been created. The same impression prevails, however, if one looks more closely at the development of quality award competitions. As Halachmi (1995) points out, one danger inherent in award competitions seems to be the effects of frustration and ``demotivation'' that emerge whenever award competitors ``fail'' to achieve some recognition. Even ``winners'' can be afflicted by frustration and demotivation, however, when they reflect on the time and money associated with being a ``role model'' for other interested agencies or being the object of attention of less successful competitors' envy, distrust, even their own employees' heightened sets of expectations. In light of such experiences, the organizers of the European Quality Award, for instance, see to it that every participant receives a ``reward'' in terms of learning outcomes and that successful competing organizations take a cooperative role in this context. In the German experience, for example, it has come to be regarded as appropriate to combine competition and cooperation and this combination is now a decisive success factor of quality award competitions. Everybody wins by having a chance to cooperate and by taking a role as a learner or as an action-learning agent. However, some problems with the implementation of this ambition exist and a stable agenda for future action remains yet to be identified. P5: The learning-effects of benchmarking are, to a very high degree, dependent on adequate organizational conditions and managerial solutions. This holds true for benchmarking activities themselves and only if the delicate relationship between competition and cooperation is cogently managed. The success of innovation rings, for example, is primarily dependent on the availability of such organizational and managerial skill. Such rings may either collapse under the pressure of mutual jealousy or be transformed into managerial complacency with a supportive, but non-learning, climate. Difficult problems of keeping a productive learning-oriented balance between competition and cooperation may also arise when the results of employee surveys are departmentally categorized and presented in a comparative way. On the one hand, the benchmarking opportunity might be quite eagerly taken by actors inside the organization. On the other hand, actor satisfaction with the results of benchmarking surveys will strongly correlate with individual assessment scores. The effect may well be that neither the high scoring nor the low scoring units will be ready to take

on a learning role. The ``winners'' may just feel justified in their selfconfidence, whereas the ``losers'' may feel highly defensive. Instead of enforced learning, a heightened level of organizational conflict can result from benchmarking if organizational or managerial skills are lacking regarding the need to harmonise ``winners'' and ``losers'' on the basis of a common commitment to learning. P6: In many cases of benchmarking, mere organizational or managerial skills will not be enough. The so-called ``upward'' assessment of superiors is possible only where a cooperative atmosphere or culture has been previously established. Benchmarking will only be a reliable learning strategy if the precarious balance of competition and cooperation is supported by a favourable framework of contextual conditions as well. Comparative assessments of people and sub-units inside organizations cannot lead to productive learning behaviour if the organization is highly centralized; one in which managers are rigorously engaged in strongly competitive ``face-saving'' and employees are regarded as ``inferiors''. The same holds true for comparisons between organizations, dominated by officials who think and act mainly in terms of achieving or defending a hierarchically defined status position. Status-seeking behaviour, as such, is not a hindrance to benchmarking and to learning. On the contrary, one finds an element of this behaviour in nearly every ``learning'' activity. A serious impediment to learning occurs, however, wherever the status-seeking behaviour of some constrains the status-seeking behaviour of others. In hierarchical organizations, in which ``vertical thinking'' (Kouzmin, 1980a; 1980b; 1983; Korac-Boisvert and Kouzmin, 1995) prevails, this will necessarily be the case to a very high degree. The conditions for successful, learning-oriented benchmarking strategies are most effective, it is postulated, where the structural and cultural characteristics of an organization permit status-seeking behaviour by everyone. The main structural pre-conditions for this can be found in decentralization, the empowering of sub-units and the use of non-hierarchical means for coordination. In view of the state of debate in organization theory (Weick, 1969; Galbraith, 1977; Argyris and Schon, 1978; Bachrach, 1989; Meyer and Zucker, 1989; Masachiro et al, 1990; Powell, 1990; Senge, 1990b; March, 1991; Davidow and Malone, 1992; Harrow and Wilcocks, 1992; Kotter and Heskett, 1992; Nohira and Eccles, 1992; Schmidt, 1993) one can formulate the paradox that the outstanding pre-condition for learning in organizations is the creation of the ``learning'' organization in structural and cultural terms. P7: Dependency on contextual conditions implies the necessity to adapt benchmarking instruments to prevailing conceptions of ``modernization'', for instance, as well as to given levels of actual development. The concept of ``modernization of the public sector'' is

Learning for agency effectiveness 133

IJPSM 12,2

134

culturally relative and the design of measurement instruments for quality award competitions, for instance, needs to be developed with such considerations very much in mind. The need to consciously define modernization targets, regarded as realistic under given conditions and compatible with elements of the international ``new public management'' movement used as models, is challenging. P8: To the casual observer, benchmarking may appear as an element of little importance within the complex framework of conditions necessary for learning. Indeed, it would be an over-estimation of benchmarking if it were elevated and regarded as the ``primary'' success factor in public sector reform. On the other hand, benchmarking is a ``necessary'' element for such public sector modernization. Benchmarking, it needs to be stressed, has to be seen as a substantial aspect of nearly every activity of public sector reform, even where it has not been developed into a separate activity. P9: The remaining question involves what might be done to foster benchmarking? Looking at the present state of public sector reform, one might easily argue that the potential of benchmarking is by no means exhausted. For instance, it is difficult to motivate public sectors to use self-assessment workshops. The use of indicators as means of comparative self-assessment is spreading, but the quality of these indicators is still rather variable. Furthermore, applications for quality awards at local, state and federal levels are less than what one would have anticipated as being worthy of successful consideration. Similarly, the creation of benchmarking cooperatives lags significantly behind the potential level of activities. Furthermore, there are considerable European differences in experiences to date with British, French or Belgian citizens' charters; the number of employee-, citizen- and clientsurveys is still very small and those surveys which have been done often lend themselves to limited implementation, if any. Finally, empirical studies of the different ways to modernize and implement reform, and to possibly become ``excellent'', are scarcely done (Klages and Haubner, 1995). Taken together, benchmarking is still a rather exceptional activity in many public sectors. The question of how to foster its use seems to be an urgent priority. The situation in the UK, for instance, is much more developed. The UK experience differs from Germany and other European public sectors by having established a citizens' charter; by demanding performance indicators from local and state authorities; by having charged audit institutions with respective controlling functions; and by having created additional bodies such as the Citizens' Charter Unit. To some extent, Britain might be regarded as a role model for other countries. On the other hand, the quality of those performance indicators used in the UK are subject to ongoing discussion and review (Pollit, 1990; Carter et al., 1992; Sanderson, 1992). A concluding message from

preliminary survey of European benchmarking and performance measurement experiences would be the call to learn from one another as much as possible; maybe with a motivation to imitate. Benchmarking organizational learning through self-assessment Given the widespread assumptions regarding the putative superior quality of private sector service delivery (Poister and Henry, 1994), self-assessment could also be helpful in placing attitudes towards government and public services into perspective, as well as providing incentives for improving current practices. For example ``accuracy'' and ``reliability'', two core components of service quality, are not regarded as high for public sector products and services (Poister and Henry, 1994) as for those of the private sector. Furthermore, ``responsiveness'' and ``sensitivity'', two other dimensions of service quality considered lacking from public programmes (Poister and Henry, 1994), can also be benchmarked. However, while direct comparisons such as benchmarking should be made where possible (for example, services provided by public health clinics and private health clinics, or private and public employment agencies), services such as social security benefits need to be self-assessed or compared with other government services such as students' assistance benefits. To facilitate adequate benchmarking or genuine self-assessment, a ``meaningful'' corporate financial information system (CFIM) must be adopted by public sector organizations to provide a basis for benchmarking (Australia, Department of Industrial Relations, 1992). This need for a CFIM system is currently receiving attention within the Australian public sector. For example, the new Australian Accounting Standard (AAS 29) required government agencies to adopt an accrual accounting system from 1995 onwards. However, it did not require the wholesale adoption of commercial accounting practice (Australia, Public Sector Accounting Standards Board (PSASB), 1993). Although the AAS 29 focuses on the cost of service delivery and direct cost recoveries, not bottom-line profit, it requires public sector organizations to prepare general purpose financial reports that disclose the resources they control (assets); the obligations they incur (liabilities); the financial characteristics of their operations during the period (the cost of service provided, cost recoveries and other revenues); and their major source of use of cash (PSASB, 1993). Accordingly, this information allows informed decision making and assists in ensuring that departments are accountable to the ultimate provider of funds and to the beneficiaries of services provided; taxpayers, customers and businesses. Furthermore, it can be used to facilitate the benchmarking of costs of public sector goods and services with industry parallels. This is a significant improvement considering that in recent history many public sector organizations did not make financial information publicly available other than through budget papers (Micallef, 1994). Client/customer or citizen surveys are useful tools for evaluating public sector goods and services (Webb and Hatry, 1973; Fitzgerald and Durant, 1980; Streib, 1990; Miller and Miller, 1991; Watson et al., 1991) on a variety of criteria,

Learning for agency effectiveness 135

IJPSM 12,2

136

such as effectiveness (Usher and Cornia, 1981) and impact (Brudney and England, 1982) as well as quality and productivity (Folz and Lyons, 1986). However, client evaluations of the specific services they receive tend to be more positive than the rating of these same services by the general public (Katz et al., 1975; Poister and Henry, 1994). This influence may be overcome by a random sampling methodology. Furthermore, citizen responses to the very general satisfaction items in citizen surveys often do not correlate with more direct, objective measures of service delivery (Stipak, 1979). Linking survey data with more objective indicators (Ostrom, 1982; Brown and Coulter, 1983; Parks, 1984) such as benchmarking can aid organizational learning processes, agenda setting and quality improvement through the balanced assessment of programme performance. Customer/citizen survey data can be especially helpful in agenda setting in seven broad areas: customer expectation; work culture; work design; work-force requirements; hours of operation; costs; remuneration and evaluation (Graham, 1994). The effects of these agenda items can be successfully benchmarked. Competence can be benchmarked on current skills, multi-skilling, skills attainment, future needs, empowerment, staff distribution, team-work and commitment. The cost of the delivered product or service can be benchmarked (or measured against a quality standard) on programme costs, staff costs and delivery costs. For example, the effects of a set agenda for cultural change through the empowerment of employees (the product is cultural change) can be benchmarked against external or industry standards or, in their absence, evaluated against internal standards based on the following items (Wilson and Durant, 1994, p. 144): . the extent to which decisions have been decentralized to lower organizational levels; . openness, closeness or the extent of hierarchical decision making (who and how many actors have freedom to decide?); . availability of opportunities for avoiding responsibility (the opportunity for ``flight''); . the extent to which organizational resources have been distributed to employees; . the extent to which a multi-level commitment exists toward change; . how well institutional incentive systems reinforce, rather than discourage, cross-functional cooperation (the project team linkage); and . the degree of integration of functional units that contribute to the production of goods or services (the patterns of structural independence). Empowerment demands continuous learning and risk-taking activities and both thrive in a climate of trust and confidence, while both wither in a climate of fear (Holpp, 1994). The fundamental challenge of empowering is the

requirement that organizational actors continuously change and learn. The activities that should be benchmarked are learning activities and change as a consequence of learning. From the behaviourist perspective, this can be achieved by ``successive approximations'', or the series of steps that bring actors closer to a fully learning organization by gradually introducing them to new tasks and procedures, helping them to learn different jobs, reinforcing changes in their behaviour and making sure that they are provided with a safety-net adequate for different learning speeds and styles. Considering that empowerment, like learning, is an incremental process; that empowerment in most cases requires culture-change programmes that take a minimum of 9-12 months and often up to 18 months (Brooks, 1994); and that established a priori standards are often absent (Sims, 1992), the effects of change may be benchmarked in stages (Wilson and Durant, 1994, p. 144): . stages of employee readiness level (level of skills, resources and commitment necessary for change); . stages in the ability of employees to cope psychologically with the change sought in long-standing behaviour (the extent of behavioural change required); and . stages in the ability of groups to accommodate change (the actors' success rate in broadening their jobs; bottom-line results delivered in terms of costs, quality and productivity). Employee readiness levels or attitudes towards change may be measured through interviews and surveys. Broadening job scope or job enrichment may be measured through job analysis, direct observation and measures of actual job accomplishment that then can be benchmarked against industry or internal standards (Holpp, 1994). Achieved productivity can be gauged when fewer resources are used to achieve more while the quality of the service or product remains as high or becomes higher. The other measure that can be used to gauge learning is that the number of significant administrative failures, or ``soft-core'' disasters, are less over the time than the industry or internal standard (Kouzmin and Korac-Boisvert, 1995). This is particularly relevant in the area of IT development where past and current failure rates are approximately 80 per cent (Legge, 1988; Dampney and Andrews, 1989; KoracBoisvert and Kouzmin, 1994) and with a predicted future failure rate of 50 per cent, with a 0.7 probability, due to organizational resistance to change (White, 1993). Gartner Group research suggests that high rates of failure will persist in the 1990s with leading-edge technology, such as client/server and distributed processing, but with a larger variety of failures (Gartner Group, 1992; 1993; Thompson, 1993). Various software matrices can also be useful benchmarking tools in the area of IT development. These stages must be well documented so that they can be used as comparative instruments. Thus, the documentation of each pre-defined stage of change serves not to entrench a process but to measure it and improve on it.

Learning for agency effectiveness 137

IJPSM 12,2

138

The culture of continuous improvement is one of ongoing re-examination and renewal of the format of work (Graham, 1994). In continuous improvement, the perceived ``best practices'' or industrial awards, job descriptions, employment categories and promotion positions change to meet client demand (Graham, 1994). Conditions of employment are necessarily flexible rather than rigidly demarked. Furthermore, changing culture to one of learning through empowerment and self-assessment often involves various re-training initiatives (Talley, 1991; Argyris, 1992) which, owing to the pragmatics involved, are often in stages. Notwithstanding, amongst the greatest threats to cultural change through training exercises are organizational defences rooted in bureaucratic routines and interests that are stubbornly resistant to change (Argyris, 1992). Other criteria can be used for evaluating quality efforts, such as the private sector's quality evaluation approach: the Malcolm Baldrige National Quality Award, which consists of seven criterion categories ± leadership; information/ analysis; strategic quality planning; human resource utilization; quality assurance of products and services; quality results; and consumer satisfaction (US National Institute of Standards and Technology, 1991). Premised on the Baldrige Award, the eight-category criteria for the public sector, developed by the Federal Quality Improvement Prototype Award, substitutes the human resource utilization item with two separate items: training/recognition and employee empowerment/team-work, each with a different weighting score (US Federal Quality Institute (FQI), 1992). Conclusion A review of job descriptions, performance agreements, performance appraisals' instructions and empirical research indicate that public sector organizations still promote actors with traditional views (James, 1992). Those who have the ability to set direction, make key decisions, energize and control staff, only rise to the fore in times of crisis (Senge, 1990a; James, 1992). The empirical evidence suggests that public sector organizations have a tendency towards high absenteeism and low turnover rates (Wooden, 1990; Callus et al., 1991). Values are rooted in an individualistic, localized culture and reinforce a focus on shortterm events and group heroes instead of systemic forces and collective learning. This evidence suggests an emergent need for organizational cultural change, where leading actors become designers, teachers and stewards (Senge, 1990a). They are responsible for building organizations where people are continually expanding capabilities for learning (Senge, 1990b). The process of periodic public sector reform followed by a significant period of consolidation is based on the premise that organizational homeostasis is the norm and that reform is the exception (Graham, 1994). This assumption needs to be replaced by one of continuous improvement, where the ongoing re-examination and renewal of the work context is the norm. This, in turn, requires the construction of an atmosphere in which participating managers and other stakeholders alike feel sufficiently free from threats to their self-

esteem to be ``open to learn''. Helping actors move towards becoming more effective and bringing about positive change in their own behaviour is not sufficient; it is equally important that they understand the relevant processes of personality development, leadership and group dynamics (Davis, 1994). Benchmarking changes which are consequences of learning can facilitate actor learning and understanding of evolving organizational changes. Notwithstanding that existing technologies are becoming better focused (customer feedback, prototyping), more disciplined (re-use, design reviews, quality matrices) and that new technologies (object-oriented tools, automated software synthesis) have emerged, a significant challenge for IT management in the public sector, for example, remains. The transfer of software technologies into organizations and bridging the gap between state-of-the-art and the stateof-practice remains problematic. Concerted efforts by both researchers and practitioners are required to engineer a bridge for the chasm. In Australia, for example, over 52 per cent of IT professional services (consulting; technical support; system integration; software development; outsourcing of training; development and information processing) are in the public sector (IDC, 1992) and, as such, beg improvement and learning from past disasters. The use of quality models and other forms of structured experience in software development, along with feedback and continuous learning, need to become a genuinely adopted practice and not be paid lip service or fobbed off as a passing trend. Client services are becoming increasingly important forces for government, industry and the wider community. In order to harness an organization's creative potential, ensuring that its actors are well equipped to exploit the increasingly important role software development is playing in society, there is a need for ``learning to learn'' (Argyris, 1982) that produces continuous improvement. Considering that most software/applications development are group activities, involving all the complexities of group dynamics such as communication, networks and organizational politics, there is a need to re-examine the human and organizational sides of software development. The social, psychological and cognitive factors in software development are all related to quality improvement failures in organizational settings, particularly in the problem definition stage (specification requirements and design). Public sector organizations are becoming not only users, but providers and exporters of global information and associated services in an increasingly globalized market. Benchmarked knowledge and information are the fundamental strategic resources of the age; access to them through electronic networks is vital to overcome the numerous ``tyrannies of distance'' afflicting the many domains of public sector organized action. References Antilla, J. (1993), ``Quality award criteria signpost the way to excellence in quality performance'', Laatuviesti (The Finnish Quality Journal, Vol. 2 No. 4, pp. 17-19. Argyris, C. (1982), Reasoning, Learning and Action, Jossey-Bass, San Francisco, CA.

Learning for agency effectiveness 139

IJPSM 12,2

140

Argyris, C. (1992), ``The next challenge for TQM: overcoming organisational defenses'', Journal for Quality and Participation, Vol. 15 No. 2, March, pp. 26-9. Argyris, C. and Schon, D.A. (1978), Organizational Learning: A Theory-in-Action Perspective, Addison-Wesley, Reading. [Australia] Department of Industrial Relations (DIR) (1992), Research Paper No. 3, AGPS, Canberra. [Australia] MAB-MIAC (1993), Performance Information and the Management Cycle, Bulletin No. 10, Canberra, February, pp. xi; 27. [Australia] Public Sector Accounting Standards Board (PSASB) (1993), ``Financial reporting by government departments'', Draft Papers, PSASB, Canberra, December. Bacharach, S.B. (1989), ``Organizational theories: some criteria for evaluation'', Academy of Management Review, Vol. 14 No. 4, October, pp. 496-515. BlaÈsing, J.P. (1992), Das QualitaÈtsbewusste Unternehmen (The Quality Conscious Company), Steinbeis Stiftung fuÈr WirtschaftsfoÈrderung, Stuttgart. Borins, S. (1995), ``Public management innovation awards in the US and Canada'', in Hill, H. and Klages, H. (Eds), Trends in Public Sector Renewal: Recent Developments and Concepts of Awarding Excellence, Peter Lang, Frankfurt, pp. 213-40. Bouckaert, G. (1992), ``Introducing TQM: some elements of measurement and control'', unpublished paper, National Centre of Public Administration, Athens, pp. 1-24. Bouckaert, G. (1995), ``Charters as frameworks for awarding quality: the Belgian, British and French experience'', in Hill, H. and Klages, H. (Eds), Trends in Public Sector Renewal: Recent Developments and Concepts of Awarding Excellence, Peter Lang, Frankfurt, pp. 185-200. Brooks, S.S. (1994), ``Moving up is not the only option'', HR Magazine, Vol. 39 No. 3, March, pp. 79-82. Broome, J.H. (1963), Rousseau: A Study of His Thought, Arnold, London. Brown, K. and Coulter, P.B. (1983), ``Subjective and objective measures of police service delivery'', Public Administration Review, Vol. 43 No. 1, January-February, pp. 50-8. Brudney, J.L. and England, R.E. (1982), ``Urban policy making and subjective service evaluation: are they compatible?'', Public Administration Review, Vol. 48 No. 3, May-June, pp. 735-42. Callus, R., Morehead, A., Cully, M. and Buchanan, J. (1991), Industrial Relations at Work, AGPS, Canberra. Camp, R. (1989), Benchmarking: The Search for Best Practices that Lead to Superior Performance, ASQC Quality Press, Milwaukee, WI. Carter, N. (1991), ``Learning to measure performance: the use of indicators in organizations'', Public Administration, Vol. 69 No. 1, Spring, pp. 85-101. Carter, N., Klein, R. and Day, P. (1992), How Organizations Measure Success: The Use of Performance Indicators in Government, Routledge, London. Dampney, D.N.G.K. and Andrews, T. (1989), ``Striving for sustained competitive advantage: the growing alignment of information systems and business'', Proceedings of the International Working Conference on Shaping Organizations ± Shaping Technology, May, Terrigal, Australia, pp. 89-129. Davidow, W.H. and Malone, M.S. (1992), The Virtual Corporation, Harper-Collins Publishers, New York, NY. Davis, G. (1994), ``Looking at developing leadership'', Management Development Review, Vol. 7 No. 1, March, pp. 16-19.

Dixon, J. and Kouzmin, A. (1994), ``The commercialization of the Australian public sector: competence, elitism or default in management education?'', International Journal of Public Sector Management, Vol. 7 No. 6, pp. 52-73 Dixon, J., Kouzmin, A. and Korac-Kakabadse, N. (1996), ``The commercialization of the Australian public service and the accountability of government: a question of boundaries'', International Journal of Public Sector Management, Vol. 9 Nos 5-6, pp. 23-36. Fitzgerald, M.R. and Durant, R.F. (1980), ``Citizen evaluations and urban management: service delivery and an era of protest'', Public Administrative Review, Vol. 40 No. 6, NovemberDecember, pp. 585-94. Folz, D.H. and Lyons, W. (1986), ``The measurement of municipal service quality and productivity: a comparative perspective'', Public Productivity Review, Vol. 10 No. 2, Winter, pp. 21-33. Frost, F.A. and Pringle, A. (1993), ``Benchmarking or the search for industry best practice: a survey of the Western Australian public sector'', Australian Journal of Public Administration, Vol. 52 No. 1, March, pp. 1-11. Galbraith, J.R. (1977), Organization Design, Addison-Wesley, Reading, MA. Gartner Group (1992), From Computing to Networking Information Technology: The Next Five Years (Executive Summary Report), Gartner Group, Stamford, CT, September. Gartner Group (1993), Trip Report ± Pacific Symposium 93, Gold Coast, Australia, November. Graham, C. (1994), ``Towards best practices: a strategic approach to agenda setting in public sector enterprise agreements'', Management, No. 1, April, pp. 17-21. Halachmi, A. (1995), ``Potential dysfunctions of quality award competitions'', in Hill, H. and Klages, H. (Eds), Trends in Public Sector Renewal: Recent Developments and Trends of Awarding Excellence, Peter Lang, Frankfurt, pp. 293-315. Harrow, J. and Willcocks, L. (1992), ``Management, innovation in organization and organizational learning'', in Wilcocks, L. and Harrow, J. (Eds), Rediscovering Public Services Management, McGraw-Hill, London, pp. 89-102. Haubner, O. (1993), ``Zur Organization des 1. Speyerer QualitaÈtswettbewerbs: von der Idee zum `Ideenmarkt''', ``On the organization of the 1st Speyer Quality Award from the idea to the `market of ideas''', in Hill, H. and Klages, H. (Eds), Spitzenverwaltungen im Wettbewerb, Nomos, Baden-Baden, pp. 48-60. Hobbes, T. (1966), Leviathan or the Matter, Form and Power of a Commonwealth, Ecclasiastical and Civil, Luchterhand Verlag, Berlin. Holpp, L. (1994), ``Applied empowerment'', Training, Vol. 31 No. 2, February, pp. 39-44. Horvath, P. and Herter, N.R. (1992), ``Benchmarking: Vergleich mit den Besten der Besten'', (``Benchmarking: comparison with the best of the best''), Controlling, Vol. 4 No. 1, JanuaryFebruary, pp. 4-11. IDC (1992), ``The services market'', Computerworld, Vol. 15 No. 1, 3 July, p. 46. James, D. (1992). ``Winners share goals but methods differ'', Business Review Weekly, 31 July, pp. 134-5. Katz, D., Gutek, B.A., Kahn, R.L. and Barton, E. (1975), Bureaucratic Encounters: A Pilot Study in the Evaluation of Government Services, Institute for Social Research, The University of Michigan, Ann Arbor, MI. Klages,H.(1993),TraditionsbruchalsHerausforderung:PerspektivenderWertewandelsgesellschaft, (The Break of Traditions as a Challenge: Perspectives of the Value Change in Society), Campus Verlag, Frankfurt.

Learning for agency effectiveness 141

IJPSM 12,2

142

È ber die Zukunft des Wertewandels'', ``Do we all Klages, H. (1994). ``Werden Wir Alle Egoisten? U become egos?: on the future of value change'', Politische Studien, Vol. 45 July-August, pp. 35-43. Klages, H. and Haubner, O. (1995), ``Strategies on public sector modernization: three alternative perspectives'', in Bouckaert, G. and Halachmi, A. (Eds), Challenges of Management in a Changing World, Jossey Bass, San Francisco, pp. 348-76. Klein, R. (1984), ``The politics of participation'', in Maxwell, R. and Weaver, N. (Eds), Public Participation in Health: Towards a Clearer View, King Edward's Hospital Fund, London, pp. 49-65. Korac-Boisvert, N. and Kouzmin, A. (1994), ``The dark side of info-age social networks in public organizations and creeping crises'', Administration Theory and Praxis: A Journal of Dialogue in Public Administration Theory, Vol. 16 No. 1, April, pp. 57-82. Korac-Boisvert, N. and Kouzmin, A. (1995), ``From soft-core disasters in IT innovation to learning capability and benchmarking failure'', in Hill, H., Klages, H. and LoÈffler, E. (Eds.), Quality, Innovation and Measurement in the Public Sector, Peter Lang, Frankfurt, pp. 203-47. Kotter, J.P. and Heskett, J.L. (1992), Corporate Culture and Performance, The Free Press, New York, NY. Kouzmin, A. (1980a), ``Control in organizational analysis: the lost politics'', in Dunkerley, D. and Salaman, G. (Eds), 1979 International Year Book of Organizational Studies, Routledge and Kegan Paul, London, pp. 56-89. Kouzmin, A. (1980b), ``Control and organization: towards a reflexive analysis'', in Boreham, P. and Dow, G. (Eds), Work and Inequality: Ideology and Control in the Capitalist Labour Process (Vol. 2), Macmillan, Melbourne, pp. 130-62. Kouzmin, A. (1983), ``Centrifugal organizations: technology and `voice' in organizational analysis'', in Kouzmin, A. (Ed.), Public Sector Administration: New Perspectives, Longman Cheshire, Melbourne, pp. 232-67. Kouzmin, A. and Korac-Boisvert, N. (1995), ``Soft-core disasters: a multiple realities crisis perspective on IT development failures'', in Hill, H. and Klages, H. (Eds), Trends in Public Sector Renewal: Recent Developments and Concepts of Awarding Excellence, Peter Lang, Frankfurt, pp. 89-132. Kouzmin, A., Korac-Kakabadse, N. and Jarman, A. (1996), ``Economic rationalism, risk and institutional vulnerability'', Risk Decision and Policy, Vol. 1 No. 2, pp. 229-57. Kouzmin, A., Leivesley, R. and Korac-Kakabadse, N. (1997), ``From managerialism and economic rationalism: towards `re-inventing' economic ideology and administrative diversity'', Administrative Theory and Praxis: A Journal of Dialogue in Public Administration Theory, Vol. 19 No. 1, pp. 19-42. Legge, J. (1988), ``IT: a hole in your corporate pocket?'', Pacific Computer Weekly, 27 May, p. 7. LoÈffler, E. (1996), ``A survey on public sector benchmarking concepts'', in Hill, H., Klages, H. and LoÈffler, E. (Eds), Quality, Innovation and Measurement in the Public Sector, Peter Lang, Frankfurt, pp. 137-59. LoÈw, K. (1986), ``Grund und Menschenrechte'' ``Basic and human rights'', in Mickel, W. (Ed.), Handlexikon zur Politikwissenschaft, (Encyclopaedia for Political Science), Vol. 237, Schriftenreihe der Bundeszentrale fuÈr Politische Bildung, Bonn, pp. 197-201. March, G. (1991), ``Exploration and exploitation in organizational learning'', Organization Science, Vol. 2 No. 1, pp. 71-87. Masachiro, A., Gustaffson, B. and Williamson, O.E. (Eds)(1990), The Firm as a Nexus of Treaties, Sage, Newbury Park, CA. Meier, A. (1993), ``Vom wert der QS-zertifikate'', ``On the value of quality assurance certificates'', Io Management Zeitschrift, Vol. 62 No. 5, pp. 76-80.

Meyer, M.W. and Zucker, L.G. (1989), Permanently Failing Organizations, Sage, Newbury Park, CA. Micallef, F. (1994), ``A new era in reporting by government departments'', Australian Accountant, Vol. 64 No. 2, March, pp. 33-4. Miller, T.I. and Miller, M.A. (1991), ``Standards of excellence: US residents evaluations of local government services'', Public Administration Review, Vol. 51 No. 6, November-December, pp. 503-14. Nohira, N. and Eccles, R. (Eds)(1992), Networks and Organizations: Structure, Form and Action, Harvard Business School Press, Boston, MA. OECD (1993), Public Management Developments: Survey 1993, OECD, Paris. Osborne, D. and Gaebler, T. (1993), Reinventing Government: How the Entrepreneurial Spirit Is Transforming the Public Sector, Plume, New York, NY. Ostrom, E. (1982), ``The need for multiple indicators in measuring the outputs of public agencies'', Policy Studies Journal, Vol. 2 No. 4, December, pp. 85-91. Parks, R. (1984), ``Linking objective and subjective measures of performance'', Public Administration Review, Vol. 44 No. 2, March-April, pp. 118-27. Peters, T.J. and Waterman, R. (1982), In Search of Excellence: Lessons From America's Best-run Companies, Harper and Row, New York, NY. Poister, T.H. and Henry, G.T. (1994), ``Citizen ratings of public and private service quality: a comparative perspective'', Public Administration Review, Vol. 54 No. 2, March-April, pp. 155-60. Pollitt, C. (1990), ``Performance indicators, roots and branch'', in Cave, M., Kogan, M. and Smith, R. (Eds), Output and Performance Measurement in Government, Kingsley, London, pp. 167-78. Powell, W.W. (1990), ``Neither market nor hierarchy: network forms of organization'', Research in Organizational Behavior, Vol. 12, pp. 295-336. Rosenthal, U. and Kouzmin, A. (1996), ``Crisis management and institutional resilience: an editorial statement'', Journal of Contingencies and Crisis Management, Vol. 4 No. 3, September, pp. 119-24. Sanderson, L. (1992), ``Defining quality in local government'', in Sanderson, I. (Ed.), Management of Quality in Local Government, Longman, Harlow, pp. 16-40. Schmidt, J. (1993), Die Sanfte Revolution: Von der Hierarchie zu Selbststeuernden Systemen (The Gentle Revolution: From Hierarchy to Auto-controlled Systems), Campus, Frankfurt/New York, NY. Senge, P.M. (1990a), The Fifth Discipline: The Art and Practice of the Learning Organization, Doubleday, New York, NY. Senge, P.M. (1990b), ``The leader's new work: building learning organizations'', Sloan Management Review, Vol. 32 No. 1, Fall, pp. 7-23. Sensenbrenner, J. (1991), ``Quality comes to City Hall'', Harvard Business Review, Vol. 69 No. 3, March-April, pp. 64-75. Sims A. (1992), ``Debate: does the Baldrige Award really work?'', Harvard Business Review, Vol. 70 No. 4, January-February, pp. 126-47. Stipak, B. (1979), ``Citizen satisfaction with urban services: potential misuse as a performance indicator'', Public Administration Review, Vol. 39 No. 1, January-February, pp. 46-52. Streib, G. (1990), ``Dusting off a forgotten management tool: the citizen survey'', Public Management, Vol. 72, August, pp. 17-79. Talley, D. (1991), Total Quality Management Performance and Cost Measures: The Strategy for Economic Survival, ASQC Quality Press, Milwaukee, WI.

Learning for agency effectiveness 143

IJPSM 12,2

144

Thompson, J. (1993), ``Gartner Conference: notes and impressions'', Notes, DEET, 29 November, Canberra. Tucker, F.G., Zivan, S.M. and Camp, R.C. (1987), ``How to measure yourself against the best'', Harvard Business Review, Vol. 65 No. 1, January-February, pp. 8-18. Walleck, A., O'Halloran, J.D. and Leader, C.A. (1991), ``Benchmarking as a world-class performance'', The McKinsey Quarterly, No. 1, pp. 3-24. Watson, D.J., Juster, R.J. and Johnson, G.W. (1991), ``Institutionalized use of citizen surveys in the budgetary and policy-making processes: a small city case study'', Public Administrative Review, Vol. 51 No. 3, May-June, pp. 232-9. Webb, K. and Hatry, H. (1973), Obtaining Citizen Feedback: The Application of Citizen Surveys to Local Government, The Urban Institute, Washington, DC. Weick, K.E. (1969), The Social Psychology of Organizing, Addison-Wesley, Reading, MA. White, C. (1993), ``Information technology transforms processes and galvanizes enterprise'', Gartner Group Pacific Symposium 1993 Trip Report, 15-17 November, Queensland, pp. 1-4. Wilson, L.A. and Durant, R.E. (1994), ``Evaluating TQM: the case for a theory driven approach'', Public Administrative Review, Vol. 54 No. 2, March-April, pp. 137-46. Wooden, M. (1990), ``The `sickie': a public sector phenomenon?'', Journal of Industrial Relations, Vol. 32 No. 4, pp. 560-75. Further reading Swiss, J. (1992), ``Adapting total quality management to government'', Public Administration Review, Vol. 52 No. 4, July-August, pp. 356-61.