WHAT IS GOVERNMENT TRANSPARENCY? New Measures and Relevance for Quality of Government

MONIKA BAUHR MARCIA GRIMES

WORKING PAPER SERIES 2012:16 QOG THE QUALITY OF GOVERNMENT INSTITUTE Department of Political Science University of Gothenburg Box 711, SE 405 30 GÖTEBORG December 2012 ISSN 1653-8919 © 2012 by Monika Bauhr and Marcia Grimes. All rights reserved.

What is Government Transparency? - New Measures and Relevance for Quality of Government Monika Bauhr Marcia Grimes QoG Working Paper Series 2012:16 December 2012 ISSN 1653-8919

ABSTRACT As policymakers and researchers focus increasing attention on the importance of government transparency for accountability and good governance more generally, the demand for greater conceptual clarity and authoritative measures of government transparency increases. Transparency advocates maintain that greater access to government information is the sine qua non of greater accountability and better quality of government in the long term. As a concept, transparency is, similar to rule of law or democracy, difficult to capture with single empirical indicators. This paper introduces a set of measures that we argue together capture key components of government transparency: government openness, whistleblower protection and likelihood of exposure (or publicity). The transparency data, collected through an expert survey carried out by the Quality of Government Institute, currently cover 52 countries with additional countries to be added in subsequent surveys. This paper explores these new measures in an effort to determine their validity and robustness.

Monika Bauhr The Quality of Government Institute Department of Political Science University of Gothenburg [email protected]

Marcia Grimes The Quality of Government Institute Department of Political Science University of Gothenburg [email protected]

2

Introduction In the last decade, the transparency movement has gained momentum; international organizations and NGOs advocate transparency as a necessary condition for better government quality, greater accountability and a more limited scope for corruption and impunity (UN-HABITAT and Transparency International, 2004; Islam, 2006; Kaufmann, et al., 2002; Kurtzman, et al., 2004; IMF, 2001; UNODC, 2004; CEPA, 2009). The theoretical arguments behind the instrumental value of transparency are by now well-established. (Stiglitz 2002) and the concept of transparency has been the subject of a number of studies in such varying fields as negotiation theory (Stasavage 2004), international security (Florini 2002), regime effectiveness (Mitchell 1998). However, an agreed upon definition of the concept of transparency is lacking (Relly & Sabharwal 2009; Florini 2007). Transparency is often conflated with “good governance” more generally which limits its applicability and lead to difficulties in finding good operationalizations of the term (Grigorescu 2003) This paper suggests a new and more theoretically founded operationalization of the concept of transparency. We suggest that transparency can be measured along three principal dimensions: government openness, whistleblower protection and publicity. These dimensions of transparency are explored using new data from the QoG expert survey. In an effort to explore and confirm the validity of these new measures, we compare our scores to currently available measures and data on transparency, and examine whether these new measures behave as relevant theories predict. In this vein, we analyze the relationship between these dimensions of transparency and a number of related areas, including corruption and quality of government, democracy and basic human rights. The analyses, still somewhat preliminary, suggest that the measures for government openness and publicity concur with theoretical predictions, but we find reason to doubt the whistleblower protections measure. Combined in an index, the three measures together perform as well as existing transparency measures but offer considerable advantages in terms of conceptual stringency and parsimony. The paper is structured as follows. Section one introduces our three central dimensions of transparency: government openness, whistleblower protection and publicity and discusses these dimensions in relation to current theoretical ideas of the concept of transparency. Section two introduces our new measure of transparency from the quality of government expert survey, as well as other data and measurements used in our analysis. Section three then presents an exploratory analysis of the association between the new measures of transparency and good government, and section four concludes

3

Dimensions of Transparency Given the political importance of transparency and the growing international interest in promoting it, it is lamentable (though perhaps not surprising) that there are currently no commonly agreed upon definition of the concept of transparency. Transparency is a multifaceted concept that is often conflated with accountability or even corruption, impartiality, and rule of law. Christopher Hood traces the history of the concept and identifies the "broadest doctrine of openness" as "...the doctrine that the general conduct of executive government should be predictable and operate according to published (and as far as possible non-discretionary) rules rather than arbitrarily" (Hood 2006, 14). Hood's analysis illustrates the tendency for transparency to bleed conceptually into other equally compelling normative standards such as rule of law, accountability, public participation, and deliberative democracy. As another example, The World Trade Organization states that ensuring “transparency” in international commercial treaties typically involves three core requirements: (i) to make information on relevant laws, regulations and other policies publicly available, (ii) to notify interested parties of relevant laws and regulations and changes to them; and (iii) to ensure that laws and regulations are administered in a uniform, impartial and reasonable manner (Bellver & Kaufman 2004). This definition includes not only making information available and accessible to stakeholders, but also that laws and regulations are administered and implemented in an impartial and predictable manner. Transparency is sometimes more narrowly defined as “the release of information which is relevant for evaluating institutions” (Bauhr & Nasiritousi forthcoming).1 Using a terminology derived from the principal agent framework, Lindstedt and Naurin make a distinction between agent controlled and non-agent controlled transparency. The release of government information by governments can be seen as a typical instance of agent controlled transparency, while the use of that information by external actors, such as the media, NGO or citizens is non-agent controlled. 2 Vishwanath and Kaufmann (1999) and Kaufmann (2002) define transparency as the “increased flow of timely and reliable economic, social and political information, which is accessible to all relevant stakeholders”. This perspective emphasizes not only the availability of information, but also its reliability and accessibility to a range of potential agents. 1

For a discussion of definitions of transparency, see Bellver and Kaufmann 2005. Cf Florini’s “regulation by revelation” as the third wave of environmental governance (Florini 1998) and “governance-by-disclosure” (Gupta 2008). 2 If this distinction is made, non agent controlled transparency can be a condition for agent controlled transparency to reduce corruption ( Lindstedt & Naurin 2010)

4

These definitions comprehensively capture a broad range of aspects related to government transparency and therefore offer valuable baseline definitions. Exploring the effects of transparency may, however, require definitions more specifically tailored to the theoretical predictions under examination. Narrowing the theoretical scope facilitates the formulation of measurement tools, and may entail focusing on a specific policy sector, or a specific aspect of the broader construct. As much of the policy literature today focuses on the role of transparency in reducing corruption, we find it compelling to explore what aspects more precisely might be necessary in order for transparency to enhance accountability and contribute to bringing about good government. A key component of the definition is the emphasis not only on the provision of information, but also the ability of external actors to demand and gain access to information not provided routinely by political and administrative institutions, i.e. both agent-controlled transparency but, importantly, also non-agent controlled transparency. We therefore define transparency as the availability of, and feasibility for actors both internal and external to state operations to access and disseminate information relevant to evaluating institutions, both in terms of rules, operations as well as outcomes. Recent decades have seen a stark increase in the number of studies using cross country data on governance. Among the most widely used and widely accepted measures of governance are the World Bank Governance indicators or TI corruption perception index Surprisingly, there is currently no similar widely accepted measure of transparency. Although the governance indexes have received a fair amount of criticism, not least for being perception based, they are widely used among both researchers and policy makers.3 No transparency index has, however, so far attained a similar level of acceptance. This deficit of data, as subsequent dearth of empirical investigations of comparative explorations of the causes and effects of varying levels of government transparency, stems in part from the empirical and analytical ambiguities, as well as normative complexity, surrounding the concept of transparency. Like many governance-related concepts, transparency contains a host of contested issues: where to draw the line between the principles of transparency and national security, individual integrity, and corporations' desire for nondisclosure; whether government offices must publish information proactively or simply provide information upon request; who should incur the cost of information provision (public offices versus those seeking access); whether access to information is 3

Studies have, for instance, called into questions their usefulness for making comparisons, in particular over time. They have also been criticized for containing biases in individual indicators and for issues pertaining to their construct validity (Arndt and Oman, 2006; Knack, 2006; Thomas, 2010) For a more comprehensive review of this critique and answers to the criticism made, see Kaufmann et al., 2007. See also Galtung, et al., 2006 and Olken 2005.

5

sufficient and meaningful absent conditions such as an independent and investigative media, and institutional arrangements for redressing abuses once brought to light. In the literature examining transparency empirically, approaches to measurement also vary considerably. Studies of the effects of transparency have employed one of three approaches to measurement, either indices comprising a broad spectrum of indicators that conceptually extend beyond transparency (Bellver and Kaufmann 2005), objective indicators (Grigorescu 2003; Hollyer, Rosendorff and Vreeland 2010) which, we argue, may be misleading if used to understand causal relations between transparency and other aspects of good government, or proxy measures, such as press freedom (Brunetti and Weder 2003) or newspaper circulation (Adesèra, Boix & Payne 2003; Besley & Burgess 2002), which capture more, but also less, than the theoretical construct of government transparency. Objective (as opposed to perception-based) measures refer to surveys of government information published (primarily) electronically, or reporting of data to international organizations (Grigorescu 2003; Hollyer, Rosendorff and Vreeland 2010; International Budget Partnership 2010; West 2005). Such measures offer considerable appeal as they eliminate measurement error resulting from perception biases and inference based evaluation, i.e. that respondents base assessments of, for example, corruption on media reporting or the overall political and economic situation in a country rather than expert or experiential knowledge. These objective measures of transparency present limitations, however, in that they only capture one aspect of transparency – a government’s propensity to publish information proactively on the Internet or upon request by international organizations. While this aspect of transparency is not unimportant, it does not capture whether citizens have the opportunity to request and receive information that goes beyond what the government has elected not to publish. Moreover, the extent to which governments effectively publish information captures the dimension of transparency that is likely to be a consequence of institutional capacity as it is to be a cause. If the ambition is to determine whether transparency leads to government effectiveness and lowers corruption –questions that neither Grigorescu (2003) nor Hollyer et al (2010) seek to address – this approach to measuring transparency is problematic. Bellver & Kaufmann present an index of transparency that instead is quite broad, and builds on a large number of indicators and measures related to transparency where: …some of the variables used to construct the indicators originate from ‘objective’ measures of transparency; for example the existence of Free-

6

dom of Information laws (Freedom info) or mandatory requirements for political funding disclosure, the availability of information on official web sites or online tools for public participation (Global e-government survey by the Centre for Public Policy or the UN E-government survey), or ‘objective’ assessments against fiscal, budget and public expenditure benchmarks (i.e. WB/OECD Budget Practices and Procedures survey or International Budget Project). Other individual variables rely on perceptionbased measures of transparency from enterprise surveys or polls of experts from international organizations. (Bellver and Kaufmann 2005:23)

The Bellver and Kaufmann index thus evaluates aspects such as the content, quality and frequency of the information provided by public institutions and whether the general public is actually getting that information, as well as whether the decision-making process in public institutions is clear and transparent. The component measures included in the index also cover several additional issues such as accountability mechanisms, audit practices, and participatory opportunities. In this paper, we seek to simplify the measurement of transparency and, taking a page from the Swedish constitution, generally regarded as the first codified transparency measure, suggest that transparency comprises three distinct but essential dimensions: government openness, whistleblower protection and publicity. Each of these dimension are discussed below.

Government openness/Access to information Government openness is defined as the information that the government releases, i.e. the extent to which governments publish information electronically or available, as well as the extent to which citizens can demand and receive information not published proactively. Studies interested in government openness sometimes use the existence of access to information (AtI) laws (Relly 2010) to capture this dimension. However, using legal frameworks to assess actual access to and availability of information is far from ideal, since countries’ level of implementation of laws varies considerably between countries. Alaistair Roberts (2006) points out that access to information depends on wellorganized records and a professional civil service. Responding to citizen requests for information requires well worked-out routines for saving documents and making them available for the public,

7

which can be costly in terms of monetary and human resources (Grigorescu 2003; Fenster 2006; Neuman & Calland 2007; Roberts 2006). Moreover, access to information laws vary tremendously in their legal and institutional strength. Mexico’s legislation has received considerable attention and praise for the ease of filing requests, the time constraints on compliance, clear appeals measures should officials refuse to comply, and an autonomous institutional responsible for enforcing the AtI law. Both citizens and non-citizens may request information, but, more importantly, a person does not need to present proof of identity when requesting information, which grants citizens protection from retaliation. Loopholes in any of these respects can render access to information legislation quite ineffectual in a political context used to operating under secrecy. In our survey, we pose a general question regarding experts' perceptions of the extent to which government documents and records are open to public access rather than trying to discern the strength of the laws and the extent to which they are enforced. In other words, we seek to determine the de facto extent of public access to government information that results from the sum of the specifics of the legal and institutional.

Whistleblower protection Whistle-blowing and whistleblower protection has receive increasing attention in recent decades, fueled by stories such as The Insider, in which Jeffrey Wigand, a tobacco company employee revealed to the media that seven major tobacco companies knew of the dangers of their products, as well as high-profile incidents in which whistle-blowing might have helped to avert disasters such as the explosion of the space shuttle Challenger and the implosion of Enron. These examples, along with much of the academic literature, treat whistle-blowing and failure to expose misconduct in corporate settings. Whistle-blowing in the public sector has, however, received much less attention. Whistle-blowing is often defined as “the disclosure by organization members (former or current) of illegal, immoral or illegitimate practices under the control of their employers, to persons or organizations that may be able to effect action” (Miceli & Near 1992; see also Miethe & Rothschild 1994; Dworkin 2002). Whistle-blower protection allows individuals to disclose information despite their individual connection to, and vulnerability to retaliation from, those in power.

8

Specific whistle blower protection laws have been established in over 30 countries today, while in other countries, whistleblower protection is regulated through labor laws and laws related to public sector employment. These legal measures vary in the extent to which they discourage the release of information to external actors (the media), and whether they stipulate rewards for exposing abuses. They also differ, for example, in terms of whether protection from retaliation is circumscribed if it is deemed that whistle-blowers did not act in good faith (Callahan, Dworkin and Lewis 2003-2004). The countries that have adopted more comprehensive whistleblower protection laws include the UK, South Africa, Ghana, Canada, Japan, New Zealand, and the United States. These laws include measures aimed at protecting whistleblowers from sanctions, incentives for people to do so and more proactive laws aimed at changing organizational culture (Banisar 2006). In a number of countries, whistleblower protection is incorporated in the national freedom of information law and freedom of press acts. Sweden has one of the oldest freedom of press acts which gives civil servants the right to anonymously report misdeeds in the public sector, and prohibits journalists from revealing the identity of sources. Whistle-blowing is one of the fundamental conditions upon which disclosure of abuses of power rests. However, despite substantial legal improvements in whistleblower protection laws, the level implementation of these laws can be expected to vary between countries. The discrepancies between legal and actual protection of whistleblowers can be expected to be particularly important. A legal right of protection is difficult to implement, since an organizations retaliation against “traitors” can be sophisticated and subtle and therefore difficult to prosecute. In our survey we ask expert to assess the risk and severity of retaliation for leakage of abuses of public power to the media (“Public sector employees risk severe negative consequences if they pass on information about abuses of public power to the media?”). Again, this means that our experts were asked to assess not only the legal framework for whistleblower protection but also the level of actual whistleblower protection.

Publicity Finally, publicity refers to the extent to which detected improprieties actually stand a reasonable chance to reach the public and relevant stakeholders. The concept of publicity goes somewhat outside the narrowest definition of government transparency, as it factors in the willingness and capac-

9

ity of media actors to address and draw attention to abuses once detected. Publicity is sometimes measured using related constructs, such as press freedom. Subhachalasai (2005) suggests that the more intense media competition and press freedom, the higher the probability that media reports corruption. Competition and press freedom would make the media more effective in finding and exposing corruption. Brunetti and Weder (2003:5) show an association between press freedom and the level of corruption across countries, and claim that “independent journalists have incentives to actively investigate any wrongdoing” including other corrupt journalists. However, as pointed to by a number of studies, a free press does not necessarily mean that it will detect and expose abuses of power. Besley and Prat (2006) outline how the nature of the market industry can easily lead to media capture and Kolstad and Wiig warn that “the media may concoct false allegations to increase profits, or use information to get access to rents.”4 Another measure used to proxy transparency and publicity is newspaper circulation. The drawback of using this measure for broader cross country analysis is its relationship with economic development. Here, we move beyond these measures and define publicity as the likelihood of abuses being exposed in the media.

About the Quality of Government Survey This paper introduces and explores data collected in an expert survey carried out by the Quality of Government Institute between March and November 2010. It covers countries in Africa, Asia, Latin America, and the Middle East (for details, see Dahlberg et al. 2011). The measurements are based on experts’ perceptions. Experts were contacted through national, regional and international organizations and through university web sites. In sum, the survey resulted in a sample of 1414 experts, with a responds rate of 31 percent, or 432 persons. Additionally, 13 persons who responded to an open link distributed to one network of scholars (where we thus cannot track the number of potential respondents). This adds up to 445 experts having provided responses for 81 countries. However, only the 52 countries for which a minimum of 3 experts weighed in are included in the analyses presented here. Latin America had the largest sample with a response rate of 37.2 percent, whereas the lowest response rate was found in the Middle East. Important to note, concerning the geographical coverage, is that African countries south of the Sahara, and also island states in the

4

Kolstad and Wiig 2009:526.

10

Pacific and the Caribbean, are greatly underrepresented, and many times absent. Figures 1-3 provide a list of the countries included.

The survey included three questions capturing the dimensions of transparency government openness, whistleblower protections, and publicity. Each of these individually is arguably insufficient to enhance accountability and a combination of the three is generally implied in policy debates as a recipe for attaining better governance. The dimensions are, however, analytically distinct and the existence of one does not necessarily imply a specific attribute along the other two dimensions. The dimensions were captured with the following statements to which expert respondents could agree (7) or disagree (1): Access to information: Government documents and records are open to public access? Public sector employees risk severe negative consequences if they pass on information about abuses of public power to the media? Publicity: Abuses of power within the public sector are likely to be exposed in the media? The whistle-blower item is reversed to avoid acquiescent responses. Responses are, however, reversed in the empirical analyses to follow so that higher values signal better whistleblower protection.

Validation against available data Verifying the validity of the survey responses can be done by various strategies including 1) validation in light of available qualitative and quantitative data available on transparency, 2) an examination of the structure of the data and relationship between the three measures, and 3) an exploration of the convergent validity of the measures. Convergent validity is met when a measure or construct correlates with variables as predicted by theory. This section deals with the first of these two and the third is dealt with in subsequent sections. The country estimates (depicted in Figures 1, 2 and 3) reveal considerable variation among the countries. Do the expert evaluations concur with available information and analyses? The govern-

11

ment openness and whistleblower protection dimensions are treated separately here. Although analytically interrelated and in some cases regulated under the same legal code, the two are generally treated in separate literatures, with whistle-blower protections figuring more heavily in discussions of law and business ethics, and access to information treated in political science and public administration studies. With respect to government openness, or access to information, the issue on which qualitative research has focused most heavily, the estimates plotted in Figure 1 seem plausible. Looking in particular at the estimates from Latin America, many of the countries that cluster in the upper end of the access to information scale (Mexico, Peru, Ecuador) have in the last decade implemented fairly strong legal measures that grant citizens access to government records and policy-related documentation. Colombia and Costa Rica, which also have strong ratings on access to information, have not put such laws in place but both have constitutional measures that guarantee citizens access to information. Constitutional measures tend to provide citizens a comparatively weaker legal footing to stand on as they generally do not specify procedures for requesting information and (if denied) filing appeals. Chile has only very recently passed an access to information law but once drafted, the law surpassed international standards in terms of the strength of legal guarantees and protections offered to citizens, suggesting a preexisting normative climate of transparency and openness. Chile and Costa Rica are also widely regarded as having a culture of openness in political institutions. Toward the other end of the scale, neither Argentina nor Paraguay has any constitutional or legal protections at the national level regarding access to information. Venezuela, the only country that has failed to ratify the Nuevo Leon Declaration regarding citizens' right to information in member countries of the OAS, ranks among the lowest of all the countries in the sample.5 Some countries' scores are, however, more difficult to account for. The United Kingdom ranks on par with Indonesia and Nepal and while the UK only relatively recently put in place legal measures to ensure citizens access to information, it seems implausible that the regime was previously as oblique as the score suggests. Such anomalies suggest that experts use different standards when evaluating access to information in a country. While respondents in Jamaica may compare government openness with, for example, other Caribbean nations, experts on the UK system may instead evaluate their own country based on a normative ideal, and therefore deem government transparency as insufficient. 5

Information on AtI legislation from an interview by Marcia Grimes with Dario Soto, legal expert on access to information at the Organization of American States, 17 December 2009 in Washington D.C.

12

The issue of whistleblower protections has received rather little attention in comparative political studies. Callahan, Dworkin and Lewis’ (2003-2004) analysis of legal measures to protect and encourage whistle-blowing in Australia, the United Kingdom and the United States offers a rare exception; Bashir and colleagues (2011) add a case study of deterrents to whistle-blowing in Pakistan. Callahan and colleagues’ deem the legal measures in all three countries studied to be comprehensive, yet also identify differences in degrees of protection and incentives for whistle-blowing. The point on which the three countries diverge the most relates to the treatment of internal versus external whistleblowing. For reasons of employer confidentiality, the U.K. law stipulates that protection from retaliation applies only if whistleblowers report misdeeds first within the organization, and resort to external channels only if “evidence of wrongdoing is likely to be concealed or destroyed” (Callahan, Dworkin and Lewis 2003-2004, 890). In Australia, only the state law in New South Wales authorizes media reports, but only condones such action as a last resort. On the whole, legislation in all three of these countries prefers internal over external whistleblowing. The single country study of Pakistan examines praxis rather than laws, and reports that while most civil servants had observed some form of wrongdoing, very few elected to report the abuse, and that the rare incidents of whistleblowing in most cases led to some form of retaliation (Bashir et al 2011, 290-291).6 The authors conclude that fear of retaliation but also strong social loyalties within public sector organizations serve as strong deterrents to disclosing abuses in Pakistan. The expert assessments of whistleblowing in the U.K. and Australia are somewhat underwhelming, with Australia rated at 2 on the 1 to 7 scale, and the U.K. at 3, and Pakistan rated as 1.67. Ranking wise, Australia only scores better than Serbia, the United Arab Emirates and Pakistan, while the U.K. score of 3 is the median for all countries. At first glance, these rankings do not seem reasonable in light of the qualitative data discussed above. The legislation in both Australia and the U.K. was deemed as comprehensive and strong. Keep in mind, however, that the survey question dealt expressly with external whistle-blowing, i.e. exposing abuses in the media, a praxis explicitly discouraged in the legislation of both countries. Whether abuses are more likely to be exposed in countries such as Jordan, Bangladesh, Uruguay and the Dominican Republic (the top four according to the expert survey) does not entirely obvious. For this reason and other factors discussed below, we regard this measure with a bit more skepticism than the other two measures.

6

82 percent of 1,762 civil servants surveyed had observed some form of wrongdoing, but only 8 percent of these had elected to report the problem (Bashir et al 2011, 290-291). Among those who had reported the wrongdoing, an overwhelming 84 percent had experienced some form of retaliation.

13

Table 1 provides descriptives of the three measures as well as a factor index that will be discussed in more detail below. The table reveals that the overall assessments on the three component measures differ considerably, with publicity, i.e. the likelihood of abuses to be exposed in the media, rated much more positively as a whole than either government openness or whistleblower protection. Considering that access to information but even more so whistleblower protection can both entail strong legal protections as well as in some countries a supportive organizational culture, this is perhaps not so surprising. Also noteworthy is that of the three measures, the government openness component shows the greatest variance among countries, which is of course a merit in measures intended for use in cross-country analyses. TABLE 1. DESCRIPTIVES OF THREE COMPONENTS OF TRANSPARENCY AND TRANSPARENCY INDEX

Mean

Std dev

Min

Max

Whistleblower Protection

3.07

0.82

1.00

5.00

Government Openness

3.60

1.16

1.25

6.06

Publicity

4.80

0.98

2.00

7.00

0

1

-2.63

1.92

Transparency Index

Available cross-country also provides a point of reference against which to gauge the validity of the expert survey assessments. Our three measures of the dimensions of transparency appear to varying extents to capture the same qualities as existing measures of transparency. Table 1 shows the bivariate associations between our three measures of dimensions of transparency, as well as factor index building on all three (discussed in next section) with six existing measures: 1) the fraction of statistics requested of states by the World Bank that in fact are reported (Hollyer, Rosendorff & Vreeland 2010), 2) the Open Budget Index, which rates the extent to which states make budget related documents available to the public in a timely and comprehensible fashion, and 3 and 4) two measures from Bellver and Kaufmann (2005) addressing economic and institutional transparency and political transparency, 5) a third measure from Bellver and Kaufmann combining these two, and 6) the findings of an access to information study conducted by the Associated Press. Each of these existing measures offers advantages and disadvantages, some of which have been discussed

14

earlier in this paper. As the Bellver and Kaufmann measures have the broadest country coverage, we revisit it briefly here. Economic and institutional transparency includes measures of “economic transparency, egovernment, access to information laws, transparency in the budget process, transparency of policy and transparency of the public sector” (Bellver and Kaufmann 2005, 20), while political transparency “includes elements such as transparency of political funding, openness of the political system and freedom of the press to monitor government’s performance and express the people’s voice” (p. 22). As discussed above, the Bellver and Kaufmann measures offer numerous advantages, foremost among which is the breadth of sources drawn upon, which increases the reliability of the measure but also introduces conceptual fuzziness. The economic and institutional transparency measure, the indicator most conceptually akin to the QoG survey measures, builds on 13 sources, eight of which capture not only the availability/ accessibility of information regarding policy and government activities, but also corruption, accountability structures, audit practices, participatory opportunities, the ability of citizens to carry out and pay for government services online, and the extent to which the populous has access to the internet. While all of these are related to transparency, they extend well beyond standard definitions of the concept, making it difficult to know if any observed associations with other factors teach us about the effects of transparency or whether they reflect a tautological association. The Open Budget Index also includes measures that span into the issues of participation, audit and accountability more generally. The Associated Press data, in contrast, captures a much more concise aspect of transparency, namely the extent to which states honor their own access to information laws. The investigation consisted of submitting requests to 105 governments regarding arrests and convictions related to terrorism. Governments’ reactions were then coded as responsive (coded numerically as 3), partially responsive useful (2), partially responsive not useful (1), unresponsive (0) or rejected (0). The description of the methodology7 leaves many questions unanswered, however, such as the extent to which requests were tailored to the specific circumstances of each country, and whether journalists followed up the requests in a similar way in all countries. Moreover, while extremely valuable for exploring de facto rather than de jure access to information, the measure builds on a single data point. More surveys of this kind would be highly welcome in the future.

7

Available at https://www.documentcloud.org/documents/267193-methodology.html

15

Examining the link between our measures and these existing measures can both increase confidence in our own measures while also shedding light on whether the existing measures concur with country experts’ assessments of government openness. Both access to information and publicity correlate fairly strongly with all the existing transparency measures (with the exception of the extent to which states submit the requested information to the World Bank, ‘Fraction of variables’, in Table 2). The correlation with each of Bellver and Kaufmann measures is around 0.6 and with their combined measure circa 0.7; with the publicity measure, the coefficients are somewhat lower. The whistleblower measure only correlates with the Hollyer et al measure, though this effect disappears if the United Arab Emirates is excluded.8 TABLE 2. BIVARIATE ASSOCIATIONS BETWEEN TRANSPARENCY MEASURES FROM QOG EXPERT SURVEY AND OTHER SOURCES (PEARSON’S R)

Government openness

Whistleblower protection

Publicity

Transparency index

Fraction of variables reported to WB

0.04

0.29*

0.12

0.17

Open Budget Index N=40

0.48*

-0.04

0.47*

0.51*

Economic & Institutional Transparency

0.59*

0.10

0.37*

0.56*

Political Transparency

0.63*

0.18

0.45*

0.65*

Overall Transparency

0.69*

0.16

0.46*

0.68*

Associated Press AtI Survey (N=37)

0.46*

0.04

0.06

0.33*

*p>.05, N=50 for all correlations with the exception for those with the Open Budget Index and the AP study Fraction of variables reported to WB: Percentage of 172 variables requested by the World Bank that are reported by a country (Hollyer, Rosendorff & Vreeland 2010, replication data available at www.jameshollyer.com). Economic & Institutional Transparency, Political Transparency, Overall Transparency: Kaufmann et al 2005 Open Budget Index N=40: Ninety-two questions inquire about the public availability, timeliness, and comprehensiveness of a country’s budget reports. Questions largely build on criteria outlined by “International Monetary Fund (IMF) in its Code of Good Practices on Fiscal Transparency, the Organization for Economic Cooperation and Development (OECD) in its Best

8

Little variation exists in the Hollyer et al measure among the countries included in these analyses, and the two-way scatter suggested that the United Arab Emirates was an outlier on both variables, therefore driving the results.

16

Practices for Budget Transparency, and the International Organization of Supreme Audit Institutions (INTOSAI) in its Lima Declaration of Guidelines on Auditing Precepts” (Open Budget Index 2010, 14-15).

Our government openness measure also correlates fairly strongly with the findings of the AP study on government responsiveness to information requests (r=0.46), which is stronger than the correlation between the AP measure and the three Bellver and Kaufmann transparency measures (not shown in the table). This finding lends credibility to the government openness measure, as the AP study very nicely captures a single instance of the more general phenomenon referred to in the Quality of Government survey question. The coefficients in Table 2 suggest that the QoG survey measures do tap into a similar phenomenon as the AP study, the Open Budget and World Bank’s transparency measures. That the association is not stronger may of course derive from measurement error in experts’ assessments, but may also arise from the fact that our measures tap into a somewhat different aspect of transparency, or the measurement noise in these other measures. A conclusion of the analyses thus far is that the whistleblower protection component has been difficult to verify in terms of available qualitative and quantitative data, and has considerably less variation among countries in the sample. This may be the product of a number of factors. The question itself requires experts to weigh in a number of factors: the risk of disclosure, which itself depends on the existence of channels and procedures via which to file complaints, but also the willingness of the media to protect the identity of sources, as well as an assessment of possible negative consequences that may range from extreme social stigmatization to loss of employment to physical safety. The mechanisms to discourage whistleblowing may to a greater extent be formed by organizational praxis than by legislative and administrative measures. A similar issue may exist with respect to the question of publicity. The question requires an assessment both of civil servants willingness to reveal abuses or conversely journalists likelihood to investigate abuses, as well as the likelihood that media channels will publicize any detected abuses. The fact that several factors weigh in to the question may account for the observed divergence in assessments. As a final step in shedding light on the validity of these measures, we examine their association with measures of basic rights and freedoms. In an attempt to explore these problems with an analysis of convergent validity – whether the measurements correlated with factors with which theory predicts – we consider the relationship to

17

measures of basic rights and freedoms. More specifically, we also hypothesize that whistleblower protections should correlate with free speech, and that the publicity component should be associated with a free press. Table 3 shows that the three dimensions of transparency examined here have strong and statistically significant associations with other measures of rights and liberties. Both Cingranelli and Richard's index of empowerment rights and Freedom House's measure of civil liberties are conglomerate measures that capture the overall extent to which basic rights are protected and de facto honored in a country, and these measures are strongly associated with publicity and in particular with our access to information measure. TABLE 3. CORRELATIONS BETWEEN TRANSPARENCY COMPONENTS AND OTHER BASIC RIGHTS (PEARSON’S R)

Empowerment Rights Index

FH Civil Liberties

FH Freedom of Expression and Belief

FH Press Freedom

Freedom of Speech

Access to Information

0.57***

0.66***

0.58***

0.67***

0.43***

Whistleblower protection

0.16

0.24

0.19

0.21

0.36***

Publicity

0.48***

0.43***

0.47***

0.49***

0.45***

N=52 *p