In this section, we provide statistical analysis from data collected in a survey conducted in three Chinese cities with a view to examining whether a cognitive basis complementary to a co-production approach to performance measurement exists in China, with reference to the four dimensions we have identified.
The survey was part of a research project that studied the effect of civil service reforms on the performance of education agencies and environmental agencies in three Chinese cities.2 The survey focused on charting the views of citizens and government officials regarding the following: (1) how they perceived the performance and measurement of local government agencies; (2) how they understood the criteria for agency performance; and (3) what they considered to be the major determinants of agency performance. We were well aware that, given China's very bureaucratic administrative structure, we might find that neither officials nor citizens would appreciate the importance of citizens' input in public service delivery. Our empirical analysis, however, enabled us to identify possible leverage points at which measures could be taken to build the necessary cognitive basis for co-production.
Given China's overtly hierarchical and bureaucratic administrative system, collecting data that can effectively gauge the cognitive basis of officials' and citizens' perceptions of administrative performance is a big challenge. Specifically, the data should (1) contain information about the views of both officials and citizens on agency performance; (2) be in-depth enough to capture the nuances of interviewees' perceptual understanding of performance; and (3) take into account the impact of the interaction between officials and citizens on their perception of agency performance.
The data that we used in the empirical analysis of this study satisfied all these requirements. We chose two important policy areas (environment and education) in the three Chinese cities, namely, Beijing, Ningbo, and Changchun. The total number of officials working in these six agencies was 388; we drew a random sample of 40 from among these officials for interview. It is warranted to point out that drawing a random sample from an exhaustive population of officials in government agencies is no easy task in the Chinese context. The sample of the 40 officials, in fact, covered a whole range of administrative ranks, which allowed us to gauge the impact of an official's position on their views on agency performance. More importantly, the six agencies chosen had relatively large clienteles. To identify citizens who had good knowledge of the agencies and also experiences of interacting with the officials in the agencies, we sought help from the officials to identify appropriate citizen-clients for interview. Among these citizen-clients were the following: in education policy, school principals, teachers, students, and parents; and, in environmental policy, factory managers, workers, and staff who are responsible for compliance with environmental protection regulations in their units.
Generally speaking, a majority of citizens in the survey commented positively on the work performed by the environmental protection agencies and the education agencies in their cities. More than 90 per cent reported that they were aware of the various activities conducted by the agencies in delivering their services. More importantly, these activities were considered by a majority of citizens to be able to address the needs of the public. In fact, more than 80 per cent of the citizens commented that there was much improvement in the performance of both the environmental protection agencies and the education agencies in their cities.
Although the citizens in general thought very highly of the performance of these agencies, most of them, however, had no idea about how performance was actually measured in these government agencies. Such a situation, of course, is not surprising. Nobody would expect an ordinary citizen to spend time and effort to find out how a government agency measures its performance just for the sake of knowing more about the measures. Yet, such a lack of knowledge about agencies' performance measures should not be interpreted as the citizens' lack of ability to understand performance measures. The survey data suggested that the more often the citizens interact with the agencies, the more likely they know about these agencies' performance measures.
In Tables 1a and 1b, we array the relationships between citizen's knowledge of agency performance measures and whether they have contacts with the agencies.3 The information suggested that citizens who had contact with agencies were more likely to know the measures that agencies used to measure their performance. Such a learning effect was particularly obvious in environmental protection. As shown in Table 2, among those who never contacted the environmental protection agency in their cities, none had any knowledge about how the agencies measured performance. Environmental protection involves much technical knowledge and may not have a direct and imminent effect on individual citizens' daily life. Ordinary citizens who had no contact with environmental protection agencies were unlikely to be able to even guess how the performance of environmental protection can be measured.
Table 1a. Relationship between citizens' knowledge of agency performance measures and officials' efforts to contact citizens
| ||Whether the government agency has contacted the citizen concerning their services|
|Whether the citizen knows about the agency's performance measures||No||Yes|
|Chi-square = 6.42; p = 0.01|| || |
Table 1b. Relationship between citizens' knowledge of agency performance measures and citizens' efforts to contact officials
| ||Whether the citizen has contacted the government agency concerning their services|
|Whether the citizen knows about the agency's performance measures||No||Yes|
|Chi-square = 4.37; p = 0.04|| || |
Table 2. Relationship between citizens' knowledge of agency performance measures and citizens' efforts to contact environmental protection officials
| ||Whether the citizen has contacted the Environmental Protection Agency concerning their services|
|Whether the citizen knows about the Environmental Protection Agency's performance measures||No||Yes|
|Chi-square = 2.86; p = 0.09|| || |
Although a majority of citizens did not know how performance was actually measured, they did have in mind the kinds of measures that should be used to assess agency performance. In the survey, citizens were asked how they would measure agency performance. More than 30 per cent suggested vague and impressionistic indicators, such as “making the environment cleaner” and “making life easier for people.” Such a pattern was consistent for the citizens of both the education agencies and the environmental protection agencies.
For those who gave answers other than impressionistic indicators, they tended to think that education agencies and environmental protection agencies should be assessed by different performance measures. In education, as shown in Table 3, more than 45 per cent of citizens considered citizen participation and citizen satisfaction to be essential measures for agency performance. In environmental protection (Table 3), on the other hand, only about 6 per cent of citizens considered citizen participation an important performance measure. Instead, most citizens (almost 60%) considered organizational objectives and technical indicators as the most appropriate measures for assessing agency performance. Such a difference in performance measures for education and environmental protection may be due to the different nature of the two policy domains. Environmental protection is considered to be technical in nature, and hence, has little to do with citizen participation and satisfaction. Education, on the other hand, is more human-oriented, and hence, citizens' involvement should be taken into account when agency performance is measured.
Table 3. Citizens' views on the appropriate agency performance measures for education agencies and environmental protection agencies
| ||Policy domains|
|What kinds of measures should be used for assessing agency performance?||Environmental protection||Education|
|Chi-square = 11.8; p = 0.02|| || |
When the citizens were asked what factors affect agency performance, almost 50 per cent ranked political support from the communist party and its leaders as the most important factor. On the other hand, about 35 per cent of citizens ranked public support as the least important factor for agency performance. Such a pattern was consistent for both education and environmental protection. Another important group of factors that were considered to be essential were organizational structure and inputs. More than 78 per cent of citizens ranked financial input to be one of the three most important factors contributing to agency performance, and 50 per cent ranked improved rules and regulations as one of the three most important factors. Interestingly, the civil service reforms were not given as much importance as expected. In fact, more than 60 per cent of the citizens ranked it to be one of the three least important factors for agency performance.
Policy domains had little effect on the ranking of these factors. The patterns of ranking were similar for both education and environmental protection, except for the factor of improved organizational management. Its ranking was much higher in the domain of education than that of environmental protection. Perhaps the citizens' ranking of the factors affecting agency performance reflected the way they perceive the nature and operation of these agencies. Government agencies were considered as the executive arm serving the purposes of political institutions by the interviewees. The support of the communist party and its leaders was still the most important determinant of agency performance. Despite all the talks about moving toward modern public management, managerial factors were still considered secondary. The citizens perceived themselves to have marginal impact on agency performance.
In the survey, a majority of officials reported that they were familiar with the objectives of their agencies. Consistent with how the citizens felt about the performance of these agencies, almost all of the officials surveyed perceived an improvement in the performance of their agencies. Although most citizens based their judgments about agency performance on some vague conceptions, the officials were able to identify the criteria, which they claimed improved their agency's performance. It is interesting that more than 82 per cent of officials cited criteria pertaining to organizational objectives, such as the missions given by superior agencies, and technical efficiency of service delivery, such as a particular air quality standards. Only 17 per cent of officials cited criteria pertaining to citizen participation or citizen satisfaction as the basis for their judgment.
The emphasis on organizational objectives was particularly eminent in environmental protection agencies. Table 4 arrays the types of measures that officials used to make judgments concerning agency performance for both the environmental protection agencies and the education agencies. Although only 50 per cent of education officials based their judgments on organizational objectives, more than 70 per cent of environmental protection officials used organizational objectives as their basis. Such a difference may have to do with the different nature of service.
Table 4. Agency performance measures used by citizens
| ||Policy domains|
|What kinds of measures do you use for assessing agency performance?||Environmental protection||Education|
|Chi-square = 8.41; p = 0.08|| || |
In the survey, the officials were asked what kinds of performance measures they would include or strengthen to better measure their agencies' performance. More than 72 per cent of them suggested measures that pertained to organizational objectives or the technical efficiency of service delivery. Only about 27 per cent suggested that citizen participation or citizen satisfaction be given more emphasis (Table 5). Such a pattern was consistent across the two policy domains of education and environmental protection.
Table 5. Officials' views on the appropriate agency performance measures for education agencies and environmental protection agencies
|What kinds of measures should be used for assessing agency performance?|| |
The officials' perceptions that citizen participation and citizen satisfaction were not important for performance measurement may be grounded upon the fact that citizens did not play a very dominant role in the process of defining the agencies' objectives. Although a majority of the officials indicated that their agencies took into account citizens' inputs when they formulated organizational objectives, the inputs were largely confined to “citizens' feedback” to the policies and operations of the agencies. More importantly, almost all the officials reported that the citizens did not play any formal role in the process of performance assessment in their organizations, although some of them argued that citizens' feedback was considered when the assessment was conducted.
Despite citizens' limited role in performance assessment, the officials had a clear idea about who their citizens were. In Table 6, we array the types of citizens as perceived by the officials for both the education agencies and the environmental protection agencies. An interesting pattern was that education officials tended to perceive their citizens as individuals (such as teachers and parents) or organizational units (such as schools). The environmental protection officials, on the other hand, tended to see their citizens as organizational units (such as factories, enterprises, or the general public). In other words, education officials, because of the human service nature of their service, tended to focus on individuals, whereas environmental protection officials tended to focus on the broader context.
Table 6. Types of citizens as perceived by officials for education agencies and environmental protection agencies
| ||Policy domains|
|Who are the citizens?||Environmental protection||Education|
|Individuals (e.g., teachers)||0||4|
|Individuals and organizations (e.g., schools)||2||8|
|Organizations and the general public||4||0|
|The general public||1||0|
|Chi-square = 12.35; p = 0.01|| || |
The data in the survey also suggested that performance measurement and assessment was not performed in a very systematic manner in the agencies. A majority of the officials surveyed indicated that, in their agencies, there was no specialized unit or individual specifically assigned the task of measuring or assessing agency performance. In almost all cases, no financial resources were specifically allocated for the task of performance measurement. All of the officials reported that the entire process of performance assessment in their organizations was purely internal and did not involve any other organizations. External monitoring of the process just did not exist. In other words, performance measurement and assessment was often seen as a routine exercise to report and review the work of government agencies. More importantly, a majority of the officials perceived that the performance assessment exercise did not have any impact on policy formulation and planning in their agencies.
The officials were also asked to rank the factors that affected agency performance. More than 60 per cent ranked political support by the communist party and its leaders to be one of the three most important factors for high agency performance. Another important factor for high performance, as perceived by officials, was rules and regulations. Given that the officials carried out their tasks and exercised their authority largely through a framework of rules, improved rules and regulations could provide a conducive and facilitative working environment for the officials. Not surprisingly, most officials ranked the support of the public to be the least important factor for agency performance. In a setting where government agencies were perceived as the service providers, and citizens as the service recipients, citizens could play at best a passive role in influencing and assessing agency performance.
The impact of the civil service reforms on agency performance, as perceived by the officials, was not as important as expected. Less than 10 per cent of the officials ranked it as the most important factor, and less than 38 per cent ranked it as one of the three most important factors affecting agency performance. Such a pattern, perhaps, indicates that the civil service reforms are still in a preliminary stage in China. Given that political support from the communist party is still considered the most important factor affecting agency performance, the technical factors (such as the civil service system) are not likely to be assigned too much importance.
We asked those who ranked the civil service reforms as one of the three most important factors affecting agency performance to rank the components of the reforms in terms of their importance in enhancing agency performance. In Table 7, we array the percentages of officials who ranked the particular component to be one of the two most important factors enhancing agency performance. Sixty-four per cent of the officials ranked examination in the recruitment process to be either the most or the second-most important component affecting performance. Examination was said to help improve the quality of recruits. Competition brought by the civil service reforms was also considered to be a very important factor facilitating performance. Interestingly, the reward and punishment mechanism brought about by the civil service reforms was considered the least relevant factor to agency performance. This is consistent with the fact that performance assessment is seldom used to reward or punish individual officials in the Chinese context.
Table 7. Officials' perceived importance of components of Civil Service Reform
|Components of civil service reform||Percentage of officials who rank the component to be one of the two most important factors for agency performance (%)|
|Rewards and punishment||10|
|Competition at work||60|
A summary of the findings
Both citizens and officials agreed that there were improvements in the performance of the environmental protection agencies and the education agencies in the three cities. Yet, they seemed to base their judgment on different criteria. Citizens in general assessed the performance of public agencies by intuition, not with reference to concrete performance measures. In fact, citizens did not care to think about performance measures, as these were often considered to be remote or sophisticated topics beyond comprehension. More contact between citizens and agencies, however, is likely to help the citizens better understand performance measurement of public agencies.
In general, citizens tended to think of performance measures as some very broad conception of what things should be like, that is, some desirable phenomena, and were unable to pin down what they meant by “desirable”. On the other hand, officials tended to focus on organizational objectives and technical measures to assess agency performance. An interesting commonality between the officials' and the citizens' views was that neither considered citizen participation and citizen satisfaction to be important measures for assessing agency performance. This may suggest that the officials and the citizens still considered their relationship as one of service provider–service recipient. The idea that public agencies are serving the needs of citizens as citizens was still a remote concept.
The nature of service made some difference in citizens' perception of agency performance. In particular, the citizens' role in performance measurement was given different degrees of attention in education than in environmental protection. Although citizen participation and satisfaction were considered not essential in assessing environmental protection agencies, they were considered to be more important in performance assessment for education agencies.
Concerning factors that affected agency performance, the views of the officials and the citizens were very similar. Both of them ranked the political support of the communist party and its leadership to be the most important determinant of agency performance. Managerial factors such as better financial inputs and improved rules and regulations were also considered to be important. The civil service reforms, however, were perceived to be only marginally relevant to agency performance. Among those officials who ranked the civil service reforms to be one of the three most important factors affecting agency performance, the improvement in the quality of staff as a result of recruitment examinations was considered to be the most important impact of the civil service reforms. It is interesting to note that both the officials and the citizens considered the support of the public to be the least important factor affecting agency performance.