Customer priorities for water and wastewater services: a comparative evaluation of three elicitation methods

Water service providers are being urged to incorporate customer preferences into their investment plans with the relative merits of different elicitation techniques being exposed to greater scrutiny. Though elicitation can be undertaken with a range of methods, there is little understanding of their comparative performance in terms of being able to generate consistent or commensurable outcomes. This study reports an evaluation of both intra and inter method consistency for three preference elicitation methods. Spearman’s rank correlation coefficient is used to measure consistency within and between elicitation methods and session transcripts provide additional evidence to support interpretation of the ranking process. Findings exposed low intramethod variation but significant variation in some intermethod comparisons. Discussion focuses on the internal dynamics of each method with conclusions calling for a wider range of methods to be studied so as to improve practitioner confidence in the use of these tools.


Introduction
Wider public involvement with commercial and public sector investment programmes is increasingly coming to characterise planning and decision-making processes. Relatively rapid adoption of such practices has taken place across a variety of geographical, sectoral and disciplinary settings. Sectors that have been particularly active include: health policy (Conklin et al., 2015); public service planning and delivery (Curry, 2012); technology development (Bogner, 2012;Ryghaug et al., 2018); environmental management (Saengsupavanich et al., 2012); community planning (Kiisel, 2013) and finally, natural resources management (Carr et al., 2012;Baker and Chapin, 2018). These often localised contributions have driven a debate on wider deliberative democracy ideals, which focus on the democratisation of global environmental policy (Berg and Lidskog, 2018).
The impressive traction secured by such participative processes has been attributed to a range of avowed benefits that Fiorino (1990) classifies as either normative, substantive or instrumental in nature. Normative benefits relate to the potential for participative processes to enhance both the procedural and distributive justice of decision-making. It has been argued that the provision of a forum for the expression of views from those affected by or interested in the outcome of a decision-making or planning process enhances democracy and promotes an empowered and active citizenship (Reed, 2008). Society's pursuit of increasingly complex objectives such as sustainable development has been argued to necessitate the incorporation of multiple perspectives, which traditional forms of governance are poorly equipped to identify, effectively characterise and incorporate (Funtowicz et al., 1999;Green, 2007;Stagl, 2007). This argument forms the basis of substantive claims for the benefits of participation. It is posited that the quality and creativity of solutions and interventions are improved by entraining citizens in the decision making process and thereby enhancing the breadth and depth of the evidence base available (Wilson and Howarth, 2002;Lach et al., 2005;Tuler and Webler, 2010). Instrumental benefits are founded on the potential for public participation to counter a reported decline in public trust and confidence in both the professional classes ('experts') and planning and decision-making processes; a result of the public's greater access to knowledge and information (Rowe and Frewer, 2004;Petts, 2005;Cass, 2006;Kuyper, 2018). Building capacity with those who have the potential to impede progress is argued to act to reduce opposition and foster a mutual understanding of, and empathy with, issues that may be barriers facilitating a smoother implementation of policy outcomes (Dean, 2017).
Recognition of the importance of public participation in the design and implementation of complex policy challenges such as those which typify natural resources management is reflected in national and international legislation where polycentric approaches to decision-making are increasingly mandated. With roots which go back at least as far as 1960s USA forestry planning and management, progressive social groupings have successfully argued for more meaningful and broader representation of public or community values in natural resource decision making. The US Clean Air Act (1972) and the World Commission on Environment and Development report 'Our Common Future' (1987) provided early political statements around this emerging consensus with the Rio Declaration on the Environment (UNCED, 1992) and Agenda 21 (UNCED, 1992) providing additional legislative weight. The principles set down in these statements have been incorporated into key water management legislation including the European Water Framework Directive (European Union, 2000) and the US Federal Clean Water Act (US EPA, 1972). Indeed, Wesselink et al. (2012) have argued that an additional benefit of public participation is its ability to capture information concerning the views and preferences of the public and stakeholders in order to satisfy formal legislative or regulatory requirements.
Polling customer opinions and preferences for services is an increasingly significant component of the water utility landscape as well. Prompted by both regulatory incentives which are designed to ensure better alignment of utility investment decisions with customer concerns, and a broader desire to engage with customers, utilities are using a range of techniques to poll consumer preferences for services, particularly where budgets are constrained. The Water Services Regulation Authority for England and Wales (Ofwat) see public consultation as delivering both outcomes that customers and society value at a price they are willing to pay (a choice driver) and improved public trust in water company operations (a credibility driver) (Ofwat, 2016). The types of choice offered to customers will depend on local circumstances and challenges but might, for example, involve a preference for investments which mitigate flood risk over those which improve water quality, or a desire for supply security over lower bills. Preferences are also increasingly being sought for specific projects where design and operation options remain to be determined or implemented and where a wider constituency of non-state actors can enrich the evidence base for and legitimise interventions (Fritsch, 2017). In this context, customer opinions and preferences are shaping both technology selection and asset investment strategies, influencing plant operation, and driving research and development. Despite this growing influence, reported experiences of customer preference elicitation tend to remain in-house with few published accounts for professionals to draw on for guidance. Important exceptions to this trend include contributions from Thacher et al. (2010), the US Water Research Foundation (2011), and, in the context of coastal flooding, Evans et al. (2017). As will be discussed below, new regulatory drivers which motivate better understanding of customer priorities have recently come to the fore in the England & Wales water sector, thereby providing a policy setting for the study reported here. However, we first move on to assess previous experiences with both the methods used to elicit citizen contributions and the substantive outcomes of those processes.

Eliciting customer priorities for water services; methods and outcomes
Even the most cursory browse through the academic literature reveals an often confusing array of engagement methods ranging from what might be termed traditional forms such as surveys and focus groups to more innovative approaches such as consensus conferences (Phadke et al., 2015) and visioning (Flower et al., 2017). The use of such a copious number of methods might be put down to a number of reasons (Rowe and Frewer, 2005). First, variety may exist due to uncertainty as to how best to enact participation. Second, the lack of clear definitions as to what participation is, widens the scope as to what may be considered as a tool to enact participation. Third, the wide definition of participation spanning a range of aims, applied in many different contexts, and a variety of types of participants, may mean that mechanisms are required to maximise effectiveness. Finally, a lack of functional methodological equivalence may prompt the adoption and development of bespoke approaches for dissimilar contexts and objectives. Here, we are concerned with a particular but common element of participative processes; the elicitation of views, opinions, choices etc. which might then be subsequently used to inform the selection or prioritisation of investments or interventions. Table 1 offers a summary of the major forms of elicitation together with representative sources for each form.  Hajkowicz and Collins (2007) This plethora of elicitation methods prompts some important queries: Is it important to select the right method? How do we know what the right method is? And does the form of elicitation method influence the outcome of the process in any way? Whilst there has been some limited commentary around the challenge of selecting appropriate elicitation methods (see e.g. Lawrence and Deagen, 2001) and a small number of studies, which examine the comparative performance of two or three elicitation options within a narrow performance lens (e.g. Jankowski et al., 2017) there is increasing concern about the appropriate selection of opinion or choice elicitation vehicles, either in isolation or in clusters (Bherer and Breux, 2012). The study reported below explores some of these concerns in the context of eliciting domestic water customer priorities for investment.
Though surveys and research to identify the priorities of customers for domestic water and wastewater services is a common practice amongst water utilities in many countries, disappointingly little finds its way into either the academic or practitioner literature. Work that has seen the light of public examination has often been conducted in the context of setting service standards (e.g. Speers et al., 2003) and has typically disaggregated water and wastewater services into distinct attributes (Randhir and Shriver, 2009). Frequently studied potable water attributes include; security of supply, interruptions to supply, drinking water quality (safety), drinking water quality (aesthetics), water pressure and leakage, whilst those for wastewater include; internal sewer flooding, external sewer flooding, nuisance from wastewater treatment works and pollution incidents (Willis et al., 2005;Consumer Council for Water, 2008). Across these two functionally different areas, a common finding is that customers do not typically possess highly differentiated priorities about the delivery of water services, on the one hand and wastewater services, on the other. It is acknowledged that consumers find it difficult to articulate priorities or even may not have considered their preferences for a service that they take for granted or rarely consider (Kelay et al., 2008).
International studies into the delivery of water services (the vast majority of which are set in the context of centralised systems in medium or high income countries) have shown that in general, participants prioritised the quality and safety of drinking water over the reliability of the supply (Kelay et al., 2008;Chenoweth et al., 2010). Chenoweth et al. (2010, p. 4339) further argue that this may reflect the relative stability of the water services in the study countries and hypothesise that 'When supply is unreliable, reliability takes precedence. Once the water supply is reliable, quality issues come to the fore as the priority of water customers'. Alternatively, some argue that this result reflects the public health factors inherent in discussions around water quality, which are less explicit in other attributes of water services (Kelay et al., 2008). It is also pertinent to consider the impact of individual customer experiences in the development of priorities for water services. Customers are strongly influenced by their sensory perceptions and may use these as the basis for assessing the perceived safety or quality of their drinking water (Falahee and MacRae, 1995) (Skellett, 1995). Conversely, it is possible that customers may only generate highly differentiated preferences for other elements of service based on their own experiences or, in some cases, those of others around them. For example, there is some evidence that customers who have experienced problems with their drinking water are more likely to seek alternatives to tap water as a way of minimising their concerns regarding potential health risks (Harding and Anadu, 2000). In comparison, the relatively sparse evidence base in the literature on customer preferences for wastewater services only highlights high levels of concern over sewer flooding incidents (Arthur et al., 2009).
The evidence described above, which is corroborated in the UK context by a 2008 study by the Consumer Council for Water, suggests that customers have relatively undifferentiated priorities for investment in water and wastewater services. Such a lack of differentiation provides significant challenge where regulation is seeking to privilege citizen opinions and priorities in utility investment decisions. So, for those professionals charged with eliciting customer preferences and mapping them on to water services provision, the need to both have confidence in the selection of priority elicitation methods and clearly discriminate different priorities and the value attached to them has never been more urgent. Whilst there is a burgeoning literature base of research seeking to identify the effectiveness of different participative and preference elicitation methods and the influence of context and participants on successful engagement (Tuler and Webler, 2010), there is relatively little which has explored the influence of varying the elicitation methodology on the outcomes of the process. Below, we report a comparative evaluation of three elicitation methods used to explore customer priorities for a range of water, wastewater and environmental attributes of their water and wastewater service. By analysing both intramethod and intermethod variation in priorities, we explore the consistency of outcomes as a function of elicitation method choice.

Methodology
In England and Wales, water utilities operate as regional monopolies and as such are subject to economic regulation by the Office for Water Services (Ofwat). They are also subject to quality regulation ensuring that they are investing to maintain and improve compliance with targets outlined in legislation such as the Bathing Waters Directive (European Commission, 2006), the Drinking Water Directive (European Commission, 1998) and the Urban Wastewater Directive (European Commission, 1991). In addition to these non-discretionary areas of investment, water utilities can choose to invest to improve levels of service beyond minimum standards if customer support can be demonstrated. On a quinquennial basis, the water services providers in England and Wales submit their investment plans to the regulator (Ofwat) for review and approval. For the 2014, Price Review Ofwat introduced an outcomes-based regulatory principle which replaced the requirement for economic cases for proposed investment to reflect customers' willingness to pay with a requirement for water utilities to generate a robust and well evidenced package of proposed performance outcomes, measures of success and outcome delivery incentives that reflect customers' views and priorities (Ofwat, 2011). The adoption of this regime, however, poses water utilities with the specific challenge of embedding customer priorities and preferences in asset planning and management practices. Success, will, in part, be dependent on the ability of water utilities to develop competencies in the design and deployment of a wider range of elicitation methods.
A review of business plans submitted for the Price Review in 2008/9 by water utilities in England and Wales (Ofwat, 2009) identified that customer preferences and priorities relating to water and wastewater services were typically elicited utilising economic valuation methods such as stated preference surveys with accompanying focus groups. These elicitation methods have been favoured by water utilities, driven in part, by a regulatory requirement to conduct cost-benefit analysis as part of the development of their investment programme. However, as indicated earlier, the emerging regulatory expectation is for water utilities to embed customer preference and priorities in asset planning and management practices thereby prompting utilities and their partners to develop their understanding of a broader range of elicitation methods. A lack of guidance regarding the tools for engaging with customers has left it open to water utilities to choose what types of elicitation method they employ.
The preference elicitation activity which provides the raw data for inter and intra method comparison was carried out by a UK water utility early in its preparations for the submission of their business plan under 2014 Price Review. It was intended to provide an initial indication of the relative priorities of domestic customers for water and wastewater services to support early policy and strategy development. Three qualitative participative approaches, each conducted in a group setting, were used to elicit customer priorities. A group setting provided the most suitable arrangement in terms of efficient use of resources and meeting sample quotas, and also offered the most flexibility in trialling different elicitation techniques. Each of three different elicitation methods were trialled three times each giving a total of nine distinct sessions. The trialled methods were; (i) individual prioritisation, (ii) group prioritisation and (iii) group budgeting (see Table 1).
Each of the nine sessions involved a common prioritisation challenge, which generated rankings of nine attributes of water services provision in order of investment precedence. Water and wastewater services were disaggregated into five water and four wastewater attributes (Table 2) with each attribute representing a significant area of customer concern as identified in previous consultation exercises. Two pilot activities were carried out in December 2011 to trial the proposed content of the sessions with modifications and improvements being made to the structure and content of the sessions due to these experiences.
Each of the nine sessions were preceded by a number of introductory activities. The function of these was to familiarise participants with the attributes of their water and wastewater service prior to undertaking the prioritisation activity. During an introductory prioritisation activity, participants were required to complete a table whereby they rated the attributes of their water and wastewater services as high, medium or low, providing a baseline set of information with regards to customer priorities. This introductory exercise was followed by discussion about each attribute and the motivations for participants' ratings. Finally, one of the three prioritisation methods were used to revise the earlier stated preferences.
In the individual prioritisation activity, participants completed a table rating attributes (listed in Table 2) as either high, medium or low importance to them. This was a repeat of the introductory prioritisation activity conducted earlier Wastewater in the session. The group prioritisation activity utilised an aggregated view of individual responses to the introductory prioritisation activity in the form of a ranked list. Time was provided for participants to exchange views on the ranked list in its existing state and they were then provided with the opportunity to make amendments to the ranked list of service attributes based on the outcomes of their discussion, outlining their rationales for any changes. For the budgeting activity, participants were shown a table that provided information showing; (i) the current level of service received by customers for the service attributes (as listed in Table 2), (ii) the level of service that could feasibly be achieved in 2020 with investment by the water utility and (iii) the associated annual bill impact for the delivery of the improvements in level of service to each customer. These participants were given a nominal budget (the totalled bill impact of all improvements) that they could use to purchase service improvements. Participants were prompted to discuss each attribute and then vote to determine whether to 'purchase' an improved level of service. Participant selection for the study involved a purposive sampling approach (privileging relative significance of group membership over simple statistical representation) with participants being drawn from a customer database, thereby avoiding the quasi-professionalism, which characterises those who serve on permanent consultation groups. Criteria for candidate selection reflected the desire to have a representative sample across the following parameters including: socio-economic group; customers in receipt of means-tested benefits; age (20-44 and 45+); household setting (urban, rural and coastal) and whether the household had a metered or un-metered supply.
Actual locations for the participative sessions represented a balance of urban, rural and coastal settings within the supply region, where good relationships had been built with recruiters thus offering an efficient method of ensuring good participation at each event. A selection questionnaire was developed for use by local face-to-face interviewers with the aim of achieving eight participants per group that corresponded to the sample quota. Participants were recruited to take part in a group discussion lasting approximately one hour and forty-five minutes. Despite the development of an initial sample design, which offered a balance of socio-economic groups, age, group and locations for each trialled technique, issues with the scheduling of venues, facilitators and recruiting, meant that amendments were made to accommodate necessary changes. A total of 65 domestic water customers participated in the study across the nine sessions (see Table 3). The sessions themselves were conducted between the 4th April 2012 and 1st May 2012 in six locations in the North of England.
A ranked list of attributes was generated from the outputs of each elicitation activity to enable the variation in priorities to be explored. Variation in priorities are assessed in groups that were exposed to the same elicitation method (Intramethod variation) and also across groups that were exposed to different methods (Intergroup variation). In order to conduct this analysis, the outputs from each elicitation method were subject to some data transformation (see Table 4).
In addition to undertaking a descriptive analysis of the data from the sessions, Spearman's rank analysis (often referred to as Spearman's rho) was employed to explore both the intramethod and intermethod variation in preferences across all data sets. This statistical test assumes a monotonic relationship between variables and is based on deviations whereas Kendall's Tau measure (a viable alternative in this context) is based on concordant and discordant pairs. We would note that our sample size is modest (n = 65 across nine groups) and results may therefore be sensitive to outlier values; a tendency that would be exaggerated if using alternatives to Spearman's rho. Transcripts were generated of all the session discussions and debates and content analysis of these transcripts provided further insights into customer priorities and enabled the extraction of quotes to support analysis where relevant.

Results
Table 5 sets out the findings of the Spearman's rank correlation coefficient (Pearson correlation coefficient between the ranked variables) analysis for both the Intramethod and intermethod cases.

Intramethod variation
The analysis of intramethod variation clearly suggests that priorities elicited by groups using the same method exhibit a low level of variation. The relationship was observed most strongly in the group prioritisation exercise (ρ = 0.87). Whilst, arguably still a strong relationship, the individual prioritisation and budgeting activities exhibited a higher level of variation with a ρ value of 0.77 and 0.78, respectively. This analysis specifically identifies variation in preferences generated using the same method. The participants were exposed to the same information and relatively similar experiences. The main variables within this sample, therefore, relate to the participant characteristics such as age, socio-economic group, whether they are a metered or unmetered customer or, perhaps most importantly, their experiences in terms of their water and wastewater services. The overarching finding from this analysis suggests that participant characteristics are not a significant driver for preference variation. Participants attending these groups generally reported no significant existing problems with their water service experience. A data set was generated for each group. This involved counting the number of High, Medium and Low ratings for each attribute across individual participant responses. Scores of three, two and one were assigned to a rating of high, medium and low, respectively. Each attribute was thereby assigned a total score. The attributes were then sorted to provide a ranked list based on the total rating score The method as set out for intramethod analysis was applied to individual responses collated across all three groups Group prioritisation No data transformation was required to the outputs of this activity. The product of each group was a ranked list of service attributes. To collate these three data sets to form one ranked list representing the views across all three groups, scores of 9 to 1 were assigned to each rank position from highest to lowest and multiplied by the number of times the attribute was ranked at each position. The attributes were then sorted to provide a ranked list based on the total rating score Group budgeting In order to convert the outputs from the budgeting activity into a ranked list a number of assumptions were applied: (a) Those items that were 'purchased' were considered to be highly important to participants (assigned a high rating) (b) Those items that were not purchased were considered to be of low importance to participants (assigned a low rating) (c) Cheaper options were observed to be more frequently purchased in the budgeting exercise indicating a preference for these options over those that had a greater potential bill impact and therefore the attributes were sorted from lowest to highest cost within the high and low classifications in order to generate a ranked list. This was justified on the basis that cheaper options were more frequently purchased indicating a preference for those improvements that had a greater potential bill impact. This process was undertaken on the outputs of each budgeting activity thereby producing a ranked list per group The number of High and Low ratings for each attribute were summed across all three group outputs, as established by applying assumptions a) and b) as for intramethod analysis. Scores of three and one were assigned to a rating of high and low, respectively. The attributes were then sorted to provide a ranked list based on the total rating score As Table 5 shows, the priorities elicited from the individual prioritisation activity and the budgeting activity do suggest a slightly higher level of variation than that observed in the group prioritisation exercise. This is, perhaps, not unexpected. In both the individual prioritisation activity and the budgeting activity, achieving consensus was not an inherent part of the elicitation methodology, unlike in the group prioritisation exercise. Participants were, therefore, able to express individual opinions, either through a survey instrument such as in the individual prioritisation activity, or by using a voting mechanism as in the budgeting activity. Stable outputs (in the form of an agreed way forward) are a (implicit or explicit) target function of collective decision making. It may also indicate that in the group prioritisation activity people are less at ease to express their true opinions; where strong opinions are pacified if they are radically different from the social norm, and thus, an increasingly stable set of outputs is generated.

Intermethod variation
In the analysis of intermethod variation a more mixed picture emerged. Outputs compared across the individual and group prioritisation activities exhibited a low level of variation (ρ = 0.82). When the ranked lists, generated based on data from the individual prioritisation and group prioritisation, were compared to the ranked list generated from the budgeting exercise, high levels of variation were exhibited with ρ values of 0.02 and 0.03, respectively.
The low variation observed when the results from the individual and group prioritisation activities were compared may be explained by an observation picked up via one of the group prioritisation session transcripts. This suggested that there was a reluctance to engage in discussion about the water and wastewater service attributes. The session gave the opportunity for participants to engage in modifying the ranked attribute list but only one group chose to do this and, when prompted, struggled to defend their proposals. This therefore meant that in two out of the three groups an aggregated view of individual priorities captured in the introductory prioritisation activity represented the outputs of the group prioritisation activity. Participants were recorded to have remarked, 'To me the way that has come out is pretty good. I don't think there is much to argue with on that -unless you try and force it in some way'. Whilst such comments suggest that participants were content with the ranked list presented to them as an aggregation of individual views expressed in the introductory prioritisation exercise, it could also be argued that it may reflect a lack of engagement in the activity (a possibility that should also be considered when interpreting the intramethod variations as well).
The high level of variation observed when the results of the budgeting exercise were compared to the outputs from the individual and group prioritisation exercises was very pronounced. The budgeting exercise differed from the individual and group prioritisation exercises by introducing additional data into the exercise including: current levels of service received by customers for each attribute, potential achievable improvements to level of service and also the cost of that potential improvement. The assumption was made that an attribute that was important to a participant meant that they were more likely to be willing to pay for it to be improved and vice versa, where an attribute was not important to participants they were less likely to be willing to pay for it to be improved. The application of this assumption to the data may be the driving factor behind the observed differences. When the raw data from each group is analysed it can be seen that choices for purchase were driven by the extent of potential bill impact. Those attributes that represented a bill impact of £1 were more likely to be chosen than those that were valued over £1 in value. Out of the twelve purchased improvements across all groups, 10 were valued at below a £1 in value. They need to generate relative prioritisations, and therefore, the need for a ranked list conceals the diversity of the total bill impacts. Across the groups this ranged from £11.18 in total to £0.60 and the number of items purchased from six service improvements to one. Table 6 illustrates this.

Discussion
We would note that this paper is, to our knowledge, the first to offer an explicit comparative analysis of the consistency of the outcomes of different preference elicitation methods within a water services context. Though previous work on techniques such as contingent valuation are to some extent pertinent to our agenda, even here there have been few attempts to explore the congruence of outcomes generated by methodological variations. However, whilst we have little previous evidence and knowledge to compare our findings with, we can make some observations about both the implications of the work and its reliability. It is in one sense encouraging that the three tested methods exhibit such internal consistency in reflecting preferences. Careful design and execution of elicitation sessions are, we argue, more important determinants of reliability than choice of method. Nurturing a relationship with customers over weeks rather than days pays strong dividends in their understanding of the participation process and their willingness to reciprocate the commitment to ensuring a productive outcome. On the contrary, it is striking that the sole precursor of significant variation across the three tested methods is the introduction of budgeting as a means to indicate preferences. Interestingly, the introduction of a monetary element to the calculation appears to influence both individual and group rankings. We have speculated that this might be attributable to participant sensitivity to bill impact, highlighting the difficulties that water utilities may face when discussing service level improvements across a range of attributes and a range of bill impacts. The provision of information with regards to how customer bills are set and the process by which this is regulated to ensure customer interests are protected may provide some assurances to customers and help to alleviate any suspicion regarding bill increases. All this suggests that re-framing these kinds of activity in order to elicit a more robust set of priorities might be possible. Such a methodological modification would replace the practice of providing participants with the opportunity to purchase improvements from a predetermined total budget with an approach where participants would be informed that their bill would be subject to a specific monetary increase and they would be asked to determine what proportions of their bill they would want to spend across each attribute. This could be achieved by providing participants with tokens each representing a nominal percentage of the total budget thereby removing the bias towards cost-based decisions, providing an improved reflection of participants' true priorities and more comparable data to accompany other prioritisation-based activities. The authors are aware of recent initiatives by several water utilities to adopt a similar token-based approach to preference allocation but look forward to future scientific studies of their comparative performance and effect.
Another thought-provoking result from this study is that participant socio-demographic characteristics do not appear to drive the observed variation in priorities. However, the results did indicate that slightly more intramethod variation was likely where participants were not required to form consensus as in the group prioritisation activity. As previously stated, this may be expected as individual voices may be more diluted than in those methods where individuals are given the opportunity to express individual opinions privately. It also highlights that in the design of customer engagement approaches consideration of the aim of the session is important in relation to whether a convergence or divergence of opinion is to be achieved. Elicitation methods that incorporate voting or an individual rating exercise are more likely to provide a spread of views across the sample. Group derived outputs where consensus has been reached is conversely more likely to exhibit less variation. Where group decision-making processes have been utilised it may be beneficial to capture information on the participant experience by providing the opportunity to complete an exit survey. This may help to reveal how easily the participants felt that they could contribute their feelings and views and will provide some assurance as to the validity of the data collected.
The low variation exhibited in comparing the individual prioritisation and group prioritisation outputs was postulated to be a feature of the participant's limited appetite to engage in the activity and the difficulty that was observed in defending their choices. Stein et al. (2006), reporting a study exploring the impact of discussion on preference formation, suggest that discussion does not have a significant impact on preferences but is highly valued by participants, providing benefits such as reassurance, improved procedural performance, improved group cohesion and satisfying curiosity. There is of course a rich and literature of long standing on the variation between individual and group choice practices and outcomes, much of which attempts to explain how the transition from a collection of varied individual preferences to coalescence around a single option occurs. For example, Laughlin (2011) surveys a number of possible explanations informed by understandings about group dynamics and communication. However, more recent commentary has pointed out that we still lack a detailed understanding of how group discussion facilitates consensus building and cooperative behaviour with three competing causal phenomena hypothesised; (i) 'group identity' where discussion creates a common bond among members thereby increasing concern for group-level rather than individual-level outcomes, (ii) 'perceived consensus' where discussion facilitates cooperation by offering group members the opportunity to be part of a majority-based group commitment, and (iii) 'group problem-solving' which argues that only a minority of active advocates is needed to deliver a cooperative decision through discussion and debate (see Meleady et al., 2013). We are unable to comment further on the mechanisms by which a greater degree of variation was observed in the individual prioritisation exercises than in the collective ones but would note that all three of the above mentioned hypotheses could equally well apply to the cases we have studied.
In future activities to explore customer priorities, it may be beneficial to ensure a wide variety of information resources to stimulate discussion. A further methodological improvement may be that a multi-session approach be employed whereby complexity is gradually introduced to the participants with time allowed between sessions for participants to reflect and deliberate on the material provided. Similar staged techniques have been used in participative approaches that incorporate deliberative, visioning or multi-criteria analysis techniques (Kallis et al., 2006;Consumer Council for Water, 2008). It is thought that this promotes more engaged participants and nurtures responses that are more considered. It is also suggested that, where a discussion-based approach is utilised, the subject being discussed is relatively specific in nature. In the case of this study, the subject being discussed was broad covering both water and wastewater services; as subject where it has been demonstrated customers hold relatively undifferentiated priorities.
There are several features of the study which warrant some comment in terms of possible constraints on its veracity and generalisability. First, whilst the results have suggested that participant characteristics did not play a major role in influencing the outputs of each session, issues with recruitment prevented accurate replication of participant characteristics across groups using different methods. Because group membership was unique for each session, it might be argued that there is no a-priori reason why the outcomes should be commensurate either within or between the three tested methods. However, several features of the study (constrained attribute set, common participant experience of service levels, single service supplier etc.) offer important reasons why similarity of priorities might well be expected. Second, the prioritisation activities that were considered in this study were time constrained and this limited the capture of richer data in relation to how participants were forming and reasoning their prioritisations. Spending greater time on these activities would nurture a more insightful comparison. Finally, due to the nature and format of the outputs from the various activities, some data transformation (as discussed above) has been implemented in order to generate comparable metrics for the three tested methods.

Conclusions
(1) Incorporating public opinion and preferences in investment decision making is both an indicator of a mature democracy and an important feature of consensual service provision. Regulatory obligations and incentives to embrace customer aspirations (and importantly to evidence robust and convincing processes for doing so) provides a non-trivial challenge to both public and private enterprises to become more competent at deploying and integrating customer views into their investment plans. The wide variety of tools and techniques used to elicit and record such preferences raise specific difficulties for practitioners around the tools' fitness for purpose and ability to generate commensurable outcomes. Our study also highlights the challenge of ensuring that those participating in preference prioritisation activities are able to provide meaningful contributions and are committed to offering their best endeavours during the activity. This raises something of a paradox in that, on the one hand, meaningful contributions might be best prompted by improving participants' awareness, understanding, empathy, and debating skills, whilst on the other, such interventions run the risk of transforming participants into atypical customers. The limited appetite to engage that we argue may have influenced some of our observations is, in all likelihood, the result of a genuinely held perspective, which views water services as a minor concern compared with more pressing personal, family, and work priorities.
(2) Our findings suggest that each of the three tested methods (group prioritisation, individual prioritisation, and group budgeting) generate broadly internally consistent outcomes. When considering outcomes across the three methods, however, those from the group budgeting activity are inconsistent with those from the other two methods. These findings provide insight into the design and deployment of customer engagement activities in the context of exposing priorities for water and wastewater services. They will also be of interest to those undertaking public participation and preference elicitation processes in other contexts and sectors where similar approaches are deployed. With very little previous work having been conducted to compare intra and inter method reliability, our findings can only really be seen as indicative and caution is required in extrapolating their inferences. Further comparative examination of a wider range of preference elicitation methods (including variants of the same method) and the development of new experimental procedures to ensure objective testing will allow practitioners to have greater confidence in the tools they use and better understand how they might influence preference ranking outcomes. We would advise that such research maintain a strong practitioner focus with evaluations being conducted in as realistic a context as possible. Credibility of process is vital to the integrity of participation processes and this contribution offers a constrained but stimulating contribution to the debate.