Quantitative tools and measurements for assessing the implementation of regulatory policies in reducing alcohol consumption and alcohol‐related harms: A scoping review

Abstract Issues Implementation of alcohol control policy is a global priority as alcohol contributes to negative individual health and societal impacts. However, there are no available reviews that comprehensively provide tools and measurements for assessing the implementation of alcohol control policy. This study reviews tools and measurements for assessing alcohol policy implementation. Policies considered include alcohol pricing and taxation, alcohol marketing control, physical availability control and drink‐driving policy. Approach We conducted a scoping review from Scopus, Web of Science and the World Health Organization's website. We included studies on policy implementation for the four most effective prevention policies published worldwide between 2000 and 2021. Key Findings The search yielded 11,654 articles and these were narrowed down to 39 included studies. Of these 39 studies, almost half assessed the implementation of a drink‐driving policy (n = 19), followed multipolicy (n = 12) and physical availability control (n = 8). There was no single study assessing policy implementation of pricing and taxation or alcohol marketing control. The majority of the studies were conducted in high‐income countries (n = 31). Globally, there is no standardised tool or guidelines for measuring the policy implementation of these four policies. The tools for measuring policy implementation mostly focused on a single policy, and few covered multiple policies. Implications We recommend developing standardised tools and measurements to monitor policy implementation across multiple policies at country levels. Conclusion This review highlighted a lack of comprehensive and standardised tools to assess policy implementation and the limited number of studies on alcohol policy implementation in low‐ and middle‐income countries.


| INTRODUCTION
Alcohol consumption has negative impacts on individual health and on society. Alcohol consumption contributes to more than 230 diseases and injuries such as liver cancer, tuberculosis, foetal alcohol spectrum disorders, alcohol dependence and suicide. Globally, there are six deaths every minute from alcohol-related diseases and injuries [1]. Not only this, but alcohol also has serious negative impacts on society as a whole, such as contributing to domestic violence and crime. In addition, alcohol consumption has resulted in productivity and economic losses, ranging from 0.45% to 5.44% of gross domestic product [2].
Policy implementation refers to transferring policies into practice. It is mainly focused on the processes of implementation (carrying out, accomplishing, fulfilling, producing and completing policy goals) [3]. Various domains that are considered important, including the content of policies or characteristics of interventions, factors within implementing agencies (e.g., resources, skills, the acceptability of the policy to the implementing agencies), contextual factors (political, social, cultural factors) [4,5]. Implementation processes include training, coordinating, raising public awareness and law enforcement [6]. Also, immediate results and short-term outcomes (e.g., knowledge about policies or law among implementers and general population, perceived of law enforcement among general population), are useful as they help to identify steps and progress in the policy implementation process.
Despite increasing evidence of the effectiveness of alcohol control policies, particularly the so-called best-buy policies [7], there has been less attention on translating those policies into practice in various settings, particularly in low-and middle-income countries. In addition, alcohol policy implementation is challenging, as efforts and commitments are beyond the health sector and include the context of powerful commercial interests [8]. The complexity of policy implementation requires sound measurements and standardised tools for measuring alcohol policy implementation to track progress and prioritise policy actions.
Measuring policy implementation is an essential step for identifying the progress of policy implementation at national and global levels. Quantitative measures and tools can help to measure the progress of policy implementation and identify potential facilitators or barriers that affect policy implementation outcomes and can be designed to be comparative across countries. To allow countries to monitor their progress requires tools to enable comprehensive measurements of policy content, implementation processes and policy implementation context [9] and to monitor policy implementation in different policy areas. This can help countries identify implementation gaps and effectively prioritise their actions and efforts to invest limited resources efficiently. As a result, it can help countries leverage and accelerate alcohol policy implementation for achieving the Sustainable Development Goal 3.5.2, reducing alcohol consumption per capita.
Globally, few tools have been developed to measure the progress of policy implementation at the national level [10][11][12]. Most of the available studies focused on the comprehensiveness and restrictiveness of policies [13][14][15][16]. Few of them included measurements of policy implementation. One example which included implementation is the Alcohol Prevention Magnitude Measure, which was developed to measure policy implementation in Sweden [12]; however, this tool has not been tested elsewhere. As most available tools focus on the comprehensiveness and restrictiveness of policies, they might not be able to capture the complexity of the factors that influence the implementation outcomes in different settings regarding social, economic, political and cultural aspects; the outcome of implementation depends on more than just an effective intervention. Other aspects include cultural aspects and social norms, political commitment, and the capacity and motivation of the implementing agency itself. Hence, a systematic review of tools for assessing the implementation of effective policies can help generate tools that capture the overall complexity of the implementation process.
At present, no systematic review or scoping review has been conducted to identify tools and measurements for assessing the implementation of alcohol control policy. Hence, this review is intended to fill this gap by focusing on the three 'best buy policies': alcohol pricing and taxation, control of alcohol marketing, control of physical availability and one effective intervention, on drink-driving [7]. This review aims to identify tools and measurements for assessing alcohol policy implementation.

| METHODS
We conducted a scoping review following the approach of the Joanna Briggs Institute (JBI) [17] and registered the protocol through the Open Science Framework. The JBI's guideline has been widely used and cited worldwide [18]. A scoping review was deemed an appropriate choice for this review to explore the breadth and depth of literature, map existing knowledge and identify gaps of knowledge [19]. A scoping review does not require quality assessment or a critical appraisal of the included studies and its' purpose is not to critically review tools, their content or application.
To ensure the accuracy of the scoping review, we followed the six approaches to conduct a scoping review provided by JBI [17].

| Stage 1: Identifying the research questions and scope of the study
The scoping review aimed to explore the evidence on tools and measurements available to assess the implementation of effective alcohol control policies. We developed a research question regarding the gap in evidence on the implementation of alcohol control policy. What tools and measurements have been used to assess the implementation of regulatory policies to reduce alcohol consumption and alcohol-related harm? We decided to focus on four regulatory measures with the strongest evidence base for effectiveness: alcohol taxation and pricing, control of marketing, control of physical availability and drink-driving measures [7]. The operational definitions of the terms used in this review are as follows. Alcohol policy included any regulatory measures implemented at the national level or subnational levels. The alcohol policies of interest included pricing and taxation policies, alcohol marketing control, physical availability control and drink-driving policy. Physical availability included regulating retail outlets, densities of retail outlets, restricting hours and days of trade, ban on public drinking, minimum purchasing age, licensing, control of social supply and online sales [20]. Alcohol marketing covers any regulatory measures that control any forms of alcohol marketing (i.e., alcohol advertisement, promotion, pricing promotion, alcohol sponsorship, products and placement) [20].
Policy implementation means carrying out, accomplishing, fulfilling, producing and completing policy goals [3]. We focused on the following key areas: first, factors inside implementing agencies included available resources (e.g., financial, materials and human resources) [4,5]. Second, policy implementation process including activities to be carried out to implement alcohol control policies (e.g., coordinating between implementing agencies, education, raising public awareness, training and law enforcement) [6]. Third, short-term outcomes of policy implementation made by governments (e.g., knowledge about law content among implementers and general population, perception towards law enforcement among general population). This review does not focus on measurement of the intermediate-term outcomes (e.g., alcohol consumption patterns) and impacts (i.e., alcohol-related harm) which has been covered by a guideline from the World Health Organization (WHO) 2000 [21]. Fourth, the context of policy implementation was also a focus (e.g., social, cultural, economic and political contexts). Last, characteristics of interventions or policy content, which focused on investigating the restrictiveness and stringency of policies and implementation were included (i.e., we excluded studies that focused only on policy content).
A quantitative tool refers to any tools used to collect data for measuring policy implementation and outcomes of policy implementation. Measurement is the process of an operationalised abstract construct (i.e., policy implementation) into concrete variables [22,23] in order to determine policy implementation processes and outcomes.

| Stage 2: Developing and conducting a search strategy
We developed a search strategy in Scopus and Web of Science. We selected Scopus as it is the largest search engine of scientific literature [24], including 100% of MEDLINE health science topics. We selected Web of Science, which covered some of the articles not covered in the Scopus. We also included grey literature from WHO website. A search strategy was developed for Scopus and revised appropriately for Web of Science and WHO's website.
The key search strategy can be found in Table S1, Supporting Information. The last date that we conducted the search strategy was 18 May 2021.

| Inclusion criteria
Inclusion criteria consisted of three aspects regarding context, concept and population. All included studies needed to meet the three criteria. First, the context, we included any literature from any setting. Second, the concepts, we included relevant studies that addressed quantitative tools or measures for assessing the implementation of one of the four policies: alcohol taxation and pricing, control of marketing, control of physical availability and drink-driving policy. The studies included in this review focused on policy implementation regarding the scope mentioned earlier. Third, the populations, we included any population group.
This review included publications published between 2000 and 2021 to ensure up-to-date evidence. We included published and grey literature. The grey literature included government reports and technical reports. Quantitative and mixed-methods were included to ensure that we included studies that used tools and quantitative measurements for assessing the implementation of alcohol control policies. We included primary and secondary research that quantified policy implementation.

| Exclusion criteria
We excluded various types of studies. First, studies of policy implementation that did not investigate the four policies listed above. Second, we excluded any studies that did not provide tools or measurements. Third, we excluded any literature that was not written in English. Last, we excluded any studies that only applied qualitative methods.

| Stage 3: Evidence screening and selection
Two reviewers (JJ and PP) independently screened titles and abstracts following the review protocol. The full texts of studies selected for inclusion were then screened. In the case of a small number of disagreements, they were discussed and a consensus was achieved.

| Stage 4: Data extraction
Based on the previous template suggested by the JBI [25], a data extraction form was developed and adapted during the protocol setting stage and piloted and adjusted during the review stage. The data extraction form was designed according to the research questions and objectives. It included authors, study country, objectives of the study, policy levels, settings, policy areas, design and methods, tools and key components of measurements assessing policy implementation. Prior to the use of the data extraction form, two reviewers independently tested the data extraction form and discussed improvements to the comprehensiveness and clarity of the form. The agreements were based on consensus between the two reviewers. The full data extraction process was done by one independent reviewer (JJ) and one reviewer (PP) verified the data for accuracy for every included study. If there were inconsistencies, the decisions were based on the consensus between the two reviewers.

| Stage 5: Data analysis
The purpose of this scoping review is to map and aggregate findings on the tools that existing studies have used for assessing the implementation of government policies. The data extracted from the studies were analysed descriptively using Stata 16. We categorised components and measurements in alignment with adapted domains from a logic model and policy implementation domains, namely policy content, inputs (e.g., resources and workforce), processes of policy implementation (e.g., cooperation, enforcement and publicising enforcement activities) and short-term outcomes (e.g., compliance with laws, perception towards law enforcement, knowledge about law content) and context of policy implementation [4,9,26]. This review investigated aspects of comprehensiveness in terms of tools in two dimensions: policy areas and policy domains (law content, inputs, process, short-term implementation outcomes and context). If there is a tool mentioned in the study, we also investigated whether its reliability and validity have been assessed (see definitions in Table 1).

| Stage 6: Presentation of the results
We followed the preferred reporting items for systematic review and meta-analysis extension for scoping review (PRISMA-ScR) checklist (see Table S2, Supporting Information).

| Search results
A total of 11,654 papers were identified from the databases and an additional search from WHO's website. After removing duplication, we screened 8,192 titles and abstracts. After excluding papers that were not in the scope of this review, we had 46 to assess for eligibility. We excluded seven articles for the following reasons: five papers were not in the scope of policy implementation, mainly focused on the restrictiveness of policies, and two papers did not provide sufficient details on tools or measurements ( Figure 1).

| Characteristics of the selected studies
Thirty-one studies were conducted in high-income countries, three were conducted in low-and middle-income countries, and five were multi-country. Regarding study design, the majority used cross-sectional surveys (n = 18), multisource data (n = 14) and mixed methods (n = 7). For policy areas, almost half of the studies were primarily focused on drink-driving policy (n = 19) and physical availability control (n = 8). Twelve studies investigated the implementation of alcohol policies across different policy areas. No study examined tools that captured the implementation of pricing and taxation policy and alcohol marketing control. Under physical availability control, most of them were minimum purchasing age (n = 6) ( Table 2).

| Tools for assessing the policy implementation
We found that there is no standardised tool available for assessing alcohol policy implementation among the four policies. Among the studies that applied tools, most of them used questionnaires (n = 32). However, only four studies assessed the validity of the questionnaires. One study assessed content validity and another study applied content validity and criterion-related validity. The study that assessed criterion-related validity was conducted in nine countries in the Western Pacific Region by investigating the association between alcohol policy score and alcohol consumption per capita [11]. Furthermore, two studies applied the known-group technique. Those two studies were conducted in Sweden and generated the Alcohol Prevention Magnitude Measure (APMM), which measured the progress of policy implementation. The studies found that the APMM varied across communities with interventions and without interventions [12,27]. In total, only five studies assessed the reliability of the tool.
In addition, the majority of studies applied a tool assessing a single policy, and we found that only two studies provided a tool for assessing implementation across the four-policy areas. These two studies included the International Alcohol Control Study, which included a component on the assessment of alcohol policy implementation using stringency and impact via surveys of, for example, price [10,28]. Another study was the Toolkit for Evaluating Alcohol Policy Stringency and Enforcement-16 (TEASE-16), which measured policy enforcement by reviewing policy enforcement from available data and government reports [11]. In addition, the tool APMM, generated and tested in Sweden, quantified the implementation of the three main policies: physical availability, brief intervention and drink-driving measures. The APMM was tested for reliability and the components of APMM were mainly focused on resources, network and communications as well as activities that carry out to implement policies [12].

| Components of measurements for assessing policy implementation among the four policies
Overall, regarding components of measurements for assessing the policy implementation, the majority of the studies assessed the context of policy implementation (n = 29) and processes of policy implementation (n = 29), followed by short-term outcomes of policy implementation (n = 16), inputs (n = 14) and law content (n = 5) ( Table 3).
Regarding the context of policy implementation, most of the studies assessed community characteristics (n = 19) (e.g., outlet density, poverty rates, percent of ethnicity, unemployment rates) and individual characteristics (n = 15) (e.g., age, gender, race, household income). Community characteristics and individual characteristics were mainly included in the studies on drink-driving policy, physical availability control and multi-policy. Beside this, 11 studies investigated social norms such as perception towards alcohol use, alcohol availability and alcohol-related harm (Table 3).

Validity
The instrument measured what it intends to measure.

Content validity
Experts in the discipline scrutinise the instrument very carefully and make value judgements regarding construct.

Criterionrelated validity
Comparing scores obtained from the instrument with external instrument (i.e., gold standard) by investigating correlation coefficient between results of the instruments.

Construct validity
Determining one construct is correlated with another relevant construct that is being measured, but not the gold standard.

Known-group technique
Instruments can differentiate outcomes that vary across different population.

Reliability
Internal consistency of instruments.
Regarding the process of policy implementation, the majority of the studies investigated the presence/absence and intensity of law enforcement (n = 27). Most of these studies investigated drink-driving policy (n = 14) and multi-policy (n = 8). Five studies investigated the presence/absence of publicity of enforcement and media advocacy for raising awareness (Table 3).
Regarding the short-term outcome components, the majority of the studies investigated short-term outcomes, including the deterrent effect of law enforcement (n = 11). The majority of studies investigating the deterrent effect of law primarily focused on drink-driving (n = 4) and multi-policy (n = 4) ( Table 3).
Regarding inputs of policy implementation, most of the studies investigated the existence of implementers and implementing agencies (n = 11). In addition, six studies investigated resources allocated for policy implementation. The majority of these studies were found among multiple policies (n = 4) and drink-driving policy (n = 2) ( Table 3).
Regarding law content, most of the studies investigated the restrictiveness and effectiveness of alcohol regulation, and among those were multi-policy studies (n = 4) ( Table 3).
Considering comprehensive studies regarding measurements, we found that none of the included studies investigated the implementation of a policy in all aspects, including content, input, process, short-term outcomes and context. Eight studies covered four components of implementation [27,[29][30][31][32][33][34][35]. Among these studies, three of them investigated policy implementation across different policies [27,29,30]. See more details of the findings of the included studies in Table S3, Supporting Information.

| DISCUSSION
This review is the first study that comprehensively reviewed tools and measurements for assessing alcohol policy implementation. This review contributes to the research community in mapping the published tools or measurements available worldwide. This study found no standardised tools or guidelines for measuring policy implementation among the four policy areas. In addition, there were few studies that assessed the validity and reliability of the tools. However, the measurement components for assessing implementation in this review reflect a wide array of policy implementation aspects, including law content, implementation inputs, implementation processes, short-term implementation outcomes and implementation context. Overall, the literature is F I G U R E 1 Article screening process dominated by the studies on the process of law enforcement effort, including the presence and absence and intensity of law enforcement. Most studies assessed the implementation of a single policy, even though more than one type of alcohol policy is usually implemented in a country. Also, studies tend to focus on just some aspects of implementation. To assess how well policy is implemented, we recommend that a more comprehensive approach be taken where multiple effective policies are measured along with a range of key implementation measures. Using a more comprehensive approach and tools will allow for both wider-assessment of how well policies are implemented at a country-level and for how well a country is doing for each type of policy, both of which can help countries prioritise their actions effectively.
When considering comprehensive studies found in this review, few studies investigated a range of policy implementation domains. As policy implementation is an interactive process [36], comprehensive studies (i.e., studies that investigate all aspects of policy domains and across multiple policies) are required to help countries prioritise their actions effectively.
We did not find a study using tools or measurements to assess policy implementation of alcohol marketing and pricing policy. The findings may reflect the policy situation globally. First, there are few countries that have implemented regulatory measures to control alcohol marketing [37]. Second, most of the studies on pricing and taxation policy focused on effectiveness, particularly impact on alcohol consumption (i.e., price elasticity) [38], which we did not include in this review. The International Alcohol Control Policy Index which included a measure of the impact of policies, intended to reflect implementation, found measures of both price and marketing were associated with per capita consumption across several countries [39].

| Implications
It is important to quantify policy implementation to evaluate its progress. Quantifying policy implementation (i.e., policy content, policy implementation process, outputs and short-term implementation outcomes, context) helps to identify potential factors that influence outcomes of policy implementation as well as assess the progress of policy implementation. Later, it contributes to evidencebased decision making to accelerate policy implementation. Our review collates tools and measurements available worldwide and reports the validity and reliability of the tools, where available. This comprehensive review of the tools can provide tools and measurements for countries to apply in their context to measure their policy implementation progress.

| Limitations
A few limitations need to be addressed in this review. First, we did not include non-English studies. Second, we assessed the validity and reliability of the tools based on the information provided by the included studies. However, authors may assess the validity and reliability of the study but not mention it in the published studies. To comprehensively assess the quality of tools, one could contact authors to ensure the accuracy of the information; however, this is not required in a scoping review. Third, this review only focused on four policy areas and regulatory policy. Therefore, it may not cover alcohol policy implementation overall but concentrate on those

(20)
Drink-driving policy 19 (49) Combining at least two policy areas 12 (31) T A B L E 3 Components of measurements for quantifying policy implementation and short-term outcomes of implementation outcomes of four policies

Sets of key components A brief description and example of relevant measurement
Policies (n) Total, n (%)

Included studies [references]
Policy content 5 (13) Stringency and effectiveness of alcohol legislation/regulation Stringency and effectiveness of alcohol legislation/regulation using policy scores, constructed from a level of stringency and effectiveness rating.
• Numbers of officers responsible for law enforcement per capita.
• Having specific unit assigned for law enforcement related to alcohol (yes/no).
• Percentage of total annual resources in agencies devoted to enforce drinkdriving laws.

(74)
Executing Cooperation with key stakeholders • Organising cooperation with relevant authorities (yes/no).
• Frequency of enforcement.
• Number of driving under the influence of alcohol arrests (proxy of enforcement).
• Number of sobriety checkpoints/annual number of traffic stops per capita/ frequency of sobriety checkpoints (weekly, monthly, less than monthly, never).
• Proportion of inspected grocery shops selling alcohol.
• Active work with media advocacy to increase awareness about alcohol (yes/no).
• Police's perception towards their skill and competence (Likert scale).
Availability (1)  Characteristics of implementers who involved in the policy implementation Characteristics of implementers Gender, age, position or rank in service, number of years were in service.

Drinking norm
• (Wet, moderate and dry). Outlet density or outlets per roadway mile in each city. Demographic/social/economic characteristics • Poverty rate.
• Percent of ethnicity.

| CONCLUSION
This review highlighted that there is no standardised tool for measuring alcohol policy implementation. Among available tools, there are few that assessed the validity and reliability of the tools. When considering measurement aspects, there is a lack of studies investigating the whole range of alcohol policy implementation domains, from policy content, policy inputs, policy implementation processes, short-term outcomes of policy implementation and policy context, and few studies assessed multiple policies simultaneously. Moreover, research on tools and measurements for assessing policy implementation is lacking in low-and middle-income country contexts; therefore, more research is needed to enhance policy implementation in these contexts.

AUTHOR CONTRIBUTIONS
Each author certifies that their contribution to this work meets the standards of the International Committee of Medical Journal Editors.