A critical review on blockchain assessment initiatives: A technology evolution viewpoint

Blockchain is considered as a major emerging technology that is having an ever‐increasing spread both in industrial and academic contexts. As the usage of blockchain keeps increasing, a fourth generation of blockchain platforms is being proposed. Thus, applications of blockchain have evolved towards wider scopes than cryptocurrency and asset management. In this context, it is important for practitioners to have deep understanding of various blockchain assessment initiatives. Therefore, this work discusses blockchain assessment initiatives from a technology evolution viewpoint. Furthermore, a mapping was conducted to identify factors that impact blockchain initiatives, synthesize available evidence, and identify gaps between relevant approaches available in the literature. As a result, nine selected works were analyzed based on applicability, research approach, assessment process, blockchain adoption process, and blockchain waves. The findings can help practitioners to understand the main assessment factors that undermine blockchain implementations.

panorama in which continuous software engineering is transforming the way users and software practitioners tackle changing requirements, 7,8 times to market are being reduced in a remarkable way. 9 This trend is affecting both tailor-made software and commercial software packages alike. 10 Blockchain technology and its different platforms are not outside technology evolution although it is true that change models and rules are considered specific in platforms. 11 Blockchain platforms enable the development of blockchain-based solutions. This distributed ledger technology (DLT) supports collaborative processes by means of a shared, distributed, and trusted dataset. There are two main blockchain models: permissioned and permissionless blockchains. Permissioned blockchains can also be called private blockchains and require permission to join. Examples are, for instance, Monax or Multichain. On the other hand, permissionless blockchain lets anyone participate in the system. Classic examples of these blockchains are Bitcoin and Ethereum and more recently Monero or Zcash. From a technology evolution perspective, the governance of both models differs in a radical way and also their control.
Blockchain appeared back in 2008 designed by Nakamoto, when the famous Bitcoin system implemented this technology to be able to track and store transactions in a decentralized way. However, the first generation of blockchain solutions (Blockchain 1.0) evolved into a variety of platforms (e.g., IBM Blockchain, Hyperledger Sawtooth, Ethereum, Hyperledger Fabric, and Quorum) now considered Blockchain 2.0. 12 These platforms introduced new services to blockchain-based applications as an evolution of the previous generation. However, as the hype in blockchain technology advanced, Blockchain 3.0 expands the blockchain focus further to incorporate decentralized applications. 13 Blockchain 3.0 is aimed to popularize blockchain-based solutions expanding the traditional sectors (finance, goods transactions, and so on) to government, supply chain management, smart energy, health, market monitoring, and education. 14,15 Therefore, the applications of blockchain have evolved to much wider scopes. However, these new applications introduce new features to the next generation platforms including key aspects such as platforms interconnection or more advanced "smart contracts." Consequently, a new generation of solutions is currently under development and will be labeled as Blockchain 4.0.
As the maturity of the technology and its applications increased, some assessment initiatives appeared both at the industry and academic levels. However, to the best of authors' knowledge, this is the first work devoted to analyze the different assessment initiatives and evaluate their applicability in the changing environment of blockchain technology.
The remaining of this work is structured as follows. In the next section, the different generations are presented and analyzed with regard to their main features. In Section 3, authors review the most important assessment initiatives devoted to analyze the applicability of blockchain in a given environment. Next, a discussion of the main aspects in the initiatives and their mapping to blockchain generations is presented. In Section 5, authors present limitations and threats to validity. In the last section, authors wrap-up the paper and suggest future works in the topic.

| BLOCKCHAIN GENERATIONS
As indicated before, and in spite this is a simplification of an already complex evolution, according to the literature, to date, there are three generations of blockchain. However, a new generation is currently under development: Blockchain 4.0. In what follows, authors will review them.

| Blockchain 1.0
Blockchain 1.0 is the seminal blockchain attached to cryptocurrencies applications. In a nutshell, Blockchain 1.0 is a way to confirm transactions on a decentralized ledger system. In 2008, Satoshi Nakamoto proposed the first initiative to implement Blockchain called Bitcoin. There, blockchain is a distributed peer-to-peer linked structure. It could be used to solve the problem of maintaining the order of transactions and to avoid the double-spending problem by means of a set of mechanisms (a chain structure composed of different blocks connected in series through hashes). Thus, blockchain manages to contain a robust and auditable registry of all transactions. 16 Blockchain 1.0 is not just about Bitcoin; it is about cryptocurrencies including the rest of cryptos (altcoins). Proof of Work (PoW), the consensus mechanism in Bitcoin, or proof of stake (PoS) as in PeerCoin or combination of both are used to achieve distributed consensus.
In blockchain, problems of efficiency in terms of computational effort, long response times, lack of interoperability, and flexibility were very soon reported as threats to blockchain adoption in wider scenarios.

| Blockchain 2.0
The new generation of blockchain started with Ethereum back in 2013. This new generation implemented the PoW mechanism; however, it incorporated a set of new and promising features enabling the possibility of applying the advantages of blockchain to other fields. 17 This new generation of blockchain is a decentralized market that provides a wider range of application scenarios by using the blockchain as a distributed ledger to record, confirm, and transfer various forms of contracts and properties. 18 Smart Contracts are one of the main new features of Blockchain 2.0, although the concept of smart contract was not initially linked to blockchain. In fact, Nick Szabo coined the term in the nineties. For Szabo, a smart contract is a computerized transaction protocol that executes the terms of a contract when certain conditions are met. In Blockchain 2.0, smart contracts are seen as scripts running in a decentralized manner and stored in the blockchain, 19 but they are not the only new feature available. Decentralized autonomous organizations (DAOs) 20 are new organization forms in which the management and operational rules are typically encoded on blockchain in the form of smart contracts and can autonomously operate without centralized control or third-party intervention. 21 In any case, and in spite of the new functionalities, several limitations are still present, as reported in the literature, for example, Bez et al. 22 The most important ones are problems of speed, in spite of the advances from Bitcoin, and scalability (for instance, in terms of high storage needs).
The main differences between Blockchain 1.0 and 2.0 are as follows: • Application of the technology to a new set of fields apart from the crypto From a more technological standpoint, new networks are aimed to enable interoperability (see the different approaches adopted, presented in Siris et al. 26 ) and increase network speed. However, the essential features of blockchain networks should be observed since Blockchain 3.0 incorporates features like immutability and transparency and no need for intermediaries, obtained by the blockchain trustless decentralization to other systems, which are built on top of blockchain technology. 25 However, from a purely technological viewpoint, there are voices claiming that IOTA, Nano, or Byteball, for instance, are not really blockchains; they are seen as post-blockchains. 27 The main differences between Blockchain 2.0 and Blockchain 3.0 are concerned with resolving issues that prevent blockchain technology from an industry-spanning application: • Scalability (IOTA) • Interoperability (ICON and Cardano) • Governance (Tezos) • Sustainability (Golem)

| Blockchain 4.0
Although the development of this new wave of platforms is still ongoing, according to Angelis and Ribeiro da Silva, 13 Blockchain 4.0 includes artificial intelligence as part of the platform, reducing the need of human management by enabling functions to make decisions and act on systems.
This integration with artificial intelligence is also present in the work of Upadhyay. 28 On the other hand, according to Ratanasopitkul, 29 Blockchain 4.0 will focus on scalability, flexibility, and usability, but the author does not specify how or in which direction. In fact, scalability and flexibility are also listed and briefly described in a survey on blockchain-based internet service architecture 30 as follows: Blockchain 4.0 wants to improve the consensus efficiency, the scalability, the energy efficiency, and so on, thereby tailoring blockchain to real, contemporary, and future environments.
Another interesting aspect is introduced in Arenas and Fernandez. 31 In this work, authors believe that the new generation will be based on the concept of blockchain-as-a-service (BaaS) given the recent developments of IBM (on Hyperledger) and Microsoft (on Ethereum).
Scientific literature fails to bring a defined set of features to the new generation of blockchain. However, professional literature 32 could shed some light into the topic. In this post, the author presents several initiatives (Multiversum, SOOM, Aergo, and Insolar) to conclude that Blockchain 4.0 will be not only BaaS but also mass-friendly, more efficient, and better performer. In support of that, another platform called Unibright claims to provide a unified framework for integrated business using blockchain technology in which anyone, who could not program, could make the business solution online just using workflow diagrams visuals.

| BLOCKCHAIN ASSESSMENT INITIATIVES
Given the popularity of the topic, literature reported a good set of assessment initiatives surrounding blockchain applicability. Studies and recommendations on this applicability contain a set of expressions including "Decision Models," "Adoption," "Applicability," "Maturity Model," or "Assessment." In what follows, the main initiatives in the topic are reviewed. Due to the popularity of the concept, there are works devoted to illustrate possibilities and assessment aspects, too. For instance, in "Blockchain Use Cases and Their Feasibility," 33 authors review different application areas, and in the last part of the paper, they also review different feasibility studies. Differently from this study, in our work, authors expand feasibility aspects to embrace also aspects of assessment and adoption. Another initiative worth mentioning is a systematic literature review on technological, organizational, and environmental considerations on blockchain adoption. 34 In this work, authors, as their main contribution, derive and rank a set of technological, organizational, and environmental considerations on the literature review. However, they do not focus on the continuous approach of blockchain adoption and assessment but report a set of factors to take into account in the assessment process.
In order to develop a study of the different assessment initiatives, authors performed a query in Google Scholar with the keywords: "Blockchain" AND "Assessment Model." The query produced around 400 results by January 2020.
From a formal perspective, the selection of studies in this paper was directed by applying a set of inclusion and exclusion criteria. The inclusion criterion was as follows: • Method: all methods were considered.
• Type: the type of study could be an article, conference paper, magazine article, or a book chapter.
• Language: studies written in English language only.
• Domain: all domains were considered.
The exclusion criteria were as follows: • The full text of the study cannot be accessed.
Authors agree that in systematic literature reviews, exclusion and inclusion criteria are more narrowed; however, given the small set of papers devoted to the topic, the set of studies was logically broad. The quality assessment of papers was not applied to try to analyze all kinds of initiatives with a proposal regardless its stage of development. However, in the final set of primary studies, this aspect was analyzed, as a factor labeled research approach (Table 1).
Mapping of the nine papers with regard to applicability, research approach, assessment process, blockchain adoption process, and blockchain waves With regard to data extraction, the main author worked independently to extract data from all primary studies, guided by an extraction form.
This form was designed to gather key information like title, authors, conference/journal, year of publication, number of pages, keywords, and abstract. After reading the abstract, the first author obtained the full-text versions and stored them in Zotero. The last filter was the full-text reading. To ensure quality and validity of our results, constant meetings among co-authors took place to validate results.
This process leaded to the nine papers presented in this paper, being the selection of this set of primary studies the main result of the literature review performed. Authors want to underline that the works of Dinh et al. 43,44 were not considered in the final set. These works are focused on analyzing private blockchains, and, although they can lead to a better understanding of different system design choices, their works are aimed to work in already configured blockchains by means of benchmarks.
The first initiative is called "A maturity model for blockchain adoption" and is dated back in 2016. 35 In this paper, authors propose a maturity model based in the approach adopted in the capability maturity model (CMM) called "blockchain maturity model" (BCMM). In their approach, they define five stages of maturity (1 to 5) and four indicators: networks, information systems, computing methodologies, and security and privacy.
Although the model is general, it is applicable to blockchain technology in particular, while introducing a light adoption procedure based on three stages: feasibility study, development, and operation.
In "A Framework for Determining Blockchain Applicability," 36  derived from previous works on blockchain taxonomies 45 and propose seven criteria to choose a specific blockchain, namely, Ecosystem Governance, Administrative Accountability, Confidentiality of Exchanged Information, Scalability and Performance, Transactions' Speed and Volume, Crash Tolerance, and, finally, Assets' Management. This approach was tested in the invoice discounting (banking) arena. Another work coming from a specific sector or application, in this case, Industry 4.0, is the one by Fernández-Caramés and Fraga-Lamas. 40 In this paper, there is a section that discusses main challenges on the implementation of blockchain in Industry 4.0, and in that, authors include a set of criteria to take into account when assessing a blockchain solution.
Two more contributions are coming from academic works of students. In the first paper, 41 the author includes a framework for blockchain adoption including a set of implementation phases, namely, discovery, implementation, operations, and disposal, along with a set of layers in an Open Systems Interconnection (OSI) style for the final composition of the solution. The second effort is coming from TU Delft, issued also back in 2017 and is designed specifically for Governmental Processes. 42 For the assessment of blockchain in an organization, this work proposes six elements: ripple effects, process factors, organizational factors, complexities, design features, and decision-making process. As a result, a blockchain assessment tool is designed based on three steps that allows a user to assess the blockchain fit, create a high-level blockchain design, and map the ripple effects.
To conclude this section, literature reported a set of frameworks and initiatives to assess aspects related to the implementation of blockchain-based solutions. In the next section, authors present a critical analysis of nine relevant initiatives based on two conventional contexts, technical and business, but also on aspects related to software evolution and more precisely, on the inclusion of the different blockchain waves.

| ANALYSIS AND DISCUSSION
In what follows, authors present a mapping of the nine selected works with regard to a set of aspects to compare and analyze their internals. To do so, in Table 1, authors will review the nine primary studies with regard to the following aspects: 1 Applicability (general purpose, specific sector, or technology).
2 Research approach adopted for the definition of the proposal. In this aspect, authors adopt the quality checklist suggested by Kitchenham and Charters 46 and Kitchenham and Brereton, 47 based on Centre for Reviews and Dissemination (CRD) Guidelines. 48 Authors adapted and employed a quality questionnaire including these aspects: • Design Is there a clear statement of the aims of the research?
• Conduct Does the paper provide relevant data related to blockchain assessment?
• Analysis How adequately are the research results documented?
• Conclusions Does the study allow answering the research questions?
The nine selected papers (primary studies) were assessed by means of this set of questions. The possible scores for each of the aspects are: Yes = 1, Partially = 0.5, and No = 0 points. Accordingly, the maximum score for a primary study is 4.
3 Assessment process definition. In this aspect, authors identify/assess if the paper introduces a way to assess the applicability of blockchain in a given situation.
4 Blockchain adoption process. Apart from the assessment process, this aspect reflects if the paper is providing information on the adoption process to guide organizations to embrace blockchain-based solutions. Despite that the focus of the paper is not the adoption process, authors want to investigate to what extent the proposal presented in the analyzed paper goes beyond selection and includes further aspects, in this case adoption process. Although adoption is a process that goes beyond selection, it is a key factor in technology acceptance. 49 5 Blockchain waves included in the initiative. The paper included aspects on Blockchain 1.0, Blockchain 2.0, and Blockchain 3.0 as different options for implementation. Blockchain 4.0 is not considered in the study performed. The possible scores are: yes, somehow, and no.
These four aspects were selected after reading the set of nine papers and adapting the aims of the research.
From Table 1, one can see that although there are works devoted to specific environments, the applicability of the framework is broad in most of the efforts. In fact, five out of nine works are published in 2019. With regard to the soundness of the approach adopted in the works, there is a majority of proposals built on their experience and not on grounded research approaches. This could undermine the applicability of the different proposals, but not with regard to our effort, given the tertiary approach adopted by researchers.
With regard to applicability, seven of the nine studies present a general approach. Two of them are focused on specific fields: Allessie 42 in government and Fernández-Caramés and Fraga-Lamas 40 in Smart Factories.
There are two works, namely, Betzwieser et al. 38 and Allessie, 42 built on research approaches. However, these works present also some caveats in the analysis and conclusions of the experiments. One more work 36 also adopts research methods in its proposal, although all aspects could be improved to make it more sound. Overall, the research aspect in the construction of the assessment process is moderate.
Regarding the assessment process, two works, namely, Wang et al. 35 and Fernández-Caramés and Fraga-Lamas, 40 just include a set of factors to consider, and one more, Kharitonov, 41 includes general guidelines. The rest of the works present a defined blockchain assessment process.
Beyond assessment, implementation is also an aspect to take into consideration. In this sense, the only paper that defines an adoption path is Wang et al. 35 while Kharitonov 41 presents some aspects to take into consideration as blockchain-based solutions evolve over time, but not a fully constructed evolution path in the adoption. However, blockchain adoption should be carefully evaluated depending on a company's sector and business goals. 14,15 Finally, regarding blockchain waves, this aspect is somehow covered in just two papers, namely, Angelis and Ribeiro da Silva 13 and Wang et al. 35 but not fully explained or developed using the current waves description. The rest of the initiatives are not considering the different blockchain waves in their proposals. Nonetheless, it is worth noting that the features of each blockchain wave impact on the potential value of blockchain use as a whole 13 and its application field. 14,15 The first analysis of results leads to a scenario in which current initiatives are not fulfilling all aspects. Some of the works are addressing a subset of the factors but present caveats in others. This leads to a conclusion that more work is needed in the topic to build a proposal able to qualify in all factors.
Within the assessment process itself, it is worthy to analyze the different aspects included in the selected papers. To do so, we divided these aspects between technical and business oriented. A total of 19 factors were analyzed; 14 of these factors are technical and five of them business oriented. Authors agree that some of the business factors present also technical aspects like, for instance, governance (including in it aspects of off-chain and on-chain governance approaches), but decided to include these factors in Business category. The selection of these factors is based on the analysis of the nine papers. The definition of each of the factors is as follows: • Immutability: To what extent the system must keep all historical data.
• Transparency: To what extent the system must be visible for all participants (some protections can be placed for privacy or anonymity).
• Trust: To what extent the system must ensure trust among participants in their interactions.
• Identity: This reflects the requirement of the system to identify who was involved in transactions.
• Distribution: To what extent the system must distribute the information in order to achieve reliability, data security, integrity, and for architectural aspects.
• Workflow: To what extent the inner capacities of blockchain are suitable for the transactions of the system.
• Transactions: To what extent the final system is transactional.
• Historical record: Longevity and immutability of records are part of the system.
• Ecosystem (Interoperability): To what extent the system will operate with others and interoperability is a key aspect to consider in the architectural design.
• Inefficiency: To what extent the system can deal with certain wastefulness because of the rigor, needed verification, and data structure.
• Scalability: To what extent the system will need to grow in size?
• Maintainability: To what extent the system will need to be repaired in a given time.
• Smart contracts and data access control: To what extent the organization and the blockchain network present the same approach.
• Permissioned versus permissionless: To what extent the assessment model is including the possibility of using permissioned and permissionless blockchains.
• Cost: Inclusion of aspects regarding the cost of the system.
• Regulations and laws: Inclusion of aspects regarding these norms.
• Efficiency: Inclusion of aspects regarding the consumption of resources (economic, energy, network, and so on) in the solution.
• Governance: Incentives and methods of coordination of the blockchain network.
• Business model adaptation: To what extent from a purely business model perspective, a blockchain solution could fit.
Authors want to clarify the different governance approaches must be taken into account in the factor and that both "on-chain" and "offchain" governance are expected to be assessed in this aspect with regard to its applicability to the given situation along with the evolution of the platform. Some blockchain projects, for example, Tezos, Dfinity, or Decred, adopted on-chain governance centred models. In on-chain governance models, rules and decision-making processes have been encoded directly into the underlying infrastructure of the blockchain. 50 Off-chain is adopted by Bitcoin or Ethereum and includes endogenous (rules coming from a reference community) and exogenous rules (coming from thirdparty).
The process of mapping was developed by researchers coding the results in a form that gathered the factors in the different papers analyzed. Table 2 shows a mapping of the aspects analyzed.
Taking into account results shown in Table 2, although there are differences among them, the Technical factors presented in Scriber 36 are included as a whole in previous studies. 13,38,39 With regard to Technical factors not covered in Scriber, 36 the presence of Scalability is more generalized among the initiatives while Maintainability is just considered in Wang et al. 35 and Kharitonov. 41 Finally, regarding Smart contracts programmability and data access control, this factor is covered just in Maranhão et al. 37 and Betzwieser et al. 38 so that is not too popular, as well. With regard to the inclusion of factors related to access restrictions, the two main models, permissioned and permissionless, are considered in five of the nine papers. However, the approach presented in Betzwieser et al. 38 is the most developed one since it treats the factor as one of the design decisions of the blockchain network.
There are also Business factors included in the initiatives as part of the assessing criteria. Cost is the first criterion and, although is part of most of assessment methods in computing, in this case is just present in four out of the nine initiatives. Regulation and laws is present in all initiatives, except in Scriber 36 and Maranhão et al. 37 Moreover, Efficiency is a factor just covered in Wang et al. 35 while Governance (although includes also Technical aspects as well as Regulations and laws) is covered in other works. [37][38][39]41 Finally, Business model adaptation is present in three initiatives, namely, Betzwieser et al. 38 Kharitonov,41 and Allessie. 42 In fact, an analysis of the coverage of the proposals in terms of factors is leading us to conclude that the most complete one is Betzwieser et al. 38 giving the set of factors it includes.
Counting the features of the different assessment factors, one can witness that the most complete effort is Betzwieser et al. 38 which includes 16 factors, followed by Fabrizio et al. 39 and Allessie 42 with a sum of 14 factors. In that, the weakest approach is Kharitonov 41 with 5 factors and Wang et al. 35 with 9. None of the factors was included in all the efforts, but several factors Distribution, Ecosystem (Interoperability), and Inefficiency were included in seven of the nine papers. Efficiency was included in just one of the papers.
One of the most important findings in the study is the fact that, although some efforts (literature 36,38,42 ) present research-based approaches ( Table 1), most of them present caveats in the construction. The second aspect, also in Table 1, is the limited coverage of Blockchain waves in the initiatives. With regard to data gathered in Table 2, it is important to note that most of the Technical factors are well-covered in the assessment of blockchain technologies, while there is a need to integrate this analysis with Business and Governance aspects.

| LIMITATIONS
This study presents several limitations that are analyzed in what follows with regard to the most common threats to validity in mapping studies. 51 Interpretive validity is accomplished when the conclusions are reasonable, given the data. This is a threat deeply connected with researcher bias. In our case, two researchers with experience in secondary studies participated in the process, and the other researcher reviewed results and conclusions.
The descriptive validity is the extent to which observations are described in an accurate and objective way. Mapping was performed by means of a form to support the recording of data and to trace aspects inside the different proposals analyzed.
Theoretical validity discusses threats in building theory out of the observed phenomenon. While it is true that this work is not covering the entire research published in the topic, this subset is an acceptable sample for the mapping conducted. Another aspect to take into account is researcher bias in the phases of selection and extraction of data. To reduce this threat, authors followed a process to involve all researchers in the mapping and measure their agreements by means of the Krippendorff alpha statistic to test agreement among them. The agreement reached between first and second authors was 68.71% being ideal values around 80% of agreement. If any disagreements exist between the two authors, the third author was involved in the discussion.
Finally, with regard to publication bias, authors expect published research to present positive aspects, while negative results are normally not published. However, authors believe that this mapping is intended to analyze the frameworks and efforts but not their results in their implementations.

| CONCLUSIONS AND FUTURE WORKS
Blockchain is considered as one of the major emerging technologies that is having an ever-increasing spread both in academic and industrial contexts. 25 This work presents the current trends in such a technology and a mapping focuses on the continuous approach of its adoption and assessment including aspects on software evolution. The main contribution of the paper is twofold. Firstly, the determination of the current studies devoted to blockchain assessment and adoption, and secondly, the analysis of these primary studies to conclude that, among current initiatives, there is room for improvement towards a solid and sound approach to guide the selection and adoption of blockchain technologies.
In the paper, authors identified factors that impact blockchain initiatives, synthesize available evidence, and identify gaps between existing approaches available in the literature. By reviewing nine selected papers for their rigor and relevance, a total of 19 factors were identified and grouped into technical (14) and business (5) contexts. It is also observed that despite the research done in this field, it lacks rigor and thus hinders the possibility of a wider adoption of blockchain technology. In fact, the quality assessment of the research approaches is significant (three out of four) only for two papers, Betzwieser et al. 38 and Allessie. 42 This calls for further studies with thorough empirical analysis, in particular validation related to software evolution and more precisely, on the inclusion of the different blockchain waves. By analyzing the coverage of the proposals with regard to the identified factors, the most complete one is Betzwieser et al.. 38 In spite of that, the only work that defines an adoption path is Wang et al. 35 while another work 41 presents some related aspects. Although the results can help practitioners to understand the main assessment factors that undermine blockchain implementations, future works are needed to build solid research approaches on blockchain assessment and implementation. To do so, authors aim to devote future efforts to the development and validation of a feasibility assessment initiative built on previous works but also on expert's opinions gathered by means of scientific methods.