Our first research question concerned the categories of Web 2.0 services implemented by academic, medical and research libraries.
Twenty-eight of the 255 articles included in our review (11%) described multiple services; therefore, the remaining 227 articles represent at least 14 different categories of Web 2.0 services implemented by libraries. Instant messaging, chat and conferencing services appeared to be the most implemented; this seems consistent with the fact that such services have been generally implemented for a longer time than most other Web 2.0 services. However, because of these small numbers, no reliable inference can be drawn with respect to the relative prevalence of specific Web 2.0 service categories in academic and medical libraries.
The same holds true for the service contexts. Because we were not able to find any standard, universally agreed categorisation of library services, both devising and applying specific codes for different service contexts proved challenging; this might also partly explain the relatively high number of articles assigned to ‘multiple’ service categories. Finally, because of the small number of articles included, it is wise not to draw inferences concerning the prevalence of implementations of these categories in the real world.
Furthermore, no one-to-one correspondence between single articles and single libraries exists in our data set. There are a number of case reports concerning single libraries, but several studies synthesise data from different libraries: in our review, no different weight was assigned to studies conveying data from more than one library, so it may well be that, in principle, a relatively under-represented service category conveys data about a bigger number of libraries than a more represented category.
Based on our data and the related limitations of collection and analysis methods, we can conclude that academic, medical and research libraries in the past 5 years have used all of the main Web 2.0 tools to deliver services to final users in different service contexts (see Tables 4 and 7). A ‘universalistic’ approach to service seems to prevail in the included articles, as Web 2.0 services tend to be offered to all members of a research/medical institution, in all topic areas of research. A more sectorial approach, aimed at specific categories of users or specialist areas, seems to be an exception.
However, no further inference about the relative prevalence of some service categories or contexts can be based on these data.
The second research question concerned the identification of study designs, measures and type of data used in the included articles to evaluate effectiveness and also asked whether the identified body of literature was amenable to a systematic review and/or meta-analysis of results. Before answering this question, some preliminary considerations seem necessary.
Although the clear prevalence of articles published in English is to be reasonably expected, our finding that certain countries and cultural–linguistic traditions appear to be over-represented raises concerns. Data emerging from this review primarily concern the English-speaking geographical and cultural area. What about the under-representation of nearly all developing, but also most other developed countries? How generalizable are these findings in geographical areas beyond the Anglo-Saxon world?
Moreover, all articles were published in LIS journals and were therefore aimed at librarians and information professionals. While this is understandable, it might be worth exploring the different dissemination channels for LIS literature on Web 2.0 service, which should be of potential interest for a variety of academic audience.
Almost all databases not primarily containing LIS literature were of little or no use for identifying articles to be included in the present review.
As for the study designs, data types and measures, it clearly emerged that, although mainly quantitative in design, very few articles used structured evaluation methods, to compare a baseline situation with expected results and verify if users’ expectations had been met. This is also clearly reflected in the absolute prevalence of the case study design in the set of included articles.
In most cases, evaluation and assessment tools had not been validated. Evaluation data in the included studies were therefore obtained on the basis of a heterogeneous range of assessment methodologies, and the types of outcomes or endpoints evaluated varies almost from article to article, even for the same typology of services.
However, evaluating ‘new’ services is typically difficult, because both providers’ and users’ expectations tend to be unclear, both from a qualitative and from a quantitative point of view.45 In our opinion, more rigorous study designs and projects with clearer hypotheses to be tested with regard to users’ needs and expectations, together with more precise baseline assessments, would greatly enhance the quality of the evidence base concerning the effectiveness of such services.
Secondly, the limitations of the current study have an impact both on the estimate of the prevalent Web 2.0 services implemented by libraries and on the evidence of effectiveness, which can be drawn from the literature. For example, both books and conference proceedings contain relevant data to answer our research question. We opted for articles published in journals to try to obtain the best possible quality from the included literature, given the time and resources available. However, it cannot be excluded that conference proceedings can produce results of comparable quality, as research in a related field show.46 Ideally, an integration of the results of the present review with conference proceedings would allow to retrieve more relevant results.
As articles usually need to undergo a peer review process during at least some months before being published in journals,47 data from the included articles are, at best, updated at the end of 2010, which can make a significant difference when studying services that are expected to have, at least in some cases, a rapidly increasing rate of adoption. However, contrary to what we expected, we did not find an increasing trend in the number of publications meeting our inclusion criteria on this topic from 2006 to 2011 (see Fig. 2).
After that, it emerged from the data that most of the Web 2.0 services were either free or not so expensive to set up and run. This surely had an impact on the ease of adoption and might have put less pressure to evaluate such services, than, for example, other pieces of library automation software or other library services that have considerable costs which constantly need to be justified.48
Finally, despite the heterogeneity of service categories, evaluation methods and outcomes, the authors recommend that a systematic review of this literature be undertaken. This seems an urgent task, because a considerable number of articles evaluating the implementation of Web 2.0 services in academic, research and health libraries have been identified, but no thorough and systematic assessment of this body of literature has been conducted so far.
Moreover, a systematic review would allow assessment of the effectiveness of such services for final users and identification of gaps in the available evidence.
This applies to our whole data set, although a specific subgroup of articles (referring to conferencing, chat and instant messaging) might represent an even more homogeneous target for a systematic review.