Library & information science literature: How much of it is research?
In Library and Information Science (LIS), while there is an awareness of the variety of journal literature, we do not have a number on the percentage of the collection that qualifies as research. A content analysis of the LIS literature available at the Simmons College Library is currently in progress. The research level collection of LIS literature in support of the Graduate School of Library and Information Science (GSLIS) program at the college makes the Library an ideal candidate for this study. Research is pre-defined and a collection method for a content analysis is planned. The entirety of the collection of articles available in 2011 will be analyzed and classified as per: 1) research versus non-research, 2) the subject of each article, and 3) the percentage of research compared to non-research. This study will benefit students, faculty, and staff with research requirements as well as librarians who guide patrons through a search. Further, it will provide a sense of the state of LIS literature for the sample year to be compared to previous studies.
The Simmons College Library is a small academic library serving the needs of all Simmons students, faculty and staff, including the College of Arts and Sciences (with both undergraduate and graduate schools), and the four Graduate Schools of: Library and Information Science (GSLIS), Nursing and Health Sciences, Management, and Social Work. The purpose of this study is to determine what percentage of the Library and Information Science (LIS) literature available to GSLIS students, faculty, and staff qualifies as research. The study will make use of the LIS databases and LIS periodicals collection available through the Simmons College Library, and will analyze the content of the collection. The researchers of this study include Library Staff from Research Services, Access Services, and a Faculty member of the GSLIS program.
Stakeholders for this study include LIS students with research requirements for completion of their degree and faculty with research requirements for tenure. Secondary stakeholders include GSLIS staff with an interest in conducting research and liaison librarians to the GSLIS program. This study will benefit both patrons and staff of the Library. Specifically, Library staff will be aware of how much the collection fits the research needs of faculty, staff, and, students of the GSLIS program. The study will benefit the profession by providing a sense of the state of LIS literature for 2011. It is a continuation of several other content analyses conducted of core LIS titles in previous decades. This study is unique in that it is the first known attempt to analyze an entire collection of LIS periodicals, including non-research based titles.
Research is a fact of academia. Both faculty and students must produce and consume research to satisfy requirements of tenure or graduation, and the LIS field is no different. As part of an evolving field, LIS programs benefit from analysis and experimentation leading to new insights – or research.
While there is an understanding of the gradations of the vast literature published in the field, there is a knowledge void on the percentage of the LIS literature that qualifies as research for a given year (Aharony, 2012; Buttlar, 1991; Jaervelin and Vakkari, 1990; Nour, 1985). There is also much speculation about the range of topics covered in LIS literature. Further, there is a gap in the relative percentage of the types of methodologies used to conduct the research.
The objective of this study is to determine a more accurate estimate of the percentage of the LIS literature published in 2011 that qualifies as research. The research questions that we seek to answer are: “1) What percentage of the LIS periodical literature available to Simmons College is research? 2) Of the titles that include research, what percentage of the articles in each title are research articles? 3) What is the subject distribution in both the research articles and non-research articles? 4) What methodologies are used in the research?” LIS periodicals, available both in print and online, will be collected, analyzed, and categorized. Article abstracts will be used to determine the subject and methodology of research articles. The LIS databases and periodicals stacks will be used to determine the size collection for this study.
The findings of this study could potentially impact how LIS Librarians support LIS researchers. A better understanding of the percentage of available literature that is research, as opposed to non-research, could potentially lead to an improved search experience. This study will also provide a snapshot of the topics covered and methodologies used in research in 2011. Librarians and established researchers mentoring new researchers would have a more concrete sense of the amount of literature available.
There have been a number of content analysis papers published regarding LIS periodical and journal literature. These studies have primarily focused on the subjects covered and methodologies used in LIS research (Feehan, Havener, and Kester, 1987; Nour, 1985; Jaervelin and Vakkari, 1990; Kumpulainen, 1991). While these studies provide valuable information regarding the trends of research literature, they tend to focus on analyzing articles from a list of core LIS research journals. These studies intentionally exclude all non-peer reviewed and referred journals (Feehan, Gragg II, Havener and Kester, 1987; Jaervelin and Vakkari, 1993; Koufogiannakis and Slater, 2004; Kumpulainen, 1991; Nour, 1985). The scope of content analyses in these past studies was on a limited list of journals with a research focus. In each case, the list of core journals was compiled after analyzing multiple indices to identify titles that are included in more than one database or index. Feehan et al. (1987) also solicited feedback from library professionals as to their opinion of the core journals in LIS. All studies explicitly excluded international journals. Only Jaervelin and Vakkari (1990, 1993) included non-English international journals.
The total list of core journals thus varied from as little as 10 journals (Arahony, 2012) to 91 (Feehan et al., 1987). This indicates that there is no consistency in what qualifies as a core journal. Another factor briefly addressed by Jaervelin and Vakkari (1993) is the nature of the publishing industry; core journal lists vary between decades because the core journals identified for one decade may cease to exist before another and new core journals may emerge since the initial year of cross-decade studies (p. 131). It is therefore generally difficult to develop an unbiased, consistent list of journals that qualify as research-based or professional, even when cross referencing lists of indexed titles as a means of developing the core list.
While part of the fluctuation can be attributed to trends in the field, it is also due in part to varying methods of conducting research (Jarvelin and Vakkari, 1990). Related to this is the fact that even when only analyzing core journals, not 100% of what is published in these research journals is research (Feehan, Gragg II, Havener and Kester, 1987; Buttlar, 1991; Nour, 1985; Jarvelin and Vakkari, 1990; Kumpulainen, 1991).
The changing lists of journal titles selected for analysis also resulted in skewed results of the percentage of research literature. Jaervelin and Vakkari found that as much as 54% of their sample qualified as research while Feehan et al. (1987) found that only 23.6% of the sample qualified as research. This discrepancy makes it difficult to develop a sense of the field. An inconsistency in titles included further exacerbates the effects of a fluctuating publishing industry.
The narrowest of the studies focused on a list of core journals of less than 20 each. Both Buttlar (1991) and Arahony's (2012) studies produced valuable information about trends in authorship of research in LIS literature. Buttlar (1991) analyzed author information including: sex, occupation, and geographic location. Aharony's (2012) most recent content analysis went beyond Buttlar's study and presented statistical descriptive analysis of research article keywords as well. Yet, the limited list of journals analyzed brings about the question of the consistency of the data.
A constant theme throughout the studies is the need to define “research” before undertaking a content analysis. Several content analyses use a consistent definition of research as established by Peritz (1980):
Research is any inquiry which is carried out, at least to some degree, by a systematic method with the purpose of [eliciting] some new facts, concepts or ideas (Feehan et al., 1987; Nour, 1985; Yontar and Yalvac, 2000).
But, as Nour suggests, even a highly accepted definition is “criticized for its lack of rigor” (p. 262). This definition is often critiqued as being too broad and not specific enough to the field (Koufogiannakis and Slater, 2004). Still, this definition endures for its inclusion of its key concepts, “method” and “purpose,” which allow a researcher to more easily distinguish research articles from other articles (Feehan et al., 1987; Nour, 1985; Yontar and Yalvac, 2000). Use of a consistent definition increases the external validity of the studies, even if their core journal lists vary drastically.
This definition has also been used in content analyses of international, non-English journals, further demonstrating its endurance and relevancy (Kajberg, 1996; Yontar and Yalvac, 2000). Moreover, the use of the same definition ensures it will still be applicable to a collection that includes international, non-English journals, as this study proposes. These international studies also varied in scope. Like the American studies, Yontar and Yalvac (1996) limited the journals included in the study. In fact, the study focused on only one journal. Still, this study demonstrated that a consistent definition produced reliable data with high internal validity. Conversely, Kajberg (1996) expanded his research to include all the Danish LIS literature published from 1957 to 1986. Unlike the American studies, the Danish studies included non-research as well as research journals, demonstrating that it is possible to compare across types of journals. These two international studies establish the validity of Peritz's definition of research in analyzing international articles.
These studies confirm the importance of analyzing the content of both research journals and trade periodicals to develop a better sense of the amount of research that exists with the body of literature. Furthermore, it has been proven that it is possible to analyze content across journal types spanning multiple years.
This study aims to determine how much of the literature qualifies as research. In this study, there are no casual variables that will affect the final measure, and thus no hypotheses. Based on previous content analysis studies, we operationalize research in this study as defined by Peritz (1980).
This research is approached as a content analysis study of articles in LIS journals. The LIS collection for the calendar year 2011 at Simmons College Library serves as the sample year. A single, most recent, year was chosen to make the study manageable. All of the available journal titles for the calendar year will be collected, classified, and analyzed. An appropriate measure has been developed to collect the data. Once the data for the selected year is compiled, it will be analyzed.
A content analysis form has been developed to categorize several key elements of each article in a journal, including:
This content analysis will focus on the entire collection of LIS periodicals available in 2011. The content analysis will be measured using a form developed based on previous studies, with slight modifications.
Before conducting the complete analysis, a comprehensive list of journal titles in the collection will be solicited from Library staff. This list will be analyzed to determine which titles were available in the collection in 2011. A list of print journals has been compiled. A comparison between the compiled list and the stacks is currently underway. A list of e-journals has also been collected. An analysis of the list to determine title availability in 2011 will follow.
Research assistants, or readers, will be recruited once there is a complete understanding of the size of the collection to be analyzed. Depending on the size, up to four readers may be recruited. Readers will be given background information regarding the project and trained on elements of the form, including:
definition of research for this project
size of collection to be analyzed
how to properly fill out the form
In the case that a periodical is clearly entirely non-research, only select articles will be analyzed for content to determine the subject. These will include 1) full length feature articles, and 2) reviews, including for a) books and b) products such as computer hardware, software, furnishings, etc.
All data will be collected using a standard form. Once the data is collected, a small sample from the literature will be randomly selected to test for reliability. A different reader will be assigned to analyze the articles in the sample. The analyses will then be compared to determine inter-rater reliability. If necessary, some articles may be selected for re-classification.
Once the data has been compiled, it will be prepared for analysis using SPSS software. The research will result in a statistical descriptive analysis of the Simmons College Library LIS journal collection for 2011.
This study will result in a content analysis of a year's worth of LIS serials literature. The research level collection of LIS literature in support of the GSLIS program at the college makes this an ideal collection for this study. The data is expected to show the distribution of the literature, including what percent is research and non-research, the methods used, and the topics covered. This one year analysis will provide a record view of the LIS literature in a particular point in time. The trending topics are expected to reflect the political climate and LIS interests around the time selected for analysis.
To be more specific, the data is also expected to show that when measured against the entirety of the published literature, research is much less then the lowest percentage calculated in previous studies. It is also expected that the topics covered in the research literature will be different from that covered in the non-research literature.
Strengths and limitations of the study
The size and nature of the collection chosen for this content analysis are both a strength and a limitation of this particular study. Because the Simmons College Library supports a graduate program in LIS that is not only top-ranked and well-established, but also the sole program in the state of Massachusetts, the collection is quite extensive. It includes all research core journals, professional and trade periodicals, popular magazines, and newsletters. This provides the opportunity to develop a complete picture of all the literature that is produced in the field.
At the same time, this will prove to be a challenge. A content analysis of an extensive, research-level collection will be very time consuming. It will require several hours of reading. Dividing up the content as a strategy to address this challenge in turn presents a new challenge. Research assistants, or readers, will all have to be trained to make sure there is a shared, baseline understanding of the definition of research. In addition, an increased number of readers increase the probability of inconsistent classification. It is likely that a large research group will present challenges to the quality of the data collection and recording. The required training time to attempt to address these issues will also add to the total time to complete the study.
The time span covered in the research will help address some of these challenges. It does also in turn present another potential limitation of the study. As a standalone one year study, we are limited in the conclusions that can be stated about the trends in LIS literature. However, the results of one year's worth of analysis will provide valuable information about the percent of research and topics covered. In future studies, a comparison to an analysis of a second year's worth of the collection would add significant value to the findings. A third data set for comparison would allow the opportunity to discover larger trends in the field.
Another strength of this study is its timing. This study is being conducted at a time when resources are readily available in a variety of formats. It is more feasible to analyze an entire body of literature for a given year when there is instant access to materials online when the print materials are missing.
CONCLUSION AND FUTURE WORK
In conclusion, this ambitious content analysis of the Simmons College Library LIS periodicals collection, while demanding, could provide useful information to researchers and librarians. One of the challenges in conducting research is overcoming the unknown. How much research already exists and what is it about? This question springboards the researcher into action. Information about the nature of the literature of the field will enhance the researcher's approach to initial inquiry.
A possible future study could include a parallel comparison of previous years. As Feehan et al., (1987) suggest, this study could lay the foundation for replicate studies to capture a wide picture of LIS research over time. With the appropriate level of resources and support, this study could expand to include an analysis of the collection for the years 2005 and 2015. This would allow for a comparison and understanding of the literature over an entire decade.
An additional study could include a content analysis of LIS literature born digital and available only online. At the time of this writing, only one study was available that measured the quality of an online journal (Beebe, 2003). The methods developed in this content analysis could be combined with Beebe's to develop a tool to analyze online content.
This research was undertaken by the authors to help answer a long-pending question in the mind of the third author. The research would not have been possible without the support and encouragement of Simmons College Library staff.