The application of text mining methods in innovation research: current state, evolution patterns, and development priorities

Unstructured data in the form of digitized text is rapidly increasing in volume, accessibility, and relevance for research on innovation and beyond. While traditional attempts to analyze text (i.e., qualitative analysis) are limited in processing large amounts of data, text mining presents a set of approaches that allow researchers to explore large-scale collections of texts in an efficient manner. Given the potential of text mining as a method of inquiry, the primary purpose of this manuscript is to enable both novice and more experienced innovation researchers to select, specify, document, and interpret text mining techniques in a way that generates valid and reliable knowledge for the innovation management community. This involved taking stock of text mining applications in the field of innovation research to date by means of a systematic review of 124 journal articles employing text mining techniques and are published in a basket of the 10 premier innovation management and 8 top general management journals. The results of the systematic manual and computational analysis of these articles do not only illustrate the state and evolution of text mining applications in our field, but also allow for evidence-based recommendations regarding their future use. Here, our paper presents methodological, conceptual, and contextual development priorities that will contribute to establishing higher methodological standards in text mining and enhance the methodological richness in our field.


Introduction
G rowing quantities of digitized text are available for researchers today. Such texts comprise among others online content from newspapers and social media, company press releases, user reviews about experiences and products as well as scientific articles and discourse. In fact, profound digitization efforts in virtually all industries will certainly continue to increase the variety and quantity of such unstructured data. There is a long tradition of drawing on text from a variety of sources in management research in general and studies on innovation in particular. Scholars have used qualitative approaches to text analysis including manual coding, discourse analytical methods, or grounded theory (Duriau et al., 2007). These manual procedures, however, are labor intensive and seem to have reached their natural limits when it comes to analyzing increasingly large amounts of text material (Jamiy et al., 2015;Kobayashi et al., 2018). Consequently, researchers have started to explore the opportunities that the computer-aided, or automated, analysis of textual data offers (Janasik et al., 2009;Wiedemann, 2013). In simple terms, the case for text mining becomes stronger the larger the text corpus and the less accessible it is to manual content analytical techniques. Whenever the text corpus is amenable to manual analysis, manual coding remains the gold standard. Studies on R&D-and innovation-related phenomena have already begun to take advantage of the potential that the computational exploration of large-scale collections of texts as a method of inquiry promises. As such, innovation scholars developed tools for technology forecasting and road mapping based on fulltext analyses of patents (e.g., Lee et al., 2008a;Choi et al., 2013), found that industries converge using newspaper articles (Kim et al., 2015), investigated the infringement risk of patents (Bergmann et al., 2008), developed patent-based profiles of inventors (Moehrle et al., 2005), and reviewed the innovation research landscape using journal publications (e.g., Antons et al., 2016).
Text mining -a field located at the intersection of computer and information science, mathematics, and (computational) linguistics -promises not only to analyze large text corpora efficiently, but also to do so in a transparent and reproducible manner (Humphreys and Wang, 2018). As such, text mining is said to fuel advances in theoretical as well as phenomenological knowledge relevant to managerial practice (Müller et al., 2016). With regard to innovation research in particular, Nambisan et al. (2017) note that the pervasive diffusion of digital technologies fundamentally changes not only the very nature of innovation, but also how we study innovation processes and outcomes. Calls from different sub-fields of business-related research share the excitement of applying computational methods to better exploit unstructured data (e.g., Agarwal and Dhar, 2014;George et al., 2014George et al., , 2016Chintagunta et al., 2016;Antons and Breidbach, 2018).
It is against this background that the present study seeks to provide the first systematic review of text mining applications in innovation research. We identify and review a set of 124 articles that have been published on an innovation-related topic in a basket of the 10 premier innovation management journals and the top 8 general management journals. Grounded in the systematic manual and computational analysis of these 124 articles, we document the state and evolution of text mining applications in innovation research and derive conceptual, methodological, and contextual priorities for future text mining applications. From a conceptual perspective, we argue that text mining applications in innovation research have now reached a state of maturity where the need for demonstrations and case studies of text mining in innovation research is rapidly declining. We provide scholars with the conceptual background needed to develop research designs that take text mining applications in innovation research to the next level. From a methodological perspective, we identify issues that warrant attention in future studies using large-scale and automated text analysis. Therefore, we provide actionable knowledge on how to employ text mining in innovation research by proposing a set of best practices. Finally from a contextual perspective, we identify areas in the wider innovation research landscape where text mining techniques could help shed light on challenging research questions.
Overall, we argue that innovation scholars have much to benefit from these evidence-based insights into the current state, evolution patterns, and development priorities of text mining applications in innovation research. This will enable both novice and more experienced researchers to select, specify, document, and interpret text mining techniques in a way that generates valid and reliable knowledge for academic research and managerial practice alike. In particular, this paper is meant to empower innovation scholars with no or little previous knowledge in computer-aided text analysis to employ text mining in ways that help them accomplish their respective research objectives. Moreover, our paper can stimulate critical reflection among innovation scholars already experienced in using text mining in view of establishing even stronger methodological standards in future research. Our review of 124 text mining applications demonstrates the value of text mining as a rapidly diffusing methodological approach whose meaningful application can contribute to advancing the field. We believe that our review and guidelines can further enhance the methodological richness of innovation management research and will allow our community to take advantage of the opportunities that the digital transformation of our field offers.

A brief overview of text mining
Text mining has developed over decades and across scientific disciplines to form a now-substantial and diverse body of literature on the computer-aided analysis of textual data. This includes, among others, fields such as statistics, computer science, (computational) linguistics, library science, and computer science (Miner et al., 2012). Moreover, different terminology has evolved including text mining, computational content analysis, and natural language processing. Despite their different emphasis, they share a clear focus on automated, computer-supported processing and analysis of text as a form of natural language. Due to the different traditions and underlying lines of thought of these disciplines, a unifying definition of text timing remains absent. However, there is broad agreement on the general process of analyzing large-scale text data, following the general process of knowledge discovery in databases (Fayyad et al., 1996). We adopt this logic and present Figure 1 as a simplified overview of the process underlying a typical text mining study.
Here, we distinguish between the process phases of data gathering, data preprocessing, content analysis, and integration of the text mining findings and results into the study. Data gathering, i.e., the collection of data from databases and archives or scraping data from websites or social media, is beyond the scope of our review on text mining techniques. 1 We, thus, focus on the steps of data preprocessing and content analysis. Please also note that Figure 1 implies that these activities might be executed in iterations to improve results by refining text preprocessing. As part of our discussion, we then discuss how innovation scholars can integrate text mining findings into their research design, for instance, by combining it with complementary qualitative or statistical analysis.
In what follows, we provide an overview of prevalent techniques used for (1) text preprocessing and natural language processing, (2) dictionary-based techniques to classify words into categories, and (3) algorithmic techniques to classify texts or textual units like sentences or paragraphs into predefined categories (so-called supervised algorithms) or to cluster texts or textual units into homogeneous groups without prior knowledge of those groups (so-called unsupervised algorithms).

Text preprocessing and natural language processing
The unifying theme linking the various text mining techniques is the idea of turning unstructured data in form of text into structured data in form of numbers so that mathematical and statistical algorithms can be applied (Miner et al., 2012). A common approach is representing text documents in a matrix, where each column represents a document and each row represents a specific term. The cells, then, contain the frequency of the term in the respective document. This approach comes with, at least, two major issues: (1) One of the essential characteristics and a potential shortcoming of a representation of textual data in such a document-term matrix is that it neglects the linguistic structure within a document treating it as a 'bag-of-words'. (2) Moreover, large collections of documents can result in a document-term matrix of very large size or, in mathematical terms, a large number of dimensions. To begin to address both limitations, various text preprocessing techniques have been developed in fields such as computational linguistics and computer and information science (Rüdiger et al., 2017). We introduce the underlying logic of these techniques below and refer to the literature in order to guide the interested reader on how to implement these techniques. Text preprocessing usually starts with tokenization, which is the process of converting a sequence of characters into tokens. For most purposes, scholars use words as tokens and separate strings at white spaces. But one could also use sentences or paragraphs as tokens. Using word tokens and converting tokens into a document-term matrix yields the above-mentioned 'bag-of-words'. This simple approach, however, fails to account for the linguistic structure of compound words such as 'innovation culture' or 'open innovation' that jointly have a specific meaning. Specific algorithms are now available (e.g., Wang et al., 2007) to detect such word combinations, also known as n-grams, in text and mark them. 2 Usually, n-grams have to be replaced before constructing a document-term matrix so that the meaning of the compound is not lost. This text preprocessing technique, therefore, addresses the first limitation introduced above.
The second major issue of large, that is, highdimensional document-term matrices results from the considerable variability of human language. For instance, we use language and words in temporal, plural, or otherwise inflected forms. In automated text analysis, this inflates the dimensionality of a document-term matrix, since each term would be added as a separate column (or row) to the matrix. Hence, preprocessing techniques have been developed to reduce such variability while preserving the word meaning. Stemming or lemmatization are techniques that reduce word variability by reducing words to their stem or their dictionary form (the lemma) (Rüdiger et al., 2017). Other techniques involve converting all text to lower cases and removing all punctuation (e.g., Antons et al., 2016;Chae and Olson, 2018), dropping stop words such as pronouns and function words that do not carry any meaning (e.g., 'the', 'and', or 'it') (e.g., Blei et al., 2003). Other scholars further reduce the document-term matrix by removing infrequent words (e.g., Hornik and Grün, 2011), or weigh words using measures such as the term frequency -inverse document frequency (tf-idf) measure (e.g., Wang et al., 2007;Antons and Breidbach, 2018;Chae and Olson, 2018). The tf-idf measure is used to spot those words that discriminate documents from each other. Terms appearing only in a focused set of documents are seen as discriminating while words that are shared across many or all documents are defined as not discriminating. Finally, scholars replace abbreviations in text with fully spelled terminology.
Beyond handling these issues, preprocessing may also add information in form of tags to words or phrases, which might be helpful to choose certain text elements or to structure texts. For instance, named entity recognition identifies those documents from a larger collection that contain names of a certain person, organization, or location. With part-ofspeech (POS) tagging, scholars may tag words based on their grammatical function (e.g., noun, verb, or adjective).
Below, we move from text preprocessing phase to the actual phase of computer-aided content analysis employing either dictionary-based techniques originating from linguistics and psychology or algorithmic techniques from statistics and computer science.

Computational content analysis with dictionary-based techniques
Text mining techniques that rely on word frequency counts to measure contextual, psychological, linguistic, or semantic concepts and constructs are among the most widely adopted approaches for computer-aided analysis of textual data in managementrelated research so far (Duriau et al., 2007;Short et al., 2010). Scholars may either employ existing dictionaries that were developed in previous research or create new dictionaries that would match their needs. From an epistemological point of view, dictionaries can be developed deductively based on existing theory (e.g., Gamache et al., 2015 used theory, construct definitions, and survey items to build a dictionary for the construct of regulatory focus), inductively generated from the corpus at hand (e.g., Henry, 2008 developed a list of positive and negative words used in earnings press releases to measure their tenor), or by combining these two approaches (e.g., Short et al., 2010 for the construct of entrepreneurial orientation). Especially to measure the general positivity or specific emotions in text, validated dictionaries already exist (e.g., Pennebaker et al., 2015). Dictionary-based text mining is sometimes referred to as sentiment analysis or automated content analysis depending on the objective pursued.

Computational content analysis with algorithmic techniques
Algorithmic content analytical techniques originating from computer science and statistics are typically categorized into classification or supervised techniques and clustering or unsupervised techniques.

Classification or supervised techniques
Researchers often seek to assign textual objects like documents or words to predefined categories. For example, Li (2010) was interested in understanding the content of forward-looking statements in the management discussion and analysis section of annual reports. After manually coding 30,000 sentences from forward-looking statements, he used a classification algorithm to categorize the content of 13 million additional forward-looking statements automatically. From a machine-learning perspective, this means that the researcher wants to generate tags. Tags represent a kind of metadata summarizing the category the text belongs in. Prior text mining literature has developed classification procedures that are able to generate tags from pre-categorized textual content. They do so by observing existing tags and textual content to learn the classification scheme that had been applied to the pre-categorized textual content. Based on this learned classification scheme, they suggest tags for new and untagged textual content. Classifiers share the fact that they are supervised, that is, they are trained on pre-categorized data.
A classifier is able to distinguish categories producing a binary outcome, very much like a yes/no decision (Tong and Koller, 2001). A well-known example of such a binary classifier is a spam filter implemented in most of the existing email services. However, other algorithms exist that are able to distinguish multiple outcomes or categories. Broadening the scope of binary classifiers, multiple binary classifiers compare input data against each class (Fung and Mangasarian, 2005). Depending on the algorithm settings, the outcome will be a classification to the class that the new document is most likely to belong to or number of weights per class indicating the likelihood of class membership per class for the new document.

Clustering or unsupervised techniques
The classification procedures described above all rely on so-called supervised algorithms, which require additional information from the researcher (e.g., a pre-classified training data set) as input. Clustering procedures, in contrast, do not require any prior human knowledge when employed for the computer-aided analysis of textual data. They rely on so-called unsupervised algorithms that group textual content based on similarity (Jain, 2010). Clustering procedures are part of the broader category of dimensionality-reduction techniques -just like exploratory factor analysis, a well-known procedure among innovation scholars developing for instance new measurement scales.
In order to meaningfully group documents in a text corpus, most approaches draw on similarity as a measure of distance between documents. When documents are represented in a document-term matrix, distance between documents understood as vectors may then be calculated using common measures such as the Euclidean, Cosine, or Manhattan distance (Ingersoll et al., 2013). Other, non-distance-based algorithms use probabilistic modeling to determine similarity. Here, similarity is calculated as a probability of membership in a cluster.
To work with the clustering result, the clusters obtained typically have to be labeled (Ingersoll et al., 2013). This usually involves a manual process that is based on inspecting documents that are at the core of the final clusters or inspecting most frequent/most distinctive terms and phrases of a cluster. Obviously, these two approaches can also be combined during the labeling process. Clustering of documents or other textual content may be done in a more fine-grained way by adding another layer of analysis. That layer takes words into account and clusters words, rather than documents, based on the assumption that words that co-occur across documents are used to express a certain latent topic. Based on those topics, document similarity can be quantified. This approach has recently been used to review and map published research (Wang et al., 2015a;Antons et al., 2016;Brust et al., 2017;Antons and Breidbach, 2018;Hopp et al., 2018), to summarize patents (Kaplan and Vakili, 2015), and to judge topical newness of research articles (Antons et al., 2019). Table 1 provides an overview of the text mining techniques discussed above with special emphasis on their value proposition to researchers. This typology will serve as an important structuring device during our subsequent systematic review of text mining applications in innovation research. In particular, it will allow us to assign each article identified to one or more of the types of analytical techniques summarized in Table 1.

A review of text mining applications in innovation research
To provide scholars with a structured overview of the state and evolution of text mining application in the field of innovation research, we conducted a systematic literature review (Mulrow, 1994;Tranfield et al., 2003;Denyer and Tranfield, 2009). We proceeded in the following way.

Search strategy and article selection
First, we defined a set of journals that can be considered the top journals publishing innovation research. To do so, we reviewed prior research discussing impactful innovation management journals (e.g., Linton and Thongpapanl, 2004;Thieme, 2007;Goffing et al., 2019) and selected the most impactful from the respective lists. We included the following 10 premier innovation management journals: (1) Research Policy, (2) Journal of Product  (8) the Strategic Management Journal. 3 As our second step, we composed a comprehensive list of keywords authors regularly use to indicate the application of a text mining technique in their article. This list comprised expressions that related to various alternative notations of the methodological domain of text mining (e.g., 'NLP'; 'computerassisted text analysis'; 'textual data mining'), some of the most common approaches (e.g., 'topic models'; 'sentiment analysis'), and specific aspects of their implementation (e.g., 'bag-of-words'; 'text preprocessing'). Table S1 in the appendix provides a full list of our search terms. We then compiled a search query based on these terms and searched the journal archives directly via their official websites. Doing so ensured that our query was performed not only on titles and abstracts but on the actual full texts of all research articles published in the 18 journals identified above.
Third, we downloaded all respective search results and the second and third authors manually inspected them. We selected only those research articles into our final sample that explicitly addressed an innovation-related topic and made substantial use of text mining broadly defined. This selection procedure resulted in 124 innovation research articles that employed text mining as a method of inquiry and were published in one of the 18 journals identified above. For each of the 124 articles, we extracted both meta-data and the full texts for analysis.

Data analysis
We applied a combination of bibliometric analysis and manual coding. 4 We extracted not only the full texts, but also meta-data for all 124 research articles from the Web of Science. Meta-data included author names, titles, keywords, abstracts, and references.
As for the bibliometric analyses, the R-package 'bibliometrix' was used. The package provides several tools for quantitative research models in bibliometrics and scientometrics. The functional scope includes importing and formatting of raw data, the actual bibliometric analyses, as well as the creation of matrices and networks for the visualization of co-citations, couplings, collaborations, and co-work analyses. After importing the data from the Web Of Science in BibTex format, we used the individual meta-information to extract the author keywords and to build a co-occurrence network. Subsequently, adopting the Louvain-clustering method, we visualized the distribution and grouping of the author keywords.
As for the manual analysis, two of the authors and a research assistant independently read the 124 papers and coded them using a standardized coding scheme informed by our overview of text mining techniques presented above. This team extracted from the full texts the type and amount of text analyzed as well as the software or algorithms used for analysis and whether it was proprietary. The team then engaged in manual coding and investigated the type of text mining approach used. Here, we distinguished between techniques of text preprocessing and natural language processing, dictionary, classification, and clustering, which we described above. For dictionaries, we also distinguished between articles using pre-defined dictionaries and those developing their own dictionaries. Some articles included in our sample were coded as using other approaches. These papers, for instance, used simple keyword extraction and illustrated the usage of keywords by means of word clouds or similar techniques. To provide an indication of the content of the papers, the authors coded the articles using the subject areas of the Product Development and Management Association (PDMA) Body of Knowledge (Griffin and Somermeyer, 2007). Many articles related to more than one subject area, so we allowed for multiple assignments. Some articles like reviews even related to all subject areas (e.g., Antons et al., 2016). Finally, we coded the type of usage of text mining using the categories of demonstration (i.e., showcasing text mining or a certain algorithm or software as a research approach), case study (i.e., using text mining to study a certain technological field), review (i.e., using text mining to review a strand of research), and variable creation (i.e., using to operationalize a variable for an econometric model). In case of disagreement during the coding procedure, a third author was consulted. The full coding results for all 124 articles are available in the appendix to this paper (see Table S3).

State and evolution of text mining applications in innovation research
The bibliometric and manual analyses conducted on our corpus of 124 articles allow us to document the state and evolution of text mining applications in innovation research below. In particular, we zoom in into (1) the journal outlets publishing innovation research informed by text mining approaches, (2) the type and amount of textual data processed, (3) the thematic content of the substantive research, (4) the text mining algorithms, (5) the specific outcome of text mining for the research performed, and (6) the reporting quality of the text mining application.

Journal outlets
Although a broad range of journals has already published innovation research using text mining, almost half of the articles (61)

Textual data
With regard to the type of textual data analyzed, we found that 48 articles investigate research papers and 44 analyze patents. Less frequently studied are texts like blog posts and forum entries (12), newspaper articles (7), tweets and other social media entries (5), and product reviews (3). Other texts like annual reports, emails, press releases, interviews have each only been analyzed once. Interestingly, we find a broad variability with regard to the amount of texts that have been used. Of the 124 articles, 58.9% use less than 10,000 texts, 22.6% use more than 10,000 but less than 100,000 documents, 4.8% analyze more than 100,000 but less than 1,000,000 texts, and 13.7% investigate even more than 1,000,000 texts.

Thematic content
To understand the themes the 124 articles sought to examine with the help of text mining, we first used the author-generated keywords of each research article to compute a keyword-network graph illustrated in Figure 3. Here, nodes indicate keywords, node size the number of occurrences in our corpus, and links showing whether terms appear together in scientific articles.
Not surprisingly, we observe that text mining is the most central and important keyword tying together the network. On the left, we see two clusters that are not connected to the other nodes. Here, the light-gray nodes refer to topics from entrepreneurship and opportunity recognition. The purple nodes refer to meaning, cohesion, and semantic concepts. The main cluster composed of the blue-colored nodes mainly covers technical terms like text mining, text clustering, topic modeling, and bibliometric analysis. Similarly, green nodes are also more methodological referring to information retrieval from scientific texts. The light-brown nodes refer to themes like content and sentiment analysis and texts such as user-generated content, product reviews, and mass media. The red nodes seem to cover themes from technology management such as technology intelligence, technology road mapping, patent analysis, and technology monitoring. Finally, the orange nodes represent topics like technological change, adoption, dominant design, and life cycle. The red and green clusters are the largest clusters not composed of merely technical terms. They refer to technology management and retrieval of information from scientific texts and patents. This is in-line with the finding that most text mining applications in innovation research nowadays take advantage of academic publications and patents.
In addition to the bibliometric analysis of author keywords, we relied on our manual coding of the article content against the PDMA subject areas to map the topic landscape of text mining applications in innovation research. Figure 4 reveals that most studies are part of the larger subject area 'strategy, planning, and decision making'. As we have seen, many studies analyze patents to monitor technological improvements and development or to develop technological road maps informed by academic literature. This clearly falls into the category of supporting strategic technology management and decision-making. It is noteworthy that only few studies are categorized as 'co-developments and alliances' or 'people, teams, and culture'.

Text mining algorithms
Although a broad range of algorithms for automated text analysis exists, most articles in our dataset apply some clustering technique. Even though the number of articles in our corpus sharply increased in recent years, the increase in clustering articles was disproportionately stronger. Figure 5 depicts this trend. Interestingly, we can also see that innovation scholars have started relatively early to develop custom dictionaries and are doing so until today to a greater extent than using predefined dictionaries.

Text mining outcomes
Text mining algorithms have been used for all four outcome categories defined in our coding scheme (demonstration, case study, review, variable). Figure 6 tracks the evolution over time. First applications focused on demonstrating the applicability and value of text mining algorithms or software. Surprisingly, until today, most articles still fall in this category of demonstrations. In 2007, scholars started using text mining techniques for reviewing academic literature as well as for informing domain-specific case studies, for instance on a particular field of technology. While the number of reviews only recently started to increase, case studies are nowadays the second most frequent type of text mining application. Only since 2011, innovation scholars have started to use text mining techniques to measure specific constructs and variables. This category has experienced a sharp increase in recent years. Interestingly, of the 18 articles using text mining techniques to measure a variable being part of a larger (econometric) model, six appeared in the premier general management journals included in the prestigious Financial Times 50-journal list. In addition, two studies appeared in Research Policy and four in the Journal of

Reporting quality
For text mining to meaningfully enhance the methodological repertoire of innovation research, its application has to be documented in a detailed manner to ensure transparency and replicability of studies. Against this backdrop, we found that of the 124 articles, roughly 10% do not report the size of the dataset they are using and 8% do not indicate the source of their data. We also found that 35 articles do not specify the software they used. Of the 89 articles reporting the software employed, 57 apply a proprietary software package and 32 use free software or packages and functions available for free programming languages like Python or R. As described above, text preprocessing is a major step in text mining preparing the text for analysis.
As any data cleaning and alteration might affect outcomes of analysis, scholars are well advised to document these steps in detail. We find that 56 articles do not describe the data preprocessing conducted.
Only 55 articles describe their preprocessing in detail and 13 at least reported the preprocessing techniques employed. As Table 2 indicates, 19 studies neither report the software used nor the preprocessing techniques applied. This limits the replicability of these studies. Interestingly, we find that of the 37 studies that report software but do not report preprocessing, 28 apply proprietary software. It seems that some proprietary packages impede introspection into their procedures limiting their scientific value. While 42 studies report both software as well text preprocessing in detail, only 37 of those also report the source and size of the dataset used. This means that only 29.8% of all articles can be regarded as fully transparent to the reader. Figure 7 depicts the cumulative number of fully transparent articles published over time. Although it is good to see that the articles that are fully transparent to the audience have increased sharply in recent years, still the majority of articles published in recent years continues to lack in transparency and, thereby, replicability. As Figure 8 shows, the standards of reporting preprocessing have increased in recent years. That said, studies continue to be published with insufficient transparency  Articles with fully transparent approach on data preprocessing. Finally, we see in Figure 9 that scholars are increasingly using non-proprietary software to conduct text mining. Here, programming languages like R and Python are mainly used offering the advantage of making the program code fully available to reviewers and fellow scholars.

Priorities for future text mining applications in innovation research
Based on our systematic review of the state and evolution of text mining applications in innovation research, we delineate conceptual, methodological, and contextual priorities for future innovation research applying text mining. We believe that these research priorities are of interest to both novices to text mining as well as experienced users. Novices may become acquainted with the state-of-the-art and readily implementable algorithms to kick-start their research projects. Experienced users, in turn, might draw on insights into the main limitations and challenges of current text mining applications to help raise the standards for rigorous and reliable text mining applications in innovation research and beyond. From a conceptual point of view, we argue that innovation scholars now have suitable exemplars at hand that showcase the meaningful application and value of text mining in innovation research. As we have seen in recent years, the number of studies that use text mining as a means to the end of generating variables for inclusion in subsequent statistical models sharply increased. For instance, Kaplan and Vakili (2015) used a topic model to generate a textbased measure of knowledge recombination that they subsequently incorporate as an independent variable into their econometric model. Similarly, Antons et al. (2019) applied LDA to generate measures of topic newness to quantitate a possible citation premium of topic newness. Angus (2019) computed document similarity measures to investigate the link between search distance and firm performance. Finally, Bednar et al. (2013) investigated how media coverage of a firm affects strategic change. We argue that this trend towards applying text mining techniques to operationalize new, previously unexplored variables or to improve the measurement of existing variables instead of simply demonstrating the techniques' usage or providing a case study can be interpreted as a sign of methodological maturity. This argument is further supported by the fact that 12 out of 18 articles operationalizing variables by means of text mining appeared in the most prestigious journals, as we have shown before. Even though the application of text mining as an innovative measurement approach is still in its early days in innovation research, we argue that it has much to offer to innovation scholars and their research endeavors.
We propose that adding text mining to an innovation scholar's toolbox allows them not only to harness the value of larger bodies of textual data, but also to increase the objectivity and reproducibility of empirical findings based on textual analysis. Text mining somehow bridges qualitative and quantitative research traditions by structuring, analyzing, and understanding textual data at scale. This link to both qualitative and quantitative research traditions is also evident in the discussion on quality criteria of text mining (e.g., Yu et al., 2011). Those text mining algorithms not including probability measures will perform in a consistent and repeatable manner on the same text as well as across texts. When the corpus and the settings of the algorithms are held stable, re-running the algorithms should yield the same results. Hence, text mining techniques are likely to produce more consistent results than human coders, especially with a growing amount of data (Yu et al., 2011). Moreover, it is possible to check the consistency of different algorithms that were designed for the same task in order to see whether they produce comparable results (McKenny et al., 2016). Going beyond reliability, it is necessary to ensure construct and criterion validity. This is of particular importance in studies testing theory and hypotheses in order to avoid both Type I errors of rejecting true hypotheses, and Type II errors of failing to reject incorrect hypotheses. Especially for those techniques that quantitate the content of text documents and whose measures are used as variables in quantitative models, it is necessary to establish construct validity. Prior research has shown that it is possible to do this by using measures derived from text mining (Short et al., 2010).
From a methodological perspective, we argue that future scholars are well advised to follow the best practice of studies that are fully transparent to readers. For us, this means that scholars need to disclose (1) the  type of texts used, (2) the source from which this data were drawn, (3) the amount of data analyzed, (4) the software used to run preprocessing as well as analyses, (5) the techniques that were used to preprocess the data including detailed descriptions of how this affected the textual basis, and (6) the kind of algorithm(s) used to analyze the text including, if applicable, choices made to fine-tune the algorithm by means of setting methodological parameters. Only such fully transparent studies (e.g., Wang et al., 2010;Kaplan and Vakili, 2015;Antons et al., 2016Antons et al., , 2018Hopp et al., 2018) enable future replication studies and the cumulative development of knowledge in our field. As part of our review, we found that scholars are increasingly using free software like R and Python and use available packages and functions to run text preprocessing and text mining algorithms. We see this as particularly advantageous, since this puts the researcher into the driver's seat. Compared to some proprietary software, this allows for full introspection of how data are handled and how analytic algorithms are used. Moreover, it enables researchers to be fully transparent to fellow scholars and reviewers by making data and code of the analysis available by means of online repositories. However, it comes at the cost of learning the required programming skills and of becoming familiar with the specific text mining packages available. To ease the process for innovation scholars, we prepared an appendix that lists popular text mining packages for R and Python by type of analysis to be performed (see Table S2).
This leads us to the contextual question of identifying the fields of innovation research for which text mining applications for variable measurement and inclusion in subsequent models appear particularly promising. Texts like product reviews, patents, customer interviews, and product preannouncements in form of press releases have always been a vital source for innovation research. As it is almost impossible to craft an all-embracing and coherent picture of research questions that can be answered by using text mining to analyze such texts, we outline fields of research that are based both on our experience of using text mining in innovation research and are particularly salient or trending in the innovation research landscape (e.g., Antons et al., 2016;Antons and Breidbach, 2018). Table 3 provides a non-exhaustive list of six exemplary research themes at the heart of our field of innovation research that we argue could benefit from the meaningful use of text mining techniques. This includes (#1) the development of novel measures for central, yet so far difficult to capture innovation concepts such as novelty using clustering techniques; (#2) the granular mapping of innovation trajectories in scientific and public discourse using custom dictionaries applied to media data; (#3) the scalable categorization of proposals (e.g., for grants, technical solutions, patents, articles) as exploratory or exploitative using classification techniques based on text-based distance measures; (#4) the fine-grained assessment of text content and sentiment (e.g., in new product announcements) as predictors of subsequent adoption using pre-defined dictionaries for positivity and/or negativity; (#5) the analysis of customer reviews on product updates using clustering and dictionary-based techniques, and (#6) the link between topic and language characteristics in product descriptions (e.g., on kick-starter) and the subsequent product success.
Overall, many of these examples illustrate distinct computational techniques can be combined to generate promising answers to discovery-oriented or theory-guided research questions. An analytical strategy can hence consist in a combination of multiple text mining techniques conducted in sequence (see #2, #4) and/or a combination of text mining techniques with econometric analyses (see #3, #6).
To find additional sweet spots for research applying text mining methodologies, we also refer the reader to Figure 4. Here, we see that some fields in innovation research, as represented by the PDMA subject areas, have not applied text mining as other areas or do not show a strong trajectory. As a case in point, research on networks, alliances, and ecosystems could benefit from using text mining. Scholars could use text mining techniques to mirror capabilities of firms active in an ecosystem by analyzing changes in their patent stocks and scientific publications to see whether these changes are associated with competitive advantages of these ecosystems.

Conclusion
Text mining, that is computer-aided analysis of textual data, offers a great opportunity to advance scholarship in innovation management. This motivated us to provide innovation scholars with an overview of available text mining methods, a systematic review of the state and evolution of text mining applications in innovation research as well as a set of priorities for those considering to apply text mining techniques in their own research. This included guidance on available R and Python libraries for data preprocessing, dictionaries, classifying and clustering. Our intent was to inform and to empower scholars to implement text mining in their research in a rigorous way, even if they are new to this methodological field. Overall, this review and the research priorities we delineate can facilitate the meaningful use of text mining by both novices and experienced scholars and R&D Management 2020 17 contribute to the methodological richness in our field of innovation research and beyond.