Current trends in flow cytometry automated data analysis software

Automated flow cytometry (FC) data analysis tools for cell population identification and characterization are increasingly being used in academic, biotechnology, pharmaceutical, and clinical laboratories. The development of these computational methods is designed to overcome reproducibility and process bottleneck issues in manual gating, however, the take‐up of these tools remains (anecdotally) low. Here, we performed a comprehensive literature survey of state‐of‐the‐art computational tools typically published by research, clinical, and biomanufacturing laboratories for automated FC data analysis and identified popular tools based on literature citation counts. Dimensionality reduction methods ranked highly, such as generic t‐distributed stochastic neighbor embedding (t‐SNE) and its initial Matlab‐based implementation for cytometry data viSNE. Software with graphical user interfaces also ranked highly, including PhenoGraph, SPADE1, FlowSOM, and Citrus, with unsupervised learning methods outnumbering supervised learning methods, and algorithm type popularity spread across K‐Means, hierarchical, density‐based, model‐based, and other classes of clustering algorithms. Additionally, to illustrate the actual use typically within clinical spaces alongside frequent citations, a survey issued by UK NEQAS Leucocyte Immunophenotyping to identify software usage trends among clinical laboratories was completed. The survey revealed 53% of laboratories have not yet taken up automated cell population identification methods, though among those that have, Infinicyt software is the most frequently identified. Survey respondents considered data output quality to be the most important factor when using automated FC data analysis software, followed by software speed and level of technical support. This review found differences in software usage between biomedical institutions, with tools for discovery, data exploration, and visualization more popular in academia, whereas automated tools for specialized targeted analysis that apply supervised learning methods were more used in clinical settings.


| INTRODUCTION
Flow cytometry (FC) is an important analytical technique for singlecell population identification and characterization. It is widely used within biotechnology, pharmaceutical and clinical laboratories, and biomanufacturing spaces. Reproducibility and rigor in results are very important, driven by the needs of regulators around the world, however, a major source of variation in FC lies within data analysis [1].
Conventional FC data analysis involves sequential manual selection (gating) of regions of interest typically in two-dimensional scatter or contour plots, viewing different combinations of parameters as axes.
The analysis is straightforward with three-to four-color immunofluorescence data but becomes significantly more complex when examining an increasing number of cellular markers, leading to increasing human operator variation and issues of reproducibility [2,3]. Current state-of-the-art flow cytometers are capable of measuring over 40 parameters, generating challenging complex, and time-consuming multidimensional data sets for manual analysis [4][5][6].
The past decade has seen a growth in the field of computational FC as researchers become increasingly motivated to solve the process bottlenecks, and reproducibility issues in manual gating, and improve standardization in immunophenotyping [7]. New automated data analysis software packages have emerged, making use of a range of different machine learning and clustering algorithms to replicate or aid manual data analysis tasks such as; data preprocessing, cell population identification and enumeration, feature extraction, and sample classification [8]. Visualization of data processed through algorithmic analyses is an essential aspect of analysis workflows, and is often embedded in the automated analysis itself, therefore making the dis- mapping of high-dimensional data to two-dimensional scatter plots representing data similarities, with color-coded cell clusters.
These data-driven automated algorithms have been demonstrated to improve the quality of flow cytometry data compared with centralized manual analysis, with potential benefits in lower technical variability in certain cell populations, reduced bias, and better efficiency [9]. Given the proliferation of such algorithms, verification methods to ensure correct choice would be recommended. It would be sensible for all users to contextually develop their own robust testing measures for automated analysis. However, this raises subjectivity issues if testing was based on users' own biological knowledge, compounded by the fact that there are no common toolsets to achieve this apart from real-world data sets which do not necessarily have an absolute cell count, and are inflexible compared with the potential of synthetic data.
Typical workflows in computational cytometry can be divided based on tools used for discovery versus targeted analysis, that is, the detection of unknown, novel cell populations compared with known well-defined ones. In both contexts, automation can help to reduce variability in the data analysis process. In discovery mode, automated tools can help uncover cell populations overlooked in sequential manual gating strategies, such as cells gated out in earlier steps. The value of automated tools in discovery mode is especially notable in facilitating the interpretation of high dimensional (>30) data, as the data can be reduced and visualized in two dimensions. These tools assist with the data exploration process, help to give an overview of the structure of the data, identify relationships between variables and offer novel insights. For comparison, in targeted analysis mode, the cell populations of interest are well characterized, the data analysis process follows a standard protocol that is likely to be validated and approved, for example, in clinical flow laboratories carrying out high throughput screening; measurement of clinical trial endpoints for hematological malignancies. The benefit of automated tools here may be in reducing the workload on users by automating the classification of healthy or disease cases, only flagging up uncertain cases for manual interpretation, thereby speeding up the data review process.
As the number of automated software increases, comparison studies have become important to provide guidance for users to determine which software to use for their analysis, and to evaluate the performance of the software. The flow cytometry: critical assessment of population identification methods (FlowCAP) consortium initiated a series of open challenges to objectively evaluate these new computational methods [10,11]. FlowCAP provided benchmarking data sets to critically assess performance in population identification and sample classification tasks and used the F-measure (the harmonic mean of precision and recall) to rank the algorithms. These rankings helped inform potential users on the quality of automated methods based on different tasks. FlowCAP demonstrated certain automated methods were able to reliably replicate manual gating.
Several other recent comparison studies have evaluated selected unsupervised clustering methods in their abilities to reproduce manual gating, detect rare cell populations and their runtimes. Among those, one study [12] identified FlowSOM [13] as the best performing clustering method along with the fastest runtimes. X-shift [14], PhenoGraph [15], Rclusterpp, and FlowMeans [16] were also mentioned to perform well across six high dimensional data sets. A separate study [17] assessed FLOCK, SWIFT, and ReFlow on their ability to detect low-frequency populations compared with central manual gating. SWIFT was found to outperform the others in terms of the identification of populations <0.1%. This study noted the difficulties in implementing a fully automated workflow without human intervention. In addition, one study [18] evaluated the reproducibility and robustness of results based on the cluster stability using the Jaccard coefficient as the performance metric. PhenoGraph was observed to generate the highest proportion of stable clusters compared with SPADE1 and FLOCK.
Despite these recent benchmarking studies, uptake of automated analysis among academic, biotechnology, pharmaceutical, clinical laboratories, and contract research organizational researchers has been slow and manual gating remains the default method and standard.
Manual analysis can be performed on instrument-packaged software (e.g., Becton Dickinson FACS Diva, BD FACS Canto, Beckman Coulter Navios) or stand-alone FC analysis software (e.g., FlowJo, FCS Express, Kaluza, VenturiOne). The primary reasons for clinical centers not employing automated analysis were recently cited as being a lack of trust/understanding and lack of resources [19]. In this regard, this novel analysis of automated software provision and use presented here is intended for researchers and process operators familiar with

| SEARCH STRATEGY
The goal of this research was to understand current trends in automated data analysis software, the characteristics of these software, and identify which software were the most popular (although this is not a measure of most effective software). Software mentioned in recent reviews [10,12,17,18,20] were included. In addition, the Web of Science (WoS) database was searched using the following keywords; flow cytometry, automated, analysis. Using this search strategy, 89 software were identified from recent reviews and 108 publications were returned from the WoS database, typically output from research, clinical and biomanufacturing facilities. The WoS search strategy was designed to be as comprehensive as possible, although some tools may have been missed due to the fragmented nature of the field, such as FLOW-MAP force-directed graphs [21] and scaffold maps [22]. Use of additional keywords such as "computational" may have highlighted more software, however, in practice, the records retrieved from the database were either too restrictive with the AND Boolean search operator, or excessively broad with the OR search operator. After removing duplicates, the software identified in the search were refined based on the following specifications.
Inclusion criteria: • Software is detailed in a publication from a peer-reviewed journal.
• Software for flow cytometry or mass cytometry.
• Software for automated cell population identification (gating).
• Software intended for identification of human or mammalian cells.
• Software source code is available, or the program is made accessible by authors.
Exclusion criteria: • Software lacking publication from a peer-reviewed journal.
• Publication type: conference proceedings, reviews, editorial material, book chapters. This exclusion criteria were applied in order to capture work that applied the data analysis software rather than just citing their use.
• Software unrelated to flow cytometry or mass cytometry technique.
• Software solely for automated data preprocessing, compensation, transformation, or other quality control feature.
• Software unrelated to the identification of human cells (e.g., beads, phytoplankton, bacterial identification) to focus the scope on cell therapy and medical applications. The number of software matching the criteria was refined to 51.
Once shortlisted, software popularity was ranked according to the number of article citations. The sum total of the number of citations across all 51 software was 2027. Citing articles were refined to those matching "cytometry" as a keyword, included articles, and excluded conference proceedings, reviews, editorial material, and book chapters.
Additional software would have been identified if the search strategy were broadened to include automated single-cell analysis approaches from other technologies (e.g., RNA-sequencing analysis software in genomics, single-cell imaging, single-cell proteomics), and indeed many tools are transferable between different omics domains, however, this was beyond the scope of this work.

| GENERAL FINDINGS AND TRENDS
As of the end of 2019, this search strategy has been completed several times on an annual basis and has currently identified 51 automated flow cytometry software (Table S1). The earliest software was released in 2008 and subsequent years saw the number of different software released ranging from 1 to 6 per year, except for 2014 when a peak of 11 software were published ( Figure 1A). When considering the country of origin, the USA has led the development with 29 software, followed by Canada with six software. Outside of North America, some European studies have come from The Netherlands, Belgium, France, and Germany (4, 2, 2, and 2 software, respectively).
Australia and Singapore have also produced two apiece ( Figure 1B).
The environment in which users interact with the software range from basic command line inputs to full graphical user interfaces (GUI).
This survey found 41% of software could be accessed with a GUI, compared with 59% without GUIs ( Figure 1C). A caveat here is that although most likely to have GUIs, as identified in section 2.0 proprietary computational tools lacking peer-reviewed publications and with unavailable source code were excluded from our survey. Many of the tools were available in multiple programming languages, offering FC analysts a choice of integrated development environments. This survey found 59% of the software were available in R, 29% in Matlab and 18% in Python ( Figure 1D).

| Most used software
The findings from the literature survey revealed the top five most cited automated data analysis software based on the search criteria and exclusion criteria were: viSNE, SPADE1, t-SNE, PhenoGraph and FLAME (Table S1). To balance out the effect of earlier software releases accumulating more citations over time, the number of citations were averaged over the number years in publication leading to an adjustment of the highest citation rates; viSNE, PhenoGraph,  [24]. The same software can be implemented and be available on more than one platform. Cost does not appear to be a deciding factor for users, because the most cited software were accessed through paid platforms ( Figure 3B). The levels of usability and software support provided typically increase in line with cost.

| Supervised learning methods
Supervised learning methods aim to solve classification and regression problems. These algorithms require training data with known outcomes to learn from, in order to build a model to classify new inputs.
In practical FC applications, manually annotated cell populations associated with healthy or diseased patients could be used as training data. Cell marker expression features that correlate with the two outcomes would be extracted from the data and then a model built to classify the disease status of new samples.
The limitation of these methods is that the algorithm is only as good as the training data sets available for it to learn from, and it is also possible to overstrain a learning algorithm. Furthermore, there are insufficient publicly available training data sets for all possible scenarios in clinical settings, especially those focused on rare cell identification. The FlowCAP-II sample classification challenge used three real-world patient data sets, half of each data set (training set) was labeled with patient clinical outcomes and the challenge was to correctly label the other half (test set). The comparison study found many algorithms achieved perfect classification accuracy on two data sets (acute myeloid leukemia detection and HIV vaccination antigen stimulation groups), but all performed poorly on a third (HIV exposure on African infants) [10]. Because the current number of supervised learning software in FC data

| Unsupervised learning methods
With unsupervised learning, no training data set is needed, and the goal is to correctly identify and quantify cell populations in FC data.
Automated gating of cell subtypes is viewed as a clustering problem.
The unsupervised learning software in this survey apply different clustering methods such as hierarchical clustering, partition clustering, model-based clustering, density-based clustering ( Figure S1).
Dimensionality reduction is also used to simplify multiparameter data sets. Below is a brief overview of the most frequently used clustering algorithms. For a comprehensive survey of clustering algorithms, see Reference [42].

| Hierarchical clustering
Hierarchical clustering has two strategies to group similar datapoints together, agglomerative, and divisive [43]. The agglomerative method follows a bottom-up approach, where neighboring datapoints are merged to form sequentially larger clusters, until only one cluster remains.
The divisive method follows a top-down approach, starting with the The second most frequently cited software in this survey, SPADE1, applies agglomerative hierarchical clustering in its algorithm [44]. A prior density-based down-sampling step is performed to equalize low density populations with high density ones. Down-sampling reduces the time complexity of the hierarchical clustering step, and also increases the prevalence of rare cell types and noise events. The SPADE1 algorithm overcomes the problem of selecting the number of clusters by overclustering the data set (e.g., instead of three nodes, set 100 nodes). The algorithm builds a minimum spanning tree (MST) from the clustered data, and then relies on expert operator manual analysis to partition the MST to determine correct number of cell populations. An improvement on the SPADE1 algorithm, SPADE3, has been released to remove the stochastic nature of the original agglomerative algorithm by implementing a deterministic K-means clustering algorithm, and to introduce a semiautomated interpretation of the MST [45], thus creating a new software (albeit with the same name) with different mathematical definitions and characteristics, and potentially different data analysis outcomes. In addition to these algorithmic differences between the versions, SPADE3 is primarily implemented in Matlab although stand-alone executable code does exist, SPADE1 and its updated version SPADE2 (better GUI and runtimes) are implemented in R and are available on Cytobank and as a plugin on FlowJo.

| K-means clustering
The K-means clustering method was first published in 1955 and is one of the most popular clustering algorithms used in pattern recognition [46]. K denotes the number of clusters, which is user defined. The K-means algorithm begins with K seed points randomly scattered in the data set acting as cluster centers. Neighboring datapoints are assigned to their nearest seed to form the initial clusters. The center of the clusters, the centroid, is calculated and repositioned. The algorithm repeats the assignment of datapoints to the updated centroid, and then updates the centroid, and so on. Further iterations to update the clustering are performed until cluster membership stabilizes. K-means is an efficient algorithm, with faster run times compared with hierarchical and model-based clustering. However, the drawbacks are its requirement for a predefined number of clusters, its limitation to spherically shaped data and sensitivity to outliers.
These are key issues that need to be addressed for correct analysis of FC data, which are usually non-convex shaped and noisy.
The software flowMeans [16] and flowPeaks [47] are based on Kmeans clustering, and attempt to solve these limitations of K-means clustering on FC data by over-clustering the data then merging nearby clusters to obtain a single population. flowMeans applies a change point detection algorithm to detect the number of clusters, whereas flowPeaks fits a Gaussian finite mixture model to the initial K-means clustered data then generates a density function to search and merge peaks. The results successfully identify nonspherical cluster shapes, however, rare clusters remain difficult to uncover.

| K-medoids clustering
K-medoids clustering, also known as partition around medoids (PAM), is similar to the K-means method, intending to partition the data set into K clusters, but instead of using centroids (the mean of the datapoints in a cluster) to assign nearby objects, K-medoids uses the representative object of a cluster with minimal average dissimilarity to its assigned objects [43]. K-medoids is less sensitive to outliers than K-means, however, its main disadvantage is the high computational cost for analyzing large data sets. Sampling of the data set is one strategy to reduce runtimes (CLARA) [43]. A modified version of PAM has been proposed for use in a clustering analysis pipeline to identify cell populations [48].

| Density-based clustering
Density-based clustering algorithms such as DBSCAN (density-based spatial clustering of applications with noise) [49] and OPTICS [50] views datapoints in high density regions as clusters, separated by regions of low density. Density-based clustering identifies core points belonging to a cluster as well as noise points. These algorithms are intended to discover clusters of arbitrary shape, such as geographical data. Key requirements are a threshold for the minimum number of points in a neighborhood and an arbitrary distance measure for the density-reachability of a point to a core point. Since the number of clusters is not a required input parameter, this method is useful for FC data analysis where the number of cell subtypes is unknown. Generically, density-based clustering algorithms appear to be a widespread strategy for software developers to identify cell populations, and are used by some software: ACCENSE [51], DensVM [25], Flock [52], flowDensity [26], Misty Mountain [53] and others [54][55][56], noting that mathematical implementations and algorithms may vary depending upon the data analysis approach.

| Model-based clustering
Model-based clustering assumes the data follows a statistical distribution and models this onto the data set.

| Spectral clustering
Spectral clustering is based on graph theory where each datapoint represents a node, and the edges are weighted based on a similarity criterion. Clustering is achieved through graph partitioning [71]. Spectral clustering is used by the software SamSPECTRAL [72] which includes a subsampling step to reduce runtimes. Wanderlust also applies a graph-based representation of data in its algorithm [73].

| Self-organizing map
The self-organizing map (SOM) is based on a model of neural network learning [74]. The premise is to construct a grid and map random datapoints one at a time onto each node of the grid. The grid selforganizes so that neighboring nodes have greater similarity, and less similar nodes are moved further away. The next input datapoint is applied to the node that matches best with it. In the end, a large high dimensional data set is reduced to a low dimensional space while retaining the global structure of the original data [75]. The resulting SOM can be clustered further to group similar nodes, using traditional methods such as hierarchical agglomerative clustering and K-means clustering [76]. The FC data analysis software FlowSOM builds a minimal spanning tree from the SOM, followed by a consensus hierarchical clustering step to give the expected number of cell types [13].

| Dimensionality reduction
Dimensionality reduction is not strictly a clustering method. The idea is to take data containing multiple parameters and reduce it to (usually) two dimensions which can be easily interpreted. Principle component analysis (PCA) is an established dimensionality reduction method, however, newer algorithms such as t-stochastic neighborhood embedding (t-SNE) are a significant improvement that preserves (to a limited extent) both the local and global structure of the highdimensional data, and generates a visual map of the data where similar points are clustered together [23]. Albeit very large data sets (>10 6 events) can cause crowding in the layouts that limit meaningful interpretation of the data, and runtimes are slow [77]. The t-SNE algorithm and its implementation in viSNE successfully visualizes a variety of large real-world data sets and appear well suited to analysis of large multidimensional FC data [78]. This is reflected in their overwhelming popularity in this survey with viSNE and t-SNE ranking first and third respectively in the software citation analysis, and their numbers combined make up 24% (488 out of 2072) of all citations. Dimensionality reduction is increasingly being used as the first step of a data analysis pipeline to extract initial clusters, followed by a clustering step to identify cell populations [79].
The benefits of data visualization and interpretation following dimensionality reduction have encouraged further development of similar data analysis tools that improve scalability, runtimes and are better able to handle large (>10 6 ) data sets and represent the global structure. These tools include hierarchical stochastic neighbor embedding (HSNE) [80], PHATE [81] and uniform manifold approximation projection (UMAP) [77].

| Preprocessing tools
Although excluded from this study, automated preprocessing tools play an important role in FC data analysis because they enable highquality input data for all the analysis approaches mentioned above.
Manual gates that exclude doublets, debris and dead cells can be imported from FlowJo into R using flowWorkspace [86], and these manual gates can also be automatically replicated using flow-Density [26].
In summary, the popularity of FC automated data analysis soft- The online survey of eight questions (Table S2) was developed to expand on the literature review to understand the potential use of automated software in clinical laboratories.

| Survey results
The survey received 49 responses out of 310 potential respondents, a response rate of 16% which is consistent with typical response rates of 15%-20% from email invitations to participate in online, non-incentivized surveys [87]. The quality of respondents is high because of the targeted nature of the survey to subscribers of an EQA programme.
Although conclusions from 49 responders should be carefully consid- The survey asked participants to identify which automated data analysis software they used ( Figure 5B). Nine software platforms were identified among the 16 respondents who used automated software, the most frequently identified of which was Infinicyt (63%).
The survey also asked participants to identify software they were aware of but do not currently use ( Figure 5C) The results gathered from this question suggest many laboratories were aware of what software was available but perhaps have not had the time or resources available to validate and implement changes to a manual gating protocol to incorporate automated analysis. It is also possible that laboratories first consider the many software packages available before committing to purchase only one software package, such as Infinicyt. Furthermore, software selection may be partly influenced by common consortium recommendations or EQA schemes.
To understand the factors which users consider important when using automated software, survey participants were asked to grade the importance of factors along a 5-point scale from "not important at all" to "extremely important" (Figure S2A,B). Results from this question revealed the most important factors for users was the software data output quality, followed by software speed, and the level of technical support. Of lesser importance, scored in decreasing order, were factors such as compatibility with other software, cost, software reputation, software availability, and, seen in the literature. The appearance of software was the lowest ranked importance factor in the survey.
To further understand the user interaction with automated software and the potential impact this has on software selection, development and data quality, the survey participants were asked in Question 8 to assess the software they were most familiar with by responding to 10 usability statements on a 1 to 5 score scale from "strongly disagree" to "strongly agree." The statements are based on the System Usability Scale (SUS) and are designed to provoke extreme disagreement or agreement among all respondents [89]. Statements that commonly lead to strong disagreement alternate with those that lead to strong agreement, to prevent response biases. This arrangement allows calculation of the SUS score, where (a) the score of each oddnumbered statement minus 1, and (b) the score of each even-numbered statement taken away from five, are summed then multiplied by 2.5 to obtain a score out of 100, with higher scores indicating better usability. Scores for individual statements are not meaningful on their own and need to be taken together to give a measure of the overall software usability.
This question received six responses ranking 5 software ( Figure S2C). From the individual surveys, AutoGate, FACS Canto and FlowMerge received SUS scores above 70, therefore were judged to have "acceptable" usability based on the benchmark provided by Bangor et al. [90]. Compass received a SUS score below 70, indicating "marginally acceptable" usability. Infinicyt received a SUS score below 50, falling into the "unacceptable" region. To our knowledge, this is the first application of the SUS to quantify the usability performance of flow cytometry automated software.
While the number of responses to this question were too low to draw conclusions from, it was interesting to note that the most identified software among the survey was also the least userfriendly, and as we anticipate the field of computational cytometry to mature and for user uptake to increase, these initial SUS scores  [92,93]. Application of these Infinicyt tools are optimized to samples acquired following fully standardized EuroFlow standard operating procedures, reagents, instrument settings, and eight-color antibody panels for hematological malignancies [94]. The database-guided tool has been shown to successfully classify acute leukemia cases using a database constructed from 656 patients [93].
The software is also designed to be integrated with a laboratory infor-  and hence users make more use of automated tools that support discovery and exploratory research.
The standardized data sets produced across clinical settings with the same experimental parameters, and crucially linked with specific patient outcomes, can be grouped to build a large database collection that allows for their use as training data sets for the development of supervised learning algorithms. In comparison, the academic space is less likely to have a large and diverse resource of labeled data to use for training purposes, and therefore is dominated by use of unsupervised learning methods. Overall, there is no "best" method.
The most suitable automated analysis tools to use will be context dependent, on factors such as cell type, the data structure, and the purpose of the analysis. The best case is to provide users with complete details of how tools work, for them to make a well-informed decision. This may call for additional benchmarking methods/results from a wider selection of data sets.
More than half of the respondents from the clinical survey never use automated analysis tools and only use manual gating protocols, suggesting barriers to adoption of software are widespread.
The questionnaire gave an insight into the clinical users' software preferences when incorporating automated workflows into their data analysis. High value was given to the data output quality, speed of software and level of technical support. The low take-up in automated software may be down to shortcomings in all three factors in the current software available. The most critical factor, quality of the data, is a major driver for the use of automated software. Tools that aid rigor and reproducibility are expected to be welcomed, so it is intriguing that adoption rates are low, but it may be down to human sentiment and trust in manual methods.
With respect to the speed of software, because results need to reported in a timely (or possibly urgent) manner for clinicians to make decisions on patient treatment strategies, the analysis time needs to be in the order of seconds and minutes rather than hours and days. Current automated software may not offer significantly faster gains in analysis times over manual analysis that would incentivize uptake. Finally, better documentation in the form of detailed user manuals, video tutorials, and troubleshooting guides would increase the level of technical support, and make automated analysis more widely used.
Regulatory requirements are a possible factor for the low uptake more complex panels will be keener adopters of automated software that offer more efficient, scalable and unbiased analyses.
The awareness of new tools can be more dated among the clinical workforce because day-to-day sample processing demands reduces the time available to keep up to date with the latest literature.
There are now trends for academic users to acquire programming skills in R, Python and Matlab to keep up with data analysis requirements. This is a less likely scenario in clinical laboratories and may be the reason for the lower uptake of tools that are executed in those programming environments.
To a certain degree, usage of these tools relies on the efforts of commonly used stand-alone software packages (e.g., FlowJo, FCSExpress) to implement automated tools as plugins integrated into their GUIs. The skills shortage presents a risk to employers, whether to train up staff to be knowledgeable in coding but lose that tacit knowledge when they leave the company, or to buy in a ready-made software with full GUI that does not require specialist training and is easy to learn for new users. Indeed, this study has shown a user preference for tools with GUI. The implication could be for high performing software without a GUI losing ground to lower quality but easier to use software.
In this paper, we have investigated the current usage trends and popularity of automated flow cytometry data analysis software. However, it is worth emphasizing that the popularity of a tool does not indicate whether it is the correct or best approach of analyzing data, and therefore a key question that has emerged from this study is whether popularity translates to quality. It is clear that challenges in the data output quality from automated software remain a hurdle to the widespread uptake of software in flow cytometry. This is an opportunity for further work to assess the actual performance of different algorithm types through a range of benchmarking real-world experimental and simulated data sets with controlled cell characteristics.

ACKNOWLEDGMENTS
The authors would like to acknowledge all participants that completed the clinical laboratory survey.