Registration of Image Modalities for Analyses of Tissue Samples Using 3D Image Modelling

Biopsies are a diagnostic tool for the diagnosis of histopathological, molecular biological, proteomic, and imaging data, to narrow down disease patterns or identify diseases. Matrix‐assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) provides an emerging state‐of‐the‐art technique for molecular imaging of biological tissue. The aim of this study is the registration of MALDI MSI data sets and data acquired from different histological stainings to create a 3D model of biopsies and whole organs.


Introduction
Biopsies represent an essential type of specimen material in medical research and diagnostic. In general, imaging techniques are used for the detailed analyses of these materials. Accordingly, there is a large number of different methods for analyses available. For example, microscopic methods like light microscopy, or fluorescence microscopy, are in general used for the analysis of tissue specimens. However, to maximize information content from these tissue specimens, a combination of different analytical methods is mandatory. Classical methods like histopathology are well-established in this context, but further molecular methods are required to reach this aim. In particular, the visualization of the 3D structure of the biopsies is of very high interest. In this context, matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) has reached the stage for both research and diagnostic applications. Advances in the development of MALDI mass spectrometers and sample preparation methods has solved the initial practical problems with 2D MALDI MSI, such as lateral resolution, speed, and handling of the instrument, allowing routine imaging of peptides and proteins from tissues. [1][2][3] Biomolecules can be identified directly from the tissue, which allows to use spatial information to correlate imaging data with protein identification experiments. Currently, the focus is on combining technologies to maximize sensitivity and fragmentation efficiency at higher spatial resolutions. Ryan et al. present an excellent overview and highlight methods and technologies specifically designed for protein identification to overcome the challenges of matrix-assisted laser desorption/ionization mass spectrometry imaging (MALDI MSI) experiments. [3] The use of high-resolution mass spectrometers with high mass accuracy has radically improved the quality and quantity of biochemical information. [4] By using MALDI MSI techniques, detailed molecular datasets of biopsies are accumulated, resulting in large data sets. Therefore, 3D MALDI MSI applications remain still challenging. However, a 3D model of the biopsies or even whole organs based on MALDI MSI data, as well as histological data sets from different sources (e.g., light microscopy, fluorescence microscopy) would be highly beneficial since such a 3D model reflects both molecular and histological information of biopsies. But image registration and multimodal imaging of MALDI MSI data and data from classical methods like histopathology are still a strong limitation in these applications. However, the combination of these methods is essential for detailed histological and molecular analyses of biopsies. Mathematical approaches are essential in order to understand the relationships between the different imaging modalities to correlate morphological tissue structures and mass spectrometric signatures resulting in histological and molecular information of the tissue specimens.
Transforming 2D MALDI MSI data into the third dimension requires three essential steps: 1) the preparation of serial sections of the analyzed specimen with a defined distance between adjacent sections appropriate for the biopsies structure, 2) the molecular and histologic analyses of the resulting sections, and 3) merging of resulting data into one single 3D representation.
In the current study, serial sections of the biopsies were prepared, and the molecular information was generated by MALDI MSI techniques. For histological analyses, a combination of hematoxylin-eosin staining (HE), Gomori staining, immunohistochemical double staining (IHC), and immunofluorescence labeling of biopsies were performed in this study. Multimodal imaging for the combination of spatially resolved molecular, histological, immunohistochemical double staining, and immunofluorescence information to relate the molecular data obtained from MSI experiments to the structural information from histological staining was used. The integration of 3D MALDI MSI data with HE, Gomori staining, and IHC images allow a correlation between histological and molecular information leading to a better understanding of the biological processes in the organ. The superposition of the images requires the same local positions of the images since two different images cannot be placed directly on one another, because of different resolutions and deformations. For the development of the necessary registration approaches, modular and problem-adapted solutions were used. [5][6][7]

Clinical Relevance
Biopsies are used for diagnostics of histopathological, molecular biological, proteomic, and/or imaging data. Registration of multimodal imaging modalities described in this study enables both the histological and the molecular analysis of such tissue samples. The visualization of several tissue slides facilitates the analysis of the tissue samples for both diagnosis and research. With the algorithm developed and established in this study, the user is able to zoom into parts of the tissue sample as well as to select the specific view angle direction into the tissue resulting in a detailed analysis at specific regions of the tissue sample. The opportunity to select the superposition variants of the image enables to focus on tissue data of interest. This mass-spectrometric-based algorithm will contribute to the translation of histological analyses to a molecular level.
Appropriate models have to be created to describe the similarity between two images and the possible deformations. The actual registration of the images onto each other is calculated using advanced mathematical optimization techniques. In conclusion, within the current study, an automatic algorithm for automatic registration of MALDI MSI data with different histological image data for the cross-process evaluation of multimodal data sets was developed, established, and validated. This multimodal image approach simplifies and improves molecular analyses of tissue specimens in both clinical research and diagnosis.

Ethical Consideration
The use of tissue specimens from C57/Bl6 mice was approved by the local ethics committee (Saarland University, Homburg, Germany; ethics vote:  and complied with the laws of the European Convention for Animal Protection.
The human vessels were obtained from body donations.

Preparation of Tissue Specimens
After sacrificing the mice, the hearts and kidneys were quickly isolated and rinsed ex vivo with 4% paraformaldehyde (PFA) solution before being isolated and fixed in PFA at 4°C overnight. The murine and human tissues were dehydrated by using an MTM automated tissue processor (Slee Medical GmbH, Mainz, Germany), embedded in paraffin, and tissue section slides (thickness 5 m) were prepared by using a microtome (Leica RM 2245, Wetzlar, Germany). For the MALDI MSI measurement, the tissue sections were mounted on indium tin oxide (ITO) conductive-coated slides (Hudson Surface Technology, New York, USA), dewaxed by using xylene for 5 min twice, and hydrated with isopropanol for 5 min with descending ethanol concentrations (100% and 96%, 70% and 50%) for 2 min each and ddH 2 O for 5 min. [1] www.advancedsciencenews.com www.clinical.proteomics-journal.com

Hematoxylin-Eosin Staining of Tissue Material
The sections of the paraffin-embedded tissue specimens were dewaxed by using xylene for 3 min and hydrated using descending ethanol concentrations (100% and 96%) for 3 min each and 70% for 2 min. The sections were hydrated by dipping in pure water. The sections were stained by a hematoxylin solution for 8 min, followed by 8 min in pure water. Next, the sections were stained by an eosin solution for 2 min, were dipped in water, and in ascending ethanol concentrations (70% and 96%) and in 100% for 2 min. After placing the sections in ethanol for 2 min and xylene for 5 min, the sections were sealed. The images of the tissue specimens were accumulated by using a microscope (Leica DM 5500 B, Leica Biosystems, Germany).

Gomori Staining of Tissue Specimens
The sections of the paraffin-embedded tissue specimens were dewaxed (as described above). The sections were stained for 15 min with Bouin´s solution at 58°C, following slides placed in water for 5 min. Next, the sections were placed in a Weigert iron hematoxylin solution (Weigert´s iron Part A 50%, Weigert´s iron Part B 50%) for 5 min, followed by placing in water for 5 min. Afterward, the sections were stained with a Trichrome solution (Chromotrope 2R 0.6%, Aniline blue 0.3%, phosphotungstic 0.8%, glacial acetic acid 0.1%) for 15 min and were placed in water for 5 min. The sections were placed in 0.5% acetic acid, water, and an ascending ethanol gradient (70%, 96%, and 100%) for 2 min each. The sections were placed in xylene for 10 min and were sealed. The images of the tissue specimens were accumulated by microscope (Leica DM 5500 B; Leica Biosystems, Germany).

Immunohistochemical Double Staining and Immunofluorescence Labeling of Tissue Specimens
The sections of the paraffin-embedded tissue specimens were dewaxed by using xylene for 10 min and hydrated with an ethanol gradient (100% ethanol for 5 min twice, 96%, and 70% ethanol 5 min each). The sections were incubated in 0.1 m citric buffer for 20 min using a microwave (Fairline MW2717; Fulltrade, Germany) at 600 W, and washed with phosphate-buffered saline (PBS). The sections were incubated in the presence of a blocking solution (1% bovine serum albumin [BSA]) in a humid incubation chamber for 30 min at room temperature. After removing the blocking solution, the sections were incubated in the presence of a primary antibody rabbit anti mouse IgG 1:100 (Cedarlane, Canada) for 24 h at 4°C. The sections were washed in PBS and the secondary antibody anti rabbit-biotin 1:100 (Vector laboratories, USA) was applied and incubated for 30 min. The sections were incubated with streptavidin-fluorescein isothiocyanate 1:33 (Vector laboratories, USA) for 30 min at 30°C. The blocking solution was applied in a humid incubation chamber for 30 min. The sections were incubated in the presence of the second primary antibody monoclonal mouse anti-human 1:100 (DAKO, USA) for 24 h at 4°C. After the sections were washed by using PBS, the sections were incubated in the presence of the secondary antibody anti mouse-Cy3 1:100 (Jackson Immunoresearch laboratories, USA) for 60 min at room temperature.
The sections were covered with Vectashield including Dapi 1:1 (Vector laboratories, USA) and sealed. The images of the tissue specimens were accumulated as described above.

MALDI Mass Spectrometry Imaging
Imaging Data Acquisition: The sections of the paraffinembedded tissue specimens were coated with a trypsin solution (25 ng L −1 in 20 mm ammonium bicarbonate), incubated for 2 h at 50°C and coated with -cyano-4-hydroxycinnamic acid (Bruker Daltonic, Germany) as matrix (10 mg mL −1 dissolved in 70% ACN/0.2% trifluoroacetic acid). The coating process was performed by using a sprayer for MALDI imaging (HTX TM-Sprayer: TMSP-M3, HTX Technologies, USA). MALDI MSI data accumulation was performed using a TOF-TOF-mass spectrometer (Rapiflex; Bruker-Daltonic, Bremen, Germany) equipped with a Smartbeam 3D laser with a 10 kHz repetition rate and controlled by the Flex-Control 4.0 (Bruker-Daltonic, Germany). Mass spectra were acquired with 200 laser shots for each raster point and a digitizer rate of 2.5 GS per s in reflector positive mode. Analyses were taken in mass range 600-1600 Da by using a raster width of 10 m.
Analysis: Data analysis was performed using SCiLS Lab software (SCiLS Lab-2019b; SCiLS GmbH; Bremen, Germany). MALDI MSI raw data sets were converted to the SCiLS SL file format. The baseline was calculated by iterative top hat, and normalization is performed based on the total ion count (TIC) method. [1] Orthogonal matching pursuit (OMP) was used to detect peaks. Representative mass signals (m/z) of pre-defined proteins were chosen for analyses of the distribution of the protein in the corresponding biopsies. The identification of peptides was performed through the MS/MS spectra using the lift option of the Rapiflex mass spectrometer (Bruker-Daltonic, Bremen, Germany). The mass spectra were calibrated and annotated using BioTools 3.2 (Bruker-Daltonic, Bremen, Germany) in combination with Swis-sProt (University of Geneva, Switzerland) and the MASCOT 2.2 database (Matrix Science, London, UK) comparing experimental mass-spectrometric data with calculated peptide masses for each entry into the sequence database. The resulting .mgf files were then searched against a mice protein database downloaded from the SwissProt database using the Mascot search engine BioTools 3.2 (Bruker-Daltonic, Bremen, Germany). The settings for the DB search were as follows: enzyme, trypsin; peptide tolerance, MS tolerance 0.5 Da; maximum missed cleavages, 1; peptide charge, 1+ ; MS/MS tolerance, 0.5 Da; MUS score, >100. The results were exported as .csv files from Mascot.
3D Image Modelling and Volume Visualization: MALDI MSI data, microscopic data of hematoxylin-eosin stained, microscopic data of Gomori-stained, immunohistochemical double-stained, and immunofluorescence-stained, respectively, of subsequent serial tissue sections (thickness 5 m each) of the tissue specimens were accumulated.
The volume visualization of different modalities was realized in MeVisLab. The registered images were passed to MeVisLab (MeVis Medical Solutions AG, Bremen, Germany) in order to reconstruct the volumes and to compute virtual sectioning. The www.advancedsciencenews.com www.clinical.proteomics-journal.com modular system of MeVisLab allowed for both visualization and registration of the different modalities. For the purpose of 3D visualization, the software relies on the object-oriented 3D toolkit "Open Inventor," developed by Silicon Graphics (SGI). [8] Hence, the reconstructed volumes as well as the registrations could be examined in different ways. This comprised free rotation of the reconstructed volume and cutting of the volume with freely movable planes. The freely movable planes especially allowed to examine the tissue from even more points-of-view than just using the standard anatomical planes (coronal, sagittal, and axial). For technical details of the visualization process, refer to ref. [8].

Data Registration
Image registration is the mathematical routine for merging data from different imaging environments. Typically, two images were not simply superimposed. Rather, the geometry of one image was changed in such a way that the images fit together as good as possible.
The typical image registration routine comprised two steps. On the one hand, a global adaptation such as rotation, translation, and scaling (rigid/affine linear registration strategies) was done as a pre-alignment routine. On the other hand, further transformations by local deformations occurred, which were compensated by a non-linear deformation model, that is, by modelling an elastic deformation.

Variational Registration Approach of Image Data
The registration of the image modalities MALDI MSI, hematoxylin-eosin, Gomori staining, and IHC imaging resulted in data sets varying in contrast, staining information, or even artifacts like deformations or cracks due to preparation like sectioning or scanning. To maximize alignments, global as well as local image information were used. The typically used variational image registration model describes the similarity of two images and the possible deformations. [7] The arbitrary images T 0 , T 1 are registered aiming to find a reasonable transformation y for minimizing the differences in T 0 and T 1. Furthermore, in order to assure a reasonable deformation y, a smoothing functional S is used. Therefore, an adjoint functional J two comprising a distance D for two images and a regularizer S is optimized. [7,9]

Global Registration Approach for Multiple Images
A global registration approach was used for multiple images to exploit the image information across all sections resulting in a complete coupling of the image frames. The global energy functional J glo was calculated as J glo (Y; T) := D glo (T Y ) + S glo (Y), with Y: vector comprising all transformations; T Y: composition of the images with the respective grids. The smoothing was a standard curvature smoother and was defined as S(u) : [7,8] For global distance measurement a feature array F(T) := (F(T 1 ),…., F(T K )) was used, comprising all vectorized feature maps of the images as columns, which is based on the presentation in ref. [10]. The used feature maps F were based on normalized gradient fields, [4] such that edges rather than intensities were used for alignment This feature map was based on a global normalization T(x)/ǁ∇Tǁ L2 , whereas the NGF distance measure itself, was based on a local normalization at each grid point, such that ∇T(x)/ǁ∇T(x)ǁ holds. [10,11] These computations assume maximal correlation of the feature images. Moreover, it is assumed that the rank of the feature array is minimal when the image frames are aligned, which was achieved by using a relaxed rank minimization. [10,12] Thus, using the so-called Schatten-q-norms, resulted in .., K, denoted the non-zero singular values of the feature array F(T). Image intensity variations were compensated by using the NGF features instead of the pixel-intensities. For computation of the numerical results, the SqN distance was used using q = 4, as proposed in ref. [10]: SqN 4 (T) := K − ǁF NGF (T)ǁ. Concludingly, the objective function J SqN was build up by the sum of the used SqN distance measure and the curvature regularizer: Figure 1A illustrates the overview of the steps for the multimodal 3D reconstruction of the model.

Histological Sections of a Whole Organ
MALDI MSI data, microscopic data of hematoxylin-eosin stained, microscopic data of Gomori stained, immunohistochemical double-stained, and immunofluorescence-stained, respectively, of subsequent serial sections (thickness 5 m each) of the tissue specimens were accumulated. The mice kidney, heart, and human vessel were sectioned, with approximately 400 single sections for one tissue specimen (120 for each image modality). Histological staining is also feasible after MALDI MSI analyses, which are more accurate than staining of subsequent sections. [1] However, this enables co-register of maximal two imaging techniques. In order to be able to use and register more than two imaging techniques, stainings were performed on successive slices in a consistent order. Four successive tissue sections were made, one for each imaging method. The order of the methods was maintained for the entire slide preparation. The first tissue section of each of the four successive tissue sections were prepared for a MALDI MSI data accumulation. The second tissue section was stained by HE, the third tissue section by Gomori, and the fourth tissue section for the immunohistochemical double staining and immunofluorescence.
MALDI MSI data as well as data from hematoxylin-eosin staining, Gomori staining, and immunohistochemical double staining and immunofluorescence were fused to a 3D reconstruction of the organ afterward ( Figure 1B).

MALDI MSI Data Accumulation
The first tissue sections of consecutive tissue sections were prepared for a MALDI MSI data accumulation. We were able to measure a complete kidney section with a raster width of 10 m within 3.5 h with the MALDI MSI mass-spectrometer ( Table 1).
For a 3D dataset of approximately 120 2D datasets (typically 23.000-375.000 spectra of about 100 000 bins) were accumulated. The range of up to nearly 100 gigabytes is needed per 2D data set.
The protocol for calculation a 3D model from 2D data was established by a three main step approach: i) preparation of serial sections of the analyzed specimen with defined distance between adjacent sections appropriate for the tissue structure to be reconstructed, ii) analysis of each section using the MSI ionization technique, and iii) merging of consecutive 2D images into a 3D representation. For the reconstruction of the organ, approximately 100 sections were used in the current study. The distance between the individual tissue sections were in the range of 15 m.

Histological Staining
To correlate the molecular data obtained from MSI experiments to the structural information from histological staining, tissue slices of consecutive tissue sections were stained by hematoxylineosin ( Table 2). This staining is a general staining for tissue samples, for example, in the diagnosis of tumors. Since tumors with a diameter of less than 0.2 mm could be screened, hematoxylin-eosin staining is often used in parallel to Gomori and immunostaining. Using 3D reconstruction, detection of pathological changes or tissue-scarring is facilitated, and the extent of tissue changes can be determined. By the additional Gomori staining, connective tissue components are stained and can therefore be quantified by the area. This often causes scarring to become apparent. The third tissue section of each of the four consecutive tissue sections were stained by Gomori.
To use the image information of the modalities MALDI MSI, HE, Gomori (histology), and IHC (immunohistochemical) similar structures of consecutive tissue sections have to be brought into correspondence. Solving the inherent problems is difficult by the fact that the image data show significant variability in terms of staining intensity, image sharpness, possible deformations, and other artefacts such as cracks or missing parts of the tissue. Variability in staining and image sharpness might occur during the staining and photography process, which has to be minimized. In addition, deformation of slices, as well as cracks and holes, are phenomena that has to be minimized as much as possible. The tissue embedded in paraffin gets quite stiff. Thus, thin slices (5 m) are likely to crack or lose small parts during sectioning. A cautious pre-processing of the data, including robust pre-alignment, and a subsequent, robust, non-linear registration, can circumvent these issues. The particular choice of gradient features for our SqN distance measure for groupwise registration meets the necessary conditions. Furthermore, due to the use of the intrinsic structure of the data across all given slices, holes and cracks do not play a significant role. These are compensated by the global point of view that the distance measure exploits. Such challenges have already arisen in the registration of high-resolution histological sectional images and have been addressed in applications. [12,13] Noteworthy are also the specific work carried out on crack handling by a specialized discretization [12] and the homogenization, including recoloration during registration. [13]

Immunohistochemical Staining
The fourth tissue section of each of the consecutive tissue sections were prepared for immunohistochemical double staining and immunofluorescence. In this study, Collagen I, cell nuclei, and smooth muscle cells were labeled. Due to the fluorescence only the specifically stained cells or tissue are clearly visible. The quantification is based in the stained area. The mass spectrometric data provide a distribution of desired substances in the tissue, which can neither be represented by specific antibodies or by other histological staining variants. Due to the high resolution of the model (Table 3), detailed, structurally based information can be made as well as a connection to the spatial information.

Segmentation
In order to remove unspecific information such as dust and dirt from the background, we opted for a powerful segmentation algorithm. A connected components approach achieved pleasing results. Component segmentation classified pixels of an image into different groups based on the intensity information. This is achieved by comparing the intensities of the pixels to the intensities of the neighboring pixels. If these are similar in a particular, manually specified tolerance, the pixels belong to the same group. In the current study, the tissue was sufficiently standing out from the background.

Linear Pre-Registration
The preprocessing of the current study also included a linear pre-registration. We have decided to use a principal axes-based pre-registration. This is due to significant differences in pixelresolution of the images (Table 4). Furthermore, there were different orientations of the individual histological slices on the photographs. The axes-based pre-registration was extended for multiple images, such that also the pre-alignment was achieved in a group-wise manner, instead of in a pair-wise manner. We used a Gaussian-based principal axes method described in ref. [9]. The aim was to identify the principal axes based on the mass-distribution of intensities. This is done by using the well-known Principal Component Analysis (PCA) as a basis, which in conclusion is a linear fitting of orthogonal axes to the given data. Based on the PCA and therefore the used eigenvalue decomposition, the scaling, as well as the orientation of the individual slices, were corrected. The alignment was based on the scaling and orientation of the respective principal axes. Figure 2 demonstrates the image correction performed by the algorithm, using the corresponding linear grid deformation.

Non-Linear Registration of Histological Images and Mass Signals
The aim of the non-linear registration is to achieve a pointto-point correspondence. For the achievement of an exact alignment, we choose the proposed group-wise registration method SqN recently described by some authors of the current study. [10,12,14] The advantage of this approach, overusing sequential approaches, is the alignment of all sections in one process rather than aligning just pairs of sections. Such a group-wise approach allows for using information for alignment across the whole stack of sections, instead of only information between pairs of images. The features used for alignment are normalized intensity gradients as they are presented for image registration in ref. [5].
To achieve this aim, the vectors, representing the gradients, are aligned by transforming them. This achieves collinearity, meaning the vectors show in the same or opposite directions, and the angle in-between is zero. The use of such gradient-based features has proven very well for multi-modal image registration in the past. By using the novel SqN approach, the gradients of all sections are compared at once. This achieves a minimization of errors like a z-drift or the banana-effect. Furthermore, each image is transformed individually. While pair-wise image registration methods need to define a specific, fixed reference image to determine the transformation between the reference and the image that is to be aligned, our group-wise approach is free of such a choice. The absence of this choice in using SqN avoids a bias introduced by the respective user ( Figure 3A). The Figure 3A shows the reconstructed 3D model for the respective used image modalities. In this state of process, the 3D image reconstruction can be modified based on the angle of interest ( Figure 3B). In addition to the incision of the organ itself, the incision plane is also shown for a better orientation.
The registration was performed by multiple resolution levels, as described in ref. [15], starting on very coarse resolutions with very few details and iteratively increasing the resolution to consider more details for even more exact alignment. Also, the process of using multiple resolutions for registration achieves a faster convergence in optimization. The high amount of detailed information on coarse resolution-levels avoids local minima in the energy function, which is to be minimized. The energyfunction is determined by the choice of the distance function and regularizer more precisely with Equation (1), as described in Section 2.4.1. For underlying mathematical details, we refer to refs. [10,12,14]. Through the independent working approach of the algorithm, tissue specimens can be reconstructed. A heart ( Figure 5) and a vessel (Figure 6) were also exemplary reconstructed within this study. Figure 6 demonstrates a calcified human vessel. By reconstructing this vessel, the calcified area can be displayed and information about the size and shape are determined. By using the mass spectrometric data, an accurate view of the composition of the plaque are given and the ratio of components are quantified. In addition, molecular modifications of the tissue samples could be identified too. By using the histological staining, the localization of these modifications is limited to certain cell types relevant for inflammation. The visualization of the molecular modification results in a more precise analyses of the tissue samples compared to 2D sections of tissue specimens.  . 3D image modelling of a mice kidney. A) 3D image modelling of data accumulated by MALDI MSI, haematoxylin-eosin, Gomori, and immunohistochemical double staining and immunofluorescence data sets. After equalization by the deformation grids, an exact overlapping of the individual sections was achieved and reconstructs the complete organ. B) 3D image modeling of a kidney from hematoxylin-eosin and Gomori staining data sets. The developed algorithm makes it possible to cut the models virtually from any angle. This allows a precise view of the regions that are relevant for corresponding investigations.
By the analyses of two or more molecular mass signals (Figure 7A), a sensitive localization, especially with regard to a colocalization is feasible. For example, the co-localization of the m/z 805.7 of the random peptide of originated from guanidine nucleotide-binding protein subunit alpha 13 (2-8) (red) and m/z 1106.8 of the random peptide of originated from transmembrane protein 14C (104-114) (green) is exemplary shown. Guanidine nucleotide-binding protein subunit alpha 13 is essential for both cell growth and pathological processes of various diseases, for example, tumor diseases or cardiovascular diseases. [16] In    addition, transmembrane protein 14C is essential for mitochondrial transport. [17] In the single display of the mass signals as well as in combination a distribution over all tissue sections is shown, and consequently a distribution over the reconstructed organ. By representation as a 3D model, changes of the co-localization in different levels can be considered. Besides the qualitative representation of the mass signals, a quantification of the signals is also possible ( Figure 7B). The ratio of the random peptide m/z 805.7 and m/z 1106.8 is shown in Figure 7B based on the measured intensities.

Discussion
There is an increasing evidence that the analysis of tissue samples like biopsies based on classical (descriptive) histological and immunohistochemical methods is insufficient for a comprehensive diagnosis as well as or for answering specific scientific questions. By combining molecular and imaging data, the information content of tissue samples is significantly increased. Therefore, the data from MALDI MSI, HE and Gomori (histology), IHC (immunohistochemical), and immunofluorescence were merged into a 3D model in the current study. The use of a 3D graphical approach of the data caused an increased significance of the generated data for the tissue sample, since specific regions of the tissue sample like mice kidney, mice heart, and human vessel can be considered individually. An advantage of this approach would be the development of a virtual stereology for the proteome, which make valid statements about quantity and distribution, for example, between renal cortex and renal medulla.
What are the main challenges to be solved in this study? Image distortion was tackled by means of mathematical approaches, aiming at overlapping of the sections of the tissue samples. The distortions in the sections were artificially caused by cutting the paraffin-embedded organs and were eliminated by global adaptation based on rotation, translation and scaling using rigid and affine linear registration strategies. Local deformations were compensated by a non-linear deformation model referred as deformable image registration. This allows merging of data despite the variability of image staining to variation in ambient lighting during the photography process, image sharpness, deformations, cracks, or lacking parts of tissue sample as already described for registration of high-resolution histological sectional images. [18] In order to detect similarities of the individual image modalities, a multimodal, group wise distance measure is defined in the current study. This distance measure is based on comparing intrinsic image features instead of simple intensities.
For registration of images from different modalities, a robust registration approach for a variety of cases and images is essential dealing with the high variability in coloration, image sharpness, deformations, and artifacts like cracks.
A large number of algorithms for image registration are available for this issue and in recent years, procedures have for the registration of histological image data that meet the requirements of high-resolved, information-rich image data have been developed. Leskovsky et al. [19] and Schönmeyer et al. [20] use automatically generated scale invariant feature transforms (SIFT) or landmarks points which are superimposed, while the method of Feuerstein et al. [21] use Markov Random Fields for registration and the method of Weiss et al. [22] achieves a registration via density distributions of cell nuclei.
Besides the modelling of the registration process, a numerical implementation of the registration procedures is essential. [21][22][23] Recently, image registration has evolved through the development of non-linear and diffeomorphic procedures under the inclusion of secondary conditions as well as the development of procedures for molecular mass, length, and volume preserving registration. [24,25] How did we tackle all these problems in our study? Image modalities could be fused by adapting and combining mathematical image registration methods. Starting with non-deformed identity-deformation-grids, the different deformations of the single images are numerically computed, and the spatial correspondence of each pixel is adjusted in the approach of the current study. After superposition of the histological sections and mass spectrometric data, data from hematoxylin-eosin staining, data from Gomori staining and immunohistochemical double staining, as well as immunofluorescence data sets were superimposed. Hereby, deep correspondence of tissue structure as well as tissue changes and biomolecule distribution were accessible. Exemplary, the detection of a random peptide of originated from guanidine nucleotide-binding protein subunit alpha 13 and the peptide of originated from transmembrane protein 14C for mice kidney were shown for demonstration ( Figure 7A). The strategy of the current study for multimodal data registration works autonomously and realizes a virtual incision of the organ from any angle and level. The 3D-reconstruction of organs based on different imaging methods offers innovative options of analysis for both clinical research and diagnosis. Based on the proposed approach, tissue samples are displayed as a whole organ and due to the different methods, the information content is far superior to the information content of individual 2D sections. The arrangement as a 3D model is preferable to the visualization of serial slices, since the 3D model enables deviations between the individual slices to be determined more easily, which leads, among other advantages, to considerable time savings. Virtual incisions could be calculated from any angle and the individual imaging components could be faded in and/or out aiming at maximization the information content of the specific sample concerned. By implementing MALDI MSI data, molecular as well as solely histological information can be generated from the respective samples. By using a mouse cursor, the insert of the tissue can be zoomed precisely to display the axial, coronal, and sagittal level. In addition, the superposition of the selected image modalities is displayed for this area. By scrolling through the slices or shifting the indicated positional axes in the figures on the left-hand side, the user can precisely visualize the current position within the tissue-volume while viewing a specific section and the respective superposition on the right-hand side.
For the used variational approach, the value of the distance measure and the regularizer is crucial. The regularizer determines a specific physical behavior when deforming the images in order to align them. The distance measure determines, how the images are compared. Applications, where images from multiple different modalities, with, for example, different staining and intensities, need to be registered, it is necessary to choose a distance measure that is capable of compensating these differences. In the current study, we decide to rely on a distance measure based on the comparison of edges within the different images, named features. The according feature maps or feature images are the functions that transform the intensity images to the "edge-images," respectively.
The definition of a distance measure is essential for the establishment of a suitable registration model. The distance measure describes the similarity of the image modalities MALDI MSI, Gomori staining, and IHC imaging.
The most common approach is the use of distance measures comparing two-images. These two-images measures assume, that one image is fixed, while the other one is transformed to match the fixed images. In most cases these measures are used www.advancedsciencenews.com www.clinical.proteomics-journal.com in a sequential manner, iterating over all pairs of images. This has two drawbacks. On the one hand, one has to choose a specific reference image all other images are aligned to. On the other hand, only pairs of images are compared for alignment. Furthermore, the sequential process favors cumulative error-effects, like the banana-effect. Since our specific application comprises more than two images, which are additionally of multiple modalities, we used a group-wise registration approach. The SqN approach exploits information across all slices in a global manner and thus avoids cumulative error-effects. [10,12,14] This approach is capable of registering all images at once. Furthermore, the distance measure is capable of comparing structural information across all images, based on the chosen features. The alignment based on these features is based on a geometric idea. The intensity changes of an image is qualified as edges or jumps from a dark to a lighter region or vice versa. The gradients describe the direction of the jump as well as its height. Since the height of the jump is different in different modalities, a normalization of the height is considered. After normalization, the idea is to align the gradient directions. Since the same or at least equal tissue is shown on the given sections, the edges should be in corresponding spots. Images are not fixed and the alignment of each image is dependent of all features of all other images. For further technical details on two-image approaches and other groupwise approaches, reference is made to, for example, refs. [7,9].
Besides the modelling of the registration, the second main focus is on the efficient numerical implementation of the registration procedures. [24] In the last years the image registration is dominated by different developments: Methodically in the further development of non-linear and so-called diffeomorphic procedures under the inclusion of secondary conditions and the development of procedures for mass, length, and volume preserving registration.
For optimization of our specific functional J glo (2), a discretizethen-optimize approach with a multi-level scheme was used. [15] We started on a coarse data resolution for which fast convergence was expected, due to the coarse and smooth representation. The numerical solution on the coarse level served as a starting guess for the next higher resolution. When reaching the original data resolution, the process was terminated. On each resolution-level, the problem was solved using the L-BFGS optimization method with Wolfe line search. [10,12] What are the measurement limits of the study? The accuracy of the MALDI TOF/TOF mass spectrometer used in this study is limited resulting in a limited accuracy of the data. [2] MALDI Orbitrap or LC-MS/MS offer a higher measurement accuracy, that is, for a better protein identification further approaches/analyses are necessary. The combination of different mass-spectrometric methods would be preferable because of the limited mass accuracy of mass-spectrometric methods in the context of identification of peptide structures. In addition, the comparison the corresponding mass-spectrometric data of the authentic peptide is an essential validation step of the identification.
However, the registration algorithm described in this study is accessible for all MALDI imaging data, independent of the mass spectrometer used. Furthermore, there are some experimental conditions affecting lateral resolution, such as diffusion of trypsin solution within the tissue sample and/or the matrix solution causing washout effects of biomolecules. These experi-mental conditions reduced the lateral information in MSI, even when working with high lateral resolution settings. However, measurements with a raster width of 10 m are feasible from the technical point of view and provide striking data in combination with other imaging techniques as shown in this study.

Conclusion
The development of an automated registration approach to generate a 3D model of tissue samples is an important step for development of histologic analyses, clinical research, and diagnosis. The representation of several levels of an organ simplifies the assessment and analysis of the tissue specimens for diagnosis and research. The ability to zoom into the tissue and cut from any angle is a further step forward for increasing accuracy tissue analysis. The function of the individual decision as to which superposition variants of the individual imaging methods are to be visualized, is the most important feature of the current approach. Due to the implementation of MALDI mass spectrometric data, this approach will contribute to translate histological analyses to a molecular level in the near future and molecular modification like post-translational modification could be analyzed, which are neglected by the currently used histological methods in diagnostics and research. By combining of this molecular approach and histological staining methods, the localization of these modifications could be associated with specific cell types. By 3D visualization, changes in the co-localization in different slides of organs could be analyzed. Besides the qualitative exposure of individual molecular mass signals, a quantification of the molecular mass signals is also applicable. This allows a more precise analysis than 2D sections of tissue specimens using different imaging methods. This approach developed and established in the current study will contribute to the further development of histological testing methods for tissue samples.