Groupwise Shape Correspondence Refinement with a Region of Interest Focus

While collections of scan shapes are becoming more prevalent in many real‐world applications, finding accurate and dense correspondences across multiple shapes remains a challenging task. In this work, we introduce a new approach for refining non‐rigid correspondences among a collection of 3D shapes undergoing non‐rigid deformation. Our approach incorporates a Region Of Interest (ROI) into the refinement process, which is specified by the user on one shape within the collection. Based on the functional map framework and more specifically on the notion of cycle‐consistency, our formulation improves the overall matching consistency while prioritizing that of the region of interest. Specifically, the initial pairwise correspondences are refined by first defining the localized harmonics that are confined within the transferred ROI on each shape, and subsequently applying the CCLB (Canonical Consistent Latent Basis) framework both on the global and the localized harmonics. This leads to an enhanced matching accuracy for both the ROIs and the overall shapes across the collection. We evaluate our method on various synthetic and real scan datasets, in comparison with the state‐of‐the‐art techniques.


Introduction
With the recent evolution in the shape capture technology, the use of captured geometry and the appearance of real-world shapes has emerged as important modelling tool in computer graphics, virtual reality, and medicine.A common prerequisite to such reconstructive shape modelling is the need for the full vertex correspondence among all captured shapes, for which a vast amount of solutions exist.Different strategies have been developed in order to circumvent the exponential complexity of the solution space and rapidly converge to the final or a number of acceptable solutions.Methods adopting the paradigm of combinatorial optimization on global and structural shape descriptors [ATCO * 10, SSGD03, BFF * 07], optimal deformation of template models [BRLB14], voting or clustering schemes to achieve collective consistency [CZ08, ATCO * 10], RANSAC (Ransom sample consensus) based algorithms to hypothesize-and-check [TBW * 09, TBW * 11], use of statistical atlas to benefit from the known commonality and variability of target shapes [ACP03, ASK * 05], and embedding based approaches [EK03,KLF11,KLCF10] where the shapes are mapped to a canonical domain before computing the correspondences in the new representation.
One of the successful embedding-based approaches has been established by the functional map framework.Originally presented in Ovsjanikov et al [OBCS * 12] and considerably extended in sev-eral follow-up works [RMOW20, RCB * 15, AGB * 21, HAGO19], it represents correspondences between shapes as a linear transformation or mapping between their respective function spaces.This allows most constraints (descriptor preservation, operator commutativity, or landmark correspondences) to be linear, making the map inference very efficient.Although an optimized functional map facilitates the transfer of real-valued functions between shapes, the conversion between functional and point-wise maps poses a challenging problem that is prone to errors, for which several solutions have been proposed, such as the map recovery as a probability density estimation [RMC17], and the extension of functional basis by using functional products [NMR * 18].Another inherent problem is related to the choice of the function basis and its size, which often relies on heuristics to achieve a favorable trade-off between the number of basis function and the precision of the mapping.To address this, iterative refinement methods such as ZoomOut [MRR * 19,RMOW20] or EFMR (Effective Functional Map Refinement) [RMWO21] have been developed.These methods progressively up-sample the functional basis, by optimizing the point-wise and the functional map alternatively.In this work, we deploy the notion of localized harmonics [MRCB17] confined in a region of interest (ROI) to maintain effective number of functional basis while preserving high-frequency accuracy.
• We enhance the cycle consistency framework [HAGO19] by incorporating local frequencies inherent to regions of interest (ROI) into the refinement process.This enables us to ensure matching consistency on both global and local scales.• While establishing the necessary cyclic maps within the cycleconsistency framework, we initialize maps in both directions, even when dealing with real-world shapes with different coverage.To ensure the presence of bidirectional maps in the case of partial correspondences, we adapt the 'Bijective ZoomOut' [RMOW20] refinement technique to handle partial cases.
The rest of this paper is organized as follows.We start by reviewing preliminaries of the functional map framework for nonrigid shape correspondence computation (Section 2).In (Section 3) we present our refinement method, which is equipped with Bidirectional Partial ZoomOut and incorporates basis extensions with a ROI focus.Next, we provide details on the dataset and the implementation, in Section 4 and Section 5.After illustrating and discussing the results we obtained on several real-world and synthetic shape collections (Section 6), we conclude the paper (Section 7).

Preliminaries
Shapes and functional representation.Assume that we are given a collection of n 3D shapes S = {S i } n−1 i=0 , represented as triangular meshes.To each shape S i , we associate a positive semi-definite Laplacian matrix L i = A −1 i W i by using the standard cotangent weight scheme [PP93], where W i is the cotangent weight matrix and A i is the diagonal lumped area matrix.The basis functions {φ k Si } k≥1 composed of the eigenfunctions of L i form an orthonormal basis of the square integrable functions on S i , denoted as L 2 (S i ), allowing to represent any function f ∈ L 2 (S i ) using a Fourier series: Functional correspondences.The functional map framework [OBCS * 12] aims at finding correspondences between shapes S i and S j as a linear transformation τ : L 2 (S i ) → L 2 (S j ) between their respective function spaces.As a linear operator, τ admits a matrix representation C = (c kl ) with coefficient determined as follows.The action of τ on a function f ∈ L 2 (S i ) can be expressed as : Truncating the Fourier series to the first K terms allows to obtain a particularly compact representation of the functional correspondences with a K × K matrix C, where K is chosen to be small (between 20 and 100 in practice).
A general approach to optimize such a matrix assumes the availability of corresponding functions gc ≈ τ fc, c = 1, . . ., Q and asks the functional map C to conserve the spectral coefficients in a least square sense: where A= (⟨φ k Si , fc⟩) and B= (⟨φ l S j , gc⟩) are K × Q matrices of Fourier coefficients of the corresponding functions, α is a scalar weight and Ereg(C) is a regularization term.
Recovering a point-wise map from a given functional map can be expressed as the solution of the following optimization problem: which can be efficiently solved by searching for each row of Φ Si its nearest-neighbor in the space of rows of Φ S j , transformed by C ji .Indeed, any point p ∈ S j can be represented by the corresponding delta function δp ∈ L 2 (S j ), expressed in Φ S j as Conversely, a functional map C ji can be obtained from a point-topoint map T i j : S i → S j using the following equation: where Π i j is the matrix representation of T i j such that Π i j (index(p), index(q)) = 1 if T i j (p) = q and 0 otherwise, with p ∈ S i and q ∈ S j .
Partial correspondence.Finding partial correspondences between two shapes, where only a subset of points or regions have a corresponding counterpart due to missing parts in either shape, is known as a highly challenging problem.present in many real data applications due to occlusion or partial views.In the functional map framework, partial-to-full correspondences have been represented using maps with a slanted diagonal structure.As demonstrated in [RCB * 15], the slope of the functional map is proportional to the ratio of surface area of the two shapes A(S j ) A(Si) , S j ⊂ S i .Based on this property, they proposed a partial shape matching approach that regularizes the functional maps by enforcing them to have a slanted-diagonal structure.This work has been extended to cluttered non-rigid correspondences [CRM * 16] and to multiple shape settings [LRB * 20].While showing encouraging results in challenging settings, the explicit representation of the part entails complicated alternating optimizations over the part indicator functions in the spatial domain and the functional map in the spectral domain.To avoid this complex optimization scheme, Litany et al. [LRBB17] proposed a fully spectral partial shape matching (FSPM) approach where they absorbed the spatial mask into a new basis for the part function space with following optimization problem: (5) Here S(K, r) is the Stiefel manifold of orthogonal k × r matrices, o f f (A) = ∑ i̸ = j a 2 i j and Ar = WrA with Wr = (I r×r 0 r×K−r ) denotes the r × K matrix containing the first rows of A. r is the rank of the partial functional map.This optimization computes a new basis, localized to a latent part of the full shape (data term) and that minimizes Dirichlet energy (smooth term) in order to obtain bases that behave as laplacian eigenfunctions.Such methods rely on hand-crafted features such as SHOT [TSDS10], which are sensitive to the mesh discretization and can lead to failure when shapes have significantly different discretizations.
Group-wise refinement.In [HWG14], Huang et al. introduce the Functional Map Networks (FMN) to compute consistent functional maps within heterogeneous shape collection.A FMN with n shapes is represented as a connected and directed graph G(F, E), where F i ∈ F represents the functional vector spaces of shape S i , and an edge (i, j) ∈ E represents a functional map C i j .Given a FMN G, Wang et al. [WHG13] proposed to extract a set of consistent latent basis (CLB) {Y i } n−1 i=0 such that C i j Y i ≈ Y j , ∀i, j through the following optimization: where ω i j is a scalar reflecting the relative confidence assigned to each functional map of the network.Indeed, since initial maps usually vary in quality, one would prefer to assign larger weights to good maps and lower ones to noisy maps.This way, the optimized latent basis are less perturbed by potential noisy maps in the shape collection.
The latent basis Y i can be seen as a functional map from some latent shape (playing the role of the average shape) to each S i .Huang et al. [HAGO19] extended the latent bases by introducing an extra normalization to the CLB construction in order to incorporate a metric information to the geometric information contained in the CLB.The latent bases thus obtained are named Canonical Consistent Latent Basis or CCLB.The use of such consistent latent basis has several advantages.First, it directly represents the 'average shape' of the collection in the functional domain, avoiding the explicit construction of a template or average shape , which can be computationally expensive.Second, the latent spaces can be used to factor a functional map between two shapes S i and S j by using the latent bases composition C i j = Y j Y −1 i .Finally, one can efficiently add a previously unseen shape S k to the latent representation by composing the functional map C ik to the latent basis Y i of the shape S i .The corresponding formula Y k = C ik Y i , improves the scalability of the latent basis construction.

Our Approach
Given a collection of n 3D shapes denoted by S = {S i } n−1 i=0 and an initial correspondence among them, our goal is to refine the initial non-rigid correspondences among them with a focus on a specific region of interest (ROI).The ROI is defined once by the user by manually selecting faces on one shape that we call as reference shape.Without loss of generality, let S 0 be the reference shape and S i ∈ S (i = 1, . . ., n − 1) represent the other shape instances.In general, each shape instance S i ∈ S (i = 1, . . ., n − 1) has a different coverage of a shape undergoing some elastic deformation and contains topological noise.Fig. 1 illustrates an example of such a shape collection.The ROI defined on S 0 is colored in cyan, and its border in yellow for the sake of visibility.To improve the quality of correspondences among multiple shapes, we adapt cycle-consistency scheme by Huang et.al. [HAGO19], requiring initial functional maps forming cycles.In the rest of this paper, we use the FMN represented in Fig. 2 to perform our correspondence refinement, composed of maps {C 0i } between

Reference shape Instance shapes Correspondence
Figure 2: Correspondence graph represents the correspondences that we use and refine in our groupwise refinement.
the reference shape and each of the instance shapes, as well as {C i j } between consecutive instance shapes.Unlike in Huang et.al., however, our shapes have different coverage, necessitating the computation of partial correspondences.Partial correspondences usually being computed in one direction (from the partial shape to the full shape), we need to compute the missing correspondences (from the full shape to the partial one) to perform the groupwise refinement.To this end, we first invert the partial-to-full pointwise maps to obtain full-to-part one.We then adapt the 'Bijective ZoomOut' [RMOW20] refinement to the partial setting so as to refine the both maps in a bidirectional manner.
By using this approach, we can obtain initial correspondences that form cycles, which serve as the basis for our groupwise refinement method.We extend 'canonical consistent latent basis (CCLB)' scheme by Huang et al [HAGO19], by incorporating an ROI focus.The ROI is a user-specified area on a shape in the collection, emphasizing the need for precise correspondences on that region.Instead of using a sufficiently large number of functional basis, we extend the basis by the use of localized harmonics (LMH) defined on the ROI.This approach allows to preserve highfrequency accuracy while maintaining a limited number of functional basis.It is worth noting that the number of manifold harmonics plays an important role in the functional map framework.As noted in Melzi et al, [MRR * 19], spectral methods can be both computationally expensive and unstable when using an excessive number of harmonics, whereas reducing the number of harmonics leads to imprecise maps that lack high-frequency details.Although the concept of LMH has been previously introduced [MRCB17] in the literature, we demonstrate how the notion of LHM can be integrated in the groupwise refinement process.Details of our method are described in what follows.

Initialization of pairwise correspondences
We first compute pointwise initial correspondences T i0 from each of the instance shapes S i to the reference shape S 0 , and T i+1,i between consecutive instance shape pairs, i.e. from S i+1 to S i .Two different approaches are adopted depending on the presence of partiality in the collection: (1) Collections of full shapes: For the synthetic dataset without partiality, we utilize the classical functional map approach (Equation 3) with three regularization terms: the Laplacian commutativity term, the orientation preserving term, and the multiplicative operator term.
(2) Collections with partial shapes: We use the 'Fully Spectral Partial Matching' (FSPM) [LRBB17] to compute partial correspondences between the shape.We use the part-to-full FSPM because using the part-to-part FSPM produced worse results in some cases while being computationally more expensive.Then, to generate the correspondences of opposite direction, we invert the previously computed pointwise correspondences T i0 and T i+1,i .Specifically, T −1 ji ≈ T i j : S i → S j is defined as follows: where p 0 ∈ S j is randomly chosen.
Note that inverting part-to-full correspondences leads to full-topart correspondences, a problem that has not been addressed before.Although it may seem unnatural to compute it, it constitutes an essential component to obtain maps forming cycles, which is required by the CCLB framework our work is based on.

Pairwise correspondence refinement
The initial pairwise correspondences obtained from Section 3.1 often need further refinement, especially in case of partial shape collections.As shown in Fig. 3 the result of the initial correspondences is sparse and noisy.The ROI transferred through the initial maps T i0 (Fig. 3a) show noisy borders, and the inverse pointwise maps T i,i+1 (Fig. 3b) reveals the sparsity.To refine the initial correspondences, we once again adopt two different approaches according to the presence of partiality in the collection.For the full-to-full cases, the classical iterative up-sampling method called 'ZoomOut' [MRR * 19] is utilized.In the partial setting, we adapt 'Bijective ZoomOut' [RMOW20] to the partial case, which we describe in what follows.
Partial Bijective ZoomOut.In the case of shapes with partiality, we extend the Bijective ZoomOut approach to the partial setting.Throughout this work, S 0 is considered as the full shape and S i as the partial one for the reference-to-instance maps.Similarly, for the consecutive maps, we designate S i as the full shape and S i+1 as the partial one assuming that their coverage is comparable, although they form part-to-part relations in a strict sense.Algorithm 1 outlines our method for bidirectional partial corre- spondence refinement.We first define a new frequency updating rule, based on the slanted diagonal structure in functional maps [RCB * 15] relating a partial shape M to a full shape N .This updating rule is then used in order to refine the functional map in n ZM iterations, starting from an initial frequency k init and ending with the final frequency k N on the full shape N .This refinement process is applied to both reference-to-instance (from S 0 to each S i ) and instance-to-instance (from S i to S i+1 ) correspondences.The refined correspondences yield denser and less noisy results, as shown in Fig. 4. The ROIs transferred through the pairwise refined maps T i0 (Fig. 4a) exhibit smooth borders, and the inverse pointwise maps T i,i+1 (Fig. 3b) become denser.However, there are still some shifted ROIs, which can be observed in the first instance shape on the top row.

Group-wise correspondence refinement
With the bidirectional map refinement implemented, we are now prepared to perform groupwise refinement on both global and local scales within the ROIs.To achieve this goal, we adapt the cycleconsistency refinement scheme [HAGO19] to our case, as presented in Algorithm 2. It consists of five steps: We first transfer the ROI defined on the reference shape S 0 to the instance shapes S i (step 1), and construct k ′ harmonics localized on each ROI by using the Localized Manifold Harmonics (LMH) [MRCB17] (step 2).We then convert each input point-wise maps T ji into two different functional maps by using Equation 4. The functional maps (4) Apply 'Bijective ZoomOut' iteration [RMOW20] with Π N M , Π MN , φ M k M , and φ N k N as input to obtain refined maps Π MN , Π N M , C N M and C MN end for maps the K manifold harmonics Φ Si to the K manifold harmonics Φ S j , while C LMH i j maps the K ′ localized harmonics Ψ Si to the K ′ localized harmonics Ψ S j (step 3).The next step consists of creating two distinct canonical consistent latent basis Y i and Z i for each shape S i .By using the CCLB [HAGO19] approach, a consistent basis Y i of size K × M, M ≤ K is extracted from the functional maps C MH i j .In the same manner, localized consistent basis Pierre Galmiche & Hyewon Seo / Groupwise Shape Correspondence Refinement with a Region of Interest Focus fined on the reference shape S 0 to each instance shape S i .In most cases, the transfer is done by assigning to each point p ∈ S i the value of its corresponding point q = Π i,0 (p) on S 0 .The resulting ROI on the target shape S i is represented by a binary mask (1 inside the ROI and 0 elsewhere).Depending on the dataset considered, however, such pointwise transfer may lead to reduced or misaligned ROIs on S i .To address this, we also incorporate the functional transfer of the ROI.Because of the low pass filter representation of the harmonics, the ROI by functional transfer is smoother and can represent a wider region than expected.Note that conversion from functional maps to pointwise maps is a challenging optimization problem in itself.
LMH computation (step 2).In this step, we compute K ′ LMH Ψ i on each shape S i .The generated harmonics are defined on the whole shape and are compatible with the standard manifold harmonics.Depending on the ROI considered and the ROI transfer method, the LMH parameters η and µ are adjusted to promote the orthogonality of the obtained harmonics with respect to the manifold harmonics and their locality on the input region of interest.
Point-wise to functional map conversion (step 3).We use the Equation (4), C ji = Φ † Si Π i j Φ S j , to convert the initial point-wise maps {Π i j } (i, j)∈E to global and local functional maps.Applying this conversion with Φ Si and Φ S j yields global functional maps C MH i j , while utilizing the localized harmonics Ψ Si and Ψ S j generates local functional maps C LMH i j .
Global and local consistency with CCLB (step 4).To ensure the cycle-consistency of the current local and global functional correspondences, we construct the CCLB [HAGO19] using the functional maps obtained in step 3 as input.The global latent bases Y i obtained with C MH i j provide a compact and coherent representation of the shapes on a global scale.Additionally, the localized latent bases Z i offer a consistent representation only focused on the ROI.
Pointwise recovery (step 5).Point-wise maps are then recovered from the CCLB Y i and Z i by using the pointwise map conversion procedure as in Huang et al [HRWO20].Denote the CCLB as {Y k i } n−1 i=0 and {Z k i } n−1 i=0 , then the problem is formulated as : with , where [•, •] represents the concatenation of the two arrays.This way, the pointwise conversion boils down to the nearest neighbor search for each row of Γ i among the rows of Γ j .By concatenating consistent local harmonics, we incorporate higher localized frequencies on ROI to the nearest neighbors search, thereby improving the accuracy of the conversion.

Dataset
Woman's torso.Our first dataset is a set of female torsos acquired from a patient undergoing post-operative breast radiotherapy.It is composed of one reference surface reconstructed from the CT image, along with multiples scans of the frontal torso surface acquired at each radiotherapy session, which were spaced approximately two weeks apart.Each scan captures different patient poses, respiratory phase, and varying degrees of breast inflation due to radiation therapy.Additionally, the scanned regions also differ from one another, making a particularly challenging setting to the problem.Animated face.A subset of a 3D animated head model has been extracted from a 3D facial expression dataset, CoMA [RBSB18].The head model represents a sequence of 3D facial scans captured from a real person demonstrating an expression ("mouth side" in this work).In addition to the raw scans composed of a variable number of vertices with noises and several holes, we also utilized registered meshes in our experiments.These meshes, obtained through a model-based optimization [LBB * 17], share the share topology as the template mesh (consisting of 5023 vertices and 9976 triangles) and contain holes for the mouth and the neck.
Organic shapes from SHREC.A number of organic shapes, shark, ant, and boy, have been chosen from the SCHREC11 and SHREC15 datasets [LGB * 11] [GDB * 15].We utilize the multiple poses available for each entity as a shape collection.

Implementation
Preprocessing.We remesh the surface meshes of the woman's torso dataset to approximately 10K vertices by using the VCG (Visualization and Computer Graphics Library) surface reconstruction filter in Meshlab [CCC * 08].Meshlab was also used to ensure that real world meshes (Woman's torso and Animated faces) were single connected components.For every shape in all datasets, we compute Laplacian eigenfunctions by using the point cloud method proposed by Sharp and Crane [SC20].
Pairwise shape matching.Two different methods have been used for the initial map depending on the dataset.For the real-world datasets with partiality (such as woman's torso), we used FSPM [LRBB17].We perform 10 iterations of FSPM part-to-full optimization, with the dense and sparse penalties set to 0.5 and 30, respectively, and the maximum number of ICP iterations to 25.As in [LRBB17], we select the top 10% of the best correspondences and use a farthest point strategy to construct delta functions as sparse but well spread descriptors to further refine the correspondences.400 delta functions with radius 7 have been used for this step.Finally, we used a 512 dimensional SHOT [TSDS10] descriptor (instead of a 352 as proposed in the paper) to better deal with the noise.
As for the others (animated face and animals), we make use of the python implementation of the functional map optimization [Mag22] to compute initial functional maps with 60 eigenfunctions, 5 subsample steps, 100 Weight Kernel Signature (WKS) descriptors and the following weights to the energy terms: w descr = w dcomm = w orient = 1, and w lap = 1e−1.
Bidirectional pairwise refinement.In case of datasets with partiality, we perform 5 iterations of the bidirectional partial ZoomOut (Sec.3.2) initialized with functional maps of size 40 × 40 obtained through FSPM.The output is rectangular functional maps of size r × 60, where r is determined by using the partiality between the two input shapes in a similar fashion to [RCB * 15].
To compute the canonical consistent latent bases (CCLB) Y i and Z i , we use the "pyFM" [Mag22] implementation using the initial functional maps containing the manifold harmonics, and the functional maps defined only on the LMH, respectively.

Initial ROI transfer
The initial ROI transfer itself can pose challenges depending on the nature of the data collection.As expected, the real-world data showed more noticeable errors overall, leaving a greater need for further improvements during the refinement steps.Fig. 5 shows the initial ROI transfer we obtained by our refined pairwise correspondences on the woman's torso dataset, real-world raw scans exhibiting a considerable level of partiality.The transferred ROIs show errors ranging from moderate to noticeable.
Figure 5: Initial ROI transfer on the woman's torso dataset: FSPM followed by the bidirectional refinement has been used to transfer ROI from S 0 (leftmost) to each S i , in a pairwise manner.
The raw scans of animated faces show another difficult case of the initial transfer.The chosen ROI (lips), constituting highfrequency features of the face, present a challenging region for accurate transfer, particularly due to the presence of large nonlinear deformations among shape instances.This becomes even more evident when comparing the transfer results on the synthetic and raw scan versions of the animated face, as shown in Fig. 6 and 7.The ROI is conserved well for the synthetic dataset (Fig. 6), both the functional transfer and the pointwise transfer results.The considered shapes are more or less similar, and the use of global harmonics is sufficient.In case of the raw scans (Fig. 7), on the other hand, the functional transfer completely failed, due to the lack of sufficiently high frequencies to represent and transfer the lips region.It confirms that the global nature of the manifold harmonics is not suited for the partial case, resulting in the correspondences of poor quality.This issue has been mitigated by using the product manifold transfer, as depicted in Fig. 8.

Consistent LMH (cLMH)
Fig. 9 and 10 show the original LMH [MRCB17] (on the left side) and our consistent LMH (on the right), for the woman's torso data and the raw scans of animated faces, respectively.Since LMH harmonics are defined independently on each shape, they are not consistent across the entire collection although some shapes possess similar frequencies.We can observe, for instance, different harmonics in the k=4 row of Fig. 9, where the second and third harmonics are of opposite sign and different from the fourth shape.With our approach, the obtained harmonics are both localized and consistent across all shapes.In Fig. 10 is shown the different harmonics obtained with a latent space reduced to M ′ = 6 or M ′ = 10.The resulting harmonics differ according to the final size of the latent space.Indeed, with M ′ = 6, the first harmonics exhibit lower frequencies, whereas with M ′ = 10, the first harmonics present higher frequencies.In both cases, we observe that the resulting harmonics exhibit consistency across all shapes in the collection, while also being concentrated on the ROI (lips).

ROI transfer with consistent LMH
Womans' torso.Fig. 11 shows the pointwise transfer of the ROI on the woman's torso collection by using different approaches.Fig. 11a and 11b present the ROI transfer results obtained by pairwise approaches.We can observe that the initial partial correspondences obtained by the FSPM approach lead to noisy transfer, often with some region shifts.The 2nd and the 8th shapes in Fig. 11a present a significant downward shift, for instance.The use of the bidirectional refinement 11b leads to smoother ROI transfers overall.Additionally, certain regions, such as the 7th and the 8th shape, are better positioned compared to the FSPM transfer.Fig. 11c shows the ROI pointwise transfer obtained with the cycle-consistent approach [HAGO19] with 30 consistent MH.
Fig. 11d depicts the pointwise ROI transfer result using our approach, by extending the latent bases with localised consistent harmonics.Incorporating cLMH in addition to the cMH leads to a ROI closer to the initial one, as can be observed on the second and third shapes, among others.Fig. 13 further shows the improved ROI transfer result by incorporating the cLMH in addition to the cMH.The inclusion of consistent LMH contributed to improved localization of the breast ROI, resulting in a rounder shape, as demonstrated in the second column.
Raw scans of animated face.Fig. 12 shows the ROI transfer results on the raw scans of animated face by using different harmonics.The initial pairwise transfer by using the standard harmonics (Fig. 12a, 1st row) fails to preserve regions.We therefore use its extension with the product harmonics [VLR * 17] (Fig. 12a row) to obtain a favorable initialization, resulting in transfers that are more localized on the lips region.The standard approach fails to transfer the region while the extended approach, while not very precise, leads to transfer that are more localized on the lips.Fig. 12b shows the groupwise refinement results with different numbers of consistent Manifold Harmonics (cMH) and Localized Manifold Harmonics (cLMH).We can observe that the functional transfer by using cMH alone ('without cLMH' on 1st row) does not conserve the ROI well, while the incorporation of merely 10 cLMH (2nd row) demonstrates effective region transfers.
Others.Fig. 14 and 15 show our results on four shape collections: shark (Fig. 14a), ant (Fig. 14b), synthetic animated face (Fig. 15a), Table 2: Mean relative pointwise cyclic error (%, normalized by the bounding box size) on four shape collections in Fig. 14 and 15, arranged in the reading order.The compared methods are the refined pairwise correspondences (Pairwise), the groupwise refinement (Groupwise), and our approach (Ours).The methods coupled with the "ROI" suffix present the error computed only on the vertices on the ROI. and boy (Fig. 15b).For each shape collection, we show the functional ROI transfer (left), pointwise ROI transfer (middle), and the colormap transfer (right).The reference shape is on the leftmost side within each cell, with the ROI highlighted in yellow.The results obtained by using the refined pairwise correspondences (top), their groupwise refinement, [HAGO19] (middle), and our method (bottom) are illustrated.
As expected, the use of the cLMH results in a more localized functional transfer of ROI.This is especially clear in the case of shark (Fig. 14a) and synthetic animated faces (Fig. 15a).For instance, the initial and groupwise correspondences computed on the shark show the functional transfer has succeeded on the tail region but failed on the fin.Moreover, the transfers lack concentration on the ROI, exhibiting non-zero function values across the entire shapes.In contrast, our method achieves an improved functional transfer, focusing on both regions while maintaining low values in other areas.On the synthetic animated face (Fig. 15a), we observe a degraded transfer when using the groupwise correspondences compared to the initial ones.This is contributable to the lack of sufficiently high frequencies to represent the lips when using truncated Manifold Harmonics (MH) alone.In contrast, the proposed method leads to a focused and localized functional ROI transfer solely on the lips, rather than being distributed around its surroundings.In the case of the ant (Fig. 14b) and the boy (Fig. 15b), the initial or groupwise correspondences already achieve accurate positioning of the region of interest in the functional transfers.This is because the ROIs are located in regions of sufficiently low frequency, thus they can be represented well using global harmonics.The improved performance of the consistent LMH can also be seen in the pointwise transfers.On the shark (Fig. 14a), the pointwise transfer by our approach has successfully included the top of the fin within the ROI, in contrast to the groupwise approach.Similarly, the noises have been corrected around the base of antennas on the ant (Fig. 14b), and on the chin of the synthetic animated face (Fig. 15a), and on the shoulder of the boy (Fig. 15b).Our approach persistently focuses on the local region, even in cases where the global map is erroneous, as can observed on some legs of the ants (Fig. 14b).Table 2 presents the mean vertex-wise cycle error on each shape of the four shape collections.The error is computed as the averaged geodesic distance between a vertex and its counterpart obtained through the best cyclic correspondence transfer, e.g. from S 0 to S 1 ,..., and back to S 0 .As expected, our method improves over other methods under comparison, especially in the ROI.As a final note, we observe that in many cases, the use of consistent cLMH alone leads to compa-rable results to combining it with cMH in terms of ROI transfer, demonstrating the effectiveness of our strategy.

Limitations
Our method still has some limitations.To ensure the effective construction of the CCLB, our method relies on initial maps of sufficiently good quality, which may be difficult to obtain in some challenging, real-world cases.Additionally, the LMH parameters have been adjusted heuristically in our experiments.Developing an algorithm to automate this process would broaden its usability across various applications and user groups.While the pointwise recovery from the functional maps has been performed using a nearest neighbor strategy in this paper, our consistent LMH are not tied to this choice.Other pointwise conversions, such as the precise maps [EBC17] or the product harmonics [VLR * 17] can benefit from this consistent functional representation of the ROI.

Conclusions
We have presented a new method for refining non-rigid correspondences within a collection of 3D shapes.It is achieved by introducing the notion of localised harmonics on a user-specified region interest and incorporating them in the Canonical Consistent Latent Basis construction.We have demonstrated their use for the consistent functional transfer of the region of interest (ROI) as well as their positive impact in the pointwise map reconstruction from the functional maps.Compared to existing works, our method demonstrates enhanced robustness to noise, and can handle partiality commonly present in real-world data.We believe that the ability to focus on a specific region, especially when dealing with a collection of partial shapes, is important for many applications in medicine, biology and engineering.Given the direct relevance to the construction of the 'limit shape' [HAGO19] through the CCLB framework, our approach can also be used for the analysis of shape collections, particularly focusing on the ROI.We intend to further investigate this in the future.

Figure 1 :
Figure 1: Front (a) and back view (b) of a shape collection S = {S i } n−1 i=0 .Left: Reference shape S 0 and its ROI in cyan.Right: Deformed instance shapes S i with different coverages.
(a) ROI transfers by using the reference-to-instance pointwise correspondences.The reference shape S 0 with the user-specified ROI is shown on the left.On the right, the instance shapes (S 1 ,..., S 8 ) are arranged from left to right, and top to bottom.(b)Colormap transfers between consecutive instance shapes by using inverse pointwise correspondences.

Figure 3 :
Figure 3: ROI pointwise transfers (a) and inverse pointwise correspondences (b) obtained by using the FSPM [LRBB17] on the woman's torso data.

©
2023 The Authors.Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.
(a) ROI transfers by using the refined (reference-to-instance) pointwise correspondences.The reference shape (S 0 ) with the user-specified ROI is shown on the left.On the right, the instance shapes (S 1 ,..., S 8 ) are arranged from left to right, and top to bottom.(b)Colormap transfers between consecutive instance shapes by using the refined inverse pointwise correspondences.

Figure 4 :
Figure 4: Pairwise correspondences after the refinement on the woman's torso data.(a) ROI pointwise transfers; and (b) inverse pointwise correspondences.

Algorithm 1 :
Bidirectional Partial ZoomOut Input : A partial shape M, a full shape N and their corresponding eigenfunctions {φ M k M } {φ N k N }, initial pointwise maps Π MN and Π N M , number of iterations n ZM , initial frequency k init Output : Refined pointwise maps Π MN and Π N M and functional maps C N M and C MN

1 i=0( 1 )
Finally, we convert the latent basis into point-wise maps by using the nearest neighbors strategy proposed by Huang et al.[HRWO20] (step 5).We provide a detailed description of each step below.Algorithm 2: Groupwise refinement with ROIInput: FMN G(E, F), with shapes S = {S i } n−1 i=0 and edges (i, j) ∈ E, corresponding initial pointwise maps {Π ji }, spectral embeddings sizes K, K ′ , M, M ′ Output : Refined point-wise maps {Π i j } (i, j)∈E , Latent Basis {Y i } n−1 i=0 , ROI localized Latent Basis {Z i } n−Transfer reference shape (S 0 ) ROI to each instance shapes S i (i = 1, . . ., n − 1).(2)For each shape S i , compute K ′ LMH on the ROI of each shape.(3) For (i, j) ∈ E, construct the LMH and MH functional maps C LMH i j and C MH i j by converting the pointwise maps Π ji by using Equation 4. (4) Compute M and M ′ global and localized latent bases Y i and Z i on each shape S i by using CCLB[HAGO19].(5) Recover refined pointwise maps Π ji for each edge (i, j) by using a neareast neighbors strategy between the latent functional embeddings.ROI transfer (step 1).We transfer the region of interest (ROI) de-© 2023 The Authors.Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

©
2023 The Authors.Computer Graphics Forum published by Eurographics and John Wiley & Sons Ltd.

FunctionalPointwiseFigure 6 :Figure 7 :
Figure 6: The initial ROI transfer from the reference (the leftmost shapes) to each of the instance shapes, on the synthetic dataset of the animated face.

Figure 8 :
Figure 8: Functional transfer on the raw scan dataset of the animated face by using the product harmonics [VLR * 17].

Figure 14 :
Functional (left) and pointwise (middle) transfers of the ROI and colormap (right) obtained on the shark and the ant collections.Each cell illustrates the results of the refined pairwise correspondences (top), the groupwise refinement (middle), and our approach (bottom).
Pierre Galmiche & Hyewon Seo / Groupwise Shape Correspondence Refinement with a Region of Interest Focus ROI transfer (functional) ROI transfer (pointwise) Colormap (a) (b) Figure 15: Functional (left) and pointwise (middle) transfers of the ROI and colormap (right) obtained on the animated face and the boy collections.Each cell illustrates the results of the refined pairwise correspondences (top), the groupwise refinement (middle), and our approach (bottom).