Recovering low‐rank tensor from limited coefficients in any ortho‐normal basis using tensor‐singular value decomposition

National Natural Science Foundation of China, Grant/Award Number: 61801513; Information Security Laboratory of National Defense Research and Experiment in 2020, Grant/Award Number: 2020XXAQ02 Abstract Tensor singular value decomposition (t‐SVD) provides a novel way to decompose a tensor. It has been employed mostly in recovering missing tensor entries from the observed tensor entries. The problem of applying t‐SVD to recover tensors from limited coefficients in any given ortho‐normal basis is addressed.We prove that ann� n� n3 tensor with tubal‐rank r can be efficiently reconstructed by minimising its tubal nuclear norm from its O(rn3n log(n3n)) randomly sampled coefficients w.r.t any given ortho‐normal basis. In our proof, we extend the matrix coherent conditions to tensor coherent conditions. We first prove the theorem belonging to the case of Fourier‐type basis under certain coherent conditions. Then, we prove that our results hold for any ortho‐normal basis meeting the conditions. Our work covers the existing t‐SVD‐based tensor completion problem as a special case.We conduct numerical experiments on random tensors and dynamic magnetic resonance images (d‐MRI) to demonstrate the performance of the proposed methods.

Chen et al. [16] proposed a LRTC method via t-SVD for image and video recovery. Zhang and Aeron [6] proved (and then Lu et al [14] corrected) that it is possible to exactly complete a tensor with high probability from the O(r min{n 1 , n 2 }n 3 log 2 (min{n 1 , n 2 }n 3 )) uniformly sampled elements by enforcing the low tubal rankness of it, given the tensor size is n 1 � n 2 � n 3 and tubal-rank is r.
In some applications such as synthetic aperture radar (SAR) imaging [17,18], dynamic magnetic resonance imaging (d-MRI) [19,20], a limited number of measurements, which are expansion coefficients with respect to some basis, are obtained to reconstruct the original images. For example, sparse sampling in MRI can accelerate imaging speed significantly [16,[21][22][23][24][25]. In recent years, we have tried to use different tensor decomposition frameworks in sparse sampling MRI. In our previous work [21], the TT decomposition framework is used to improve the quality of dynamic MR image reconstruction under sparse sampling. In ref. [22], we combined the partially separable scheme with tensor low tubal rank constraint to further reduce sampling data. In the conference paper [25], we gave the result of using t-SVD model and the sparsity for MRI reconstruction. Different from ref. [25], we focus on theoretical analysis and proof, and give the theorem and coherence conditions for the tensor reconstruction from limited coefficients in any ortho-normal basis. In addition, we demonstrate the experimental results of the proposed method and its variants substantially.
We address the problem of applying t-SVD to reconstruct tensors from limited coefficients in any given ortho-normal basis. The tensor recovery problem we focus on can be seen as a generalisation of the tensor completion problem in ref. [6]. With our theoretical support, the prior low tubal rankness based on t-SVD framework can not only be applied to video completion problems like Zhang and Aeron did in [6,7], but also can be applied to other applications. We propose a tensor reconstruction model based on t-SVD and derive theoretical performance bounds for it. The main theoretical result is Theorem 2 in Section 3. Specifically, we extend the matrix coherent conditions in [26] to tensor coherent conditions and prove that our results hold for any ortho-normal basis meeting the conditions. In the proof, we first validate our results in the case of Fourier-type basis. Then, we generalise the proof to any ortho-normal basis using mathematical induction with two modifications. Similar to ref. [6], we also have to handle the block diagonal constraint in the proof procedure, which leads to subtle differences with matrix reconstruction problems in ref. [26]. We adopted the Golfing scheme 1 in our proof which is widely used after [26] first introduced. The difference is in our generalisation, the critical step is how to verify that the condition (2, b) in Proposition 1 still holds under any ortho-normalised basis. Lastly, we conduct numerical experiments (including dynamic MR image reconstruction) to demonstrate the performance of the proposed methods.
The remainder is organized as follows. In Section 2, t-SVD is reviewed briefly. In Section 3, the main theoretical result is given. In Section 4, the proofs of the main theoretical result are provided. In Section 5, the experimental results are presented to demonstrate the effectiveness of the proposed method. In Section 6, the conclusion is made. In Section 7, the appendices is present.

| Notations and preliminaries
A summary of symbols and notations is shown in Table 1. ∈ C n 1 n 3 �n 2 n 3 ð1Þ In the scenario of tensor decomposition, a slice is defined as a matrix obtained by fixing every index but two indexes of a tensor. A fibre is defined by fixing every index but one. For an order-3 tensor A, we use the MATLAB notation Aðk; :; :Þ, Að:; k; :Þ and Að:; :; kÞ to denote the k th horizontal, lateral and frontal slices, and Að:; i; jÞ, Aði; :; jÞ and Aði; j; :Þ to denote the (i,j)th mode-1, mode-2, mode-3 fibre. We use b A ¼ fftðA; ½ �; 3Þ as the tensor obtained by applying the 1D-FFT along the third dimension of A. Especially, we use A ðkÞ to represent Að:; :; kÞ, and use b A ðkÞ to represent b Að:; :; kÞ. Let A denote the block-diagonal matrix of the tensor b A in the Fourier domain, as shown in Equation (1).
The following preliminaries will be used frequently.

| Tensor singular value decomposition
The introduction to t-SVD below is for the reader to understand our main result, more information about t-SVD can be found in refs. [6,9,[27][28][29].

Definition 2
Tensor transpose [28,29]. We denote the transpose of an n 1 � n 2 � n 3 tensor A as A T of size n 2 � n 1 � n 3 . A T can be obtained from A by transposing each of the frontal slices first, then reversing the order of transposed frontal slices 2 through n 3 .

Theorem 1
Tensor singular value decomposition (t-SVD) [9,27]. The t-SVD of A ∈ R n 1 �n 2 �n 3 is given by where U and V are orthogonal tensors of size n 1 � n 1 � n 3 and n 2 � n 2 � n 3 , respectively, S is a rectangular f-diagonal tensor of size n 1 � n 2 � n 3 . The t-SVD for the order-3 tensor case is shown in Figure 1.

Definition 6
Tensor multi-rank and tubal-rank [9]. The multi-rank of a tensor A ∈ R n 1 �n 2 �n 3 is a vector m ∈ R n 3 �1 with its ith entry as the rank of the ith frontal slice in Fourier domain, that is Þ. The tensor tubal-rank rðAÞ, is defined as the number of non-zero singular tubes of S, where A ¼ U * S * V T . The symbol # below describes the number of i satisfying the condition Sði; i; :Þ ≠ 0.
In t-SVD, the main information features of tensors can be grasped by keeping only the non-zero singular tubes in S, for example, by truncating t-SVD to achieve dimensionality reduction, and it can approximate the original tensor. According to Theorem 1, the reduced (truncated) t-SVD of A is given by where U r and V r are orthogonal tensors of size n 1 � r � n 3 and n 2 � r � n 3 , respectively, S r is a rectangular f-diagonal tensor of size r � r � n 3 . If r = min{n 1 , n 2 }, the third-order tensor A is of full tubal-rank.

Definition 7
The tensor nuclear norm (TNN) [6,9]. TNN norm kAk T NN is defined as the sum of the singular values of all frontal slices of b A, and is a convex relaxation of the tensor tubal-rank [7,8].
Definition 8 Inner product of tensors [6]. The inner product between the order-3 tensors A ∈ R n 1 �n 2 �n 3 and B ∈ R n 1 �n 2 �n 3 is defined as where 1 n 3 comes from the normalisation constant of the FFT. Then, the Frobenius norm kAk F is induced as: Ak F . The definition of tensor inner product makes it possible to simplify tensor analysis with matrix analysis techniques.
Remark 1 Tensor operator via t-product [9]. For a tensor operator denoted by a tensor C with the size of n 4 � n 1 � n 3 , which means mapping tensor B with the size of n 1 � n 2 � n 3 to A with the size of n 4 � n 2 � n 3 via t-product, we can transform it into the equivalent form in Fourier domain for computational efficiency as follows, where A is a matrix of size n 4 n 3 � n 2 n 3 , B is a matrix of size n 1 n 3 � n 2 n 3 , and C is a matrix of size n 4 n 3 � n 1 n 3 .

Definition 9
Ortho-normal basis. The ortho-normal basis fW a g n 1 n 2 n 3 a¼1 composed of the basis W a with size n 1 � n 2 � n 3 meets the orthonormality, that is kW a k F = 1 for all a = 1, 2, …, n 1 n 2 n 3 and W i • W j = 0, i≠j, where • denotes Hadamard product and i, j ∈{1, 2, …, n 1 n 2 n 3 }.
Definition 10 Standard operator basis [26]. The basis fW a g n 1 n 2 n 3 a¼1 composed of the basis W a with size n 1 � n 2 � n 3 is standard operator basis, if each basis W a has only non-zero element 1 at the corresponding a th location index of W a ð:Þ 2 . The standard operator basis meets the orthonormality, is a special case of orthonormal basis.

| THEOREM AND METHOD
Given the fact that an unknown tensor X can be expanded in terms of an ortho-normal basis fW a g n 1 n 2 n 3 a¼1 (referred to as an arbitrary operator basis 3 ) below: where 〈, 〉 denotes tensor inner product defined in Definition 8. W a and X have the same size of n 1 � n 2 � n 3 . The expansion coefficients of X with respect to the basis fW a g n 1 n 2 n 3 a¼1 is ⟨W a ; X ⟩, where a = 1, 2, …, n 1 n 2 n 3 . If the expansion coefficients are partially known, how can X be reconstructed from them? Obviously, this is an ill-posed problem. Generally, regularizations are employed to enforce the prior knowledge of X to solve this problem. Based on the observation that X is a low-rank tensor, we propose to reconstruct X by solving the following convex optimization problem: where ⟨W a ; N ⟩ is the partial known coefficients, X is the unknown tensor to be recovered, both X and N are order-3 tensors with the size of n 1 � n 2 � n 3 . Ω ⊂ [1, n 1 n 2 n 3 ] is a random index set of size m, which means that the coefficients ⟨W a ; N ⟩ are known for all a ∈Ω. Here, tensor nuclear norm (TNN) is used to enforce the tensor tubal-rank defined in t-SVD framework.
If fW a g n 1 n 2 n 3 a¼1 is the standard operator basis, the coefficients are elements of the tensor itself. Paper [6] has given the answer to this case by solving the optimization problem below.
As shown in Figure 2, TNN norm kX k T NN is defined as the sum of the singular values of all frontal slices of b X . We use block diagonal procedure for the calculation of TNN norm, which saves computation time compared with directly constraining the tubal rank. The procedure of block diagonal constraint is shown in Figure 2. The rank of block diagonal matrix satisfies: rankðX Þ ≤ minðn 1 n 3 ; n 2 n 3 Þ. This concept runs through the procedure of proof. When tensor X is rearranged into the tensor with the size of n 2 � n 1 � n 3 , we get a new matrix X , the rank of this new X is the same as that of the previous X . But when tensor X is rearranged into the tensor with the size of n 2 � n 3 � n 1 or n 3 � 2 The arbitrary operator basis here means any basis meeting the orthonormality. 3 A detailed description of Golfing scheme is provided in Appendix 7.3.

MA ET AL.
n 2 � n 1 or n 1 � n 3 � n 2 or n 3 � n 1 � n 2 , the rank of the new X is not the same as that of the previous X . This is also the reason why we use (n, n, n 3 ) to simplify the representation of (n 1 , n 2 , n 3 ) in some places, where n = min(n 1 , n 2 ).
In order to recover N from the partially known coefficients by solving the problem of Equation (6), the question needs be addressed is: given that the tensor tubal-rank r ⩽ n and n = min(n 1 , n 2 ), how many randomly chosen coefficients are needed to efficiently reconstruct tensor X of size n 1 � n 2 � n 3 ?
Based on previous conclusions from the low-rank matrix completion [30,31], it is easy to deduce that tensor X cannot be reconstructed if X has very few non-zero coefficients with respect to the basis fW a g. To ensure that each coefficient contains enough non-trivial information, some conditions are needed to characterise the incoherence between the tensor and the basis. Intuitively, tensors with small operator norm are 'incoherent' to all low-tubal rank tensors simultaneously. The 'incoherence conditions' have been proposed in tensor completion using t-SVD [6]. The 'incoherent' has a wellknown analogue in compressed sensing [32][33][34]. There, one uses the fact that 'vectors with small entries' are incoherent to 'sparse vectors'. Ref. [26] proved that matrices with small operator norm are 'incoherent' to all low rank matrices simultaneously. The definition of coherence stated below is closely related to, but more general than, the parameter μ 0 used in ref. [6]. We follow the form of definitions in ref. [26] and define coherence conditions of a tensor, which are extensions of coherence condition of a matrix. Then, we give the answer to the above question under the defined tensor coherence conditions.
To state our results clearly, we need to introduce some notation. We introduce the orthogonal decomposition x → ; y → are arbitrary tensor columns with the size of n 1 � 1 � n 3 , n 2 � 1 � n 3 , respectively, and k = 1, 2, …, r. The orthogonal projection P T onto T is given as follows, and the projections onto the orthogonal complement T ⊥ is given as follows, Define a random variable δ a = 1 a∈Ω where 1 (⋅) is the indicator function. Let R Ω : R n 1 �n 2 �n 3 → R n 1 �n 2 �n 3 be a random projection as follows, where p ¼ m n 2 n 3 is the probability of that a ∈ [1, n 1 n 2 n 3 ] is included in Ω. Then, we have P Ω ðAÞ ¼ ∑ a δ a ⟨W a ; A⟩W a . As in the previous work [6,30,31], the sampling model we use is the Bernoulli model. As we know, there are also other commonly used random models, like sampling with replacement and sampling without replacement. Similar to matrix completion problems, researchers can get corresponding recovery guarantees by slightly changing the proof procedure [31,35].
Next, we present our definition of coherence conditions and new theorem for tensor reconstruction problem (Equation 6) using tensor-singular value decomposition (t-SVD).

Definition 11
Tensor coherence conditions. The order-3 tensor M ∈ R n 1 �n 2 �n 3 has coherence ν with respect to an operator basis fW a g n 1 n 2 n 3 or the two estimates hold, where n = min{n 1 , n 2 } and U � S � V T is the reduced t-SVD of M. The first inequality in Definition 11 is the tensor coherence condition for reconstructing low tubal-rank tensors from limited Fourier-type coefficients. Inequalities (Equations 12,13) are suitable for the general case that is reconstructing low 166tubal-rank tensors from the limited coefficients with respect to any ortho-normal basis.
Theorem 2 Suppose M ∈ R n 1 �n 2 �n 3 and its reduced t-SVD is given by M ¼ U � S � V T where U ∈ R n 1 �r�n 3 , S ∈ R r�r�n 3 and V ∈ R n 2 �r�n 3 . Suppose M satisfies the coherence condition above with parameter ν > 0. Then there exist constants c 0 , c 1 , then M is the unique minimizer to Equation (6) with probability at least 1 − c 1 ððn 1 þ n 2 Þn 3 Þ −c 2 . Theorem 2 is the main theoretical result.
Reference [6] concentrated only on the problem of tensor completion where it aims to recover a low-tubal rank tensor from randomly selected tensor elements. While our paper extends tensor completion to a more general scenario where the coefficients under any ortho-normalised basis are randomly sampled to recover the tensor. The tensor recovery problem we focus on can be seen as a generalisation of the tensor completion problem in ref. [6]. Similar to ref. [6], we also have to handle the block diagonal constraint in the proof procedure, which leads to subtle differences with matrix reconstruction problems in ref. [26]. We adopted the Golfing scheme3 in our proof which is widely used after [26] first introduced. The difference is in our generalisation, the critical step is how to verify that the condition (2, b) in Proposition 1 below still holds under any ortho-normalised basis. In addition, in our proof, we use the conclusion in Lemma 3 and mathematical induction to get Equation (36) in Appendix 7.2.
The algorithm is designed for solving Equation (6). We turn the constraints ⟨X ; where ∀a ∈ Ω and • denotes Hadamard product and R is the binary tensor contains 0 and 1. The index set of nonzero elements in R is Ω. ψ is an operator, which acts on X to get the expansion coefficients of X with respect to basis fW a g, for all a = 1, 2, …, n 1 n 2 n 3 . B is the undersampled coefficients. Then, we rewrite Equation (6) as an unconstrained convex optimization problem: where λ is a regularisation parameter. We develop an alternating direction method of multipliers (ADMM) algorithm to solve the problem Equation (15). We introduce an auxiliary variable Z ¼ X , the augmented Lagrangian function of Equation (15) is given by where Q denotes the Lagrangian multiplier, and ρ > 0 is called the penalty parameter. By applying ADMM, each sub-problem is performed at each iteration as follows: The variables Z and Q are initialized as tensors full filled with zeros before the above sub-problems can be alternately solved. In sub-problem (Equation 17) (linear least-squares problem), the value of X at the (t + 1) th iteration step is X tþ1 , which can be obtained by taking derivative of Equation (16) with respect to X and making this derivative equal to zeros: where ψ 0 is the inverse operator of ψ, ψ 0 ψ = I, and I denotes the tensor with every entry being 1.
The details of solving sub-problem (Equation 18) are as follows. It is well known that block circulant matrices can be block diagonalized by using the Fourier transform. This property can be used to simplify the calculation of t-SVD [9,27,28]. We compute t-SVD decomposition using matrix SVDs in Fourier domain and give the algorithm for solving the problem (Equation 18) as shown in Algorithm 1, where shrink1 ρ ðSÞ is singular value thresholding (SVT) function [36,37].

Algorithm 1 For solving problem Equation (18)
The whole developed ADMM algorithm named as t-SVD method for solving the problem (Equation 15) is shown in Algorithm 2.

| PROOF OF THEOREM 2
In order to prove Theorem 2, we need to prove that kM þ Zk T NN > kMk T NN , for any Z supported in Ω c . Since kM þ Zk T NN > kMk T NN , for any X ≠ M obeying  (1) and (2) in Proposition 1 are satisfied with probability at least 1 − c 1 ððn 1 þ n 2 Þn 3 Þ −c 2 . So, we drive that the M in Theorem 2 would be the unique minimizer to Equation (6) with probability at least 1 − c 1 ððn 1 þ n 2 Þn 3 Þ −c 2 . Then, the proof of our Theorem 2 is finished. (14), then tensor M is the unique minimizer to Equation (6) if the following conditions hold:

Proposition 1 Suppose p satisfies Equation
There exists a dual certificate tensor Y; P Ω ðYÞ ¼ Y and aÞ k P T ðYÞ − U � V T k F ≤ 1 4nn 2 3 bÞ k P T ⊥ ðYÞ k ≤ 1 2 Lemma 1 Suppose kP T R Ω P T − P T k op ≤ 1 2 , then for any tensor Z and P Ω ðZÞ ¼ 0, we have For the proof of Proposition 1 and Lemma 1, we first give the proof for Fourier-type basis in this section, then present the relatively simple modifications to cover the general case in Appendix 7.2.
As we all know, Fourier basis is a typical vector basis with small vector spectral norm which satisfies the incoherent condition. In ref. [26], the matrix basis is called Fourier-type basis as long as the operator norm satisfies relatively small conditions. Similarly, we define the tensor basis of the condition in the operator norm satisfying Equation (11) as the Fourier-type basis. First, we prove that Theorem 2 holds in the case of Fourier-type basis. Then, we modify the proof in two places and propose two new lemmas (Lemma 3 and Lemma 4) to prove that Theorem 2 holds in the case of any ortho-normal basis.
As mentioned above, the inequality (Equation 11) in Definition 11 is the coherence condition for reconstructing low tubal rank tensors from limited Fourier-type coefficients. Next, we show that the construction of Y using Golfing scheme continues to work if assumption (Equation 11) on the operator norm of the basis' elements.
Before continuing, the theorem below will be used in the proof of Proposition 1(1) and Lemma 1-4, which was first developed in ref. [38].
Theorem 3 [38] Let X 1 , X 2 ,…, X L be independent zero-mean random matrices of dimension d 1 � d 2 . Suppose almost surely for all k. Then for any τ > 0, Theorem 3 is a corollary of Chernoff bound for finite dimension operators developed from ref. [39]. Usually Equation (22) is called Non-commutative Bernstein Inequality (NBI). Reference [40] gave an extension of Theorem 3, stated that if ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi ffi for any c > 0. Then Equation (22) becomes

| Proof of Proposition 1(1)
Proof: As in the previous work [6,30,31], we employ the Bernoulli model as the sampling model. Then ER Ω � ¼ I . Note that which gives and Given any tensor Z of size n � n � n 3 , we can decompose P T ðZÞ as the following

This gives
Define operator T a which maps Z to 1 p δ a ⟨Z; P T W a ⟩P T W a , then observe that kT a k op ¼ kT a k ≤ 1 p kP T W a k 2 F , and kP T k op ¼ kP T k ≤1, we have We derive the above inequality by using the coherence assumption (Equation 12) and the fact that if A and B are positive semi-definite matrices, then kA − Bk ≤ max{kAk, kBk}.
From Equation (25) we have ET a � ¼ 1 with some constant β > 1. The inequality holds given p satisfying Equation (14) with c 0 large enough. Using Theorem 3, we have which finishes the proof. MA ET AL.

| Proof of Lemma 1
Proof: Given any tensor Z as a perturbation with P Ω ðZÞ ¼ 0 and according to Proposition 1(1): On the other hand, according to Equation (9), we observe that Combining the last two display equations gives So, under the conditions of Theorem 2, p ≥ 1 2ðnn 3 Þ 2 holds clearly, which finish the proof.

| Proof of Proposition 1(2)
Before validating Proposition 1(2), we need to introduce Lemma 2 which will be used in this part. (14) in Theorem 2, and Z ∈ T sized n � n � n 3 . Then for some constant c 3 , we have

Proof of Proposition 1(2):
a. Here, we construct a dual certificate Y and show it satisfies both conditions, that is, Proposition 1(2). We use the approach called Golfing scheme introduced in ref. [26] and construct the tensor dual certificate Y iteratively follows the idea in refs. [6,41]. A more detailed description of Golfing scheme is provided in Appendix 7.3.
Let Ω be a union of smaller sets Ω t such that Ω ¼ ⋃ t 0 t¼1 Ω t where t 0 ≔ 2 log 2 (2nn 3 ). For each t, we assume and is independent of all others. Clearly, this Ω t is equivalent to the original Ω in our Bernoulli model.
Let G 0 ≔ 0 and for t = 1, 2, …, t 0 , then set a tensor Y ¼ G t 0 and we have P Ω ðYÞ ¼ Y by the above construction.
Note that Ω t is independent of D t , and if p satisfies Equation (14) in Theorem 2, then q ≥ p/t 0 ≥ c 0 νr log(2nn 3 )/n. According to Proposition 1(1), we have for each t. Applying the above inequality recursively, we get holds with probability at least 1 − c 0 ð2nn 3 Þ −c 00 by the union bound, for some positive constants c 0 , c 00 large enough.
holds with probability at least 1 − c 1 ð2nn 3 Þ −c 2 by union bound for some large enough constants c 1 , c 2 > 0.

| Proof of Lemma 2
Observe that We use the Bernoulli model. So, E½δ a � ¼ E½δ 2 a � ¼ p, and E½R Ω � ¼ I . Note that if Z ∈ T , then P T ⊥ Z ¼ 0, At the end of the above inequalities, we use the inequality Equation (11). Using Equation (11), we also have F I G U R E 3 Reconstruction of order-3 tensor (32 � 32 � 20) with different tubal rank r from their coefficients in db4 basis with different sampling rates. In the figures on the left, the white cell stands for exact reconstruction, and black one stands for the failure. The figures on the right depict the RLNE curves of one typical run of the simulation. The value of each cell is the RLNE of the recovery under the corresponding sampling rate and tubal rank. The colour scale ranges from 0 to 0.55 holds with probability at least 1 − ð2nn 3 Þ −ðc−1Þ for any constant c, given p satisfying Equation (14), which finish the proof of Lemma 2.

F I G U R E 4
Reconstruction of order-3 tensors with different tubal rank r from their coefficients in Fourier basis with different sampling rates. In the figures on the left, the white cell stands for exact reconstruction, and black one stands for the failure. The figures on the right depict the RLNE curves of one typical run of the simulation. The value of each cell is the RLNE of the recovery under the corresponding sampling rate and tubal rank. The colour scale ranges from 0 to 0.55 We conduct experiments on random tensors and dynamic MR images to estimate the performance of our methods. All simulations were carried out on Windows 10 and MATLAB R2016a running on a PC with an Intel Core i5 CPU 3.2 GHz and 12 GB of memory. For quantitative evaluation, the reconstruction quality was measured by the relative least normalised error (RLNE) and structural similarity (SSIM).
RLNE is a standard image quality metric indicating the difference between the reconstruction X * and the original tensor X : The structural similarity (SSIM) index 4 [42] attempts to measure the change in luminance, contrast, and structure between X * ð:; :; iÞ and X ð:; :; iÞ: Here, i = 1, 2, …, N 3 , μ 1,ii and μ 2,ii are mean intensities at the iith local window of X * ð:; :; iÞ and X ð:; :; iÞ, while σ 1,ii and σ 2,ii are the corresponding SDs. σ 12,ii denotes the covariance and the constants ϵ 1 , ϵ 2 are included to avoid instability. Then, we use the mean SSIM ζ SSIM ¼ 1 N 3 ∑ N 3 i ζ SSIM i as a metric for the quality of the reconstructed tensor.

| Random tensor reconstruction
In order to evaluate the performance of our t-SVD methods, we conduct numerical simulations on real-valued random order-3 tensors with tubal rank exactly r. To create such a tensor, we generate an order-3 tensor with a i.i.d. Gaussian distribution first, and then find its tubes via t-SVD. We keep the r largest tubes but set the rest to zeros, then get the tensor X according to Equation (4). In the signal recovery application, the Fourier base is the most commonly used, while the application under other orthogonal bases is seldom used. We only show the experimental results under the orthogonal Fourier base and the db4 wavelet basis. Data acquisition is simulated by undersampling Fourier coefficients and db4 wavelet coefficients of the frontal slices X ð:; :; kÞ. The number of the measurements is quantified in terms of the percentage of the number of fully sampled coefficients, referred to as sampling rate (SR). Here, we use variable sampling masks with different sampling rates. We reconstruct these random tensors using t-SVD method and compute the relative least normalised error (RLNE). If the RLNE ≤0.05, we claim the reconstruction is exact. The experimental results under the Fourier transform and db4 wavelet transform are shown in Figures 3 and 4, respectively. In the figures on the left, the white cell stands for exact reconstruction, and black one stands for the failure. The figures on the right depict the RLNE curves of one typical run of the simulation. The value of each cell is the RLNE of the recovery under the corresponding sampling rate and tubal rank. The colour scale ranges from 0 to 0.55.
As shown in Figures 3 and 4, more accurate results can be obtained from a widely range of lower tubal rank and lower sampling rate, while more accurate results can also be obtained from a widely range of higher tubal rank and higher sampling rate. Under the same tubal rank and sampling rate, the RLNE under orthogonal Fourier basis is smaller than that of db4 wavelet, which means that the orthogonal Fourier basis is better than that of db4 wavelet in the case studies.
The performance of the proposed t-SVD method depends on the regularisation parameter pair (λ, ρ). We first generate a random order-3 tensor of size 30 � 30 � 20 with tubal rank r = 5. Then, the parameters (λ, ρ) were optimised based on this tensor using 1/2 undersampling variable density mask. We set the maximum number of iteration t max = 200 and convergence condition η tol = 10 −5 . Under the orthogonal Fourier transform base, Figure 5 shows the parameter optimization based on RLNE from different pairs of λ and ρ. It can be seen that the optimization setting for λ is less than 0.1 while the low-rank regularisation setting ρ is from 0.1 to 50.

| Image reconstruction
Based on Definition 6, we can show the low tubal rankness of the d-MRI data of size n 1 � n 2 � n 3 by plotting {δ i } which is defined as follows. Considering dynamic MR images 5 have low tubal-rank as shown in Figure 6, we enforce low tubal rank to reconstruct dynamic MR images from highly undersampled k-t space data. This problem can be formulated as below.
where X denotes a dynamic magnetic resonance image with the size of n 1 � n 2 � n 3 , F s is the spatial Fourier operator, that is F s X denotes the Fourier coefficients along the spatial dimension. R is the undersampling tensor mask, which selects 0 or 1 as its elements randomly. • denotes Hadamard product.
B is the undersampled k-t space measurements with the size of n 1 � n 2 � n 3 . In dynamic MR image reconstruction, the sampling should be different along the n 3 dimension of the k-t space.
The state-of-the-art CS d-MRI methods including L1-TV [44], k-t SLR [20], RPCA-DMRI [45], TuckerDMRI [19] and PS-L1 [23] etc. We have conducted various comparisons of k-t tSVDTV (the t-SVD method combined with sparsity), k-t SLR, RPCA-DMRI, L1-TV, and TuckerDMRI methods in our previous papers [21,25]. Zero-filled method fills zeros into the undersampled k-space, then gets the reconstructed image is the inverse operator of F s . In most low-rank-based d-MRI methods such as k-t SLR [20], RPCA-DMRI [45] and PS-L1 [23], a 3D dynamic MRI data was unfolded into a 2D matrix, then low matrix-rankness that reflected the inherent spatiotemporal correlation was enforced to reconstruct images. For example, the k-t SLR method first unfolded the 3D dynamic MRI data into a 2D matrix, then low matrix-rankness that reflected the inherent spatiotemporal correlation was enforced to reconstruct images. Simultaneously, it integrated total variation (TV) as sparsity constraint. We call the reconstruction method that only constrains low rankness of the unfolded matrix as unfolding method. Instead of unfolding, tensor decomposition can be used to discover the inherent structural features of dynamic MRI data. In ref. [19], the authors proposed Tucker decomposition based dynamic MRI method (TuckerDMRI). TuckerDMRI treated the dynamic images as the sum of a sparse and a low rank component, and then enforced the sparsity of the sparse component and low rankness of the other component by constraining the low rankness of mode-n matrices. We name the method that only constrains the low rankness of mode-n matrices as Tucker method. We conduct comparisons of three methods exploiting only low-rankness: (1) t-SVD method. (2) Tucker method. (3) Unfold method.
The key parameters of the above methods were chosen empirically by minimising the RLNE of reconstruction results over a range of possible values. In the following experiments of t-SVD method, we set the maximum number of iteration t max = 200, convergence condition η tol = 10 −5 , λ = 1 and ρ = 0.003. The sampling mask and the original d-MR images used in the following experiments are shown in Figure 7. Experiments were conducted on both complex-valued and real-valued cardiac data sets. The Cardiac MR images 1: Obtained from Bio Imaging and Signal Processing Lab (http:// bispl.weebly.com/), this data set contains N 3 = 25 temporal frames of size N 1 = N 2 = 256 with a 345 � 270 mm 2 field of view (FOV) and 10 mm slice thickness. The acquisition sequence was steady-state-free precession (SSFP) with a flip angle of 50°and TR = 3.45 ms. The Cardiac MR images 2: A numerical human cardiac MR phantom with quasi-periodic heartbeats provided in http://mri.beckman.illinois.edu/software.html. The phantom was created from real human cardiac MRI data which were collected using retrospective ECG-gating during a single breath-hold and used to generate a time series of images representing a single prototype cardiac cycle [23]. Acquisition parameters: acquisition matrix size = 200 � 256, field-of-view (FOV) = 273 � 50 mm, effective spatial resolution 1.36 � 1.36 mm, slice thickness = 6 mm, and TR = 3 ms.
The RLNEs (%) and SSIMs of the reconstructed images by the four methods with different SRs are shown in Table 2. The RLNEs and SSIMs of every spatial frame under 30% sampling mask for Cardiac MR images 1 reconstruction are shown in Figure 8. The visual comparisons of the reconstructed results by different methods under the 30% sampling mask are shown in Figures 9 and 10. We can see that the Unfold method yields better performances over other methods. The t-SVD method demonstrates better resluts than Tucker method and comparable results as the unfold method, which verifies that t-SVD framework can be used to reduce the amount of the required sampling data in d-MRI.
The unfold method uses the correlation between spatial frames (frontal slices), which is an effective method for d-MRI, but when all the sampling mask of the frontal slices are the same, as shown in Table 3, the unfold method does work well. Moreover, when the reconstructed data are random tensors with low tubal rank rather than the similar images, the unfold method does not work well either. Our t-SVD method can overcome the above problems. We can see that Tucker method is comparable to t-SVD method when all the sampling mask of the frontal slices are the same. More numerical results related with the t-SVD method can be found in our previous papers [21,22,25].
All the above experiments demonstrate that an unknown tensor can be efficiently reconstructed from randomly coefficients by minimising tensor nuclear norm.
Computational complexity: given an 3-order tensor X n 1 �n 2 �n 3 , the computational complexity in Tucker method mainly depends on the nuclear norm minimization of X ðnÞ . In t-SVD method, it needs the nuclear norm minimization of each block in the block-diagonal matrix. The two methods require computing singular value decompositions (SVD) at each iteration step. The computation burden increases rapidly as matrix sizes and ranks increase. Assuming n i = I for all i = 1, 2, 3, the computational complexity in Tucker method is O(3I 3 ), while in t-SVD method is O(3I 2 ). Therefore, the overall complexity is O(C3I 3 ), O(C3I 2 ) respectively, where C is the number of iterations.

| CONCLUSION
Enforcing low rankness was proved to be effective in sparse sampling. We addressed the problem of applying t-SVD to reconstruct tensors from a small number of coefficients in any given ortho-normal basis.
The contributions are as follows: 1. We proposed a tensor reconstruction model based on t-SVD and gave the theorem and coherence conditions for the tensor reconstruction from limited coefficients in any ortho-normal basis. 2. Specifically, we extended the matrix coherent conditions to tensor coherent conditions and prove that our results hold for any ortho-normal basis meeting the conditions. 3. In the proof, we first validated our results in the case of Fourier-type basis. Then, we generalised the proof to any orthonormal basis using mathematical induction with two modifications. 4. Lastly, we applied t-SVD to reconstruct random tensors and d-MR images from undersampled measurements. In the reconstruction of random tensors, the tensor structure can be maintained successfully. The reconstruction of dynamic MR images demonstrated that t-SVD framework can be used to reduce the amount of the required sampling data.
Nowadays, MR images are commonly multi-channel. But to simplify the application, this work studies the application of our method in the traditional MRI method (single channel). We will consider the application of parallel imaging in future.