Latent Signal Models: Learning Compact Representations of Signal Evolution for Improved Time-Resolved, Multi-contrast MRI

Purpose: Training auto-encoders on simulated signal evolution and inserting the decoder into the forward model improves reconstructions through more compact, Bloch-equation-based representations of signal in comparison to linear subspaces. Methods: Building on model-based nonlinear and linear subspace techniques that enable reconstruction of signal dynamics, we train auto-encoders on dictionaries of simulated signal evolution to learn more compact, non-linear, latent representations. The proposed Latent Signal Model framework inserts the decoder portion of the auto-encoder into the forward model and directly reconstructs the latent representation. Latent Signal Models essentially serve as a proxy for fast and feasible differentiation through the Bloch-equations used to simulate signal. This work performs experiments in the context of T2-shuffling, gradient echo EPTI, and MPRAGE-shuffling. We compare how efficiently auto-encoders represent signal evolution in comparison to linear subspaces. Simulation and in-vivo experiments then evaluate if reducing degrees of freedom by inserting the decoder into the forward model improves reconstructions in comparison to subspace constraints. Results: An auto-encoder with one real latent variable represents FSE, EPTI, and MPRAGE signal evolution as well as linear subspaces characterized by four basis vectors. In simulated/in-vivo T2-shuffling and in-vivo EPTI experiments, the proposed framework achieves consistent quantitative NRMSE and qualitative improvement over linear approaches. From qualitative evaluation, the proposed approach yields images with reduced blurring and noise amplification in MPRAGE shuffling experiments. Conclusion: Directly solving for non-linear latent representations of signal evolution improves time-resolved MRI reconstructions through reduced degrees of freedom.


INTRODUCTION
Efficient time-resolved Magnetic Resonance Imaging (MRI) enables a wide range of applications such as MRI spectroscopy 1 , quantitative parameter mapping 2,3 , motion-resolved imaging 4,5 , and blur-free, multi-contrast imaging from fast-spin-echo 6 (FSE) and Magnetization Prepared Rapid Acquisition Gradient Echo 7,8 (MPRAGE) acquisitions. In this manuscript, we focus on applications that reconstruct individual echo-images with differing contrasts from acquisitions with evolving signal evolution over an echo-train 7 .
Time-resolving MRI requires lengthy scan times using traditional methods. Compressed sensing 9,10 and parallel imaging 11,12 reduce acquisition times in structural MRI, but these techniques in isolation do not enable clinically feasible time-resolved MRI. Machine learning provides another avenue for acceleration. Models, typically neural networks trained in a supervised fashion, impose custom regularization or directly reconstruct undersampled data [13][14][15][16][17] . Some work demonstrates applications of supervised and unsupervised machine learning in dynamic imaging 18,19 , but the cost of acquiring fully sampled data hampers widespread use of supervised learning in many time-resolved acquisitions.
Time-resolved MRI employs a variety of different sequence and reconstruction techniques to reduce acquisition times in the presence of the additional imaging dimension. Some techniques model signals with an analytic formula and then resolve the underlying tissue parameters that characterize this model [20][21][22][23][24][25] , introducing a nonlinear optimization problem that may not account for slice-profile effects, stimulated-echoes, and B1+ inhomogeneity 26 . Other techniques build the Bloch-equations into the forward model and solve for the underlying parameters governing the model [27][28][29] , but require extensive computation or approximation of gradients.
For example, the highly-under-sampled shuffling 6,7,37,38 and echo-planar-time-resolvedimaging 2,39 (EPTI) techniques resolve multi-contrast images from fast-spin-echo 40  Recently, auto-encoders trained on dictionaries of simulated signal in MRI spectroscopy and diffusion acquisitions learn compact latent representations of signal space 43,44 . The proposed MRSI and diffusion techniques employ the trained auto-encoders as regularization in reconstruction.
We propose Latent Signal Models, combining ideas from Bloch-equation models, linear subspaces, and latent representations for improved time-resolved MRI reconstruction. Latent Signal Models simulate dictionaries of signal evolution to train an auto-encoder to learn a compact representation of signal. The proposed reconstruction framework then incorporates the decoder portion, of the trained auto-encoder, into the imaging forward model and solves for the learned latent representation of signal. This improves reconstruction quality with reduced degrees of freedom in comparison to linear techniques. The decoder produces the time-series of multi-contrast images from the reconstructed latent representation. Latent Signal Models learn to represent the Bloch-equations with a simple neural-network that can be inserted into the forward model enabling reconstruction with the sophisticated autodifferentiation tools developed for machine learning 45 . In this way, we feasibly, quickly, and conveniently solve a reconstruction problem that maintains the benefits of and circumvents the demanding optimization usually induced by incorporating the Bloch-equations into the forward model.
We begin by introducing and characterizing our Latent Signal Model framework through application in T2-shuffling. We show that auto-encoders learn more compact representations of FSE signal evolution in comparison to linear models. Then, simulation and retrospectively under-sampled in-vivo reconstruction experiments suggest that the reduced degrees of freedom afforded by the Latent Signal Model framework improves reconstruction accuracy in comparison to linear subspace constraints. Additional experiments analyze the stability of the technique across varying noise instances. Next, we empirically verify that the trained autoencoder essentially serves as a fast and feasible proxy for optimization through the simulation of signal evolution. Finally, we demonstrate versatility through improved reconstructions in invivo, retrospectively under-sampled gradient-echo EPTI, and under-sampled MPRAGE shuffling experiments.

Time-resolved MRI with Linear Subspace Constraints
Sequences that acquire multiple k-space lines after excitation or inversion across a gradientecho or spin-echo train concatenate data from all echoes into a single k-space matrix, often of dimension × × × , where , , represent read-out, phase-encode, and partition dimensions respectively and represents the number of coils. Parallel imaging then produces a single image which suffers from blurring 46 or distortion 47 artifacts due to signal decay and phase evolution during the echo train.
To improve image sharpness and reduce distortion, techniques like EPTI and shuffling aim to resolve ∈ ! × $ × % × & , a time-series of multi-contrast images, where represents the number of echoes or timepoints, by considering the temporal representation of acquired kspace, ∈ ! × $ × % × ' × & , which assigns k-space points to their associated time of acquisition 2,6 . However, the resultant reconstruction problems become heavily underdetermined as resolving the time dimension increases the number of unknowns by a factor of .
To produce tractable reconstruction problems, linear subspace techniques generate a dictionary, , of simulated signal evolution from realistic tissue relaxation parameters for the desired sequence of interest. Then the first singular vectors of form a low dimensional subspace ∈ × that can be inserted into the forward model to produce: Where and represent the under-sampled Fourier and Coil-sensitivity operators applied to each time-point, represents temporally varying phase (if necessary), ∈ × × × represents the subspace coefficients the optimization problem solves for, and represents a spatial-regularization function applied to the subspace coefficients. The estimated coefficients * yields the time-series of images with * = * . This formulation reduces the number of unknowns by & / . For example, T2-shuffling reconstructs data from an FSE sequence with = 80 and utilizes subspaces with = 4, reducing unknowns by a factor of 20×.

Latent Signal Models Reconstruction Framework for Time-resolved MRI
We propose learning a nonlinear latent representation of the signal evolution dictionary, . Let 0 and 1 represent fully-connected neural-networks for the encoder and decoder of an autoencoder 48,49 . The auto-encoder learns a latent representation of signal evolution by minimizing the following with respect to its weights and biases, and : * , The trained decoder can then be inserted in the forward model, to reconstruct the latent representation * ∈ !×$×%×5 , proton density * ∈ !×$×% , and time-series of images with * = * 1 * ( * ), where is the number of latent variables. Unlike previous work 43, 44 , the decoder prior does not require tuning an additional regularization parameter, and separate spatial regularization can be applied directly to the unknown latent representation and density. Figure 1 visualizes the Latent Signal Model framework in comparison to the standard linear approach through an exemplar T2-shuffling setting.
With complex-valued coefficients , linear reconstructions resolve × × × × 2 unknowns; while the real-valued latent representation and the complex-valued proton density yield × × × ( + 2) unknowns in the proposed framework. If + 2 < 2 × , the Latent Signal Model framework produces a reconstruction problem with fewer unknowns. In subsequent experiments, linear models require = 2, 3, while our method uses = 1, corresponding to a 1.5× to 2.5× reduction of unknowns.
In essence, the decoder neural-network serves as a proxy for the simulation process used to generate the dictionary of signal evolution and synergizes with auto-differentiation tools for fast and feasible optimization.

T 2 -shuffling Experiments
Experiments begin in T2-shuffling 6 settings for demonstration and characterization, while subsequent results showcase the technique's versatility through gradient-echo EPTI and MPRAGE-shuffling.

Dictionary Compression
We compare how efficiently SVD-generated linear-subspaces and a trained auto-encoder represent FSE signal evolution. Extend-phase-graph (EPG) simulations 50  Linear subspaces and auto-encoders compressed and reconstructed signal in the testing dictionary, and the resultant reconstruction accuracies were compared.

Simulated Reconstruction Experiments
Next, we simulated T2-shuffling acquisitions from a numerical phantom with T1, T2, proton density, and 8-channel coil-sensitivity maps 52 . EPG simulations produced a time-series of kspaces (with added noise) for each echo of a FSE sequence: = 80 echoes, 5.56 ms echo spacing, and 160 degree refocusing pulses. We applied an undersampling mask that mimics a 2D T2-shuffling acquisition with 4-shots that samples a random phase-encode line at each echo and labeled the dataset simulated_t2shfl1. This corresponds to 320 sampled k-space lines,

Simulated Auto-encoder Hyperparameter Ablation Experiments
With identical sequence parameters, we generated an additional dataset, labeled simulated_t2shfl2, modeling a 3D T2-shuffling acquisition by assuming each echo-k-space corresponds to phase and partition encode dimensions and sampling 256 × 256 k-space points randomly throughout the 80 echoes 6 . An ablation study trained auto-encoders on all combinations of the following hyperparameters and compared the resultant un-regularized Latent Signal Model reconstructions. Since these experiments do not employ regularization, we utilized a dataset with more incoherence to better condition the reconstruction problem.

Retrospective in-vivo reconstruction experiments
All imaging protocols were performed with approval from the local institutional review board with written informed consent. A volunteer was scanned with a Siemens 3T Trio (Siemens Healthineers, Erlangen, Germany) system using a 12-channel receive head coil. A fully-sampled spatial and temporal multi-echo T2-weighted dataset with field-of-view = 180 mm x 240 mm, matrix size = 208 x 256 matrix size, slice thickness = 3 mm, echo spacing = 11.5 ms, and = 32 echoes was acquired.
The applied undersampling mask generated acquisitions with 7 shots that samples a random phase-encode line at each echo, and we label the dataset invivo_t2shfl1. This corresponds to 224 sampled k-space lines, resulting in an overall acceleration of = 256 * 32 The experiment compares linear subspace reconstructions with {4,6} degrees of freedom and wavelet or locally-low-rank regularization to the proposed Latent Signal Model reconstruction with 3 degrees of freedom and wavelet regularization. Additionally, the proposed approach utilized an auto-encoder with 3 fully-connected layers and tanh activation functions. We chose all experimentally optimized regularization parameters to minimize error, and the autodifferentiation and BART frameworks solved the proposed and linear problems respectively.

Retrospective Auto-encoder Hyperparameter Ablation Experiments
On a similar 8-shot dataset, labeled invivo_t2shfl2, an ablation study trained auto-encoders on combinations of the following hyper-parameters and compared the resultant un-regularized Latent Signal Model reconstructions. Since these experiments do not employ regularization, we utilized a dataset with more shots to better condition the reconstruction problem.

Analyzing the effects of reduced degrees of freedom and evaluating reconstruction stability across various noise instances
The aforementioned experiments combined temporal constraints, spatial regularization, and tuned hyper-parameters for best reconstruction performance. Thus, the following experiment isolates and characterizes the improvements afforded by the reduced degrees of freedom from Latent Signal Models and evaluates stability of the reconstruction across various noise instances in the spirit of g-factor 12 analysis.
We added 250 instances of realistic gaussian noise to the simulated_t2shfl2 dataset, described in the simulation hyper-parameter experiment, and 2D-, 7-shot, in-vivo invivo_t2shfl1 dataset.
Then, the proposed Latent Signal Model framework with 3 and linear subspaces with {4,6} degrees of freedom reconstructed the 250 k-spaces in both the simulation and retrospective regimes without regularization. We report the mean and standard deviation of the normalizedroot-mean-squared-error (NRMSE) at each reconstructed echo time and display mean reconstructed error maps.

Signal Simulations in a Reconstruction Setting
The subsequent experiment demonstrates that the trained auto-encoder serves as a proxy for fast and feasible optimization through the signal simulation process directly in the reconstruction problem.
Let represent the simulation function that produces signal evolution given an input T2 and proton density in T2-shuffling. The simulation-based reconstruction problem becomes: Eq4 which solves for the 2 ∈ ! × $ map and density ∈ .
We show that minimizing Eq3 in the Latent Signal Model framework produces a solution that also minimizes the simulation-based reconstruction problem in Eq4. We use the proposed framework to reconstruct the simulated_t2shfl2, and invivo_t2shfl2 datasets (both without regularization). The optimization runs for 1000 iterations, applying the following procedure every 50 iterations: • Extract the current latent variable and density estimates, 7 and 7 .
Finally, we compare computation times in optimizing the Latent Signal Model and simulationbased frameworks, and we try applying the simulation based forward model (Eq4) to reconstruct the simulated T2-shuffling dataset without using the proposed framework's solution as initialization.
We implement with a PyTorch EPG algorithm 57 that enables computation of gradients with auto-differentiation.

Gradient-Echo EPTI Experiments
Here, we demonstrate application of Latent Signal Models in 2D-gradient-echo (GE) EPTI 2,58 which continuously measures k-space during T2 * dominated signal decay after an initial 90degree excitation pulse. GE-EPTI resolves signal dynamics from the highly-undersampled kx-ky-t dataset through B0-informed linear subspace reconstructions.
A fully-sampled spatial and temporal multi-echo gradient-echo EPTI dataset with matrix size = 216 x 216, slice thickness = 3mm, 1.1 x 1.1 mm in-plane resolution, echo spacing = .93 ms, = 40 echoes, and 32 coils was acquired. To produce a challenging case, the under-sampling mask modeled a GE-EPTI acquisition with two shots. This corresponds to 80 sampled k-space lines, resulting in an overall acceleration factor of = 216 * 40 80 = 108.
The first experiment generated from the fully-sampled dataset. Different from T2-shuffling and MPRAGE, EPTI models temporally varying phase, so both the linear and Latent Signal Model reconstructions might be more sensitive to bias in B0 phase estimation in this highly undersampled experiment. We started with more accurate phase estimates to evaluate reconstructions independent of the B0-estimation algorithm.
We then repeated the experiment using a B0-map estimated from the central 49 k-space lines of the first 6 echoes to characterize performance with low-resolution phase. These central kspace lines were treated as a calibration pre-scan and not amongst the 80 lines used for reconstruction.
The experiments compare linear subspace reconstructions with {4,6} degrees of freedom and locally-low-rank regularization to the proposed reconstruction with 3 degrees of freedom (1 latent variable) and wavelet regularization. The proposed approach trained an auto-encoder with two-fully connected layers for both the encoder and decoder with tanh activations on a dictionary of simulated GE-EPTI signal evolution with T2 * values in the range of 10 -300 ms. We chose all experimentally optimized regularization parameters to minimize error.

MPRAGE Shuffling Experiments
MPRAGE-shuffling 7 employs linear subspaces to resolve multiple image contrasts across the MPRAGE echo train. We apply the proposed framework to model MPRAGE signal and reconstruct under-sampled MPRAGE-shuffling data.

Dictionary Compression
In Figures 2 (A)

Retrospective in-vivo reconstruction experiments
In-vivo, retrospectively under-sampled experiments in Figure 4 compare  Additionally, the proposed approach (B,D) achieves lower average NRMSE across all echoes, and maintains comparable variance in reconstruction accuracy.  In this challenging case with high under-sampling, the linear reconstructions suffer from bias and artifacts, while the proposed approach significantly improves reconstruction quality. In (B), Supporting Figure 3 displays the same reconstructions from Figure 7 with phase estimates calibrated from low-resolution data. Linear reconstructions remain poor, while the proposed approach still achieves significant relative improvement.

MPRAGE Dictionary Compression
In Figures 8 (A)   plots of NRMSE versus T1-value on individual signal-evolution entries from the testing dictionary. The proposed auto-encoder with 1 latent variable represents signal evolution as effectively as the linear subspaces with 3 coefficients.  Our proposed method builds upon previous work that utilizes auto-encoders to regularize Without regularization, the proposed approach significantly reduces noise amplification and improves reconstruction quality. With applied regularization, linear + wavelet suffers increased blurring while linear + locally-low-rank still exhibits noise amplification. The proposed approach reduces reconstruction artifacts, particularly at TI ≈ 500 ms, improves image sharpness in comparison to linear + wavelet, and reduces noise amplification in comparison to linear + locally-low-rank. In the settings explored, signal largely depends on a single underlying tissue parameter (T2 for FSE, T2 * for GE-EPTI, T1 for MPRAGE shuffling), so we suspect that the auto-encoder successfully compresses dictionaries to just one latent variable in these applications for this reason.

MPRAGE Reconstruction Experiments
However, our technique does not explicitly build this knowledge into the learning algorithm; rather, the model implicitly learns to compress signal into one latent variable from the dictionary of simulated signal evolution.
Since we simulate the dictionary, the proposed Latent Signal Model framework does not require acquisition of any fully-sampled datasets for model training. In addition, the autoencoder is trained once and can be reused with subsequent acquisitions from the same sequence.
Informed by Supporting Figures 1 and 2, we used the hyperbolic tangent nonlinearity with either two or three fully connected layers in the encoder and decoder. This extends well into EPTI and MPRAGE-shuffling, but ideal hyper-parameters may change in other applications.
Most of the reconstruction experiments in this work employ regularization in the proposed and linear reconstructions to make comparisons as fair and competitive as possible. We included In the T2-shuffling setting, Figure 6 demonstrated that the proposed technique finds local minima of reconstruction problems that incorporate EPG-simulations in the forward model. In comparison to solving the EPG-based reconstruction problem, we found that our proposed approach reduced reconstruction time per iteration by at least an order of magnitude.
Additionally, when not initialized with the proposed framework's solution, we found that the EPG based reconstruction problem does not converge with the simulated data. Thus, we envision our approach being advantageous when differentiation through signal simulation requires significant computation. For example, applications that require iso-chromat based simulations 27,60 might benefit significantly from the proposed approach. Future work will explore improved B0 estimation and refinement 47,61,62 techniques to improve phase-estimates for Latent Signal Model EPTI reconstructions.

Limitations
The presence of neural networks in the proposed framework induces a non-convex reconstruction problem. Thus, the optimization may converge to a suboptimal local minimum with an improperly trained auto-encoder. For example, the LeakyRelu autoencoders, explored in Supporting Figures 1 and 2, yield significant variability in performance from different random initialization of model weights. We found that tanh activations produce reconstruction problems that reliably terminate at reasonable local minima. Avoiding sub-optimal local minima in other applications may require changes to the auto-encoder structure.
In the linear forward model, from Eq1, the Fourier and coil operator, , commutes with the temporal operator, 6           Without regularization, the proposed approach significantly reduces noise amplification and improves reconstruction quality. With applied regularization, linear + wavelet suffers increased blurring while linear + locally-low-rank still exhibits noise amplification. The proposed approach reduces reconstruction artifacts, particularly at TI ≈ 500 ms, improves image sharpness in comparison to linear + wavelet, and reduces noise amplification in comparison to linear + locally-low-rank. Even with a worse phase-estimate, the proposed approach still produces significantly higher quality images with lower NRMSE in comparison to the linear techniques.