双视角超分辨重建 磁共振图像

Because of the inherent constraints in the hardware capabilities of magnetic resonance imaging (MRI), the process of image acquisition is comparatively sluggish, and the resolution among slices is subpar, leaving clinicians with incomplete structural details about specific bodily regions. Current approaches to super-resolution interpolation for medical slices predominantly cater to individual slices, neglecting the valuable correlational information that exists between slices. This oversight substantially hampers the efficacy of interpolation synthesis. Furthermore, mainstream super-resolution techniques are tailored to specific up-sampling scenarios, complicating their application in clinical settings. To overcome these hurdles, this paper introduces an MRI spatial-aware interpolation synthesis method utilizing asymmetric mask encoding. This method employs an asymmetric network model to concurrently achieve high-resolution interpolation synthesis across multiple interpolation factors. The proposed MRI spatial-aware interpolation network, powered by asymmetric mask encoding, effectively harnesses correlational insights from diverse MRI image perspectives. Validation of this approach was conducted using a subset of data from the MICCAI (Medical Image Computing and Computer-Assisted Intervention) BraTS (Brain Tumor Segmentation) 2018 challenge T1 dataset, yielding high-resolution images with multiple interpolation factors.

The incorporation of physical information in machine learning frameworks is transforming medical image analysis (MIA). Integrating fundamental knowledge and governing physical laws not only improves analysis performance but also enhances the model’s robustness and interpretability. This work presents a systematic review of over 100 articles on the utility of PINNs dedicated to MIA (PIMIA) tasks. We propose a unified taxonomy to investigate what physics knowledge and processes are modeled, how they are represented, and the strategies to incorporate them into MIA models. We delve deep into a wide range of image analysis tasks, from imaging, generation, prediction, inverse imaging (super-resolution and reconstruction), registration, and image analysis (segmentation and classification). For each task, we thoroughly examine and present the central physics-guided operation, the region of interest (with respect to human anatomy), the corresponding imaging modality, the datasets used for model training, the deep network architectures employed, and the primary physical processes, equations, or principles utilized. Additionally, we also introduce a novel metric to compare the performance of PIMIA methods across different tasks and datasets. Based on this review, we summarize and distill our perspectives on the challenges, and highlight open research questions and directions for future research.

Medical image super-resolution reconstruction is a key technology to improve diagnostic precision in healthcare systems where hardware constraints tend to degrade image quality. This paper proposes a new medical image super-resolution technique that combines edge detection algorithms with deep learning networks to retain key anatomical structures while enhancing image resolution. The method encompasses a multi-scale edge detection module tailored to medical imaging characteristics, a dual-pathway encoder-decoder network structure that processes content and structural information in parallel pathways, and an adaptive gating fusion approach that fuses edge guidance and semantic features intelligently. Extensive experimental verification was performed on the NIH chest X-ray dataset under varied pathological conditions. The method outperformed with PSNR gains of 30.21 dB for $2\times $ scaling, SSIM of 0.8689, and LPIPS of 0.1423, clearly outperforming state-of-the-art methods. The edge detection module effectively retained anatomical boundaries with 94.2% rib edge continuity and 91.7% heart boundary definition. Cross-dataset generalization experiments showed robustness across different imaging protocols with minimal performance degradation. The computational efficiency of 28.7 milliseconds per image patch supports practical clinical deployment while preserving diagnostic quality required for accurate pathological evaluation. The approach enhances diagnostic value extraction from lower-cost imaging hardware through computational post-processing, with inference operations optimized for execution on standard GPU infrastructure.

… of imaging tasks, including super-resolution [43] enhancement of MRI and X-ray images [46]… cross-modality translation (eg, MRI-to-CT [29, 47] or T1-weighted to T2-weighted MRI [28]). …

—The image super-resolution is utilized for the image transformation from low resolution to higher resolution to obtain more detailed information to identify the targets. The super-resolution has potential applications in various domains, such as medical image processing, crime investigation, remote sensing, and other image-processing application domains. The goal of the super-resolution is to obtain the image with minimal mean square error with improved perceptual quality. Therefore, this study introduces the perceptual loss minimization technique through efficient learning criteria. The proposed image reconstruction technique uses the image super-resolution generative adversarial network (ISRGAN), in which the learning of the discriminator in the ISRGAN is performed using the EfficientNet-v2 to obtain a better image quality. The proposed ISRGAN with the EfficientNet-v2 achieved a minimal loss of 0.02, 0.1, and 0.015 at the generator, discriminator, and self-supervised learning, respectively, with a batch size of 32. The minimal mean square error and mean absolute error are 0.001025 and 0.00225, and the maximal peak signal-to-noise ratio and structural similarity index measure obtained are 45.56985 and 0.9997, respectively.

The study explored a deep learning image super-resolution approach which is commonly used in face recognition, video perception and other fields. These generative adversarial networks usually have high-frequency texture details. The relevant textures of high-resolution images could be transferred as reference images to low-resolution images. The latest existing methods use transformer ideas to transfer related textures to low-resolution images, but there are still some problems with channel learning and detailed textures. Therefore, the study proposed an enhanced texture transformer network (ETTN) to improve the channel learning ability and details of the texture. It could learn the corresponding structural information of high-resolution texture images and convert it into low-resolution texture images. Through this, finding the feature map can change the exact feature of images and improve the learning ability between channels. We then used multi-scale feature integration (MSFI) to further enhance the effect of fusion and achieved different degrees of texture restoration. The experimental results show that the model has a good resolution enhancement effect on texture transformers. In different datasets, the peak signal to noise ratio (PSNR) and structural similarity (SSIM) were improved by 0.1–0.5 dB and 0.02, respectively.

Multi-modal Magnetic Resonance Imaging (MRI) super-resolution (SR) and reconstruction aims to obtain a high-quality target image from corresponding sparsely sampled signals …

In multi-modal magnetic resonance imaging (MRI), the tasks of imputing or reconstructing the target modality share a common obstacle: the accurate modeling of fine-grained inter-modal differences, which has been sparingly addressed in current literature. These differences stem from two sources: 1) spatial misalignment remaining after coarse registration and 2) structural distinction arising from modality-specific signal manifestations. This paper integrates the previously separate research trajectories of cross-modality synthesis (CMS) and multi-contrast super-resolution (MCSR) to address this pervasive challenge within a unified framework. Connected through generalized down-sampling ratios, this unification not only emphasizes their common goal in reducing structural differences, but also identifies the key task distinguishing MCSR from CMS: modeling the structural distinctions using the limited information from the misaligned target input. Specifically, we propose a composite network architecture with several key components: a label correction module to align the coordinates of multi-modal training pairs, a CMS module serving as the base model, an SR branch to handle target inputs, and a difference projection discriminator for structural distinction-centered adversarial training. When training the SR branch as the generator, the adversarial learning is enhanced with distinction-aware incremental modulation to ensure better-controlled generation. Moreover, the SR branch integrates deformable convolutions to address cross-modal spatial misalignment at the feature level. Experiments conducted on three public datasets demonstrate that our approach effectively balances structural accuracy and realism, exhibiting overall superiority in comprehensive evaluations for both tasks over current state-of-the-art approaches. The code is available at https://github.com/papshare/FGDL.

… In this paper, we propose a novel multi-modal HR MRI generation framework based on deep learning techniques. Specifically, we construct a CNN based on multi-resolution analysis to …

Super-resolution (SR) is primarily tailored for single-modal medical images. While in many applications of magnetic resonance imaging (MRI), multimodal images with diverse parameters are available. Current single-modal SR approaches are limited in fully exploring and exploiting the correlations between these cross-modal images, leading to degraded reconstruction performance. Thus, this article presents a novel multiscale (MS) fusion approach for cross-modal MRI SR reconstruction. The proposed method develops domain-specific convolutions to spatially decouple MRI into different subspaces and task-specific modules for reconstruction. Specifically, a CNN-based framework is constructed to explore the mapping between a low-resolution T2-weighted (LR T2w) image and a high-resolution (HR) T2w one, by incorporating an HR T1-weighted (T1w) image. In the proposed network, the low-frequency filtering modules have been integrated into it to remove the low-frequency components of the HR T1w while extracting its high-frequency information. Therefore, by fusing the detailed features of the HR T1w and the LR T2w at two different scales, the network generates the HR T2w image. Extensive results across benchmark MRI datasets demonstrate the effectiveness of the proposed method in MRI SR reconstruction.

Super-resolving medical images can help physicians in providing more accurate diagnostics. In many situations, computed tomography (CT) or magnetic resonance imaging (MRI) techniques capture several scans (modes) during a single investigation, which can jointly be used (in a multimodal fashion) to further boost the quality of super-resolution results. To this end, we propose a novel multi-modal multi-head convolutional attention module to super-resolve CT and MRI scans. Our attention module uses the convolution operation to perform joint spatial-channel attention on multiple concatenated input tensors, where the kernel (receptive field) size controls the reduction rate of the spatial attention, and the number of convolutional filters controls the reduction rate of the channel attention, respectively. We introduce multiple attention heads, each head having a distinct receptive field size corresponding to a particular reduction rate for the spatial attention. We integrate our multimodal multi-head convolutional attention (MMHCA) into two deep neural architectures for super-resolution and conduct experiments on three data sets. Our empirical results show the superiority of our attention module over the state-of-the-art attention mechanisms used in super-resolution. Moreover, we conduct an ablation study to assess the impact of the components involved in our attention module, e.g. the number of inputs or the number of heads. Our code is freely available at https://github.com/lilygeorgescu/MHCA.

… methods utilizing these properties of multi-modal. That is, in … the existing multi-modal MRI image super-resolution (MMSR) … Therefore, how to design a simple but effective multi-modal …

… can be achieved, the multi-modal reconstruction network is … proposed a novel 3D multi-modal reconstruction method, … built two networks to reconstruct super-resolution MR images: (1) …

… To transfer this to the task of super-resolution, we present the combination of the guided filter, … to the input image, with a guidance map that is learnd end-to-end from multi-modal input. …

… As is known that MRI is a native multi-modal imaging technique, thus we can flexibly obtain a lot of desired information of different modalities as available references for MRI image …

Magnetic resonance (MR) image enhancement technology can reconstruct high-resolution image from a low-resolution image, which is of great significance for clinical application and scientific research. T1 weighting and T2 weighting are the two common magnetic resonance imaging modes, each of which has its own advantages, but the imaging time of T2 is much longer than that of T1. Related studies have shown that they have very similar anatomical structures in brain images, which can be utilized to enhance the resolution of low-resolution T2 images by using the edge information of high-resolution T1 images that can be rapidly imaged, so as to shorten the imaging time needed for T2 images. In order to overcome the inflexibility of traditional methods using fixed weights for interpolation and the inaccuracy of using gradient threshold to determine edge regions, we propose a new model based on previous studies on multi-contrast MR image enhancement. Our model uses framelet decomposition to finely separate the edge structure of the T2 brain image, and uses the local regression weights calculated from T1 image to construct a global interpolation matrix, so that our model can not only guide the edge reconstruction more accurately where the weights are shared, but also carry out collaborative global optimization for the remaining pixels and their interpolated weights. Experimental results on a set of simulated MR data and two sets of real MR images show that the enhanced images obtained by the proposed method are superior to the compared methods in terms of visual sharpness or qualitative indicators.

Improving the resolution of magnetic resonance (MR) image data is critical to computer-aided diagnosis and brain function analysis. Higher resolution helps to capture more detailed content, but typically induces to lower signal-to-noise ratio and longer scanning time. To this end, MR image super-resolution has become a widely-interested topic in recent times. Existing works establish extensive deep models with the conventional architectures based on convolutional neural networks (CNN). In this work, to further advance this research field, we make an early effort to build a Transformer-based MR image super-resolution framework, with careful designs on exploring valuable domain prior knowledge. Specifically, we consider two-fold domain priors including the high-frequency structure prior and the inter-modality context prior, and establish a novel Transformer architecture, called Cross-modality high-frequency Transformer (Cohf-T), to introduce such priors into super-resolving the low-resolution (LR) MR images. Experiments on two datasets indicate that Cohf-T achieves new state-of-the-art performance.

… Multi-modal medical image fusion integrates complementary … medical image fusion and super-resolution while supporting flexible … More importantly, the fused super-resolution images …

… images is beneficial to image super-resolution. Our multi-modal MR image super-resolution method can achieve high-quality super-resolution results of two modalities simultaneously in …

This paper introduces a multimodal medical image super-resolution framework using the combined power of Diffusion Models and Vision Transformers. In the diffusion model, the technique is applied to generate high-resolution images progressively by removing the noise present in the input images. Vision Transformers are then used to provide the model withthe ability to capture contextual information. Since the modelis multimodal, by using a combination of multiple imaging modalities such as MRI, CT, and PET images, the approachis able to benefit from the complementary information presentin different images. The model is trained using pairs of low-resolution images and their corresponding high-resolution images, where feature fusion is performed before the diffusion phase. The experiments are carried out using publicly available multimodal medical image datasets, and the performance ofthe model is evaluated using metrics such as MSE, PSNR, andSSIM. When the results are compared with bicubic interpolation and CNN-based super-resolution methods, the differences are quite noticeable. The proposed method achieves improvements of up to 7 dB in average PSNR, while the average SSIM shows an improvement of more than 0.14. These experimental results validate the proposed diffusion transformer framework, which generates super-resolved medical images in a reliable and high-quality manner. This further supports clinical interpretation and can also be extended to several other applications in downstream medical image analysis and automated diagnostic systems.

Due to time and cost limitations, Magnetic Resonance (MR) imaging often employs anisotropic scanning with large slice spacing and thickness. This causes blurring in views perpendicular to the slices, which adversely affects clinical diagnosis and research. Taking into account the complementary information from the reference modality, deep learning (DL) based multi-contrast methods have become a focal point of research. These methods aim to reconstruct the isotropic target MR image with the auxiliary high-resolution (HR) reference modality. However, most of the methods primarily concentrate on the structural restoration of the target low-resolution (LR) image, neglecting the crucial aspect that the coexisting structural and modality differences between target and reference modalities can impede effective restoration. Additionally, these methods are designed for a fixed upsampling scale, not accounting for the practical scenario of varying slice thickness. In this work, we propose a joint Super-resolution and Modality translation network (SMNet) for multi-contrast arbitrary-scale isotropic MRI reconstruction. The modality translation branch includes the Modality-Specific-Augmented Alignment (MSAA) block, which eliminates modality distribution disparities and enhances modality-specific regions on the reference feature before fusion. And the super-resolution branch employs the Reliability-based Spatial Fusion (RSF) block for the structural restoration of the target LR feature using a reliability prior. The outputs from these two branches are then ensembled to obtain the final reconstructed result. Extensive experiments on both a private dataset and the Brasts2021 dataset demonstrate the effectiveness and generalizability of the proposed method. Our code is available at https://github.com/11710615/smnet.

Given a low-resolution (LR) image, multi-modal image super-resolution (MISR) aims to find the high-resolution (HR) version of this image with the guidance of an HR image from another modality. In this paper, we use a model-based approach to design a new deep network architecture for MISR. We first introduce a novel joint multi-modal dictionary learning (JMDL) algorithm to model cross-modality dependency. In JMDL, we simultaneously learn three dictionaries and two transform matrices to combine the modalities. Then, by unfolding the iterative shrinkage and thresholding algorithm (ISTA), we turn the JMDL model into a deep neural network, called deep coupled ISTA network. Since the network initialization plays an important role in deep network training, we further propose a layer-wise optimization algorithm (LOA) to initialize the parameters of the network before running back-propagation strategy. Specifically, we model the network initialization as a multi-layer dictionary learning problem, and solve it through convex optimization. The proposed LOA is demonstrated to effectively decrease the training loss and increase the reconstruction accuracy. Finally, we compare our method with other state-of-the-art methods in the MISR task. The numerical results show that our method consistently outperforms others both quantitatively and qualitatively at different upscaling factors for various multi-modal scenarios.

Ultrasound image super-resolution (SR) aims to transform low-resolution images into high-resolution ones, thereby restoring intricate details crucial for improved diagnostic accuracy. However, prevailing methods relying solely on image modality guidance and pixel-wise loss functions struggle to capture the distinct characteristics of medical images, such as unique texture patterns and specific colors harboring critical diagnostic information. To overcome these challenges, this paper introduces the Multi-Modal Regularized Coarse-to-fine Transformer (M2Trans) for Ultrasound Image SR. By integrating the text modality, we establish joint image-text guidance during training, leveraging the medical CLIP model to incorporate richer priors from text descriptions into the SR optimization process, enhancing detail, structure, and semantic recovery. Furthermore, we propose a novel coarse-to-fine transformer comprising multiple branches infused with self-attention and frequency transforms to efficiently capture signal dependencies across different scales. Extensive experimental results demonstrate significant improvements over state-of-the-art methods on benchmark datasets, including CCA-US, US-CASE, and our newly created dataset MMUS1K, with a minimum improvement of 0.17dB, 0.30dB, and 0.28dB in terms of PSNR.

… This work focuses on improving existing MRI reconstruction techniques for low undersampling rates. Many prior methods perform well at moderate sampling fractions but performance …

Purpose To develop and evaluate a super-resolution approach to reconstruct time-resolved four-dimensional magnetic resonance imaging (TR-4DMRI) with a high spatiotemporal resolution for multi-breathing cycle motion assessment. Methods and Materials A super-resolution approach was developed to combine fast 3D cine MRI with low-resolution during free breathing (FB) and high-resolution 3D static MRI during breath hold (BH) using deformable image registration (DIR). A T1-weighted, turbo field echo sequence, coronal 3D cine acquisition, partial Fourier approximation, and SENSE parallel acceleration were employed. The same MRI pulse sequence, field of view, and acceleration techniques were applied in both FB and BH acquisitions; the intensity-based Demons DIR method was used. Under an IRB-approved protocol, seven volunteers were studied with 3D cine FB scan (voxel size:5x5x5mm3) at 2Hz for 40s and a 3D static BH scan (2x2x2mm3). To examine the image fidelity of 3D cine and super-resolution TR-4DMRI, a mobile gel phantom with multi-internal targets was scanned at three velocities and compared with the 3D static image. Image similarity among 3D cine, 4DMRI, and 3D static was evaluated visually using difference image and quantitatively using voxel intensity correlation and Dice index (phantom only). Multi-breathing-cycle waveforms were extracted and compared in both phantom and volunteer images using the 3D cine as the references. Results Mild imaging artifacts were found in the 3D cine and TR-4DMRI of the mobile gel phantom with a Dice index of >0.95. Among seven volunteers, the super-resolution TR-4DMRI yielded high voxel-intensity correlation (0.92±0.05) and low voxel-intensity difference (<0.05). The detected motion differences between TR-4DMRI and 3D cine were −0.2±0.5mm (phantom) and −0.2±1.9mm (diaphragms). Conclusion Super-resolution TR-4DMRI has been reconstructed with adequate temporal (2Hz) and spatial (2x2x2mm3) resolutions. Further TR-4DMRI characterization and improvement are necessary before clinical applications. Multi-breathing cycles can be examined, providing patient-specific breathing irregularities and motion statistics for future 4D radiotherapy.

Super-resolution reconstruction (SRR) of isotropic fetal brain MR images is critical for prenatal examinations but is hindered by fetal motion and misalignment of thick-slice scans. To address these challenges comprehensively, we introduce an innovative deep learning model, namely 3D-WISE, a 3D Weighted Interpolation for Super-resolution Estimation of fetal brain MRI. The model generates high-quality isotropic fetal brain MR images by learning the interpolation weights to correct misalignments between slices and volumes. These misalignments are estimated by extracting deep features from multiple motion-corrupted stacks. Specifically, 3D-WISE incorporates two key components: (1) a weight learning module for multi-view interpolation and (2) a feature extraction module guided by multi-type attention mechanisms. The weight learning module first maps motion-corrupted thick-slice stacks into latent feature spaces. The resulting features are then fed to an implicit decoding block to estimate interpolation weights of the surrounding points for a given coordinate. We further enhance our approach by incorporating convolutional block attention and atlas-induced cross-attention mechanisms. Extensive experiments on two benchmark datasets show that our 3D-WISE achieves remarkably improved performance compared to the widely adopted registration-reconstruction framework. We also extend the experiments on anatomical structure reconstruction and achieve promising results, highlighting the significant potential of our 3D-WISE for fetal brain MR images SRR in clinical settings.

Purpose: To develop T2-weighted (T2w) time-resolved (TR) 4DMRI reconstruction technique with higher soft-tissue contrast for multiple breathing-cycle motion assessment by building a super-resolution (SR) framework using the T1w TR-4DMRI reconstruction as guidance. Methods: The multi-breath T1w TR-4DMRI was reconstructed by deforming a high-resolution (HR:2x2x2mm3) volumetric breath-hold (BH, 20s) 3DMRI image to a series of low-resolution (LR:5x5x5mm3) 3D cine images at a 2Hz frame rate in free-breathing (FB, 40s) using an enhanced Demons algorithm, namely [T1BH→FB] reconstruction. Within the same imaging session, respiratory-correlated (RC) T2w 4DMRI (2x2x2mm3) was acquired based on an internal navigator to gain HR T2w (T2HR) in three states (full exhalation and mid and full inhalation) in ~5 minutes. Minor binning artifacts in the RC-4DMRI were automatically identified based on voxel intensity correlation (VIC) between consecutive slices as outliers (VIC<VICmean-σ) and corrected by deforming the artifact slices to interpolated slices from the adjacent slices iteratively until no outliers were identified. A T2HR image with minimal deformation (<1cm at the diaphragm) from the T1BH image was selected for multi-modal B-Spline DIR to establish the T2HR-T1BH voxel correspondence. Two approaches to reconstruct T2w TR-4DMRI were investigated: (A) T2HR→[T1BH→FB]: to deform T2w HR to T1w BH only as T1w TR-4DMRI was reconstructed, and combine the two displacement vector fields (DVFs) to reconstruct T2w TR-4DMRI, and (B) [T2HR←T1BH]→FB: to deform T1w BH to T2w HR first and apply the deformed T1w BH to reconstruct T2w TR-4DMRI. The reconstruction times were similar, 8-12 minutes per volume. To validate the two methods, T2w- and T1w-mapped 4D XCAT digital phantoms were utilized with three synthetic spherical tumors (ϕ=2.0, 3.0 and 4.0cm) in the lower or mid lobes as the ground truth to evaluate the tumor location (the center of mass, COM), size (volume ratio, %V), and shape (Dice index). Six lung cancer patients were scanned under an IRB-approved protocol and the T2w TR-4DMRI images reconstructed from the two methods were compared based on the preservation of the three tumor characteristics. The local tumor-contained image quality was characterized using the VIC and structure similarity (SSIM). Results: In the 4D digital phantom, excellent tumor alignment after T2HR-T1HR DIR is achieved: ΔCOM=0.8±0.5 mm, %V=1.06±0.02, and Dice=0.91±0.03, in both deformation directions using the DIR-target image as the reference. In patients, binning artifacts are corrected with improved image quality: average VIC increases from 0.92±0.03 to 0.95±0.01. Both T2w TR-4DMRI reconstruction methods produce similar tumor alignment errors ΔCOM=2.9±0.6 mm. However, method B ([T2HR←T1BH]→FB) produces superior preservation in preserving more T2w tumor features with a higher %V=0.99±0.03, Dice=0.81±0.06, VIC=0.85±0.06, and SSIM=0.65±0.10 in the T2w TR-4DMRI images. Conclusion: This study has demonstrated the feasibility of T2w TR-4DMRI reconstruction with high soft-tissue contrast and adequately-preserved tumor position, size, and shape in multiple breathing cycles. The T2w-centric DIR (method B) produces a superior solution for the SR-based framework of T2w TR-4DMRI reconstruction with highly preserved tumor characteristics, including position, size, and shape, which is useful for tumor delineation and motion management in radiation therapy.

The use of MRI for prostate cancer diagnosis and treatment is increasing rapidly. However, identifying the presence and extent of cancer on MRI remains challenging, leading to high variability in detection even among expert radiologists. Improvement in cancer detection on MRI is essential to reducing this variability and maximizing the clinical utility of MRI. To date, such improvement has been limited by the lack of accurately labeled MRI datasets. Data from patients who underwent radical prostatectomy enables the spatial alignment of digitized histopathology images of the resected prostate with corresponding pre-surgical MRI. This alignment facilitates the delineation of detailed cancer labels on MRI via the projection of cancer from histopathology images onto MRI. We introduce a framework that performs 3D registration of whole-mount histopathology images to pre-surgical MRI in three steps. First, we developed a novel multi-image super-resolution generative adversarial network (miSRGAN), which learns information useful for 3D registration by producing a reconstructed 3D MRI. Second, we trained the network to learn information between histopathology slices to facilitate the application of 3D registration methods. Third, we registered the reconstructed 3D histopathology volumes to the reconstructed 3D MRI, mapping the extent of cancer from histopathology images onto MRI without the need for slice-to-slice correspondence. When compared to interpolation methods, our super-resolution reconstruction resulted in the highest PSNR relative to clinical 3D MRI (32.15 dB vs 30.16 dB for BSpline interpolation). Moreover, the registration of 3D volumes reconstructed via super-resolution for both MRI and histopathology images showed the best alignment of cancer regions when compared to (1) the state-of-the-art RAPSODI approach, (2) volumes that were not reconstructed, or (3) volumes that were reconstructed using nearest neighbor, linear, or BSpline interpolations. The improved 3D alignment of histopathology images and MRI facilitates the projection of accurate cancer labels on MRI, allowing for the development of improved MRI interpretation schemes and machine learning models to automatically detect cancer on MRI.

Recently, diffusion models (DM) have been applied in magnetic resonance imaging (MRI) super-resolution (SR) reconstruction, exhibiting impressive performance, especially with regard to detailed reconstruction. However, the current DM-based SR reconstruction methods still face the following issues: (1) They require a large number of iterations to reconstruct the final image, which is inefficient and consumes a significant amount of computational re-sources. (2) The results reconstructed by these methods are often misaligned with the real high-resolution images, leading to remarkable distortion in the reconstructed MR images. To address the aforementioned issues, we propose an efficient diffusion model for multi-contrast MRI SR, named as DiffMSR. Specifically, we apply DM in a highly compact low-dimensional latent space to generate prior knowledge with high-frequency detail information. The highly compact latent space ensures that DM requires only a few simple iterations to produce accurate prior knowledge. In addition, we design the Prior-Guide Large Window Trans-former (PLWformer) as the decoder for DM, which can ex-tend the receptive field while fully utilizing the prior knowledge generated by DM to ensure that the reconstructed MR image remains undistorted. Extensive experiments on public and clinical datasets demonstrate that our DiffMSR11Code: https://github.com/GuangYuanKK/DiffMSR outperforms state-of-the-art methods.

… The slice-to-volume registration stage aligns multi-resolution and multi-view input slices with the tentatively-reconstructed volume by rigid registration. However, accurate registration is …

Applications in materials and biological imaging are limited by the ability to collect high-resolution data over large areas in practical amounts of time. One solution to this problem is to collect low-resolution data and interpolate to produce a high-resolution image. However, most existing super-resolution algorithms are designed for natural images, often require aligned pairing of high and low-resolution training data, and may not directly incorporate a model of the imaging sensor. In this paper, we present a Multi-resolution Data Fusion (MDF) algorithm for accurate interpolation of low-resolution electron microscope data at multiple resolutions up to 8x. Our approach uses small quantities of unpaired high-resolution data to train a neural network prior model denoiser and then uses the Multi-Agent Consensus Equilibrium (MACE) problem formulation to balance this denoiser with a forward model agent that promotes fidelity to measured data. A key theoretical novelty is the analysis of mismatched back-projectors, which modify typical forward model updates for computational efficiency or improved image quality. We use MACE to prove that using a mismatched back-projector is equivalent to using a standard back-projector and an appropriately modified prior model. We present electron microscopy results at 4x and 8x interpolation factors that exhibit reduced artifacts relative to existing methods while maintaining fidelity to acquired data and accurately resolving sub-pixel-scale features.

To develop a super‐resolution technique using convolutional neural networks for generating thin‐slice knee MR images from thicker input slices, and compare this method with alternative through‐plane interpolation methods.

… for medical image super-resolution. Additionally, we discuss the applications of super-resolution techniques across various medical imaging modalities, including CT and MRI. Finally, …

The purpose of super-resolution approaches is to overcome the hardware limitations and the clinical requirements of imaging procedures by reconstructing high-resolution images from low-resolution acquisitions using post-processing methods. Super-resolution techniques could have strong impacts on structural magnetic resonance imaging when focusing on cortical surface or fine-scale structure analysis for instance. In this paper, we study deep three-dimensional convolutional neural networks for the super-resolution of brain magnetic resonance imaging data. First, our work delves into the relevance of several factors in the performance of the purely convolutional neural network-based techniques for the monomodal super-resolution: optimization methods, weight initialization, network depth, residual learning, filter size in convolution layers, number of the filters, training patch size and number of training subjects. Second, our study also highlights that one single network can efficiently handle multiple arbitrary scaling factors based on a multiscale training approach. Third, we further extend our super-resolution networks to the multimodal super-resolution using intermodality priors. Fourth, we investigate the impact of transfer learning skills onto super-resolution performance in terms of generalization among different datasets. Lastly, the learnt models are used to enhance real clinical low-resolution images. Results tend to demonstrate the potential of deep neural networks with respect to practical medical image applications.

Background Cardiac MRI is limited by long acquisition times, yet faster acquisition of smaller-matrix images reduces spatial detail. Deep learning (DL) might enable both faster acquisition and higher spatial detail via super-resolution. Purpose To explore the feasibility of using DL to enhance spatial detail from small-matrix MRI acquisitions and evaluate its performance against that of conventional image upscaling methods. Materials and Methods Short-axis cine cardiac MRI examinations performed between January 2012 and December 2018 at one institution were retrospectively collected for algorithm development and testing. Convolutional neural networks (CNNs), a form of DL, were trained to perform super resolution in image space by using synthetically generated low-resolution data. There were 70%, 20%, and 10% of examinations allocated to training, validation, and test sets, respectively. CNNs were compared against bicubic interpolation and Fourier-based zero padding by calculating the structural similarity index (SSIM) between high-resolution ground truth and each upscaling method. Means and standard deviations of the SSIM were reported, and statistical significance was determined by using the Wilcoxon signed-rank test. For evaluation of clinical performance, left ventricular volumes were measured, and statistical significance was determined by using the paired Student t test. Results For CNN training and retrospective analysis, 400 MRI scans from 367 patients (mean age, 48 years ± 18; 214 men) were included. All CNNs outperformed zero padding and bicubic interpolation at upsampling factors from two to 64 (P < .001). CNNs outperformed zero padding on more than 99.2% of slices (9828 of 9907). In addition, 10 patients (mean age, 51 years ± 22; seven men) were prospectively recruited for super-resolution MRI. Super-resolved low-resolution images yielded left ventricular volumes comparable to those from full-resolution images (P > .05), and super-resolved full-resolution images appeared to further enhance anatomic detail. Conclusion Deep learning outperformed conventional upscaling methods and recovered high-frequency spatial information. Although training was performed only on short-axis cardiac MRI examinations, the proposed strategy appeared to improve quality in other imaging planes. © RSNA, 2020 Online supplemental material is available for this article.

Artificial intelligence (AI)-based image enhancement has the potential to reduce scan times while improving signal-to-noise ratio (SNR) and maintaining spatial resolution. This study prospectively evaluated AI-based image enhancement in 32 consecutive patients undergoing clinical brain MRI. Standard-of-care (SOC) three-dimensional (3D) T1 precontrast, 3D T2 fluid-attenuated inversion recovery, and 3D T1 postcontrast sequences were performed along with 45% faster versions of these sequences using half the number of phase-encoding steps. Images from the faster sequences were processed by a Food and Drug Administration-cleared AI-based image enhancement software for resolution enhancement. Four board-certified neuroradiologists scored the SOC and AI-enhanced image series independently on a five-point Likert scale for image SNR, anatomic conspicuity, overall image quality, imaging artifacts, and diagnostic confidence. While interrater κ was low to fair, the AI-enhanced scans were noninferior for all metrics and actually demonstrated a qualitative SNR improvement. Quantitative analyses showed that the AI software restored the high spatial resolution of small structures, such as the septum pellucidum. In conclusion, AI-based software can achieve noninferior image quality for 3D brain MRI sequences with a 45% scan time reduction, potentially improving the patient experience and scanner efficiency without sacrificing diagnostic quality. Keywords: MR Imaging, CNS, Brain/Brain Stem, Reconstruction Algorithms © RSNA, 2022.

Abstract Introduction To explore and evaluate the performance of MRI‐based brain tumor super‐resolution generative adversarial network (MRBT‐SR‐GAN) for improving the MRI image resolution in brain tumors. Methods A total of 237 patients from December 2018 and April 2020 with T2‐fluid attenuated inversion recovery (FLAIR) MR images (one image per patient) were included in the present research to form the super‐resolution MR dataset. The MRBT‐SR‐GAN was modified from the enhanced super‐resolution generative adversarial networks (ESRGAN) architecture, which could effectively recover high‐resolution MRI images while retaining the quality of the images. The T2‐FLAIR images from the brain tumor segmentation (BRATS) dataset were used to evaluate the performance of MRBT‐SR‐GAN contributed to the BRATS task. Results The super‐resolution T2‐FLAIR images yielded a 0.062 dice ratio improvement from 0.724 to 0.786 compared with the original low‐resolution T2‐FLAIR images, indicating the robustness of MRBT‐SR‐GAN in providing more substantial supervision for intensity consistency and texture recovery of the MRI images. The MRBT‐SR‐GAN was also modified and generalized to perform slice interpolation and other tasks. Conclusions MRBT‐SR‐GAN exhibited great potential in the early detection and accurate evaluation of the recurrence and prognosis of brain tumors, which could be employed in brain tumor surgery planning and navigation. In addition, this technique renders precise radiotherapy possible. The design paradigm of the MRBT‐SR‐GAN neural network may be applied for medical image super‐resolution in other diseases with different modalities as well.

Magnetic resonance (MR) images with both high resolutions and high signal-to-noise ratios (SNRs) are desired in many clinical and research applications. However, acquiring such images takes a long time, which is both costly and susceptible to motion artifacts. Acquiring MR images with good in-plane resolution and poor through-plane resolution is a common strategy that saves imaging time, preserves SNR, and provides one viewpoint with good resolution in two directions. Unfortunately, this strategy also creates orthogonal viewpoints that have poor resolution in one direction and, for 2D MR acquisition protocols, also creates aliasing artifacts. A deep learning approach called SMORE that carries out both anti-aliasing and super-resolution on these types of acquisitions using no external atlas or exemplars has been previously reported but not extensively validated. This paper reviews the SMORE algorithm and then demonstrates its performance in four applications with the goal to demonstrate its potential for use in both research and clinical scenarios. It is first shown to improve the visualization of brain white matter lesions in FLAIR images acquired from multiple sclerosis patients. Then it is shown to improve the visualization of scarring in cardiac left ventricular remodeling after myocardial infarction. Third, its performance on multi-view images of the tongue is demonstrated and finally it is shown to improve performance in parcellation of the brain ventricular system. Both visual and selected quantitative metrics of resolution enhancement are demonstrated.

Low-field MRI scanners are significantly less expensive than their high-field counterparts, which gives them the potential to make MRI technology more accessible all around the world. In general, images acquired using low-field MRI scanners tend to be of a relatively low resolution, as signal-to-noise ratios are lower. The aim of this work is to improve the resolution of these images. To this end, we present a deep learning-based approach to transform low-resolution low-field MR images into high-resolution ones. A convolutional neural network was trained to carry out single image super-resolution reconstruction using pairs of noisy low-resolution images and their noise-free high-resolution counterparts, which were obtained from the publicly available NYU fastMRI database. This network was subsequently applied to noisy images acquired using a low-field MRI scanner. The trained convolutional network yielded sharp super-resolution images in which most of the high-frequency components were recovered. In conclusion, we showed that a deep learning-based approach has great potential when it comes to increasing the resolution of low-field MR images.

High resolution magnetic resonance (MR) images are desired in many clinical and research applications. Acquiring such images with high signal-to-noise (SNR), however, can require a long scan duration, which is difficult for patient comfort, is more costly, and makes the images susceptible to motion artifacts. A very common practical compromise for both 2D and 3D MR imaging protocols is to acquire volumetric MR images with high in-plane resolution, but lower through-plane resolution. In addition to having poor resolution in one orientation, 2D MRI acquisitions will also have aliasing artifacts, which further degrade the appearance of these images. This paper presents an approach SMORE1 based on convolutional neural networks (CNNs) that restores image quality by improving resolution and reducing aliasing in MR images.2 This approach is self-supervised, which requires no external training data because the high-resolution and low-resolution data that are present in the image itself are used for training. For 3D MRI, the method consists of only one self-supervised super-resolution (SSR) deep CNN that is trained from the volumetric image data. For 2D MRI, there is a self-supervised anti-aliasing (SAA) deep CNN that precedes the SSR CNN, also trained from the volumetric image data. Both methods were evaluated on a broad collection of MR data, including filtered and downsampled images so that quantitative metrics could be computed and compared, and actual acquired low resolution images for which visual and sharpness measures could be computed and compared. The super-resolution method is shown to be visually and quantitatively superior to previously reported methods.

Super‐resolution is an emerging method for enhancing MRI resolution; however, its impact on image quality is still unknown.

Background Deep learning (DL) reconstructions can enhance image quality while decreasing MRI acquisition time. However, DL reconstruction methods combined with compressed sensing for prostate MRI have not been well studied. Purpose To use an industry-developed DL algorithm to reconstruct low-resolution T2-weighted turbo spin-echo (TSE) prostate MRI scans and compare these with standard sequences. Materials and Methods In this prospective study, participants with suspected prostate cancer underwent prostate MRI with a Cartesian standard-resolution T2-weighted TSE sequence (T2C) and non-Cartesian standard-resolution T2-weighted TSE sequence (T2NC) between August and November 2022. Additionally, a low-resolution Cartesian DL-reconstructed T2-weighted TSE sequence (T2DL) with compressed sensing DL denoising and resolution upscaling reconstruction was acquired. Image sharpness was assessed qualitatively by two readers using a five-point Likert scale (from 1 = nondiagnostic to 5 = excellent) and quantitatively by calculating edge rise distance. The Friedman test and one-way analysis of variance with post hoc Bonferroni and Tukey tests, respectively, were used for group comparisons. Prostate Imaging Reporting and Data System (PI-RADS) score agreement between sequences was compared by using Cohen κ. Results This study included 109 male participants (mean age, 68 years ± 8 [SD]). Acquisition time of T2DL was 36% and 29% lower compared with that of T2C and T2NC (mean duration, 164 seconds ± 20 vs 257 seconds ± 32 and 230 seconds ± 28; P < .001 for both). T2DL showed improved image sharpness compared with standard sequences using both qualitative (median score, 5 [IQR, 4-5] vs 4 [IQR, 3-4] for T2C and 4 [IQR, 3-4] for T2NC; P < .001 for both) and quantitative (mean edge rise distance, 0.75 mm ± 0.39 vs 1.15 mm ± 0.68 for T2C and 0.98 mm ± 0.65 for T2NC; P < .001 and P = .01) methods. PI-RADS score agreement between T2NC and T2DL was excellent (κ range, 0.92-0.94 [95% CI: 0.87, 0.98]). Conclusion DL reconstruction of low-resolution T2-weighted TSE sequences enabled accelerated acquisition times and improved image quality compared with standard acquisitions while showing excellent agreement with conventional sequences for PI-RADS ratings. Clinical trial registration no. NCT05820113 © RSNA, 2023 Supplemental material is available for this article. See also the editorial by Turkbey in this issue.

BACKGROUND AND OBJECTIVE With the high-resolution (HR) requirements of medical images in clinical practice, super-resolution (SR) reconstruction algorithms based on low-resolution (LR) medical images have become a research hotspot. This type of method can significantly improve image SR without improving hardware equipment, so it is of great significance to review it. METHODS Aiming at the unique SR reconstruction algorithms in the field of medical images, based on subdivided medical fields such as magnetic resonance (MR) images, computed tomography (CT) images, and ultrasound images. Firstly, we deeply analyzed the research progress of SR reconstruction algorithms, and summarized and compared the different types of algorithms. Secondly, we introduced the evaluation indicators corresponding to the SR reconstruction algorithms. Finally, we prospected the development trend of SR reconstruction technology in the medical field. RESULTS The medical image SR reconstruction technology based on deep learning can provide more abundant lesion information, relieve the expert's diagnosis pressure, and improve the diagnosis efficiency and accuracy. CONCLUSION The medical image SR reconstruction technology based on deep learning helps to improve the quality of medicine, provides help for the diagnosis of experts, and lays a solid foundation for the subsequent analysis and identification tasks of the computer, which is of great significance for improving the diagnosis efficiency of experts and realizing intelligent medical care.

… -task learning approaches based on super-resolution of MR image. Furthermore, we also present the background of the super-resolution … of existing MRI super-resolution methods and …

Magnetic resonance imaging (MRI) is widely used in the detection and diagnosis of diseases. High-resolution MR images will help doctors to locate lesions and diagnose diseases. However, the acquisition of high-resolution MR images requires high magnetic field intensity and long scanning time, which will bring discomfort to patients and easily introduce motion artifacts, resulting in image quality degradation. Therefore, the resolution of hardware imaging has reached its limit. Based on this situation, a unified framework based on deep learning super resolution is proposed to transfer state-of-the-art deep learning methods of natural images to MRI super resolution. Compared with the traditional image super-resolution method, the deep learning super-resolution method has stronger feature extraction and characterization ability, can learn prior knowledge from a large number of sample data, and has a more stable and excellent image reconstruction effect. We propose a unified framework of deep learning -based MRI super resolution, which has five current deep learning methods with the best super-resolution effect. In addition, a high-low resolution MR image dataset with the scales of ×2, ×3, and ×4 was constructed, covering 4 parts of the skull, knee, breast, and head and neck. Experimental results show that the proposed unified framework of deep learning super resolution has a better reconstruction effect on the data than traditional methods and provides a standard dataset and experimental benchmark for the application of deep learning super resolution in MR images.

Abstract Super resolution problems are widely discussed in medical imaging. Spatial resolution of medical images are not sufficient due to the constraints such as image acquisition time, low irradiation dose or hardware limits. To address these problems, different super resolution methods have been proposed, such as optimization or learning-based approaches. Recently, deep learning methods become a thriving technology and are developing at an exponential speed. We think it is necessary to write a review to present the current situation of deep learning in medical imaging super resolution. In this paper, we first briefly introduce deep learning methods, then present a number of important deep learning approaches to solve super resolution problems, different architectures as well as up-sampling operations will be introduced. Afterwards, we focus on the applications of deep learning methods in medical imaging super resolution problems, the challenges to overcome will be presented as well.

BACKGROUND AND OBJECTIVE With the rapid development of medical imaging and intelligent diagnosis, artificial intelligence methods have become a research hotspot of radiography processing technology in recent years. The low definition of knee magnetic resonance image texture seriously affects the diagnosis of knee osteoarthritis. This paper presents a super-resolution reconstruction method to address this problem. METHODS In this paper, we propose an efficient medical image super-resolution (EMISR) method, in which we mainly adopted three hidden layers of super-resolution convolution neural network (SRCNN) and a sub-pixel convolution layer of efficient sub-pixel convolution neural network (ESPCN). The addition of the efficient sub-pixel convolutional layer in the hidden layer and the small network replacement consisting of concatenated convolutions to address low-resolution images but not high-resolution images are important. The EMISR method also uses cascaded small convolution kernels to improve reconstruction speed and deepen the convolution neural network to improve reconstruction quality. RESULTS The proposed method is tested in the public dataset IDI, and the reconstruction quality of the algorithm is higher than that of the sparse coding-based network (SCN) method, the SRCNN method, and the ESPCN method (+ 2.306 dB, + 2.540 dB, + 1.089 dB improved); moreover, the reconstruction speed is faster than its counterparts (+ 4.272 s, + 1.967 s, and + 0.073 s improved). CONCLUSION The experimental results show that our EMISR framework has improved performance and greatly reduces the number of parameters and training time. Furthermore, the reconstructed image presents more details, and the edges are more complete. Therefore, the EMISR technique provides a more powerful medical analysis in knee osteoarthritis examinations.

Super-resolution reconstruction (SRR) from motion-corrupted thick-slice stacks of fetal brain MR images is essential for comprehensive prenatal examination and precise quantification of fetal brain development. Conventional approaches for fetal brain SRR require human intervention to extract cerebral structures from 2D slices, and their performance is often limited by blurred and less informative acquisitions due to fast scanning or irregular fetal movement. In order to overcome these challenges, we propose an automatic fetal brain SRR framework by integrating both individual- and group-level priors of fetal brain MRI for improved SRR. Specifically, we develop a robust fetal brain extraction approach based on Segment Anything Model (SAM). The extracted fetal brain region is segmented into brain tissues to serve as anatomical prior for the follow-up SRR. The SRR is performed based on an iterative optimization scheme by alternatingly performing slice-to-volume registration and volumetric reconstruction. Specifically, we ingeniously integrate anatomical priors obtained by tissue segmentation into both slice-to-volume registration and volumetric reconstruction, which emphasizes boundary alignment during registration and mitigates misalignment stemming from indistinct boundaries of cerebral tissues. Furthermore, we harness the available longitudinal fetal brain atlases to serve as specific guidance for volumetric reconstruction, thereby enriching structural details of the reconstructed images and also circumventing reconstruction of outliers. Experimental results on 184 clinical fetal brain MR images show that our proposed framework largely outperforms state-of-the-art methods for fetal brain SRR quantitatively and qualitatively.

… anatomical structure maps for MRI slices. By conditioning the pretraining of the discrete latent space on anatomical priors, we develop an anatomical-… We further constrain the matching …

… constraint (DBTV), and the second one, the Tri-modal regularizer, is designed to benefit from a priori knowledge on intensities of brain MRI… the continuity of the brain anatomy in all three …

Magnetic Resonance Imaging (MRI) is pivotal in clinical diagnostics and neurological research, providing high-contrast, non-invasive imaging. However, the acquisition of high-resolution MRI is hampered by hardware limitations, extended scan durations, and low signal-to-noise ratios (SNR). To overcome these challenges, super-resolution (SR) techniques have emerged as a promising alternative, enhancing the resolution of low-resolution (LR) brain MRI images. This systematic review evaluates the current landscape of SR methods applied to brain MRI, analyzing recent studies’ algorithms, datasets, and evaluation metrics. We systematically retrieved and examined relevant literature from five major databases: Scopus, PubMed, IEEE Xplore, Google Scholar, and ACM Digital Library. Our analysis identifies key methodologies, including convolutional neural networks (CNNs), generative adversarial networks (GANs), and Transformer-based models, significantly contributing to SR advancements. The review also highlights the challenges in accurately reconstructing high-resolution images, particularly in maintaining fine details at higher scaling factors. Evaluation metrics such as Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) are discussed to quantify the performance of these SR models. This comprehensive assessment provides insights into the current state of brain MRI SR, guiding future research and development efforts to improve diagnostic accuracy and reduce scan times.

The interpretation and analysis of Magnetic resonance imaging (MRI) benefit from high spatial resolution. Unfortunately, direct acquisition of high spatial resolution MRI is time-consuming and costly, which increases the potential for motion artifact, and suffers from reduced signal-to-noise ratio (SNR). Super-resolution reconstruction (SRR) is one of the most widely used methods in MRI since it allows for the trade-off between high spatial resolution, high SNR, and reduced scan times. Deep learning has emerged for improved SRR as compared to conventional methods. However, current deep learning-based SRR methods require large-scale training datasets of high-resolution images, which are practically difficult to obtain at a suitable SNR. We sought to develop a methodology that allows for dataset-free deep learning-based SRR, through which to construct images with higher spatial resolution and of higher SNR than can be practically obtained by direct Fourier encoding. We developed a dataset-free learning method that leverages a generative neural network trained for each specific scan or set of scans, which in turn, allows for SRR tailored to the individual patient. With the SRR from three short duration scans, we achieved high quality brain MRI at an isotropic spatial resolution of 0.125 cubic mm with six minutes of imaging time for T2 contrast and an average increase of 7.2 dB (34.2%) in SNR to these short duration scans. Motion compensation was achieved by aligning the three short duration scans together. We assessed our technique on simulated MRI data and clinical data acquired from 15 subjects. Extensive experimental results demonstrate that our approach achieved superior results to state-of-the-art methods, while in parallel, performed at reduced cost as scans delivered with direct high-resolution acquisition.

Super-resolution techniques provide a route to studying fine scale anatomical detail using multiple lower resolution acquisitions. In particular, techniques that do not depend on regular sampling can be used in medical imaging situations where imaging time and resolution are limited by subject motion. We investigate in this work the use of super-resolution technique for anisotropic fetal brain MR data reconstruction without modifying the data acquisition protocol. The approach, which consists of iterative motion correction and high resolution image estimation, is compared with a previously used scattered data interpolation-based reconstruction method. To optimize acquisition time, an evaluation of the influence of the number of input images and image noise is also performed. Evaluation on simulated MR images and real data show significant improvements in performance provided by the super-resolution approach.

Fast magnetic resonance imaging slice acquisition techniques such as single shot fast spin echo are routinely used in the presence of uncontrollable motion. These techniques are widely used for fetal magnetic resonance imaging (MRI) and MRI of moving subjects and organs. Although high-quality slices are frequently acquired by these techniques, inter-slice motion leads to severe motion artifacts that are apparent in out-of-plane views. Slice sequential acquisitions do not enable 3-D volume representation. In this study, we have developed a novel technique based on a slice acquisition model, which enables the reconstruction of a volumetric image from multiple-scan slice acquisitions. The super-resolution volume reconstruction is formulated as an inverse problem of finding the underlying structure generating the acquired slices. We have developed a robust M-estimation solution which minimizes a robust error norm function between the model-generated slices and the acquired slices. The accuracy and robustness of this novel technique has been quantitatively assessed through simulations with digital brain phantom images as well as high-resolution newborn images. We also report here successful application of our new technique for the reconstruction of volumetric fetal brain MRI from clinically acquired data.

… magnetic resonance imaging (MRI), super-resolution … This approach employs second-order gradient constraints in … brain images with significant anatomical differences. However…

… As a reference for the anatomical structures … constrained ourselves to studying the resolution-SNR-acquisition time relationship, and kept all other parameters fixed. These constraints …

There is a growing demand for high-resolution (HR) medical images for both clinical and research applications. Image quality is inevitably traded off with acquisition time, which in turn impacts patient comfort, examination costs, dose, and motion-induced artifacts. For many image-based tasks, increasing the apparent spatial resolution in the perpendicular plane to produce multi-planar reformats or 3D images is commonly used. Single-image super-resolution (SR) is a promising technique to provide HR images based on deep learning to increase the resolution of a 2D image, but there are few reports on 3D SR. Further, perceptual loss is proposed in the literature to better capture the textural details and edges versus pixel-wise loss functions, by comparing the semantic distances in the high-dimensional feature space of a pre-trained 2D network (e.g., VGG). However, it is not clear how one should generalize it to 3D medical images, and the attendant implications are unclear. In this paper, we propose a framework called SOUP-GAN: Super-resolution Optimized Using Perceptual-tuned Generative Adversarial Network (GAN), in order to produce thinner slices (e.g., higher resolution in the ‘Z’ plane) with anti-aliasing and deblurring. The proposed method outperforms other conventional resolution-enhancement methods and previous SR work on medical images based on both qualitative and quantitative comparisons. Moreover, we examine the model in terms of its generalization for arbitrarily user-selected SR ratios and imaging modalities. Our model shows promise as a novel 3D SR interpolation technique, providing potential applications for both clinical and research applications.

… robustness of the created superresolution images from these … conclude that the proposed super-resolution techniques can both … on a minimization algorithm with a constraint term on the …

Highlights • A fused attentive generative adversarial networks framework is proposed for MR image super-resolution.• A combination of channel attention and self-attention is used to calculate the weight parameters of the input features.• Spectral normalization process is introduced to make the discriminator network stabler.• The proposed FA-GAN method is superior to the state-of-the-art reconstruction methods.

… on Transformer and generative adversarial network (GAN), with Tansformer approach and … T-GAN model can be employed directly for super-resolution MRI image reconstruction, and …

Single image super-resolution (SISR) reconstruction for magnetic resonance imaging (MRI) has generated significant interest because of its potential to not only speed up imaging but to improve quantitative processing and analysis of available image data. Generative Adversarial Networks (GAN) have proven to perform well in image recovery tasks. In this work, we followed the GAN framework and developed a generator coupled with discriminator to tackle the task of 3D SISR on T1 brain MRI images. We developed a novel 3D memory-efficient residual-dense block generator (MRDG) that achieves state-of-the-art performance in terms of SSIM (Structural Similarity), PSNR (Peak Signal to Noise Ratio) and NRMSE (Normalized Root Mean Squared Error) metrics. We also designed a pyramid pooling discriminator (PPD) to recover details on different size scales simultaneously. Finally, we introduced model blending, a simple and computational efficient method to balance between image and texture quality in the final output, to the task of SISR on 3D images.

Challenges arise in achieving high-resolution Magnetic Resonance Imaging (MRI) to improve disease diagnosis accuracy due to limitations in hardware, patient discomfort, long acquisition times, and high costs. While Convolutional Neural Networks (CNNs) have shown promising results in MRI super-resolution, they often don't look into the structural similarity and prior information available in consecutive MRI slices. By leveraging information from sequential slices, more robust features can be obtained, potentially leading to higher-quality MRI slices. We propose a multi-slice two-dimensional (2D) MRI super-resolution network that combines a Generative Adversarial Network (GAN) with feature fusion and a pre-trained slice interpolation network to achieve three-dimensional (3D) super-resolution. The proposed model requires consecutively acquired three low-resolution (LR) MRI slices along a specific axis, and achieves the reconstruction of the MRI slices in the remaining two axes. The network effectively enhances both in-plane and out-of-plane resolution along the sagittal axis while addressing computational and memory constraints in 3D super-resolution. The proposed generator has a in-plane and out-of-plane Attention (IOA) network that fuses both in-plane and out-plane features of MRI dynamically. In terms of out-of-plane attention, the network merges features by considering the similarity distance between features and for in-plane attention, the network employs a two-level pyramid structure with varying receptive fields to extract features at different scales, ensuring the inclusion of both global and local features. Subsequently, to achieve 3D MRI super-resolution, a pre-trained slice interpolation network is used that takes two consecutive super-resolved MRI slices to generate a new intermediate slice. To further enhance the network performance and perceptual quality, we introduce a feature up-sampling layer and a feature extraction block with Scaled Exponential Linear Unit (SeLU). Moreover, our super-resolution network incorporates VGG loss from a fine-tuned VGG-19 network to provide additional enhancement. Through experimental evaluations on the IXI dataset and BRATS dataset, using the peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM) and the number of training parameters, we demonstrate the superior performance of our method compared to the existing techniques. Also, the proposed model can be adapted or modified to achieve super-resolution for both 2D and 3D MRI data.

… In this paper, based on the neural network model termed as GAN-CIRCLE (… super-resolution for both MRI and CT. In this study, we demonstrate two-fold resolution enhancement for MRI …

Abstract Generative Adversarial Models (GANs) have been quite popular and are currently and active area of research. They can be used for generative new data and study adversarial samples and attacks. We have used the similar approach to apply super-resolution to medical images. In Radiology MRI is a commonly used method to produce medical imaging but the limitations of lab equipment and health hazard of being in an MRI radiation environment to obtain good quality scans lead to lower quality scans and also it takes a lot of time to get a high-resolution data. This problem can be solved by using super-resolution using deep learning as a post-processing step to improve the resolution of the scans. Super-resolution is a process of generating higher resolution images from lower resolution data. For this, we are proposing a generative adversarial network architecture which is a dual neural network designed to generate lifelike images. In this deep learning algorithm, two neural networks compete with each other to improve alternatively. Given a training set, this technique learns to generate new data with the same statistics as the training set. To apply this technique to our problem statement we are using generator as the network to improve the resolution and discriminator as a network to train generator better. We used transfer learning in our generative neural network and training our discriminator from scratch and using the perceptual loss [1] to train our network. This will help in improving the performance of the network. We are using Lung MRI scans of tuberculosis with a set of 216 MRI samples containing around 60-130 channels each and each channel having 512x512 dimensions.

MRI super-resolution (SR) and denoising tasks are fundamental challenges in the field of deep learning, which have traditionally been treated as distinct tasks with separate paired training data. In this paper, we propose an innovative method that addresses both tasks simultaneously using a single deep learning model, eliminating the need for explicitly paired noisy and clean images during training. Our proposed model is primarily trained for SR, but also exhibits remarkable noise-cleaning capabilities in the super-resolved images. Instead of conventional approaches that introduce frequency-related operations into the generative process, our novel approach involves the use of a GAN model guided by a frequency-informed discriminator. To achieve this, we harness the power of the 3D Discrete Wavelet Transform (DWT) operation as a frequency constraint within the GAN framework for the SR task on magnetic resonance imaging (MRI) data. Specifically, our contributions include: 1) a 3D generator based on residual-in-residual connected blocks; 2) the integration of the 3D DWT with 1 × 1 convolution into a DWT+conv unit within a 3D Unet for the discriminator; 3) the use of the trained model for high-quality image SR, accompanied by an intrinsic denoising process. We dub the model "Denoising Induced Super-resolution GAN (DISGAN)" due to its dual effects of SR image generation and simultaneous denoising. Departing from the traditional approach of training SR and denoising tasks as separate models, our proposed DISGAN is trained only on the SR task, but also achieves exceptional performance in denoising. The model is trained on 3D MRI data from dozens of subjects from the Human Connectome Project (HCP) and further evaluated on previously unseen MRI data from subjects with brain tumours and epilepsy to assess its denoising and SR performance. Our code is available at https://github.com/wqlevi/DISGAN.

… In this work, GAN based multi image MRI super-resolution that utilizes the correlated … Network (GAN)-based deep neural network tailored for the super-resolution of multiple MRI images …

Time-resolved three-dimensional phase-contrast MRI (4D Flow MRI) enables non-invasive quantification of blood flow and derivation of hemodynamic parameters. However, its clinical application is limited by low spatial resolution and noise, particularly affecting velocity measurements near vessel walls. Machine learning-based super-resolution has shown promise in addressing these limitations, but challenges remain, not least in recovering near-wall velocities. Generative adversarial networks (GANs) offer a compelling solution, having demonstrated strong capabilities in restoring sharp boundaries in non-medical super-resolution settings. Yet, their application in 4D Flow MRI remains unexplored, with implementation challenged by known issues such as training instability and non-convergence. In this study, we investigate GAN-based super-resolution and denoising in 4D Flow MRI. Training and validation were conducted using patient-specific cerebrovascular in-silico models, converted into synthetic images via an MR-true reconstruction pipeline, with complementary validation on in-vivo acquisitions. A dedicated GAN architecture was implemented and evaluated across three adversarial loss functions: Vanilla, Relativistic, and Wasserstein. Our results demonstrate that the proposed GAN improved near-wall velocity recovery compared to a non-adversarial reference (vector Normalized Root Mean Square Error (vNRMSE): 6.9% vs. 9.6%); however, implementation specifics are critical for stable network training. While Vanilla and Relativistic GANs proved unstable compared to generator-only training (vNRMSE: 8.1% and 7.8% vs. 7.2%), a Wasserstein GAN demonstrated optimal stability and incremental improvement (vNRMSE: 6.9% vs. 7.2%). Moreover, strong in-vivo performance supports clinical translation. Together, these findings highlight the potential of GAN-based super-resolution in enhancing 4D Flow MRI, particularly in challenging cerebrovascular regions, while emphasizing the importance of carefully selecting adversarial training strategies.

The high-resolution magnetic resonance image (MRI) provides detailed anatomical information critical for clinical application diagnosis. However, high-resolution MRI typically comes at the cost of long scan time, small spatial coverage, and low signal-to-noise ratio. The benefits of the convolutional neural network (CNN) can be applied to solve the super-resolution task to recover high-resolution generic images from low-resolution inputs. Additionally, recent studies have shown the potential to use the generative advertising network (GAN) to generate high-quality super-resolution MRIs using learned image priors. Moreover, existing approaches require paired MRI images as training data, which is difficult to obtain with existing datasets when the alignment between high and low-resolution images has to be implemented manually.This paper implements two different GAN-based models to handle the super-resolution: Enhanced super-resolution GAN (ESRGAN) and CycleGAN. Different from the generic model, the architecture of CycleGAN is modified to solve the super-resolution on unpaired MRI data, and the ESRGAN is implemented as a reference to compare GAN-based methods performance. The results of GAN-based models provide generated high-resolution images with rich textures compared to the ground-truth. Moreover, results from experiments are performed on both 3T and 7T MRI images in recovering different scales of resolution.

… methods typically concatenate features from different … Fusion Network (CMF-Net) that performs joint MRI reconstruction and super-resolution by enabling mutual propagation of feature …

… brain magnetic resonance imaging (MRI) super-resolution (SR… ) Most methods overlook feature fusion in the frequency domain… range, and struggle to fuse similar features effectively. (2) …

Magnetic resonance imaging (MRI) can present multicontrast images of the same anatomical structures, enabling multi-contrast super-resolution (SR) techniques. Compared with SR reconstruction using a single-contrast, multicontrast SR reconstruction is promising to yield SR images with higher quality by leveraging diverse yet complementary information embedded in different imaging modalities. However, existing methods still have two shortcomings: (1) they neglect that the multi-contrast features at different scales contain different anatomical details and hence lack effective mechanisms to match and fuse these features for better reconstruction; and (2) they are still deficient in capturing long-range dependencies, which are essential for the regions with complicated anatomical structures. We propose a novel network to comprehensively address these problems by developing a set of innovative Transformer-empowered multi-scale contextual matching and aggregation techniques; we call it McMRSR. Firstly, we tame transformers to model long-range dependencies in both reference and target images. Then, a new multi-scale contextual matching method is proposed to capture corresponding contexts from reference features at different scales. Furthermore, we introduce a multi-scale aggregation mechanism to gradually and interactively aggregate multi-scale matched features for reconstructing the target SR MR image. Extensive experiments demonstrate that our network outperforms state-of-the-art approaches and has great potential to be applied in clinical practice. Codes are available at https://github.com/XAIMI-Lab/McMRSR.

Multi-contrast MRI super-resolution assisted by auxiliary anatomical guidance has emerged as a pivotal strategy for accelerating clinical imaging protocols. While diffusion models (DMs…

BACKGROUND AND OBJECTIVES High-resolution magnetic resonance imaging (MRI) is essential for diagnosing and treating brain diseases. Transformer-based approaches demonstrate strong potential in MRI super-resolution by capturing long-range dependencies effectively. However, existing Transformer-based super-resolution methods face several challenges: (1) they primarily focus on low-frequency information, neglecting the utilization of high-frequency information; (2) they lack effective mechanisms to integrate both low-frequency and high-frequency information; (3) they struggle to effectively eliminate redundant information during the reconstruction process. To address these issues, we propose the Cross-fusion Adaptive Feature Enhancement Transformer (CAFET). METHODS Our model maximizes the potential of both CNNs and Transformers. It consists of four key blocks: a high-frequency enhancement block for extracting high-frequency information; a hybrid attention block for capturing global information and local fitting, which includes channel attention and shifted rectangular window attention; a large-window fusion attention block for integrating local high-frequency features and global low-frequency features; and an adaptive sparse overlapping attention block for dynamically retaining key information and enhancing the aggregation of cross-window features. RESULTS Extensive experiments validate the effectiveness of the proposed method. On the BraTS and IXI datasets, with an upsampling factor of ×2, the proposed method achieves a maximum PSNR improvement of 2.4 dB and 1.3 dB compared to state-of-the-art methods, along with an SSIM improvement of up to 0.16% and 1.42%. Similarly, at an upsampling factor of ×4, the proposed method achieves a maximum PSNR improvement of 1.04 dB and 0.3 dB over the current leading methods, along with an SSIM improvement of up to 0.25% and 1.66%. CONCLUSIONS Our method is capable of reconstructing high-quality super-resolution brain MRI images, demonstrating significant clinical potential.

… extraction and feature fusion in SGF-MRI. The M4Raw brain dataset results further validate the effectiveness of our approach across different anatomical structures. For 4×and 2×…

In clinical diagnosis, magnetic resonance imaging (MRI) allows different contrast images to be obtained. High‐resolution (HR) MRI presents fine anatomical structures, which is important for improving the efficiency of expert diagnosis and realising smart healthcare. However, due to the cost of scanning equipment and the time required for scanning, obtaining an HR brain MRI is quite challenging. Therefore, to improve the quality of images, reference‐based super‐resolution technology has come into existence. Nevertheless, the existing methods still have some drawbacks: (1) The advantages of different contrast images are not fully utilised. (2) The slice‐by‐slice scanning nature of magnetic resonance imaging is not considered. (3) The ability to capture contextual information and to match and fuse multi‐scale, multi‐contrast features is lacking. In this paper, we propose the multi‐slice aware matching and fusion (MSAMF) network, which makes full use of multi‐slice reference images information by introducing a multi‐slice aware module and multi‐scale matching strategy to capture corresponding contextual information in reference features at other scales. To further integrate matching features, a multi‐scale fusion mechanism is also designed to progressively fuse multi‐scale matching features, thereby generating more detailed super‐resolution images. The experimental results support the benefits of our network in enhancing the quality of brain MRI reconstruction.

Due to the unique environment and inherent properties of magnetic resonance imaging (MRI) instruments, MR images typically have lower resolution. Therefore, improving the resolution of MR images is beneficial for assisting doctors in diagnosing the condition. Currently, the existing MR image super-resolution (SR) methods still have the problem of insufficient detail reconstruction. To overcome this issue, this paper proposes a multi-level feature transfer network (MFTN) based on MRI-Transformer to realize SR of low-resolution MRI data. MFTN consists of a multi-scale feature reconstruction network (MFRN) and a multi-level feature extraction branch (MFEB). MFRN is constructed as a pyramid structure to gradually reconstruct image features at different scales by integrating the features obtained from MFEB, and MFEB is constructed to provide detail information at different scales for low resolution MR image SR reconstruction by constructing multiple MRI-Transformer modules. Each MRI-Transformer module is designed to learn the transfer features from the reference image by establishing feature correlations between the reference image and low-resolution MR image. In addition, a contrast learning constraint item is added to the loss function to enhance the texture details of the SR image. A large number of experiments show that our network can effectively reconstruct high-quality MR Images and achieves better performance compared to some state-of-the-art methods. The source code of this work will be released on GitHub.

… This paper explores impacts of super-resolution on DCE-MRI. The main … for MRI super-resolution reconstruction. A multi-scale deformable transformer for knee MRI super-resolution [26]. …

… and feature fusion module. After HLFB extraction and enhancement, it employs Global Feature Fusion(GFF) [20] for fusion. … operation to complete the fusion through convolution, the …

Multi-contrast magnetic resonance imaging (MRI) reflects information about human tissues from different perspectives and has wide clinical applications. By utilizing the auxiliary information from reference images (Refs) in the easy-to-obtain modality, multi-contrast MRI super-resolution (SR) methods can synthesize high-resolution (HR) images from their low-resolution (LR) counterparts in the hard-to-obtain modality. In this study, we systematically discussed the potential impacts caused by cross-modal misalignments between LRs and Refs and, based on this discussion, proposed a novel deep-learning-based method with Deformable Attention and Neighborhood-based feature aggregation to be Computationally Efficient (DANCE) and insensitive to misalignments. Our method has been evaluated in two public MRI datasets, i.e., IXI and FastMRI, and an in-house MR metabolic imaging dataset with amide proton transfer weighted (APTW) images. Experimental results reveal that our method consistently outperforms baselines in various scenarios, with significant superiority observed in the misaligned group of IXI dataset and the prospective study of the clinical dataset. The robustness study proves that our method is insensitive to misalignments, maintaining an average PSNR of 30.67 dB when faced with a maximum range of ±9°and ±9 pixels of rotation and translation on Refs. Given our method's desirable comprehensive performance, good robustness, and moderate computational complexity, it possesses substantial potential for clinical applications.

Magnetic resonance imaging (MRI) is an essential radiology technique in clinical diagnosis, but its spatial resolution may not suffice to meet the growing need for precise diagnosis due to hardware limitations and thicker slice thickness. Therefore, it is crucial to explore suitable methods to increase the resolution of MRI images. Recently, deep learning has yielded many impressive results in MRI image super-resolution (SR) reconstruction. However, current SR networks mainly use convolutions to extract relatively single image features, which may not be optimal for further enhancing the quality of image reconstruction. In this work, we propose a multi-level feature extraction and reconstruction (MFER) method to restore the degraded high-resolution details of MRI images. Specifically, to comprehensively extract different types of features, we design the triple-mixed convolution by leveraging the strengths and uniqueness of different filter operations. For the features of each level, we then apply deconvolutions to upsample them separately at the tail of the network, followed by the feature calibration of spatial and channel attention. Besides, we also use a soft cross-scale residual operation to improve the effectiveness of parameter optimization. Experiments on lesion-free and glioma datasets indicate that our method obtains superior quantitative performance and visual effects when compared with state-of-the-art MRI image SR methods.

Deep learning (DL) models, which have significantly promoted medical imaging, typically assume that training and testing data come from the same domain and distribution. However, these models struggle with unseen testing variations, like different imaging scanners or protocols, leading to suboptimal results from distribution mismatches between training and testing data. Despite extensive research, the issue of distribution mismatch in DL-based medical imaging has been largely overlooked in current literature. To improve the performance with mismatched testing data, this article proposes an orthogonal meta-learning (OML) framework for test-time adaptation (TTA) in medical imaging. Specifically, during training, we develop supervised meta-training reconstruction tasks to guide the self-supervised meta-testing task. Additionally, we introduce an orthogonal learning strategy to enforce orthogonality of pretrained parameters during training, which accelerates convergence during TTA and enhances performance. During the testing stage, the fine-tuned meta-learned parameters effectively reconstruct new, unseen testing data. Extensive experiments on magnetic resonance imaging and computed tomography datasets were conducted to validate our method’s effectiveness against other state-of-the-art methods, including supervised ones, in various mismatch scenarios.

Accurate segmentation of brain tumor plays an important role in MRI diagnosis and treatment monitoring of brain tumor. However, the degree of lesions in each patient's brain tumor region is usually inconsistent, with large structural differences, and brain tumor MR images are characterized by low contrast and blur, current deep learning algorithms often cannot achieve accurate segmentation. To address this problem, we propose a novel end-to-end brain tumor segmentation algorithm by integrating the improved 3D U-Net network and super-resolution image reconstruction into one framework. In addition, the coordinate attention module is embedded before the upsampling operation of the backbone network, which enhances the capture ability of local texture feature information and global location feature information. To demonstrate the segmentation results of the proposed algorithm in different brain tumor MR images, we have trained and evaluated the proposed algorithm on BraTS datasets, and compared with other deep learning algorithms by dice similarity scores. On the BraTS2021 dataset, the proposed algorithm achieves the dice similarity score of 89.61%, 88.30%, 91.05%, and the Hausdorff distance (95%) of 1.414 mm, 7.810 mm, 4.583 mm for the enhancing tumors, tumor cores and whole tumors, respectively. The experimental results illuminate that our method outperforms the baseline 3D U-Net method and yields good performance on different datasets. It indicated that it is robust to segmentation of brain tumor MR images with structures vary considerably.

Capturing an enclosing volume of moving subjects and organs using fast individual image slice acquisition has shown promise in dealing with motion artefacts. Motion between slice acquisitions results in spatial inconsistencies that can be resolved by slice-to-volume reconstruction (SVR) methods to provide high quality 3D image data. Existing algorithms are, however, typically very slow, specialised to specific applications and rely on approximations, which impedes their potential clinical use. In this paper, we present a fast multi-GPU accelerated framework for slice-to-volume reconstruction. It is based on optimised 2D/3D registration, super-resolution with automatic outlier rejection and an additional (optional) intensity bias correction. We introduce a novel and fully automatic procedure for selecting the image stack with least motion to serve as an initial registration target. We evaluate the proposed method using artificial motion corrupted phantom data as well as clinical data, including tracked freehand ultrasound of the liver and fetal Magnetic Resonance Imaging. We achieve speed-up factors greater than 30 compared to a single CPU system and greater than 10 compared to currently available state-of-the-art multi-core CPU methods. We ensure high reconstruction accuracy by exact computation of the point-spread function for every input data point, which has not previously been possible due to computational limitations. Our framework and its implementation is scalable for available computational infrastructures and tests show a speed-up factor of 1.70 for each additional GPU. This paves the way for the online application of image based reconstruction methods during clinical examinations. The source code for the proposed approach is publicly available.

Accurate characterization of in utero human brain maturation is critical as it involves complex and interconnected structural and functional processes that may influence health later in life. Magnetic resonance imaging is a powerful tool to investigate equivocal neurological patterns during fetal development. However, the number of acquisitions of satisfactory quality available in this cohort of sensitive subjects remains scarce, thus hindering the validation of advanced image processing techniques. Numerical phantoms can mitigate these limitations by providing a controlled environment with a known ground truth. In this work, we present FaBiAN, an open-source Fetal Brain magnetic resonance Acquisition Numerical phantom that simulates clinical T2-weighted fast spin echo sequences of the fetal brain. This unique tool is based on a general, flexible and realistic setup that includes stochastic fetal movements, thus providing images of the fetal brain throughout maturation comparable to clinical acquisitions. We demonstrate its value to evaluate the robustness and optimize the accuracy of an algorithm for super-resolution fetal brain magnetic resonance imaging from simulated motion-corrupted 2D low-resolution series compared to a synthetic high-resolution reference volume. We also show that the images generated can complement clinical datasets to support data-intensive deep learning methods for fetal brain tissue segmentation.

… achieving competitive MRI super-resolution quality on three … estimation and super-resolution of 4D-MRI, addressing low … TTA method which is build upon the base IQA-TTA algorithm […