Multimodal large language models generate natural-language responses from visual inputs, yet may mention objects absent from an image. In medication assistance, accessible perception, and environmental decision-making, such hallucinations can create real-world safety risks. We propose Semantic-Spatial Agreement Verification (SSAV), a training-free method for verifying object claims. A visually grounded claim should remain stable across semantically equivalent queries and repeatedly localize to the same image region. SSAV aggregates multiple prompts to estimate semantic support and reduce sensitivity to query wording. Query-Induced Regional Verification (QIRV) combines cross-query region persistence, spatial overlap, and relative candidate dominance to identify isolated high responses and dispersed localizations. A geometric mean fuses semantic and spatial evidence, lowering the verification score when either branch lacks support. Experiments on three base models and multiple evaluation protocols show that SSAV effectively mitigates object hallucination. On LLaVA-1.5-7B, accuracy averaged across COCO, A-OKVQA, and GQA improves by 1.81 and 3.17 percentage points under POPE Popular and Adversarial, respectively, while CHAIRs decreases from 49.40% to 32.80%. These results show that cross-query semantic stability and regional consistency provide interpretable external visual evidence for object claims.
Reliability under sparse and heterogeneous failures remains a fundamental challenge for medical image segmentation. High average accuracy can conceal a small set of structurally distinct and clinically consequential errors. Existing post-hoc correction methods alleviate this problem, but typically estimate false-positive and false-negative corrections from the same fixed prediction. This ignores the dynamic evolution of error states and limits the correction of complex cases. Inspired by iterative error feedback in structured prediction, we propose Failure-Aware Progressive Repair (FAPR). FAPR represents the current segmentation mask as a dynamic failure state and models each repair operation as a state-transition operator. Each accepted correction forms a new prediction state for subsequent error diagnosis and repair, enabling later operations to adapt to preceding changes. Conditional routing selectively activates necessary state transitions, while failure replay exposes the model to rare error states. By keeping the base segmentor frozen, FAPR preserves its established segmentation capability while improving difficult cases. Across three public ultrasound lesion segmentation benchmarks, FAPR improves mean DSC by 1.52%. On the very-hard subsets of BUSI and TN3K, the average gain reaches 13.77%.
Reference labels used to train medical image classification models are not always as certain as they may appear, and this uncertainty has implications on performance metrics. In this study, we propose a framework to analyze model performance for periventricular Fazekas score prediction that goes beyond conventional metrics. The Fazekas score is an ordinal visual rating scale used to assess the severity of white matter hyperintensities and is known to be affected by inter-rater variability. While the best Fazekas score prediction model achieved a Matthews correlation coefficient (MCC) of 0.70, performance varied across data splits and loss functions, making interpretation of model capabilities difficult. Rather than interpreting epistemic uncertainty of a model's prediction as an isolated scalar value, our approach of uncertainty mapping relates uncertainty to its position within the learned feature representation. This highlights regions of class-boundary transitions where cases appear more ambiguous and misclassifications are more likely. It also identifies potential label disagreement, including low-uncertainty misclassified cases that expert review found to be inconsistent with the original reference Fazekas score. Therefore, uncertainty mapping allows model behaviour to be examined in relation to class separation and potential model-label disagreement. Loss function choice also influenced the uncertainty profile, with some models showing clearer class separation and more localized uncertainty in ambiguous regions than others. These findings suggest that uncertainty mapping for Fazekas score predictions can support model interpretation and targeted dataset review when reference labels are affected by ambiguity/ inter-rater variability.
All-in-One medical image restoration (MedIR) aims to address diverse tasks across modalities and degradation types using a single universal model. Existing methods typically prioritize modeling inter-task heterogeneity (e.g., distinct data distributions and degradation types). However, they largely neglect the inherent homogeneity present in medical images, such as widely shared anatomical structures within and across modalities, which can be leveraged to ease model training and improve generalization. To this end, we propose UniH3, a novel framework that Unifies Hierarchical Homogeneity and Heterogeneity for all-in-one medical image restoration. Specifically, to comprehensively exploit homogeneity, we introduce a Hierarchical Homogeneity Memory (H2M) module that progressively distills intra- and inter-task homogeneity priors from high-quality images during training, and adaptively retrieves the most relevant priors tailored to the input for guided restoration. These retrieved priors are then injected into the restoration pipeline via an efficient Homogeneity-Guided Attention (HGA) mechanism. Furthermore, to comprehensively address heterogeneity, we design a Hierarchical Heterogeneity Balancer (H2B) that mitigates both inter- and intra-task conflicts during optimization, facilitating balanced and effective multi-task learning. Extensive experiments on two large-scale benchmarks, MedIR-2D-500K and MedIR-3D-3K, demonstrate that UniH3 achieves state-of-the-art performance on both all-in-one and single-task medical image restoration. We hope this work establishes a strong benchmark and advances the development of general-purpose medical image restoration models. Code is available at https://github.com/Yaziwel/UniH3.
Large-scale public medical imaging datasets contribute critically to translational research. When accompanied by rich clinical and multi-omics data, they can stimulate exploratory research and enable secondary analyses. Expert annotations of such imaging collections can support the development of new image analysis tools. Continuous enrichment of images with image-derived data makes them more usable for researchers without expertise in image analysis or access to large-scale computational resources. The National Lung Screening Trial (NLST) released a rich longitudinal dataset that includes Computed Tomography (CT) images for over 26,000 patients. We introduce three Digital Imaging and Communications in Medicine (DICOM) formatted datasets, complementing NLST CT images, shared as analysis results in the National Cancer Institute Imaging Data Commons (IDC). Two of those (IDC NLSTSeg and IDC NLSTSybil) contain DICOM-harmonized annotations and extracted measurements (for 581 and 601 NLST patients, respectively) shared earlier using research formats (Sybil and NLSTseg). The third one (TotalSegmentator-CT-Segmentations) contains volumetric segmentations generated using TotalSegmentator and radiomics features for each segment for 26,194 NLST patients.
Segmenting bruises is a challenging task in medical imaging due to limited data and annotations, diffuse boundaries, and highly variable appearance. In this work, we propose BruNet, a segmentation framework that combines a ViT-based visual encoder (a self-supervised DINOv3 or a pretrained LingBot-Vision backbone) with a SAM-based mask decoder. BruNet is trained on the HAM10000 skin lesion dataset and evaluated on a separate bruise dataset without additional fine-tuning. Although a small number of prior studies have explored machine learning and computer vision for bruise analysis, existing work has primarily focused on detection, classification, or colour analysis rather than pixel-level localisation. To the best of our knowledge, this is the first study to address automatic bruise segmentation. Our results show that BruNet outperforms CNN-based models, state-of-the-art segmentation models, ChatGPT-4o/5-assisted SAM2 zero-shot baselines, and the medical-oriented MedSAM model, demonstrating strong cross-domain generalisation to bruise segmentation.
Adapting natural-image foundation models like DINOv3 to multi-modal medical imaging is challenging due to the significant domain gap between natural color images and multi-channel medical scans. We present a unified, patch-based framework that processes raw multimodal imaging through training-free registration, automated localization, and mask-filtered patch extraction. This architecture culminates in a hierarchical strategy that aggregates patch-level insights into subject-level diagnostics. Using liver fibrosis staging as a case study, we evaluate four patch-level feature representations: handcrafted Radiomics features, learned ResNet features, pre-trained foundation model SAM-Med2D features, and frozen DINOv3 features. To ensure a controlled comparison, all models utilize the same lightweight MLP head and are evaluated across both rigid and deformable registration settings. Our training protocol focuses on mild fibrosis (S1) and cirrhosis (S4) classes only, enabling a single classifier to address both substantial fibrosis detection and cirrhosis staging. Evaluated via 10 random train (90%)/ test (10%) splits on 360 subjects from the CARE 2025 Liver Track 4 cohort, our DINOv3-based framework significantly outperforms all baselines, achieving the best classification accuracy of 78.4% for S1 and 75.8% for S4.
Brain metastases exhibit high inter-lesion variability in size, enhancement pattern, and post-treatment appearance, making volumetric segmentation of both pre- and post-treatment cases the central challenge of the BraTS 2026 Task 1 (Brain Metastases). We build a pragmatic pipeline on a 5-fold nnU-Net ResEnc-L ensemble, in which each fold is trained independently for 1,000 epochs with the standard Dice + cross-entropy loss on 1,296 four-modality training cases. This ensemble is followed by a rule-based post-processing cascade tuned for the lesion-wise Dice similarity coefficient (LW-DSC), a detection-oriented metric that behaves very differently from the traditional global Dice. The final pipeline reaches an LW-DSC of 0.733 / 0.751 / 0.713 / 0.549 on the enhancing tumour (ET), tumour core (TC), whole tumour (WT), and resection cavity (RC) sub-regions on the official validation leaderboard. Rather than trusting these leaderboard gains, we audit every post-processing stage with a five-fold out-of-fold (OOF) analysis with no model-training leakage over all 1,296 training cases, scored with the official BraTS evaluation code (BraTS_evaluation): it confirms two stages as robust, per-fold-consistent improvements while the third improves only the leaderboard and does not reproduce out-of-fold. We further provide a mechanistic analysis of the LW-DSC metric that explains why recall-recovering post-processing carries low risk whereas component deletion does not, and we report thirteen negative results spanning loss engineering, alternative backbones, and inference-time settings, several of which run counter to widely held intuitions. Source code is released under Apache-2.0 at https://github.com/hornbeamliu/brats2026-met.
Accurate segmentation of polyps and skin lesions is pivotal for clinical diagnosis, yet existing methods struggle with low contrast, ambiguous boundaries, and cross-domain distribution discrepancies. Discriminative networks and most diffusion-based segmentation approaches predict standalone binary masks, leaving the visual priors of large-scale pretrained generative models largely unexploited. We propose InstEditSeg, a unified generative framework that reformulates medical segmentation as an instruction-driven image editing problem. Instead of emitting a mask, the model renders a color-coded overlay on the original image, conditioned on a textual instruction, so that the edited output aligns with the natural image distribution learned by latent diffusion models and mitigates the domain gap between natural and medical imagery. To recover fine anatomical structures, we introduce DINOv3 as an auxiliary visual encoder and a DINO Feature Guidance Block that builds a multi-scale feature pyramid. The pyramid is fused into the diffusion U-Net by channel concatenation and zero-initialized convolution so that hierarchical discriminative priors can be injected without perturbing the pretrained weights. A dual-branch classifier-free guidance strategy requiring only two forward passes per denoising step reduces inference cost. On polyp and skin lesion benchmarks the framework achieves accuracy competitive with strong discriminative baselines, and it further demonstrates concrete advantages of the generative formulation: notably better cross-domain generalization on unseen data, more complete multi-lesion segmentation, instruction-conditioned task control, and sampling flexibility. We also analyze the strengths and limitations of the paradigm, including its color sensitivity and unsupported attribute-conditioned selection. Code is available at: https://github.com/wincharm001/InstEditSeg.
The human heart lacks cardiac stem cells and the ability to reestablish the loss of ventricular cardiomyocytes (vCMs) after myocardial infarction (MI) resulting in heart dysfunction. vCM replacement using intracardiac transplantation of human induced pluripotent stem cell (iPSC) derived vCMs represent a promising strategy for targeting MI, but low engraftment remains a challenge. Herein, we investigated whether formulation of ventricular iPSC-CMs (iPSC-vCM) as spheres (iPSC-vCMSphere) improves cell retention and cardiac function as directly compared to single-cells (iPSC-vCMSC) after transplantation in a new severely immunocompromised mouse model with MI. Recipient survival immediately after MI and intervention was poor for both Vehicle (50%) and iPSC-vCMSC (36%) groups, whereas 92% of iPSC-vCMSphere treated mice survived, which is comparable to non-immune deficient C57bl/6 mice (91%). All surviving and engrafted animals retained iPSC-vCMs in the infarct border zone at 8-weeks, and heart function remained similar between groups, although minor improvements were observed for iPSC-vCMSphere animals. Surprisingly, iPSC-vCMSpheres were less mature than iPSC-vCMSCs. Thus, our data suggest that iPSC-vCM sphere formulation may offer some benefits for intracardiac delivery as compared with single cell formulated iPSC-vCMs, and thus remains a promising candidate for reestablishing the lost CMs after MI in the future.
Noninvasive live-cell imaging in deep human tissues is crucial for exploring the cellular biological and pathogenic processes, but remains a significant unmet challenge. Diffusion magnetic resonance imaging (dMRI) promises to narrow this gap by noninvasively providing cellular-level microstructural information. Within a single crowded voxel containing millions of living cells, the intricate cellular-level microstructures create numerous microcompartments, each characterized by a specific diffusivity. However, conventional dMRI methods relying on voxel-averaged macroscopic parameters, merely reflect aggregate microstructural properties and fail to quantify this distribution of microcompartment-specific diffusivity within a voxel, thereby obscuring microstructural details. Here, we propose an intravoxel diffusivity probability distribution (IDPD) model to resolve a wealth of essential microstructural information via quantifying microcompartment-specific diffusivity distribution, thereby enabling direct cellular-level characterization. This exceptional capability is realized through a multi-tiered analytical workflow spanning targeted single-voxel or region of interest (ROI) analysis to global visualization using dynamic videos and statistic parametric maps. Ultimately, the IDPD model enables noninvasive cellular-level microstructure imaging, offering a promising avenue to evaluate living cell functions in vivo.
Normative modeling in neuroscience aims to characterize interindividual variation in brain phenotypes and establish reference ranges, or brain charts, against which individuals can be compared. Normative models are typically limited to coarse spatial scales due to computational constraints, limiting their spatial specificity. Furthermore, dependence on fixed parcellation atlases limits their adaptability to alternative parcellation schemes. To overcome these key limitations, we propose spectral normative modeling (SNM), which leverages brain eigenmodes to efficiently generate normative ranges for arbitrarily defined regions of interest. Training SNM on over 78,000 healthy brain scans, we generate accurate lifespan thickness growth charts across different spatial scales, from millimeters to the whole brain. These charts reveal three principal thickness growth gradients, aligning neurotypical cortical change with established anatomical, genetic, and functional hierarchies. We further demonstrate SNM's utility by elucidating high-resolution individual cortical atrophy patterns that characterize the heterogeneous expression of neurodegeneration in Alzheimer's disease. SNM lays the groundwork for a new generation of spatially precise brain charts, offering substantial potential to drive advances in individualized precision medicine.
Background: Magnetic Resonance Imaging (MRI) is the mainstay for neurosurgical oncology but not for informing us about glioma biology. An obstacle to developing MR-based glioma biomarkers is the absence of rigorous correlation between MRI features and glioma biology as assessed in multi-regional biopsies, within and across patients. Methods: We directly addressed this obstacle by collating a unique cohort of 202 MRI-localized biopsies from 58 patients. We define a low-dimensional transcriptional pseudotime continuum along which heterogeneous high-grade glioma (HGG) samples organize both within and across patients. Results: We observe three polarized transcriptional tissue states: infiltrated brain, immune/inflammatory, and proliferative associated with patterns of cohabitation of cellular subpopulations. The states and deconvolved populations show correlation with enhancement status on T1Gd MRI. Moreover, discrete MRI habitats, regions sharing common imaging features, defined as combinations of high or low signal intensity across multiparametric MRI revealed 14 MRI habitats. We order the MRI habitats according to the average pseudotime on the transcriptional continuum. We find that MRI habitats with low pseudotime (associated with early tumor development and diffusely invaded brain tissue) localized at the periphery of the tumor whilst high pseudotime either proliferative or immune/inflammatory states were towards the core of the lesion. We find that composition of MRI habitats is impacted by MGMT status. Conclusion: This suggests that ongoing aggregation of MRI-localized biopsies may augment our projection of biology onto MRI habitats to support the noninvasive identification of cellular ecologies within and across each patient's tumor.
Background and Objectives: Automatic segmentation of the nasal cavity and paranasal sinuses from CT aids diagnosis and surgical planning, but clinical datasets in this domain remain small, which can affect training stability and evaluation validity. This study investigates whether the frequency-domain mechanisms of the Adaptive Frequency-Spatial Dual-Stream Network (AFS-DSN) can be transferred, in a lightweight form, into the self-configuring nnU-Net framework, and whether the resulting performance improvement survives rigorous statistical validation. We integrated the AFS-DSN frequency branch and cross-domain attention mechanism into nnU-Net, and corrected a zero-initialization gating deadlock in the original design, using the corrected model (v2) as a common base. On this base, we evaluated three variants: X1, an input-conditioned learnable spectral gate applied to 24 wavelet sub-bands; X2, which adds a boundary-distance training objective; and X3, combining X1 and X2. Experiments used the 130-volume NasalSeg CT dataset, under a full 91/19/20 protocol and a rebuilt, genuine 3-fold cross-validation in which the training set for each fold was reduced by approximately 19%, from 91 to 73-74 cases. Under the full protocol, all variants showed a small but statistically detectable improvement over a matched nnU-Net baseline. X3 achieved a Dice score of 95.78% versus 95.59% for the baseline, an improvement of 0.19 percentage points (95% CI [0.09, 0.29]; Holm-adjusted p = 0.004), and X3 also reduced average surface distance (ASD) from 0.252 mm to 0.235 mm. X3 was statistically equivalent to either individual component within a margin of +/-0.10 percentage points, suggesting a shared performance ceiling rather than a complementary gain. Under reduced training data, baseline Dice dropped by 2.71 percentage points (95.59% to 92.88%), while X3 remained essentially unchanged (95.78% to 95.75%), a cross-fold advantage of +2.88 percentage points (fold-level 95% CI [0.80, 4.96]) that held across all 20 test cases (sign test, p = 1.9e-6) and was corroborated by average surface distance. The practical value of this combined approach lies primarily in within-distribution data efficiency, rather than in maximizing peak segmentation accuracy.
Transcranial low-intensity focused ultrasound (LIFU) offers a rare ability to noninvasively modulate human brain activity in superficial and deep neural structures. Yet the field still cannot reliably predict how a given acoustic protocol will affect a specific brain target. We synthesize 230 LIFU experiments targeting the human cerebrum from 136 articles reporting sufficient acoustic information and neuromodulatory outcomes. We compare associations between acoustic parameters and reported outcomes across cortical and subcortical targets. This reveals divergent parameter-effect landscapes across cortical and subcortical targets. Cortical outcomes follow a nonmonotonic organization across pulse repetition frequency and duty cycle, with distinct zones associated with facilitation, suppression, and null or ambiguous effects. Subcortical outcomes instead vary predominantly along duty cycle, shifting from suppression at lower values toward facilitation at higher values. Intensity and dose factors do not provide a consistent target-independent explanation for outcome polarity, and current evidence is insufficient to identify parameters promoting lasting effects. These findings challenge the assumption that acoustic protocols have generalizable effects across the brain and identify anatomical target as a central dimension that should inform LIFU protocol design. We provide SonoMap, an open, interactive web resource that supports a more systematic, evidence-based approach to human LIFU neuromodulation.
Training deep learning-based medical image segmentation models is challenging with limited curated datasets. For AGITG TOPGEAR, a gastric cancer trial, the Clinical Target Volume (CTV) is complex and defined by multiple anatomical landmarks, making upfront training data preparation difficult for an automated contour QA segmentation model. We investigate anatomical priors, derived from surrounding organ segmentations, to provide spatial context and improve TOPGEAR CTV segmentation accuracy. We also evaluate active learning, iteratively expanding the training dataset by selecting cases expected to improve performance. One hundred TOPGEAR CT scans were retrospectively analyzed. An initial set of 10 expert-contoured cases was used to train an nnU-Net model. TotalSegmentator generated a voxel-wise anatomical prior map from surrounding structures as an additional input channel. Active learning was simulated over four iterations, selecting cases by model uncertainty and segmentation performance. All models used five-fold cross-validation for an ensemble uncertainty measure. Evaluation used a hold-out testing set of 50 cases. The anatomical prior improved CTV segmentation accuracy, increasing mean Dice Similarity Coefficient (DSC) from 0.84 to 0.86. Active learning similarly improved performance to 0.86, with greatest benefit in the final round. Combining the anatomical prior with active learning achieved the highest accuracy, with a DSC of 0.87. Model uncertainty correlated with DSC, supporting its use in identifying suboptimal predictions and guiding active learning. Anatomical priors and active learning each improved CTV segmentation accuracy and generalizability, with their combination achieving the best performance, supporting integration into segmentation model development for automated contour QA in radiotherapy clinical trials.
A large share of late-stage clinical trial failures reflects not the underlying biology of the target but the composition of the enrolled population: trials recruit patients in whom the drug cannot work. Methods that identify likely responders before treatment therefore address a failure mode that better target selection alone cannot. We applied interpretable machine learning to gene-expression data from a treatment-response cohort in lupus nephritis (GSE224705; 21,914 genes across 319 samples) covering four regimens: mycophenolate mofetil (MMF), azathioprine (AZA), hydroxychloroquine (HC) and standard of care (SOC). We independently reconstructed the expression matrix and metadata, rebuilt the treatment-specific cohorts, and derived compact multi-gene programs that separate responders from non-responders within each treated population. Two results follow. First, discriminative performance is strongly graded by regimen. Com pact programs of five to ten genes achieved patient-level AUROC of 0.847 (MMF) and 0.866 (AZA), but only 0.718 (HC) and 0.623 (SOC); the SOC programs performed close to chance (MCC 0.119, balanced accuracy 0.555). A regimen in which response is not transcriptionally discriminable is an actionable finding for trial design rather than a null result. Second, the programs proved considerably more stable than the differential-expression lists that generated them: reconstructed counts of significant genes differed markedly from the published analysis (222 vs. 46 for MMF; 4,455 vs. 157 for AZA; 6 vs. 24 for HC; 5 vs. 11 for SOC), yet the dominant biology and the predictive performance were preserved. Programs were also non-redundant: re moving a single gene (TUBB2A) from the MMF program reduced AUROC by approximately 0.17. At the pathway level, 13 cross-treatment enrichment relationships remained significant after adjustment, indicating that response landscapes are treatment-specific yet coupled. Patient-generalisable programs of this kind offer a concrete near-term route to enrichment style trial design, identifying before enrolment which patients a given therapy suits. Our results also caution that the number of differentially expressed genes is a poor proxy for the strength or stability of a response signal.
Uncertainty Quantification (UQ) plays a vital role in enhancing the reliability of deep learning model predictions, especially in scenarios with high-dimensional output spaces. This paper addresses the dual nature of uncertainty -- aleatoric and epistemic -- focusing on their joint integration in high-dimensional regression tasks. For example, in applications like medical image segmentation or restoration, aleatoric uncertainty captures inherent data noise, while epistemic uncertainty quantifies the model's confidence in unfamiliar conditions. Modeling both jointly enables more reliable predictions by reflecting both unavoidable variability and knowledge gaps, whereas modeling only one limits transparency and robustness. We propose a novel approach that approximates the resulting joint uncertainty using a low-rank plus diagonal covariance structure, capturing essential output correlations while avoiding the computational burdens of full covariance matrices. Unlike prior work, our method explicitly combines aleatoric and epistemic uncertainties into a unified second-order distribution that supports robust downstream analyses like sampling and log-likelihood evaluation. We further introduce stabilization strategies for efficient training and inference, achieving superior UQ in the tasks of image inpainting, colorization, optical flow, and depth estimation.
Accurate brain tumor segmentation from magnetic resonance imaging (MRI) is essential for diagnosis, treatment planning, surgical guidance, and disease monitoring. However, developing automated segmentation models that generalize across diverse tumor characteristics, imaging protocols, acquisition sites, and patient populations remains challenging. Variations in tumor morphology and imaging distributions can substantially degrade performance outside the training domain. Consequently, improving the robustness and generalization of deep learning-based segmentation models has become a key objective in medical image analysis. To improve segmentation robustness, we propose Multi-Stage Dynamic Prompt nnU-Net, a prompt-conditioned extension of nnU-Net. Three independent dynamic prompt modules are inserted into the deepest encoder stages. Each module contains a learnable bank of ten 256-dimensional prompt vectors and uses globally pooled encoder features to generate image-specific prompt representations. These representations are projected into feature-wise scaling $(γ)$ and shifting $(β)$ parameters that modulate encoder feature maps through Feature-wise Linear Modulation (FiLM), enabling adaptive feature conditioning at multiple semantic levels. Evaluation on the BraTS GOAT validation dataset demonstrated that the proposed Multi-Stage Dynamic Prompt nnU-Net outperformed the baseline nnU Net across the majority of evaluated metrics and tumor subregions. The proposed model achieved average lesion-wise Dice scores of 76.16% (ET), 80.04% (TC), and 86.42% (WT), compared with 74.38%, 78.14% and 84.01% for the baseline model. The results demonstrate that multi-stage dynamic prompt conditioning improves segmentation accuracy and boundary delineation for brain tumor segmentation.
🏫 主要单位：Dept. of Otorhinolaryngology, Sleep Surgery Center, Sleep Disorders Center, Istituto Auxologico Italiano IRCCS, Milan, Italy, Dept. of Clinical Sciences and Co ｜ 主要作者：Federico Leone、Leone, F.
Background: Drug-Induced Sleep Endoscopy (DISE) is the current reference standard for identifying the anatomical site of upper airway obstruction in obstructive sleep apnea (OSA), but its invasiveness and limited availability restrict its routine use. Because snoring is generated by vibration of the obstructing upper-airway structures, its acoustic characteristics may provide a non-invasive biomarker of the anatomical site of obstruction. This study investigated whether machine learning could reliably distinguish palatal from epiglottic snoring using acoustic information alone. Methods: A retrospective analysis was performed on 159 DISE recordings obtained from adult patients with moderate, non-positional OSA. Snoring events were independently identified and anatomically classified by two blinded expert examiners. Only events with complete inter-observer agreement were included. The corresponding audio segments were extracted and characterized using spectral, cepstral, temporal and harmonic acoustic features. Feature selection was performed using recursive feature elimination. Support Vector Machine (SVM) and Multilayer Perceptron (MLP) classifiers were developed using a patient-independent nested cross-validation framework. Results: The final dataset comprised 1,759 snoring events, including 1,054 palatal and 705 epiglottic recordings. Both classifiers demonstrated robust discrimination between the two anatomical classes under patient-independent validation, achieving ROC AUC values of 0.90 +/- 0.03 (SVM) and 0.90 +/- 0.03 (MLP), with balanced accuracies of 0.83 and 0.81, respectively. The most informative predictors were spectral features, particularly spectral energy distribution, spectral flux, Mel-Frequency Cepstral Coefficients (MFCCs), and spectral flatness, whereas fundamental frequency contributed minimally to classification. Conclusions: Snoring contains reproducible acoustic information reflecting the anatomical origin of upper-airway obstruction. By combining high-confidence DISE-derived anatomical labels with a rigorous machine learning framework, this study demonstrates the feasibility of non-invasive acoustic phenotyping of clinically relevant obstruction sites. Rather than proposing a novel artificial intelligence algorithm, our work establishes a clinically oriented framework that may support future decision-support tools for patient selection, treatment planning, and multicentre development of comprehensive acoustic phenotyping models.
Segmentation of curvilinear anatomical structures in 3D medical images remains challenging due to complex topology, severe class imbalance, weak contrast, and large variations in structure morphology. While deep learning approaches for 3D curvilinear segmentation have been proposed, they are often tailored to specific anatomies or modalities, limiting generalization across clinical settings and leaving room for improvement. Recent generative models have shown the benefits of iterative prediction for structured segmentation tasks, yet diffusion-based methods suffer from computationally expensive sampling, hindering their use on high-resolution 3D volumes. We present 3D-CurvSegFlow, a flow matching-based model for 3D curvilinear structure segmentation. The model learns a continuous transformation from a simple source distribution to the target vascular representation, enabling progressive refinement of complex curvilinear geometries with efficient inference. We evaluate our method on Three public challenging datasets covering distinct anatomies and modalities: portal vein, cerebral vessel, and coronary arteries. Using a common architecture and training strategy across all tasks, our method outperforms general-purpose and vessel-specific approaches, with strong preservation of thin branches and vascular continuity. This work not only advances the state-of-the-art in 3D curvilinear segmentation but also opens new avenues for efficient, generalizable, and clinically applicable methods in medical image analysis.
Uncertainty estimation is critical for the safe clinical deployment of deep learning in medical image segmentation, with aleatoric uncertainty theoretically designed to capture irreducible data ambiguity. However, whether entropy-based measures reflect clinically meaningful ambiguity, i.e. case-level disagreement about whether a pathology is present at all, remains poorly understood. Contrary to most prior work, which focused on pixel-wise boundary disagreement, we systematically evaluate how well aleatoric uncertainty captures presence ambiguity. Our evaluation spans 3D lung nodule segmentation across four architectures with Monte Carlo dropout and deep ensembles, on LIDC-IDRI and an external validation cohort (LNDb). We find that entropy-based uncertainty maps align with boundary noise and minor drawing variation but carry insufficient discriminative signal for presence ambiguity. In contrast, a lightweight supervised ambiguity head trained on frozen segmentation features substantially outperforms all entropy-aggregation-based baselines across architectures, metrics, and both cohorts, and matches or exceeds methods that explicitly model ambiguity under disagreement supervision (Probabilistic U-Net, Annotator-Confusion 3D-UNet). A qualitative feature-space analysis shows that presence ambiguity is already encoded in the frozen encoder features of pixel-wise-trained networks, only to be discarded by the segmentation output and its entropy aggregation. Our findings expose a fundamental mismatch between the theoretical promise of aleatoric uncertainty and its practical behavior, and suggest that practitioners should not rely on entropy-based uncertainty as a proxy for clinical ambiguity in safety-critical applications.
Purpose: Magnetic resonance spectroscopic imaging (MRSI) is a versatile technique to investigate the spatial distribution of in vivo metabolism. However, processing MRSI data is demanding, and only a few software packages support end-to-end analysis. The goal of this study was to implement fully automated, end-to-end MRSI analysis into the open-source 'Osprey-MRSI' software package. Methods: MRSI-specific analysis and visualization capabilities were implemented, building on the existing Osprey workflow. Modifications included spatial transformation and filtering operations, automated brain masking and tissue segmentation of the MRSI data, improved lipid filtering, rapid integral maps, linear-combination modeling with explicit B0 frequency-shift correction, and generation of quality-control maps and metabolic images. A fully interactive GUI and semi-interactive HTML reports provide a user-friendly way to inspect each step of the analysis. All analysis derivatives are also exported in NIfTI and NIfTI-MRS format for easy visualization and synergies with other toolboxes and modalities. Results: The automated MRSI workflow was successfully used to analyze short- and medium-TE 3T in vivo MRSI datasets from all major vendors (Philips, GE, Siemens) across multiple sites. Correct coregistration of MRSI data and MR images was validated using phantom data from each vendor and existing MRSI processing tools. Conclusion: Osprey-MRSI offers state-of-the-art methods with minimal user interaction available for non-expert users. The modularity of the workflow and the modeling algorithm will foster innovation and development of novel MRSI-specific analysis methods.
Purpose: Deep learning-based medical image segmentation has achieved remarkable success, yet purely data-driven approaches often fail to exploit the rich mathematical structure inherent in medical images. We investigate whether explicit mathematical inductive biases, specifically matrix spectral analysis and vector calculus operators, can enhance segmentation beyond data-driven learning alone. Methods: We propose M-Net (Math-Augmented Network), which integrates three complementary mathematical priors into U-Net: (1) continuous spectral features derived from the condition number of centered local pixel matrices, providing a differentiable measure of texture ill-conditioning; (2) physical field operators (divergence and a discrete curl-like boundary irregularity operator) computed from image gradient fields, capturing focal intensity extrema and edge non-smoothness; and (3) a Math-Attention Gate (MAG) that adaptively fuses mathematical features with CNN-extracted deep features at skip connections. Results: Experiments on three benchmarks (LiTS, KiTS, and BraTS) show that M-Net achieves Dice scores of 78.42%, 76.15%, and 83.67%, outperforming baseline U-Net by 12.37%, 3.52%, and 5.55% on liver, kidney, and brain tumor segmentation, respectively. Ablations reveal that the condition-number feature contributes a 2.14% gain over binary invertibility features, while MAG adds 1.45% over simple concatenation. Conclusion: M-Net establishes that mathematical inductive biases provide effective complementary information for medical image segmentation. The continuous condition-number feature offers superior gradient information over discrete alternatives, and MAG preserves these priors throughout the network. This work opens avenues for integrating linear algebra and vector calculus into deep architectures for medical imaging.
Forecasting anatomical changes such as tumor growth and neurodegeneration is a challenging generative vision task. Morphological evolution is subtle relative to static anatomy, highly patient-specific, and inherently stochastic. Existing methods struggle with several issues: deterministic networks ignore biological stochasticity, while standard diffusion models require computationally prohibitive multi-pass sampling to quantify uncertainty. We propose MUMINS (Metadata-conditioned Uncertainty-aware Medical Image Next-state Synthesis), an efficient diffusion framework that jointly diffuses a baseline scan and its follow-up residual, summed to synthesize the follow-up scan, while concurrently predicting a spatial uncertainty map, in a single reverse diffusion process. Conditioned on the time interval and relevant metadata, it preserves fine-grained anatomy by dynamically re-injecting the baseline as a soft anchor at every denoising step, and a negative-log-likelihood head learns the uncertainty map to explicitly flag error-prone regions. Designed without organ-specific heuristics, the same architecture is reused across anatomies via separate, dataset-specific retraining. Extensive evaluations demonstrate that dataset-specific retraining of MUMINS matches or outperforms dedicated, domain-specific state-of-the-art methods on lung CT (PNG) and brain MRI (OASIS-3). Project page: https://github.com/aolivtous/MUMINS.
Acquiring high quality annotated medical image data is critical for training deep learning models; however, annotation is expensive, time consuming, and requires domain expertise. Conditional diffusion models, such as ControlNet, offer an alternative by generating images conditioned on semantic masks and text. However, existing approaches fail to capture fine grained properties (e.g., intensity and texture), as well as semantic consistency expected by domain experts, limiting their effectiveness for downstream tasks. Recent attempts to address these issues using reinforcement learning fine-tuning remain limited due to the reliance on a single scalar reward, which conflates diverse failure modes and provides weak corrective signals. We propose PRISM, a Compositional Reward Model (CRM) framework for conditional medical image generation. Instead of assigning a single reward, we decompose image quality into verifier grounded stages, each evaluating a distinct aspect of correctness from fine to coarse properties, including low level attributes (intensity and texture), structural alignment with conditioning inputs, and high level semantic fidelity. These stage wise rewards are composed through a Hierarchical Constrained Propagation (HCP) mechanism that enforces a fine to coarse notion of correctness, ensuring that lower level deficiencies are resolved before higher level rewards are accrued, preventing easier objectives from masking critical failures. We evaluate PRISM across three datasets spanning diverse medical imaging tasks: PanNuke (multi-class cell segmentation), CeDeM (villi/crypt detection and measurement), and ISIC (skin lesion classification). Training downstream models with data generated by PRISM yields improvements over closest baselines, including a 2.3% increase in mDice on PanNuke, a 8.5% reduction in Mean Relative Error (MRE) on CeDeM, and increases ISIC F1 by 5.9%.
Deep learning models have achieved impressive performance in medical image diagnosis, yet their deployment in clinical settings remains constrained by limited explainability. Counterfactual images provide one means of auditing model behavior by showing how an image would need to change for a classifier to produce a different prediction. Existing approaches typically generate such explanations using auxiliary models, including generative adversarial networks and diffusion models. While often capable of producing visually realistic images, these methods explain one black-box model using another, making it difficult to separate the classifier's decision-making process from the inductive biases of the generator. We propose a novel counterfactual-generation framework that requires no generative model. Instead, counterfactuals are constructed directly from causal evidence extracted from the classifier. The resulting approach is deterministic, requires no additional model training, and enables controllable edits within user-specified regions of interest. Experiments on real-world medical imaging datasets demonstrate that the proposed method successfully changes classifier predictions while remaining closer to the original image than generative baselines, providing a more direct and transparent view of the classifier's decision boundary.
Image super-resolution, which aims to reconstruct high-resolution images from their low-resolution observations, is fundamental to medical imaging, remote sensing, surveillance, microscopy, and scientific visualization. Traditional model-based methods formulate super-resolution as an inverse problem with hand-crafted regularization priors. While interpretable and theoretically grounded, they rely on fixed assumptions and require computationally intensive iterative solvers. Deep learning methods offer data-driven flexibility by learning nonlinear mappings from low- to high-resolution images, among which diffusion models have achieved particularly impressive perceptual quality. However, the standard diffusion training objective is a pixel-domain noise-prediction loss that does not explicitly enforce perceptual fidelity, which can lead to oversmoothing and loss of fine image structure. To address these limitations, we propose a perceptually regularized diffusion framework that incorporates prior knowledge through perceptual-loss-based regularization, improving training convergence and encouraging the recovery of meaningful image features. Experiments on benchmark datasets demonstrate improved perceptual quality and competitive distortion metrics, highlighting the effectiveness of regularization for diffusion-based super resolution.
Over the years, Convolutional Neural Networks (CNNs) have demonstrated strong capability in cancer detection and classification using medical images. However, CNN-based models often struggle to capture long-range contextual dependencies. In such scenarios, integrating Compact Convolutional Transformer (CCT) architectures after the CCT layer allows CNN-extracted features to reshape into compact patch tokens using a CCT tokenizer, followed by the addition of positional embeddings to preserve spatial structure. Using 5-fold cross-validation, the model was tested on 3 sets of breast cancer mammography. With only 250,435 parameters, the model achieved 99%-100% accuracy across 3 datasets, indicating robust generalization. Explainable AI (XAI) was integrated into the model to explain the breast cancer classification process to enhance clinical trust. The results indicate that the proposed framework is suitable for computer-aided diagnosis systems, particularly in resource-constrained clinical environments. The novelty of the proposed CNN-integrated CCT overcomes the limitation of CNN's gradient degradation in the last layers by integrating convolutional tokenization with transformer-based learning. Lighter than ViT, which is effective in capturing long-range dependencies, the model has also proven efficient in breast cancer classification by capturing long-range dependencies among breast tissue regions.
No Free Lunch theorem implies that any performance gains achieved by a classifier on a particular image distribution are necessarily offset by a loss of performance over the set of all possible problems; thus, no single model is universally optimal. Selecting the most suitable classifier for image datasets is a critical yet challenging task due to the intrinsic complexity and diversity of images. This paper proposes a meta-learning framework that leverages a comprehensive set of meta-features capturing dataset complexity to predict classifier performance without exhaustive training. By extracting and selecting features using methods such as autoencoders, pre-trained networks, and dimensionality reduction techniques, we train regression models to efficiently estimate classifier accuracies. Additionally, clustering techniques are employed to group classifiers with similar performance patterns, simplifying the recommendation process. The datasets used span a wide range of concepts, including nature, animals, numbers, motorcycles, medical images, and human bodies, to ensure broad generalization. Evaluated on 56 diverse image datasets, our approach achieves an average ranking prediction accuracy exceeding 86%, demonstrating its effectiveness in guiding model selection. This scalable and interpretable framework provides a practical solution to improve classification performance while reducing computational costs.
Early diagnosis of melanoma is critical for improving patient survival rates. However, accurately distinguishing melanoma from other skin lesions remains a significant clinical challenge due to the high visual similarity among lesion types and variability in image acquisition conditions. Artificial intelligence, particularly machine learning, has emerged as a promising tool to support dermatological diagnosis by automating feature extraction from medical images. Among the available approaches, convolutional neural networks (CNNs) have demonstrated strong performance in image classification tasks, making them well-suited for analyzing both dermatoscopic and histopathological images, given their ability to capture hierarchical visual patterns relevant to lesion characterization. Nevertheless, despite numerous pre-trained CNN architectures having been proposed, selecting the most appropriate one for a given imaging modality remains an open challenge. In this study, we evaluate pre-trained convolutional neural networks (CNNs) for skin lesion classification using dermatoscopic and histopathological image datasets. Experiments were conducted on the HAM10000, ISIC 2018, and CR-AI4SkIN datasets, evaluating the ResNet50, VGG16, VGG19, MobileNet, and InceptionV3 architectures under the same training protocol. The experimental evaluation showed that the models achieved accuracies ranging from 71% (InceptionV3 on ISIC 2018) to 84% (ResNet50 on HAM10000) on dermatoscopic images. For histopathological images, accuracies ranged from 72% (VGG19) to 83% (ResNet50) on the CR-AI4SkIN dataset. The results demonstrate that model performance differs between dermatoscopic and histopathological image modalities, showing that architectures exhibiting similar performance on dermatoscopic images exhibit different performance on histopathological data.
We address few-shot object detection with vision-language models (VLMs) in out-of-domain settings such as aerial, industrial, and medical imagery, using only ten annotated images for supervision. Existing adaptation methods are discrete prompt optimization and LoRA fine-tuning. We revisit a third option: soft prompting, where a small number of continuous prompt tokens are optimized while the pretrained backbone remains frozen. We identify two key design choices. First, placing prompt tokens at the cross-modal boundary between visual and text tokens outperforms other placements (10.0 vs. 8.4 mAP). Second, initializing prompts from the empty space token outperforms semantic and random initialization. With these choices, one to three learned tokens (7,168 parameters on average) match the best LoRA configuration on Roboflow20-VL (14.2 mAP, 10-shot) while training over 20,000x fewer parameters. Soft prompting remains harder to optimize, exhibiting higher variance across random seeds. Unlike LoRA, however, it causes no forgetting: the LoRA rank matching our accuracy reduces NaturalBench VQA accuracy by 35% relative, rising to 56% at the largest rank, whereas soft prompting leaves pretrained performance unchanged. The learned tokens behave like prompts rather than weights. They transfer to a newer model without retraining (+0.8 mAP on Qwen3.5-9B) and can be verbalized into readable prompts competitive with prompt-search methods (matching DetPO and outperforming GEPA). The approach also extends beyond detection. On RoboCasa manipulation tasks, the frozen $π_{0.5}$ vision-language-action policy benefits from soft prompting, matching the LoRA baseline on two of three tasks when tokens are placed at the gradient bottleneck. These results suggest modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.
The heightened prevalence of respiratory disorders, particularly exacerbated by a significant upswing in fatalities due to the novel coronavirus, underscores the critical need for early detection and timely intervention. This imperative is paramount, possessing the potential to profoundly impact and safeguard numerous lives. Medically, chest radiography stands out as an essential and economically viable medical imaging approach for diagnosing and assessing the severity of diverse Respiratory Disorders. However, their detection in Chest X-Rays is a cumbersome task even for well-trained radiologists owing to low contrast issues, overlapping of the tissue structures, subjective variability, and the presence of noise. To address these issues, a novel analytical model termed Exponential Pixelating Integral is introduced for the automatic detection of infections in Chest X-Rays in this work. Initially, the presented Exponential Pixelating Integral enhances the pixel intensities to overcome the low-contrast issues that are then polar-transformed followed by their representation using the locally invariant Mandelbrot and Julia fractal geometries for effective distinction of structural features. The collated features labeled Exponential Pixelating Integral with dually characterized fractal features are then classified by the non-parametric multivariate adaptive regression splines to establish an ensemble model between each pair of classes for effective diagnosis of diverse diseases. Rigorous analysis of the proposed classification framework on large medical benchmarked datasets showcases its superiority over its peers by registering a higher classification accuracy and F1 scores ranging from 98.46 to 99.45% and 96.53-98.10% respectively, making it a precise and interpretable automated system for diagnosing respiratory disorders.
Detecting brain metastases in magnetic resonance imaging (MRI) remains challenging because lesions vary widely in size and appearance, with very small metastases occupying only a minute fraction of a three-dimensional input. We investigate whether combining different spatial fields of view (FOVs) improves lesion detection in multimodal MRI and present a scale-aware 3D deep-learning framework. The method uses independently trained $96^3$ and $64^3$ 3D U-Nets whose whole-volume probability maps are combined by weighted late fusion. This design allows us to study the effect of spatial context separately from image resolution and modality choice. On a 97-patient development cohort, cross-FOV fusion improved lesion-level precision and F1 while substantially reducing false positives relative to the individual models. A same-FOV ensemble control showed that these gains were not explained solely by averaging independently trained networks, supporting a contribution from complementary spatial context. An exploratory cross-FOV agreement filter reduced false positives but did not improve overall F1. These results support cross-FOV probability fusion as a simple and computationally practical strategy for improving the precision-false-positive trade-off in 3D brain-metastasis detection.
Lung cancer is one of the leading causes of death worldwide, and its early diagnosis is crucial to improving patients prognosis and quality of life. However, the process of interpreting medical images for the detection of lung cancer is complex and requires trained experts. In this context, artificial intelligence (AI) and deep learning (DL) emerge as potential tools to automate and optimize image analysis. The objective of this work is to review the most recent and relevant applications of AI and DL in the field of radiology for the detection of lung cancer. To this end, an exhaustive search was carried out in scientific databases such as PubMed,IEEEXPLORE, Scopus and Web of Science, and 96 articles published from 2015 to the present addressing the use of AI and DL in biomedical engineering were selected. Emphasis is placed on the use of convolutional neural networks (CNN) with transfer learning and Data Augmentation as promising techniques to improve the accuracy and efficiency of the image interpretation process. The results show that the use of AI and DL can offer an effective alternative for the early diagnosis of lung cancer, with high sensitivity and specificity. However, current limitations and challenges that must be addressed to guarantee its responsible and safe application in clinical practice are also identified, such as the lack of standardized data, the ex plainability of the models, patient privacy, and the ethical and social implications. It is concluded that the use of AI and DL can have a positive impact on the care of patients with lung cancer, but further research and regulation are required to ensure its quality and reliability.
Lung cancer causes more deaths than any other malignancy, and low-dose CT screening is the main pathway to early diagnosis. That pathway hinges on the smallest lesions, yet nodules below six millimeters remain hard to detect, because most methods treat a nodule as a generic object and ignore the imaging physics behind its appearance. We show that this appearance is highly regular. Intensity peaks at the geometric center of a nodule and decays radially in a Gaussian pattern, and a fit to 18,218 annotated lesions from three public benchmarks yields a mean radial coefficient of determination above 0.86 in every dataset and size stratum. A square convolution samples both axes uniformly and is mismatched to this radial signal, most severely for small nodules. Guided by this evidence, we propose GRIPNet (Gaussian Radial Intensity Prior Network), a detector in which every module maps to a measurable property of the intensity distribution. Pinwheel convolutions decompose radial gradients, a dual-frequency module separates boundary detail from structural context, dilated masked attention matches the decay extent, and an adaptive loss reweights samples by conspicuity. GRIPNet raises mAP@0.5 to 95.3, 91.6 and 97.9 percent on KanserSet, LUNA16 and Lung-PET-CT-Dx while sharpening high-IoU localization at real-time speed.
The detection performance of liver metastases decreases with the reduction of radiation dose, but misses are heterogeneous. Previous eye tracking work has characterized missed metastases into two categories: search errors i.e., the eyes never land on the lesion, and decision errors i.e., the lesion is seen but not recognized as malignant. We integrated three prior reader studies to answer this question. In all studies, radiologists interpreted the same set of 40 contrast enhanced abdominal CT exams containing 91 liver metastases whose locations had been previously marked. In two studies, the workstation recorded their gaze and eye movements. Using eye dwell times, metastases were classified as search-error-dominant (majority of misses had <2 sec gaze time) or decision-error-dominant (>2 sec gaze time). In the third study, exams were interpreted both at 120 and 200 quality reference mAs (QRM) by ten radiologists. The third study did not include eye tracking. Out of 91 liver metastases, we excluded 16 that were never missed in the eye tracking studies and used 75 liver metastases for the present study.
Object detectors have shown remarkable performance in various fields, among these medical imaging, surveillance, and autonomous driving. However, they are prone to overconfidence when encountering unseen objects in real-world deployments, causing potential safety issues. To address this, detecting out-of-distribution (OOD) objects is essential for reliable object detection. Modern approaches leverage the broad semantic knowledge of foundation models such as CLIP for post-hoc few- and zero-shot OOD detection. However, these methods typically perform OOD assessment in feature space, which can be sensitive to object detector localization errors and variations in object appearance. Moreover, the current state-of-the-art (SOTA) zero-shot method performs computationally costly diffusion in inference. In this work, for our proposed zero-shot object OOD detection method SAM3-O2D2, we employ the SAM3-image foundation model in an efficient manner. Specifically, we prompt SAM3 only with the object detector's predicted classes and compare the predictions of the object detector and SAM3. An object is in-distribution (ID), if SAM3 also detects an object at the corresponding location. If SAM3 does not detect the prompted object, this indicates a mismatch between the detector's prediction and the image content, suggesting that the object is OOD. Experimental results show that our method significantly surpasses the so-far zero-shot SOTA method. Specifically, we achieve new SOTA AuROC and FPR95 metrics over both ID datasets Pascal-VOC and BDD100K and both OOD datasets MS-COCO and OpenImages.
Automated electrocardiogram analysis has advanced largely through digital waveforms, yet many emergency-care workflows rely on ECGs available only as printed tracings, scanned reports, PDFs or mobile photographs. We developed an image-based deep learning system for emergency ECG classification and evaluated it in InCor-EMG, an expert-adjudicated dataset of 18,519 emergency ECGs spanning 12 ECG categories, with labels from 19 cardiologists. On the held-out test set, the final ConvNeXt ensemble achieved a macro F1-score of 0.807 (95% CI, 0.788-0.825), compared with 0.820 (95% CI, 0.805-0.832) for annotating cardiologists, and higher F1-scores than Mortara Veritas in most evaluated categories. Performance was associated more strongly with inter-reader agreement than with training sample size and remained informative across scanned and photographed ECGs, with supportive performance in model-enriched temporal and heterogeneous public-image evaluations. These findings support ECG image classification when digital waveforms are unavailable.
As artificial intelligence is increasingly integrated into chest X-ray (CXR) interpretation, triage, and clinical decision support, understanding its vulnerability to adversarial manipulation is critical for safe deployment. Existing robustness evaluations, however, predominantly rely on pixel-space attacks that introduce numerically constrained perturbations but may not represent plausible radiographic variation. This limitation is particularly important in multi-disease CXR classification, where models simultaneously evaluate multiple overlapping pathologies and adversarial failures may alter several diagnostic predictions. We propose a text-guided diffusion-based adversarial framework that optimizes learnable text conditioning while keeping the diffusion generator and target classifier frozen, enabling adversarial generation through a learned image prior rather than direct pixel manipulation. We evaluate the framework across multiple classifier architectures in both binary atelectasis and multi-disease CXR classification and compare it with FGSM, PGD, and Carlini-Wagner attacks. Our approach consistently produced the greatest degradation in classifier performance, reducing AUROC to 0.3885-0.5646 in binary classification and 0.4441-0.4878 in the multi-disease setting, while achieving superior image fidelity (SSIM 0.9080, LPIPS 0.1670, FID 51.23). Importantly, clinician interpretation remained unchanged for 95.9% of binary and 73.8% of multi-disease adversarial images despite substantial changes in model predictions. These findings reveal a clinically important discrepancy between human and machine interpretation and demonstrate the need to extend medical AI robustness evaluation beyond conventional pixel-space attacks toward generative threat models that can expose failures under visually and clinically plausible image variations.
With the rising prevalence of age-related memory impairments, efficiently detecting and monitoring decline is increasingly urgent. Unfortunately, traditional assessment methods fall short of these needs, as they typically require in-person administration and cannot be repeated frequently. Here, we demonstrate that remote identification and monitoring of abnormal memory function is possible by combining an online assessment platform with computational phenotyping, allowing repeatable, unsupervised remote observations from patients. Fifty-one well-characterized older individuals, including 24 patients with amnestic mild cognitive impairment and 27 age- and education-matched healthy controls, completed a series of longitudinal, unsupervised, remote weekly 8-minute online memory assessments for up to one year. Weekly test data were fit to a formal model of memory consolidation and forgetting, yielding an individualized index of memory function, the Seattle-Groningen Memory Assessment (SGMA) score. The SGMA score was found to be reliable, with a mean correlation of r = 0.70 across assessments. The score was also found to be stable across different study materials, and only barely affected by practice effects, which averaged to a 0.2% increase per assessment. Finally, the SGMA score was found to be diagnostic, being capable of detecting mild cognitive impairment with up to 87% accuracy. These findings show that model-based, adaptive assessments can support high-frequency, remote detection and scalable longitudinal monitoring of early memory decline, providing a new way to assess memory decline trajectories in healthy aging and dementia.
🏫 主要单位：Cancer Data Science Laboratory, Center for Cancer Research, National Cancer Institute, National Institutes of Health, Bethesda, MD, USA ｜ 主要作者：Eytan Ruppin、Mukherjee, S.
Accurately predicting which patients will respond to immune checkpoint blockade (ICB) remains a major challenge. Here, we present TIME_ACT, an unsupervised 66-gene transcriptomic signature of tumor immune activation derived from TCGA (The Cancer Genome Atlas) melanoma data. First, we demonstrate that TIME_ACT scores accurately identify tumors with activated immune microenvironments across different cancer types. Further, analysis of spatial features reveals that tumor microenvironment regions with dense lymphocyte infiltration near tumor cells have high TIME_ACT scores, successfully marking localized immune activation. Second, across 25 transcriptomic ICB cohorts encompassing nine cancer types, TIME_ACT achieves a mean AUC of 0.76 and a mean odds ratio of 5.77, significantly outperforming 30 established transcriptomic signatures and prediction methods for ICB response, including a recently developed foundation model for immunotherapy response prediction. Third, we show that TIME_ACT scores can be accurately inferred from routine tumor histopathology slides and that slide-inferred TIME_ACT scores predict ICB response across nine new independent patient cohorts spanning eight cancer types, achieving a mean AUC of 0.72 and a mean odds ratio of 4.99. These findings establish TIME_ACT as a robust, pan-cancer biomarker that enables accurate, low-cost, and clinically scalable prediction of ICB response from routine histopathology.
Multiple myeloma is bone-marrow-predominant, motivating marrow plasma cell-free DNA (cfDNA) as a tumor-proximal liquid biopsy. We tested whether marrow plasma cfDNA better represents the myeloma genome than peripheral blood cfDNA by profiling matched marrow plasma cells and both cfDNA compartments from 74 patients across 372 cfDNA sample time points. Tumor fractions correlated between compartments (r=0.84), but marrow plasma cfDNA was not a superior surrogate: peripheral-blood cfDNA matched or exceeded marrow-plasma cfDNA for marrow-defined CNA and mutation recovery (77.4% vs 72.0%; 69.2% vs 65.4%), with high specificity. Disease-associated features were detected in 69/74 peripheral-blood and 68/74 marrow-plasma profiles. BCR clonotypes were the highest-recovery feature, with BM-dominant clonotypes recovered in 31/34 callable profiles per compartment. At high tumor fraction, adverse-lesion sensitivity reached 100% in both cfDNA compartments. Marrow plasma cfDNA showed distinct, handling-sensitive fragmentomic architecture (LOOCV AUC=0.910), demonstrating performance depends on compartment, biomarker class, tumor fraction and processing context.
Objective. To evaluate task-dependent selection of local and cloud models for structured extraction from multilingual non-English stroke discharge summaries, distinguishing entity detection, record assembly, and raw-document processing. Methods. This retrospective system evaluation compared multilingual encoders, locally fine-tuned Qwen3-4B models, and zero-shot GPT-5.5. Primary test cohorts comprised 332 section cases, 149 medication cases with 2,475 reference records after identity-based exclusions, and 191 laboratory cases. Outcomes were section-span F1, drug-name F1, normalized seven-field record recovery, test-name F1, and laboratory quintuple F1. Paired human comparisons used a common second-annotator reference on 50 cases per task. Saved cascade outputs were compared with curated-section controls using document-bootstrap intervals. Results. Section F1 was 0.919 for the encoder, 0.926 for GPT-5.5, and 0.932 for Qwen. GPT-5.5 led medication detection (0.966 versus 0.940 for Qwen), whereas Qwen led normalized medication recovery (0.381 versus 0.311; difference 0.070, 95% CI 0.026-0.114) and laboratory quintuple F1 (0.892 versus 0.822). With matched inference stacks, medication recovery declined from 0.377 on curated inputs to 0.204 for single-window and 0.246 for all-block cascades; quintuple F1 on the original laboratory cohort declined from 0.898 to 0.811. Corpus-wide extraction processed 193,101 summaries in 13.3 H100 GPU-hours. Conclusion. Local models achieved strong detection performance without frontier-scale inference in this setting. Fine-tuned local LLMs were advantageous for record assembly under the evaluated recipes; annotation conventions, normalization, and upstream section extraction remained important constraints.
Background: Glial fibrillary acidic protein (GFAP) is a promising blood biomarker that can distinguish intracerebral hemorrhage (ICH) from acute ischemic stroke (AIS). We aimed to summarize the existing evidence regarding: 1) the diagnostic accuracy of GFAP to differentiate ICH from AIS, and from undifferentiated suspected stroke, and 2) characterize GFAP's diagnostic accuracy across time strata from stroke onset. As a secondary objective, associations between GFAP and clinical and radiological outcomes were narratively summarized. Methods: Five databases [(MEDLINE/EMBASE/EBM Reviews/CINAHL Complete/Google Scholar] were searched (03/2026) for studies reporting blood GFAP levels in acute stroke. GFAP levels were analyzed according to time from onset to sampling. Risk of bias was assessed using QUADAS-2 for studies reporting diagnostic accuracy data. Forest plots, pooled sensitivities and specificities were calculated. Results: Two reviewers independently screened 6,336 studies; 64 were included for systematic review and 18 for meta-analysis. Risk of bias was low overall among diagnostic studies. For differentiating ICH from AIS, pooled AUC was 0.86 (95% CI: 0.76-0.92), pooled sensitivity 74.4% (95% CI:63.2-84.2%) and pooled specificity 87.2% (95% CI:77.0-94.7%). For ICH versus undifferentiated suspected stroke, pooled AUC was 0.88 (95% CI: 0.79-0.95), pooled sensitivity 75.4% (95% CI: 63.6-83.3%) and pooled specificity 90.5% (95% CI:88.0-96.2%). Diagnostic precision improved in earlier time strata: <6h: AUC 0.91, sensitivity 75.0%, specificity 92.7%; <2h: AUC 0.96, sensitivity 76.7%, specificity 96.4%. GFAP correlated with ICH volume in 11/12 studies, although smaller hematomas (<10 mL) were frequently associated with GFAP values below limits of detection. Few studies (n=3) suggest higher GFAP levels correlate with greater ischemic injury in AIS. Conclusions: As a blood biomarker in acute stroke, GFAP has strong rule-in value (i.e. high specificity) for ICH, which is preserved in earlier time windows. Nevertheless, sensitivity remains limited, restricting its use as a rule-out test, particularly for smaller ICH volumes. Knowledge gaps remain regarding GFAP's potential in assessing ischemic injury extent in AIS.
The detection of small and subtle lesions in high-resolution 3D volumes is a highly relevant, yet far from solved task in biomedical imaging. We here address a specific task in detecting certain types of epileptogenic lesions through our novel semi-supervised spatially context aware transformer (SpyCAT) approach to anomaly detection. SpyCAT is modeling-based in the sense that it builds on specific assumptions that constitute what is normal and what constitues relevant deviations from normality. We explicitly use these assumptions to justify the inductive bias of our anomaly detection approach. The resulting SpyCAT system is patch-based and uses a transformer architecture to process discrete tokens obtained from a vector quantizing variational autoencoder, which produces counterfactual patches through full 3D convolutions of each patch. We evaluate our approach on the grounds of point-annotations of two subtypes of epileptogenic lesions, using validation measures that build on the Metrics Reloaded framework, showing that SpyCAT can reliably identify and localize the lesion types under consideration, and outperforms state-of-the-art reference methods.
Large language models (LLMs) show promise in medical image interpretation but suffer from hallucination, limited accuracy, and run-to-run inconsistency. We developed and validated an agentic AI framework integrating LLMs with specialized deep learning tools for glaucoma detection from fundus photography. The workflow had three steps: (1) LLM initial assessment; (2) function calling to invoke specialized tools for image quality (QAModel, FundaQ-8), glaucoma classification (SwinV2-Tiny), and optic disc/cup segmentation (SegFormer-B0); and (3) LLM reflection integrating the initial impression with tool outputs. Two LLMs (Gemini 2.5 Flash, GPT-5.4 mini) were evaluated on two public datasets (ORIGA, n=100; RIM-ONE-v3, n=100) under uncropped and cropped fields of view; all images were independently graded by a masked fellowship-trained glaucoma specialist. The agentic workflow improved classification accuracy by 16 to 47 percentage points across all conditions, reaching within 6 points of the specialist; on RIM-ONE-v3 the best configurations matched the specialist accuracy of 88%. LLM-alone approaches failed in two ways: GPT-5.4 mini showed positive bias (sensitivity 95-100%, specificity 0-5%), while Gemini 2.5 Flash varied stochastically between runs; the agentic workflow corrected both. Cup-to-disc ratio error fell 15-50% (MAE 0.156-0.228 to 0.104-0.132), and correlation with specialist grading rose from weak (r=0.12-0.39) to moderate-strong (r=0.59-0.84). Run-to-run consistency rose from near-random (kappa as low as -0.01) to near-perfect (kappa up to 0.96). Integrating LLMs with specialized tools addressed key limitations of LLM-alone approaches, including over-diagnosis and run-to-run variability. Gains held for both LLMs, suggesting generalizability across backbones, and may signal a shift from monolithic models toward orchestrated multi-agent systems in medical AI.
Developing competitive deep learning baselines for medical imaging remains a highly iterative process requiring literature review, implementation, experimentation, and expert refinement. Existing automation approaches typically optimize isolated components, such as architecture search or hyperparameter tuning, rather than the complete baseline development process. We present an agentic AI Scientist workflow that combines literature-guided reasoning, automated code generation, and hypothesis-driven experimentation to generate competitive baseline models for medical imaging challenges. The framework is evaluated on four public benchmarks spanning segmentation, classification, and detection. Across all tasks, the Experimentation Pipeline consistently improves validation performance, achieving competitive leaderboard results, including 6th place on both PUMA tracks (15 teams) and 31st place on MILK10k (125 teams). On MIDOG25, the resulting model also demonstrates strong domain generalization across scanners, tumor types, and species. Using the same workflow across all challenges without task-specific redesign, we demonstrate that skill-based, literature-guided agentic workflows can substantially reduce the engineering effort required to develop competitive medical imaging baselines.
Intraoperative 2D/3D registration, which aligns live X-ray images with preoperative volumes, is essential for image-guided interventions. Previous regression-based methods suffer from limited generalization, thus requiring time-consuming patient-specific retraining. Inspired by recent geometry foundation models such as DUSt3R, we propose XPos3R, a generalizable pose regression method that eliminates preoperative preparation. Unlike existing geometry models designed for homogeneous inputs, XPos3R extends this paradigm to multi-modal inputs, namely 2D X-rays and 3D volumes. Specifically, we introduce an asymmetric encoder-decoder architecture that improves cross-modal feature alignment while maintaining computational efficiency. To scale training under limited medical data, we adopt an anatomy-specific data curation strategy and construct million-scale synthetic datasets. Evaluated on real-world benchmarks, a single pretrained XPos3R surpasses patient-specific methods in both accuracy and robustness. With test-time optimization completed in seconds, it further reduces the 3D error to <4 mm and the reprojection error to <1 mm. The strong generalization, accuracy, and efficiency of XPos3R highlight its clinical potential, while its asymmetric framework may inspire broader cross-modal vision geometry tasks.
Diffeomorphic image registration is central to medical image analysis, enabling anatomically consistent alignment across subjects. Most learning-based diffeomorphic methods model autonomous ODEs(ordinary differential equations) by parameterizing a stationary velocity field and recovering deformations via scaling-and-squaring. While non-autonomous ODEs with time-dependent velocities increase expressiveness, existing approaches rely on numerical integration to implicitly enforce flow structure that entangles model expressiveness with discretization accuracy. We propose a framework to directly learn the continuous-time solution of a non-autonomous ODE formulated as a two-parameterflow map. By enforcing cocycle consistency, a fundamental structural property of time-varying flows, we learn the flow maps without time discretization and velocity integration during training. The framework recovers diffeomorphic mappings at inference using a small number of compositions. Our proposed framework seamlessly incorporates standard registration backbones and improves alignment accuracy consistently across nine datasets while preserving diffeomorphic structure. Notably, the proposed method achieves an average Dice improvement of 2.1% on brain MRI benchmarks, a 12% TRE reduction on lung CT, and a 2.6% Dice gain on cardiac MRI and ultrasound datasets.
Multi-phase contrast-enhanced computed tomography (CT) is the gold standard for renal cell carcinoma (RCC) characterization, yet clinical interpretation relies on subjective visual comparison across phases. We present PRISM (Phase-Resolved Isotropic Subtraction Mapping), an open-source automated pipeline that transforms multi-phase CT acquisitions into registered digital subtraction angiography (DSA) volumes with color-coded enhancement maps. PRISM integrates six sequential processing stages: (1) DICOM loading with automated contrast-phase classification, (2) deep learning-based isotropic interpolation via RIFE, (3) automated kidney segmentation using TotalSegmentator v2, (4) enhancement-based tissue detection, (5) three-step deformable registration (rigid, affine, B-spline) using SimpleITK, and (6) dual-channel digital subtraction visualization. We present a systematic parameter optimization study comprising 200 registrations across five patients and five experiments. Key findings: We identify an efficient registration configuration combining a 40 mm B-spline grid (within 6% of the 30 mm quality optimum at 36% lower computational cost), 5% metric sampling (equivalent quality to 25% at 3.2x speedup), and a single-level multi-resolution pyramid (avoiding the 5.4x overhead of a 4,2 pyramid with no quality benefit); we show that registration quality is effectively independent of interpolation target spacing from 0.5-3.0 mm, enabling a coarse-register/fine-apply strategy that computes the full transform at 3.0 mm (approximately 4 minutes per phase) and applies it to 0.5 mm volumes for high-resolution visualization. We also determine that a 40 HU subtraction noise threshold optimally balances signal-to-noise ratio (2.00) against sensitivity (14.2% enhancing volume retained), with higher thresholds (60-80 HU) favoring specificity and lower thresholds (20 HU) favoring sensitivity.
Purpose: To quantify the dosimetric effect of residual intrafraction motion during gated MR-guided prostate stereotactic body radiotherapy with simultaneous integrated boost to the dominant intraprostatic lesion, and to derive and evaluate target-specific, population-based margins designed to mitigate motion-induced coverage loss. Materials and Methods: Thirty patients were treated on a 1.5-T MR-Linac with 36.25, 40, and 45 Gy prescribed to the planning target volume, clinical target volume (CTV), and gross tumor volume (GTV), in 5 fractions. Across 150 fractions, segment-level dose matrices synchronized with time-resolved target displacements were combined to reconstruct per-fraction and cumulative dose. Directional prescription-isodose deviations were used to derive internal target volume margins, which were evaluated through offline replanning, gating-envelope analysis, and dose reconstruction. Results: Median 95th-percentile motion ranges were 0.5, 1.4, and 2.9 mm in the left-right, anterior-posterior, and superior-inferior directions; these exceeded 3 mm in 0.7%, 13.2%, and 47.7% of fractions, respectively. The median treatment time was 13.2 min (IQR, 12.0-15.1 min) with a median gating duty cycle of 93.2% (80.6%-97.8%). Per-fraction median GTV deviations were -1.2% (-2.4% to -0.1%) for D95 and -5.6% (-15.3% to -0.2%) for V45Gy; corresponding CTV deviations were -0.5% (-1.1% to -0.1%) and -2.1% (-4.1% to -0.4%). Cohort-derived margins of 2/2/1/2/1/2 mm for the GTV and 1/1/1/2/1/2 mm for the CTV in the left/right/anterior/posterior/superior/inferior directions achieved sufficient coverage in approximately 94% of fractions in each direction. Gated delivery using the proposed margins improved coverage but may reduce duty cycle by an average of 6.9 +/- 4.7 %. Conclusion: Residual intrafraction motion produced directionally asymmetric coverage loss, with the largest effect on the boosted GTV. Motion-inclusive dose reconstruction enabled derivation of practical asymmetric margins that improved robustness while identifying the tradeoff between target coverage and delivery efficiency.