ale算法

We investigate how Accumulated Local Effects (ALE), a model-agnostic explanation method, can be adapted to visualize the influence of node feature values in link prediction tasks using Graph Neural Networks (GNNs), specifically Graph Convolutional Networks and Graph Attention Networks. A key challenge addressed in this work is the complex interactions among nodes during message passing within GNN layers, which complicate the direct application of ALE. Since a straightforward solution of modifying only one node at a time substantially increases computation time, we propose an approximate method that mitigates this issue. Our findings reveal that although the approximate method offers computational efficiency, the exact method yields more stable explanations, particularly when smaller data subsets are used. However, the explanations produced by the approximate method are not significantly different from those obtained by the exact method. Additionally, we analyze how varying parameters affect the accuracy of ALE estimation for both approaches.

In many machine learning contexts, tasks are often treated as interconnected components with the goal of leveraging knowledge transfer between them, which is the central aim of Multi-Task Learning (MTL). Consequently, this multi-task scenario requires addressing critical questions: which tasks are similar, and how and why do they exhibit similarity? In this work, we propose a multi-task similarity measure based on Explainable Artificial Intelligence (XAI) techniques, specifically Accumulated Local Effects (ALE) curves. ALE curves are compared using the Fr\'echet distance, weighted by the data distribution, and the resulting similarity measure incorporates the importance of each feature. The measure is applicable in both single-task learning scenarios, where each task is trained separately, and multi-task learning scenarios, where all tasks are learned simultaneously. The measure is model-agnostic, allowing the use of different machine learning models across tasks. A scaling factor is introduced to account for differences in predictive performance across tasks, and several recommendations are provided for applying the measure in complex scenarios. We validate this measure using four datasets, one synthetic dataset and three real-world datasets. The real-world datasets include a well-known Parkinson's dataset and a bike-sharing usage dataset -- both structured in tabular format -- as well as the CelebA dataset, which is used to evaluate the application of concept bottleneck encoders in a multitask learning setting. The results demonstrate that the measure aligns with intuitive expectations of task similarity across both tabular and non-tabular data, making it a valuable tool for exploring relationships between tasks and supporting informed decision-making.

Accumulated Local Effects (ALE) is a widely-used explainability method for isolating the average effect of a feature on the output, because it handles cases with correlated features well. However, it has two limitations. First, it does not quantify the deviation of instance-level (local) effects from the average (global) effect, known as heterogeneity. Second, for estimating the average effect, it partitions the feature domain into user-defined, fixed-sized bins, where different bin sizes may lead to inconsistent ALE estimations. To address these limitations, we propose Robust and Heterogeneity-aware ALE (RHALE). RHALE quantifies the heterogeneity by considering the standard deviation of the local effects and automatically determines an optimal variable-size bin-splitting. In this paper, we prove that to achieve an unbiased approximation of the standard deviation of local effects within each bin, bin splitting must follow a set of sufficient conditions. Based on these conditions, we propose an algorithm that automatically determines the optimal partitioning, balancing the estimation bias and variance. Through evaluations on synthetic and real datasets, we demonstrate the superiority of RHALE compared to other methods, including the advantages of automatic bin splitting, especially in cases with correlated features.

Accumulated Local Effects (ALE) is a model-agnostic approach for global explanations of the results of black-box machine learning (ML) algorithms. There are at least three challenges with conducting statistical inference based on ALE: ensuring the reliability of ALE analyses, especially in the context of small datasets; intuitively characterizing a variable's overall effect in ML; and making robust inferences from ML data analysis. In response, we introduce innovative tools and techniques for statistical inference using ALE, establishing bootstrapped confidence intervals tailored to dataset size and introducing ALE effect size measures that intuitively indicate effects on both the outcome variable scale and a normalized scale. Furthermore, we demonstrate how to use these tools to draw reliable statistical inferences, reflecting the flexible patterns ALE adeptly highlights, with implementations available in the 'ale' package in R. This work propels the discourse on ALE and its applicability in ML and statistical analysis forward, offering practical solutions to prevailing challenges in the field.

Accumulated Local Effect (ALE) is a method for accurately estimating feature effects, overcoming fundamental failure modes of previously-existed methods, such as Partial Dependence Plots. However, ALE's approximation, i.e. the method for estimating ALE from the limited samples of the training set, faces two weaknesses. First, it does not scale well in cases where the input has high dimensionality, and, second, it is vulnerable to out-of-distribution (OOD) sampling when the training set is relatively small. In this paper, we propose a novel ALE approximation, called Differential Accumulated Local Effects (DALE), which can be used in cases where the ML model is differentiable and an auto-differentiable framework is accessible. Our proposal has significant computational advantages, making feature effect estimation applicable to high-dimensional Machine Learning scenarios with near-zero computational overhead. Furthermore, DALE does not create artificial points for calculating the feature effect, resolving misleading estimations due to OOD sampling. Finally, we formally prove that, under some hypotheses, DALE is an unbiased estimator of ALE and we present a method for quantifying the standard error of the explanation. Experiments using both synthetic and real datasets demonstrate the value of the proposed approach.

No abstract available

Although the geometric manufacturing deviations of compressor blades are typically small in magnitude (generally on the order of millimeters), they can significantly disturb the flow field structure, leading to notable performance degradation. To investigate the uncertainty in compressor stage performance caused by deviations in leading-edge radius, this study develops a high-precision gated recurrent unit-Kolmogorov–Arnold network (GRU-KAN) neural network surrogate model. The model output is interpreted using SHapley additive exPlanations and accumulated local effects methods to provide multidimensional insights. Results indicate that the combined presence of leading-edge radius deviations in both rotor and stator blades has the most pronounced impact on compressor performance, particularly under peak-efficiency conditions where the sensitivity is heightened. Moreover, deviations at the 50% span of the rotor blade exhibit the greatest influence on both stage performance and flow structure, with effects propagating across adjacent spans. In contrast, leading-edge radius deviations at 95% span primarily induce localized disturbances in the flow field. Under near-stall conditions, the impact of stator leading-edge radius deviations across various spans becomes more prominent, especially in terms of their contribution to total pressure ratio prediction. Overall, leading-edge radius deviations impose systematic and cumulative negative effects on compressor performance. As the deviation amplitude increases, the performance deterioration becomes increasingly severe.

Abstract Previous research on hit-and-run crashes employed regression methods or machine learning techniques. However, regression methods necessitate preestablished model formulations, making it challenging to accommodate intricate nonlinear effects. In contrast, machine learning methods are characterized as black box systems, lacking interpretability. Thus, we propose an innovative analytical framework that combines data-driven machine learning algorithms with emerging interpretation techniques. The complex nonlinear effects of various factors on hit-and-run crashes are investigated by employing post hoc interpretation techniques, specifically, Shapley Additive exPlanations and accumulated local effect. The results demonstrate that machine learning algorithms are superior in accounting for complex relationships among influencing factors and identifying hit-and-run crashes. The quantitative importance of various factors is estimated and compared to reveal key determinants such as visibility, road location, and accident liability. The complex effects of different factors on hit-and-run crashes are unveiled, delineating quantitative piecewise nonlinear patterns. These patterns, which are difficult to capture using conventional regression models with predefined formulations, shed light on the nuanced dynamics of hit-and-run crashes. This research offers quantitative analysis and data-supported insights for transportation agencies and police departments to proactively mitigate hit-and-run crashes.

Deep artificial neural networks show high predictive performance in many fields, but they do not afford statistical inferences and their black-box operations are too complicated for humans to comprehend. Because positing that a relationship exists is often more important than prediction in scientific experiments and research models, machine learning is far less frequently used than inferential statistics. Additionally, statistics calls for improving the test of theory by showing the magnitude of the phenomena being studied. This article extends current XAI methods and develops a model agnostic hypothesis testing framework for machine learning. First, Fisher's variable permutation algorithm is tweaked to compute an effect size measure equivalent to Cohen's f2 for OLS regression models. Second, the Mann-Kendall test of monotonicity and the Theil-Sen estimator is applied to Apley's accumulated local effect plots to specify a variable's direction of influence and statistical significance. The usefulness of this approach is demonstrated on an artificial data set and a social survey with a Python sandbox implementation.

Breast cancer remains a global health burden, with an increase in deaths related to this particular cancer. Accurately predicting and diagnosing breast cancer is important for treatment development and survival of patients. This study aimed to accurately predict breast cancer using a dataset comprising 1208 observations and 3602 genes. The study employed feature selection techniques to identify the most influential predictive genes for breast cancer using machine learning (ML) models. The study used K-nearest Neighbors (KNN), random forests (RF), and a support vector machine (SVM). Furthermore, the study employed feature- and model-based importance and explainable ML methods, including Shapley values, Partial dependency (PDPS), and Accumulated Local Effects (ALE) plots, to explain the genes’ importance ranking from the ML methods. Shapley values highlighted the significance of some of the genes in predicting cancer presence. Model-based feature ranking techniques, particularly the Leaving-One-Covariate-In (LOCI) method, identified the ten most critical genes for predicting tumor cases. The LOCI rankings from the SVM and RF methods were aligned. Additionally, visualization methods such as PDPS and ALE plots demonstrated how individual feature changes affect predictions and interactions with other genes. By combining feature selection techniques and explainable ML methods, this study has demonstrated the interpretability and reliability of machine learning models for breast cancer prediction, emphasizing the importance of incorporating explainable ML approaches for medical decision-making.

Explainable AI is an emerging field of research since the spread of AI in multifarious fields. The opacity and inherent black-box nature of the advanced machine learning models create a lack of transparency in them leading to the insufficiency in societal recognition. The increasing dependence on AI across diverse sectors has created the need for informed decision-making of the numerous predictive models used. XAI strives to close this divide by providing an explanation of the decision-making process, promoting trust, ensuring adherence to regulations, and cultivating societal approval. Various post-hoc techniques including well-known methods like LIME, SHAP, Integrated Gradients, Partial Dependence Plot, and Accumulated Local Effects have been proposed to decipher the intricacies of complex AI models. In the context of post hoc explanatory methods for machine learning models there arises a conflict known as the Disagreement problem where different explanation techniques provide differing interpretations of the same model. In this study, we aim to find whether reducing the bias in the dataset could lead to XAI explanations that do not disagree. The study thoroughly analyzes this problem, examining various widely recognized explanation methods. Our method aims to understand the effect of bias versus the effect after removing it. It also aims to understand the effect of bias on the explainability of the model predictions in light of the work of other authors on bias removal and fairness.

Transport is a major contributor to anthropogenic greenhouse gases, making accurate assessment of vehicle emissions essential for climate change mitigation. This study develops a comparative machine learning framework to predict CO2 emissions from internal combustion engines (ICEs) and hybrid electric vehicles (HEVs), using data from the UK Vehicle Certification Agency. In addition to standard technical variables, the study considers noise level, a factor seldom integrated into emission modeling, reflecting potential interactions between acoustic conditions and vehicular emission patterns. Explainable machine learning techniques, including accumulated local effects, are employed to clarify how engine capacity, fuel consumption and pollutant indicators influence CO2 outputs under different driving conditions. Results show that medium- and high-speed driving dominate ICE emissions, whereas HEVs maintain lower emissions except under high power demand. By combining predictive modeling with interpretability, the study advances environmental informatics and provides actionable insights for low-carbon vehicle design, emission standards and sustainable transportation policies aligned with global climate goals.

No abstract available

Accurate prediction of reservoir temperature is critical for optimizing geothermal energy systems, yet the complexity of geothermal data poses significant challenges for traditional modeling approaches. This study conducts a comprehensive comparative analysis of advanced machine learning models, including support vector regression (SVR), random forest (RF), Gaussian process regression (GP), deep neural networks (DNN), and graph neural networks (GNN), to evaluate their predictive performance for reservoir temperature estimation. Enhanced feature engineering techniques, including accumulated local effects (ALE) and SHAP value analysis, are employed to improve model interpretability and identify key hydrogeochemical predictors. Results demonstrate that RF outperforms other models, achieving the lowest mean squared error (MSE = 66.16) and highest R2 score (0.977), which is attributed to its ensemble learning approach and robust handling of nonlinear relationships. SVR and GP exhibit moderate performance, while DNN and GNN show limitations due to overfitting and sensitivity to hyperparameter tuning. Feature importance analysis reveals that SiO2 concentration as the most influential predictor, aligning with domain knowledge. The study highlights the interplay between model complexity, dataset size, and predictive accuracy, offering actionable insights for optimizing geothermal energy systems. By integrating advanced machine learning with enhanced feature engineering, this research provides a robust framework for improving reservoir temperature prediction, contributing to the sustainable development of geothermal energy in alignment with sustainable energy development.

Background: Patients with both diabetes mellitus (DM) and atrial fibrillation (AF) face elevated mortality in intensive care units (ICUs), yet models targeting this high-risk group remain limited. Objective: To develop an interpretable machine learning (ML) model predicting 28-day mortality in ICU patients with concurrent DM and AF using early-phase clinical data. Methods: A retrospective cohort of 1,535 adult ICU patients with DM and AF was extracted from the MIMIC-IV database. Data preprocessing involved median/mode imputation, z-score normalization, and early temporal feature engineering. A two-step feature selection pipeline-univariate filtering (ANOVA F-test) and Random Forest-based multivariate ranking-yielded 19 interpretable features. Seven ML models were trained with stratified 5-fold cross-validation and SMOTE oversampling. Interpretability was assessed via ablation and Accumulated Local Effects (ALE) analysis. Results: Logistic regression achieved the best performance (AUROC: 0.825; 95% CI: 0.779-0.867), surpassing more complex models. Key predictors included RAS, age, bilirubin, and extubation. ALE plots showed intuitive, non-linear effects such as age-related risk acceleration and bilirubin thresholds. Conclusion: This interpretable ML model offers accurate risk prediction and clinical insights for early ICU triage in patients with DM and AF.

Objective To develop an accurate and interpretable height estimation model for children and adolescents using body composition variables and explainable artificial intelligence approaches. Methods A light gradient boosting method was employed on a dataset of 278,301 measurements from 54,374 children and adolescents aged 6–18 years. The model incorporated anthropometric and body composition measures. Model interpretability was enhanced through feature importance analysis, Shapley additive explanations, partial dependence plots, and accumulated local effects. Results The models achieved high accuracy with mean absolute percentage errors of 1.64% and 1.63% for boys and girls, respectively. Soft lean mass (SLM), body fat mass percentage (BFMP), skeletal muscle mass, and skeletal muscle mass percentage were consistently identified as key factors influencing height estimation. Analysis revealed a positive correlation between SLM and estimated height, while BFMP exhibited an inverse relationship with height projections. Conclusion These findings provide valuable insights into the relationship between body composition and height, underlining the potential of body composition variables as accurate height predictors in children and adolescents. The model's interpretability and accuracy make it a promising tool for pediatric growth assessment and monitoring.

The paper introduces the Explainable AI methodology to assess the global feature importance of the Machine Learning models used for heat demand forecasting in intelligent control of District Heating Systems (DHS), with motivation to facilitate their interpretability and trustworthiness, hence addressin g the challenges related to adherence to communal standards, customer satisfaction and liability risks. Methodology involves generation of global feature importance insights by using four different approaches, namely intrinsic (ante-hoc) interpretability of Gradient Boosting method and selected post-hoc methods, namely Partial Dependence, Accumulated Local Effects (ALE) and SHAP and qualitative analysis of those insights in context of expected behavior of DHS and comparative analysis. None of the selected methods assume feature permutation or perturbations which can introduce bias due to introduction of random unrealistic values of data instances. ALE and SHAP have been found as most reliable methods for determining the feature importance, taking into account feature interactions and nonlinearities. ALE plots with transmitted energy across the range of ambient temperatures closely resemble the shape of the control curve, which is the evidence of accurate model, as well as suitability of explanation method. By providing the insights which align with the domain expertise, the discussion confirms the value of using Explainable AI stack as mandatory layer in assessing the performance of ML models, especially in high-risk AI systems, such as those whose use is anticipated in the DHS.

Due to the outstanding thermal stability, inherent high melting points, and elevated temperature strengths, refractory high-entropy alloys (RHEAs) have been widely used for extreme environments in aerospace, nuclear energy, and advanced propulsion systems. Herein, we present an integrated design and simulation framework for RHEAs, combining machine learning potentials, supervised regression models, and multiobjective optimization algorithms. Utilizing a universal neuroevolution potential version 1 (UNEP-v1), the framework significantly enhances the accuracy of atomic-scale simulation while substantially reducing computational cost. High-throughput molecular dynamics simulations generate melting points and ultimate tensile strengths at 1000 K for various alloy compositions. Supervised regression models enable a rapid performance prediction. Integrating Shapley Additive exPlanations, Partial Dependence Plots, Accumulated Local Effects, and Individual Conditional Expectation analysis can provide a comprehensive interpretability toolkit. Validation of the proposed method in the TiVCrZrMo alloy system demonstrates its efficacy in designing high-strength, high-temperature resistant alloys. We not only develop a precise and interpretable predictive modeling paradigm but also establish procedural frameworks, promoting the integration of atomic-scale simulations with data-driven approaches for RHEAs in extreme environments.

Compound drought and heatwave events (CDHWs) are increasing in frequency across dryland ecosystems, yet their impacts on vegetation remain insufficiently studied. In this study, we develop a new CDHWs index that integrates a four-day standardized soil moisture index with heatwaves index. Employing an interpretable machine learning model, we quantitatively analyzed the effects of various environmental factors on vegetation during CDHWs. Results demonstrate significant upward trends in CDHW frequency, intensity, and maximum duration across global drylands. Analysis indicates that 65% of CDHWs negatively affect vegetation, 20% produce positive effects, and 15% result in minimal impact. Accumulated local effects and partial dependence plot analyses identify maximum temperature as the primary factor driving negative vegetation responses, while soil moisture emerges as the predominant factor associated with positive responses. Projections under multiple coupled model intercomparison project phase 6 scenarios (SSP245 and SSP585) indicate continued increases in CDHW frequency, intensity, and maximum duration through 2054. These findings underscore the necessity for fine-scale monitoring of compound extremes and offer actionable insights for adaptive management in dryland regions.

Machine learning models are increasingly used for streamflow prediction due to their promising performance. However, their data-driven nature makes interpretation challenging. This study explores the interpretability of a Random Forest model trained on high streamflow events from a hydrological perspective, comparing methods for assessing feature influence. The results show that the mean decrease accuracy, mean decrease impurity, Shapley additive explanations, and Tornado methods identify similar key features, though Tornado presents the most notable discrepancies. Despite the model being trained with events of considerable temporal variability, the last observed streamflow is the most relevant feature accounting for over 20% of importance. Moreover, the results suggest that the model identifies a catchment region with a runoff that significantly affects the outlet flow. Accumulated local effects and partial dependence plots may represent first infiltration losses and soil saturation before precipitation sharply impacts streamflow. However, only accumulated local effects depict the influence of the scarce highest accumulated precipitation on the streamflow. Shapley additive explanations are simpler to apply than the local interpretable model-agnostic explanations, which require a tuning process, though both offer similar insights. They show that short-period accumulated precipitation is crucial during the steep rising limb of the hydrograph, reaching 72% of importance on average among the top features. As the peak approaches, previous streamflow values become the most influential feature, continuing into the falling limb. When the hydrograph goes down, the model confers a moderate influence on the accumulated precipitation of several hours back of distant regions, suggesting that the runoff from these areas is arriving. Machine learning models may interpret the catchment system reasonably and provide useful insights about hydrological characteristics.

Background: Vancomycin, a key antibiotic for severe Gram-positive infections in ICUs, poses a high nephrotoxicity risk. Early prediction of kidney injury in critically ill patients is challenging. This study aimed to develop a machine learning model to predict vancomycin-related creatinine elevation using routine ICU data. Methods: We analyzed 10,288 ICU patients (aged 18-80) from the MIMIC-IV database who received vancomycin. Kidney injury was defined by KDIGO criteria (creatinine rise>=0.3 mg/dL within 48h or>=50% within 7d). Features were selected via SelectKBest (top 30) and Random Forest ranking (final 15). Six algorithms were tested with 5-fold cross-validation. Interpretability was evaluated using SHAP, Accumulated Local Effects (ALE), and Bayesian posterior sampling. Results: Of 10,288 patients, 2,903 (28.2%) developed creatinine elevation. CatBoost performed best (AUROC 0.818 [95% CI: 0.801-0.834], sensitivity 0.800, specificity 0.681, negative predictive value 0.900). Key predictors were phosphate, total bilirubin, magnesium, Charlson index, and APSIII. SHAP confirmed phosphate as a major risk factor. ALE showed dose-response patterns. Bayesian analysis estimated mean risk 60.5% (95% credible interval: 16.8-89.4%) in high-risk cases. Conclusions: This machine learning model predicts vancomycin-associated creatinine elevation from routine ICU data with strong accuracy and interpretability, enabling early risk detection and supporting timely interventions in critical care.

Predicting seafood consumption behavior is essential for fishing companies to adjust their production plans and marketing strategies. To achieve accurate predictions, this paper introduces a model for forecasting seafood consumption behavior based on an interpretable machine learning algorithm. Additionally, the Shapley Additive exPlanation (SHAP) model and the Accumulated Local Effects (ALE) plot were integrated to provide a detailed analysis of the factors influencing Shanghai residents’ intentions to purchase salmon. In this study, we constructed nine regression prediction models, including ANN, Decision Tree, GBDT, Random Forest, AdaBoost, XGBoost, LightGBM, CatBoost, and NGBoost, to predict the consumers’ intentions to purchase salmon and to compare their predictive performance. In addition, Bayesian optimization algorithm is used to optimize the hyperparameters of the optimal regression prediction model to improve the model prediction accuracy. Finally, the SHAP model was used to analyze the key factors and interactions affecting the consumers’ willingness to purchase salmon, and the Accumulated Local Effects plot was used to show the specific prediction patterns of different influences on salmon consumption. The results of the study show that salmon farming safety and ease of cooking have significant nonlinear effects on salmon consumption; the BO-CatBoost nonlinear regression prediction model demonstrates superior performance compared to the benchmark model, with the test set exhibiting RMSE, MSE, MAE, R2 and TIC values of 0.155, 0.024, 0.097, 0.902, and 0.313, respectively. This study can provide technical support for suppliers in the salmon value chain and help their decision-making to adjust their corporate production plan and marketing activities

No abstract available

Poverty remains a multidimensional challenge in Central Java, necessitating robust data-driven approaches to identify its socioeconomic determinants. This study applied six machine learning models, specifically Extreme Gradient Boosting (XGBoost), Random Forest, CatBoost, LightGBM, Elastic Net Regression, and a Stacking ensemble using district-level data from Statistics Indonesia covering demographics, education, labor, infrastructure, and household welfare. Model evaluation combined an 80:20 hold-out split, 10-fold cross-validation, and noise perturbation tests. Results show that XGBoost achieved the best individual performance (MAE = 2,180.01; RMSE = 3,512.07; R² = 0.931), while the Stacking ensemble surpassed all single learners (MAE = 2,640.99; RMSE = 3,202.79; R² = 0.942). Interpretability was ensured through SHAP (Shapley Additive Explanations), Partial Dependence Plots (PDP), and Accumulated Local Effects (ALE), consistently identifying Number of Households, Per Capita Expenditure, and Uninhabitable Houses as the most influential predictors. Counterfactual simulations indicated that increasing per capita expenditure by 10% could reduce the poverty index by 9.9%, while reducing household size by 10% lowered it by 11.3%. Robustness checks revealed Brebes as an influential district shaping model stability. Overall, the findings demonstrate that boosting and stacking ensembles, when combined with explainable AI tools, not only enhance predictive accuracy but also provide transparent, policy-relevant evidence to strengthen poverty alleviation programs in Central Java. This study contributes both methodological advances in explainable machine learning and practical insights for targeted poverty reduction strategies.

This paper introduces a novel framework for enhancing insurance underwriting through Explainable Artificial Intelligence (XAI) methodologies. The approach addresses critical challenges in the insurance industry by automating risk assessment while maintaining full transparency for regulators, underwriters, and customers. Our framework incorporates multiple complementary XAI techniques including SHAP values, accumulated local effects, counterfactual explanations, rule extraction, and natural language generation to provide comprehensive understanding of model decisions. The system delivers personalized policy recommendations across multiple dimensions including coverage optimization, exclusion refinement, deductible customization, risk prevention guidance, bundle optimization, and payment structure flexibility. Experimental validation across auto, commercial property, and life insurance demonstrates significant improvements in operational efficiency, risk assessment accuracy, customer satisfaction, and regulatory compliance. The integration of explainability with advanced personalization capabilities proves that transparency and sophisticated AI-driven underwriting can be achieved simultaneously, creating a blueprint for next-generation insurance systems that balance innovation with trust and regulatory requirements.

ABSTRACT The integration of Artificial Intelligence (AI) into smart farming, particularly Crop Recommendation System (CRS), has propelled significant advancements but is often hindered by the ‘black box’ nature of models, which limits transparency and trust. This study aims to enhance smart farming by embedding explainable Artificial Intelligence (XAI) techniques – specifically Contrastive Explanation Method (CEM) and Accumulated Local Effects (ALE) – within CRS, empowering farmers to understand AI-generated crop suggestions. Implemented an XAI-driven CRS, utilising Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), CEM, and ALE for comprehensive explainability. Notably, CEM provides farmers with actionable contrastive explanations, while ALE details the average influence of environmental factors. To address data scarcity, Generative Adversarial Networks (GANs) were used to augment the dataset with synthetic data, and an interactive, explainable interface was developed using Streamlit. Results show a substantial improvement in system interpretability and user trust, evidenced by clearer, actionable explanations for farmers. Quantitatively, incorporating GAN-augmented data improved the Random Forest model’s Area Under the Receiver Operating Characteristic curve (AUROC) from 0.94 to 0.985 and F1-score from 0.93 to 0.98. This research is the first to integrate CEM and ALE in CRS, establishing a new benchmark for transparent and effective AI-powered agricultural decision-making.

Background: Hypertensive kidney disease (HKD) patients in intensive care units (ICUs) face high short-term mortality, but tailored risk prediction tools are lacking. Early identification of high-risk individuals is crucial for clinical decision-making. Methods: We developed a machine learning framework to predict 30-day in-hospital mortality among ICU patients with HKD using early clinical data from the MIMIC-IV v2.2 database. A cohort of 1,366 adults was curated with strict criteria, excluding malignancy cases. Eighteen clinical features-including vital signs, labs, comorbidities, and therapies-were selected via random forest importance and mutual information filtering. Several models were trained and compared with stratified five-fold cross-validation; CatBoost demonstrated the best performance. Results: CatBoost achieved an AUROC of 0.88 on the independent test set, with sensitivity of 0.811 and specificity of 0.798. SHAP values and Accumulated Local Effects (ALE) plots showed the model relied on meaningful predictors such as altered consciousness, vasopressor use, and coagulation status. Additionally, the DREAM algorithm was integrated to estimate patient-specific posterior risk distributions, allowing clinicians to assess both predicted mortality and its uncertainty. Conclusions: We present an interpretable machine learning pipeline for early, real-time risk assessment in ICU patients with HKD. By combining high predictive performance with uncertainty quantification, our model supports individualized triage and transparent clinical decisions. This approach shows promise for clinical deployment and merits external validation in broader critical care populations.

Precise process control is essential in semiconductor manufacturing to maintain product quality and reduce variability. This paper proposes an adaptive Run-to-Run (R2R) control algorithm that integrates a virtual metrology model with interpretable machine learning techniques. The algorithm combines Accumulated Local Effects (ALE) and Shapley Additive Explanations (SHAP) to guide parameter adjustments in a double Exponentially Weighted Moving Average (dEWMA) controller. ALE gradients are utilized to refine adjustment directions and magnitudes, while SHAP values prioritize critical variables, enabling dynamic and precise control of nonlinear processes across batch runs. Comparative studies with traditional EWMA and dEWMA controllers highlight the proposed method's superior prediction accuracy and control performance, providing a robust and intelligent solution for advanced semiconductor process optimization.

Disentangling the individual and interactive effects of soil water content (SWC) and vapor pressure deficit (VPD) on vegetation growth remains challenging owing to tight land-plant-atmosphere interaction and lagged water stress effects. We developed a Random Forest-based dual-interpretation framework (SHAP-ALE, Shapley Additive Explanations – Accumulated Local Effects) to quantify the main effects of SWC, VPD, and historical SWC (HSWC), as well as interaction effects between water and heat stress, across six plant function types (PFTs) using the FLUXNET2015 dataset. The performance of modes was enhanced by incorporating HSWC, with an increase in the average determination coefficient of 0.13. Dual interpretation methods reveal that HSWC, which significantly affects GPP with high intensity and a wider effective range, is the dominant water stress for PFTs except for savanna and grassland, for which VPD is the dominant one. SWC benefits GPP intensely only below critical thresholds, while VPD exerts a gradual, widespread negative impact. Furthermore, heat-water stress interactions outweigh VPD-SWC interactions for most PFTs. These biome-specific effects provide insights into the refinement of vegetation parameterization in land surface models, thereby advancing ecosystem response quantification to environmental stresses.

This study aimed to evaluate the performance of machine learning (ML) models in the diagnosis of Hepatitis C Virus (HCV) patients and to identify clinical biomarkers using explainable artificial intelligence (XAI) approaches. Black box algorithms - Random Forest (RF) and Extreme Gradient Boosting (XGBoost) were used in the study, and XAI methods - SHapley Additive Explanations (SHAP) and Accumulated Local Effects (ALE) were applied to increase the interpretability of the models. There were 615 patients in the dataset, and the output variable included various liver disorders including HCV. The results showed that RF and XGBoost models exhibited high performance with 93.75% and 92.38% accuracy rates, respectively. SHAP and ALE analyses revealed the importance and interactions of the factors (ALT, AST, bilirubin, albumin, age) underlying model decisions. This study demonstrates the potential of ML models in early diagnosis of HCV infection and how they can be integrated with XAI methods to make them more reliable in medical applications.

Efforts to green cities for cooling are succeeding unevenly because the same vegetation that cools surfaces can also intensify how hot the air feels. Previous studies have identified humid heat as a growing urban hazard, yet how physiologically active vegetation governs this trade-off between cooling and moisture accumulation remains poorly understood, leaving mitigation policy and design largely unguided. Here we quantify how vegetation structure and function influence the Heat Index (HI), a combined measure of temperature and humidity in 138 Indian cities spanning tropical savanna, semi-arid steppe, and humid subtropical climates, and across dense urban cores and semi-urban rings. Using an extreme-aware, one kilometre reconstruction of HI and an interpretable machine-learning framework that integrates SHapley Additive Explanations (SHAP) and Accumulated Local Effects (ALE), we isolate vegetation-climate interactions. Cooling generally strengthens for EVI>= 0.4 and LAI>= 0.05, but joint-high regimes begin to reverse toward warming when EVI>= 0.5, LAI>= 0.2, and fPAR>= 0.5,with an earlier onset for fPAR>= 0.25 in humid, dense cores. In such environments, highly physiologically active vegetation elevates near-surface humidity faster than it removes heat, reversing its cooling effect and amplifying perceived heat stress. These findings establish the climatic limits of vegetation-driven cooling and provide quantitative thresholds for climate-specific greening strategies that promote equitable and heat-resilient cities.

Abstract Premise Here we demonstrate the application of interpretable machine learning methods to investigate intraspecific functional trait divergence using diverse genotypes of the wide‐ranging sunflower Helianthus annuus occupying populations across two contrasting ecoregions—the Great Plains versus the North American Deserts. Methods Recursive feature elimination was applied to functional trait data from the HeliantHOME database, followed by the application of the Boruta algorithm to detect the traits that are most predictive of ecoregion. Random forest and gradient boosting machine classifiers were then trained and validated, with results visualized using accumulated local effects plots. Results The most ecoregion‐predictive functional traits span categories of leaf economics, plant architecture, reproductive phenology, and floral and seed morphology. Relative to the Great Plains, genotypes from the North American Deserts exhibit shorter stature, fewer leaves, higher leaf nitrogen content, and longer average length of phyllaries. Discussion This approach readily identifies traits predictive of ecoregion origin, and thus the functional traits most likely to be responsible for contrasting ecological strategies across the landscape. This type of approach can be used to parse large plant trait datasets in a wide range of contexts, including explicitly testing the applicability of interspecific paradigms at intraspecific scales.

This study incorporates a random forest (RF) approach to probe complex interactions and nonlinearity among predictors into an item response model with the goal of using a hybrid approach to outperform either an RF or explanatory item response model (EIRM) only in explaining item responses. In the specified model, called EIRM-RF, predicted values using RF are added as a predictor in EIRM to model the nonlinear and interaction effects of person- and item-level predictors in person-by-item response data, while accounting for random effects over persons and items. The results of the EIRM-RF are probed with interpretable machine learning (ML) methods, including feature importance measures, partial dependence plots, accumulated local effect plots, and the H-statistic. The EIRM-RF and the interpretable methods are illustrated using an empirical data set to explain differences in reading comprehension in digital versus paper mediums, and the results of EIRM-RF are compared with those of EIRM and RF to show empirical differences in modeling the effects of predictors and random effects among EIRM, RF, and EIRM-RF. In addition, simulation studies are conducted to compare model accuracy among the three models and to evaluate the performance of interpretable ML methods.

In this study, interpretable machine learning (ML) models and econometric models were compared to analyze individual travel mode choice behavior using data from the 2017 National Household Travel Survey in the San Francisco Bay Area. Two ML models, Random Forest (RF) and eXtreme Gradient Boosting (XGB), were implemented, along with three econometric models: Multinomial Logit (MNL), Nested Logit (NL), and Cross-Nested Logit (CNL). Higher predictive accuracies of 0.853 for RF and 0.846 for XGB were achieved, particularly under class imbalance conditions. Model interpretability was ensured through the use of feature importance and Accumulated Local Effects (ALE) plots. Promising temporal transferability was observed, as PM travel modes were accurately predicted using AM-trained models, with overall accuracies of 0.83 for RF and 0.83 for XGB, and balanced accuracies between 0.63 and 0.69. The findings indicate that the complementary strengths of ML and econometric approaches can enhance the understanding of travel behavior.

We propose a novel, distribution-free framework for estimating option Greeks using machine learning. Unlike traditional parametric approaches, our method employs accumulated local effects to capture data-driven sensitivities without imposing functional form restrictions. Validation on synthetic Black–Scholes prices confirms the method’s ability to accurately recover theoretical Greeks. When applied to a comprehensive dataset of 14 million European S&P 500 options, we document notable empirical deviations: Vega sensitivity plateaus at high volatility levels, Gamma exhibits sharper peaks near at-the-money strikes, and Theta decay follows nonlinear patterns. These findings reveal important limitations of conventional models in capturing market-implied sensitivities. The model-agnostic framework, illustrated here with gradient-boosted trees, offers practitioners a robust, parametric assumption–free alternative for risk measurement. Its versatility suggests potential for extension to hedging American options, exotic derivatives, and complex portfolios.

The objective of this article is to introduce a fairness interpretability framework for measuring and explaining the bias in classification and regression models at the level of a distribution. In our work, we measure the model bias across sub-population distributions in the model output using the Wasserstein metric. To properly quantify the contributions of predictors, we take into account favorability of both the model and predictors with respect to the non-protected class. The quantification is accomplished by the use of transport theory, which gives rise to the decomposition of the model bias and bias explanations to positive and negative contributions. To gain more insight into the role of favorability and allow for additivity of bias explanations, we adapt techniques from cooperative game theory.

No abstract available

Traditional text representations like embeddings and bag-of-words hinder rule learning and other interpretable machine learning methods due to high dimensionality and poor comprehensibility. This article investigates using Large Language Models (LLMs) to extract a small number of interpretable text features. We propose two workflows: one fully automated by the LLM (feature proposal and value calculation), and another where users define features and the LLM calculates values. This LLM-based feature extraction enables interpretable rule learning, overcoming issues like spurious interpretability seen with bag-of-words. We evaluated the proposed methods on five diverse datasets (including scientometrics, banking, hate speech, and food hazard). LLM-generated features yielded predictive performance similar to the SciBERT embedding model but used far fewer, interpretable features. Most generated features were considered relevant for the corresponding prediction tasks by human users. We illustrate practical utility on a case study focused on mining recommendation action rules for the improvement of research article quality and citation impact.

Lubricant film thickness is a tribological parameter that significantly impacts the performance of a tribo-system, primarily by affecting the resulting friction and wear. Measuring or predicting film thickness is therefore crucial for any hydrodynamically lubricated machine element. Numerical techniques are often employed to solve the film thickness under specified operating conditions. However, those approaches are computationally expensive and time-consuming. Machine learning methods have been proven to be a highly accurate and faster alternative to predicting lubricant film thickness. Still, these methods have their own benefits and drawbacks with regards to accuracy, speed, and model interpretability. In this paper, a speed/accuracy comparison is provided between the numerical solution, artificial neural network, and other machine learning approaches for determination of lubricant film thickness. It is found that non-ANN methods can be ideal in cases where prediction time is prioritized while high accuracy is still desired; the tested non-ANN models are orders of magnitude faster than both the numerical solution and the ANN approach, while accuracy is still high with a maximum R2 score of 0.997. This finding, combined with the possible benefit of easy and accurate model interpretability, shows that non-ANN methods can be very useful for this application.

Nowadays, most of the health expenditure is due to chronic patients who are readmitted several times for their pathologies. Personalized prevention strategies could be developed to improve the management of these patients. The aim of the present work was to develop local predictive models using interpretable machine learning techniques to early identify individual unscheduled hospital readmissions. To do this, a retrospective, case-control study, based on information regarding patient readmission in 2018–2019, was conducted. After curation of the initial dataset (n = 76,210), the final number of participants was n = 29,026. A machine learning analysis was performed following several algorithms using unscheduled hospital readmissions as dependent variable. Local model-agnostic interpretability methods were also performed. We observed a 13% rate of unscheduled hospital readmissions cases. There were statistically significant differences regarding age and days of stay (p < 0.001 in both cases). A logistic regression model revealed chronic therapy (odds ratio: 3.75), diabetes mellitus history (odds ratio: 1.14), and days of stay (odds ratio: 1.02) as relevant factors. Machine learning algorithms yielded better results regarding sensitivity and other metrics. Following, this procedure, days of stay and age were the most important factors to predict unscheduled hospital readmissions. Interestingly, other variables like allergies and adverse drug reaction antecedents were relevant. Individualized prediction models also revealed a high sensitivity. In conclusion, our study identified significant factors influencing unscheduled hospital readmissions, emphasizing the impact of age and length of stay. We introduced a personalized risk model for predicting hospital readmissions with notable accuracy. Future research should include more clinical variables to refine this model further.

Machine learning (ML) techniques applied to quantum many-body physics have emerged as a new research field. While the numerical power of this approach is undeniable, the most expressive ML algorithms, such as neural networks, are black boxes: The user does neither know the logic behind the model predictions nor the uncertainty of the model predictions. In this work, we present a toolbox for interpretability and reliability, agnostic of the model architecture. In particular, it provides a notion of the influence of the input data on the prediction at a given test point, an estimation of the uncertainty of the model predictions, and an extrapolation score for the model predictions. Such a toolbox only requires a single computation of the Hessian of the training loss function. Our work opens the road to the systematic use of interpretability and reliability methods in ML applied to physics and, more generally, science.

Many datasets are of increasingly high dimension- ality, where a large number of features could be irrelevant to the learning task. The inclusion of such features would not only introduce unwanted noise but also increase computational complexity. Deep neural networks (DNNs) outperform machine learning (ML) algorithms in a variety of applications due to their effectiveness in modelling complex problems and handling high-dimensional datasets. However, due to non-linearity and higher-order feature interactions, DNN models are unavoidably opaque, making them black-box methods. In contrast, an interpretable model can identify statistically significant features and explain the way they affect the model’s outcome. In this paper, we propose a novel method to improve the interpretability of blackbox models in the case of high-dimensional datasets. First, a black-box model is trained on full feature space that learns useful embeddings on which the classification is performed. To decompose the inner principles of the black-box and to identify top-k important features (global explainability), probing and perturbing techniques are applied. An interpretable surrogate model is then trained on top-k feature space to approximate the black-box. Finally, decision rules and counterfactuals are derived from the surrogate to provide local decisions. Our approach outperforms tabular learners, e.g., TabNet and XGboost, and SHAP-based interpretability techniques, when tested on a number of datasets having dimensionality between 54 and 20,5311.1GitHub: https://github.com/rezacsedu/DeepExplainHidim

Research in mechanistic interpretability seeks to explain behaviors of machine learning models in terms of their internal components. However, most previous work either focuses on simple behaviors in small models, or describes complicated behaviors in larger models with broad strokes. In this work, we bridge this gap by presenting an explanation for how GPT-2 small performs a natural language task called indirect object identification (IOI). Our explanation encompasses 26 attention heads grouped into 7 main classes, which we discovered using a combination of interpretability approaches relying on causal interventions. To our knowledge, this investigation is the largest end-to-end attempt at reverse-engineering a natural behavior"in the wild"in a language model. We evaluate the reliability of our explanation using three quantitative criteria--faithfulness, completeness and minimality. Though these criteria support our explanation, they also point to remaining gaps in our understanding. Our work provides evidence that a mechanistic understanding of large ML models is feasible, opening opportunities to scale our understanding to both larger models and more complex tasks.

The analysis of the interplay between the feature selection and the post-hoc local interpretation steps in a machine learning workflow followed for IoT botnet detection constitutes the research scope of the present paper. While the application of machine learning-based techniques has become a trend in cyber security, the main focus has been almost on detection accuracy. However, providing the relevant explanation for a detection decision is a vital requirement in a tiered incident handling processes of the contemporary security operations centers. Moreover, the design of intrusion detection systems in IoT networks has to take the limitations of the computational resources into consideration. Therefore, resource limitations in addition to human element of incident handling necessitate considering feature selection and interpretability at the same time in machine learning workflows. In this paper, first, we analyzed the selection of features and its implication on the data accuracy. Second, we investigated the impact of feature selection on the explanations generated at the post-hoc interpretation phase. We utilized a filter method, Fisher's Score and Local Interpretable Model-Agnostic Explanation (LIME) at feature selection and post-hoc interpretation phases, respectively. To evaluate the quality of explanations, we proposed a metric that reflects the need of the security analysts. It is demonstrated that the application of both steps for the particular case of IoT botnet detection may result in highly accurate and interpretable learning models induced by fewer features. Our metric enables us to evaluate the detection accuracy and interpretability in an integrated way.

Despite recent medical advancements, breast cancer remains one of the most prevalent and deadly diseases among women. Although machine learning-based Computer-Aided Diagnosis (CAD) systems have shown potential to assist radiologists in analyzing medical images, the opaque nature of the best-performing CAD systems has raised concerns about their trustworthiness and interpretability. This paper proposes MT-BI-RADS, a novel explainable deep learning approach for tumor detection in Breast Ultrasound (BUS) images. The approach offers three levels of explanations to enable radiologists to comprehend the decision-making process in predicting tumor malignancy. Firstly, the proposed model outputs the BI-RADS categories used for BUS image analysis by radiologists. Secondly, the model employs multi-task learning to concurrently segment regions in images that correspond to tumors. Thirdly, the proposed approach outputs quantified contributions of each BI-RADS descriptor toward predicting the benign or malignant class using post-hoc explanations with Shapley Values.

No abstract available

Providing accurate diagnosis of diseases generally requires complex analyses of many clinical, biological and pathological variables. In this context, solutions based on machine learning techniques achieved relevant results in specific disease detection and classification, and can hence provide significant clinical decision support. However, such approaches suffer from the lack of proper means for interpreting the choices made by the models, especially in case of deep-learning ones. In order to improve interpretability and explainability in the process of making qualified decisions, we designed a system that allows for a partial opening of this black box by means of proper investigations on the rationale behind the decisions; this can provide improved understandings into which pre-processing steps are crucial for better performance. We tested our approach over artificial neural networks trained for automatic medical diagnosis based on high-dimensional gene expression and clinical data. Our tool analyzed the internal processes performed by the networks during the classification tasks in order to identify the most important elements involved in the training process that influence the network’s decisions.We report the results of an experimental analysis aimed at assessing the viability of the proposed approach.

No abstract available

Abstract Machine learning (ML) models are widely employed for crash severity modelling, yet their interpretability remains underexplored. Interpretation is crucial for comprehending ML results and aiding informed decision-making. This study aims to implement an interpretable ML to visualize the impacts of factors on crash severity using 5 years of freeways data from Iran. Methods including classification and regression trees (CART), K-nearest neighbours (KNNs), random forest (RF), artificial neural network (ANN) and support vector machines (SVM) were applied, with RF demonstrating superior accuracy, recall, F1-score and ROC. The accumulated local effects (ALE) were utilized for interpretation. Findings suggest that light traffic conditions (volume/capacity<0.5) with critical values around 0.05 or 0.38, and higher proportion of large trucks and buses, particularly at 10% and 4%, are associated with severe crashes. Additionally, speeds exceeding 90 km/h, drivers younger than 30 years, rollover crashes, collisions with fixed objects and barriers, nighttime driving and driver fatigue elevate the likelihood of severe crashes.

With increasing interest in explaining machine learning (ML) models, this paper synthesizes many topics related to ML explainability. We distinguish explainability from interpretability, local from global explainability, and feature importance versus feature relevance. We demonstrate and visualize different explanation methods, how to interpret them, and provide a complete Python package (scikit-explain) to allow future researchers and model developers to explore these explainability methods. The explainability methods include Shapley additive explanations (SHAP), Shapley additive global explanation (SAGE), and accumulated local effects (ALE). Our focus is primarily on Shapley-based techniques, which serve as a unifying framework for various existing methods to enhance model explainability. For example, SHAP unifies methods like local interpretable model-agnostic explanations (LIME) and tree interpreter for local explainability, while SAGE unifies the different variations of permutation importance for global explainability. We provide a short tutorial for explaining ML models using three disparate datasets: a convection-allowing model dataset for severe weather prediction, a nowcasting dataset for sub-freezing road surface prediction, and satellite-based data for lightning prediction. In addition, we showcase the adverse effects that correlated features can have on the explainability of a model. Finally, we demonstrate the notion of evaluating model impacts of feature groups instead of individual features. Evaluating the feature groups mitigates the impacts of feature correlations and can provide a more holistic understanding of the model. All code, models, and data used in this study are freely available to accelerate the adoption of machine learning explainability in the atmospheric and other environmental sciences.

The objective of the study is to investigate the effect of Nuchal Fold (NF) in predicting Fetal Growth Restriction (FGR) using machine learning (ML), to explain the model's results using model-agnostic interpretable techniques, and to compare the results with clinical guidelines. This study used second-trimester ultrasound biometry and Doppler velocimetry were used to construct six FGR (birthweight < 3rd centile) ML models. Interpretability analysis was conducted using Accumulated Local Effects (ALE) and Shapley Additive Explanations (SHAP). The results were compared with clinical guidelines based on the most optimal model. Support Vector Machine (SVM) exhibited the most consistent performance in FGR prediction. SHAP showed that the top contributors to identify FGR were Abdominal Circumference (AC), NF, Uterine RI (Ut RI), and Uterine PI (Ut PI). ALE showed that the cutoff values of Ut RI, Ut PI, and AC in differentiating FGR from normal were comparable with clinical guidelines (Errors between model and clinical; Ut RI: 15%, Ut PI: 8%, and AC: 11%). The cutoff value for NF to differentiate between healthy and FGR is 5.4 mm, where low NF may indicate FGR. The SVM model is the most stable in FGR prediction. ALE can be a potential tool to identify a cutoff value for novel parameters to differentiate between healthy and FGR.

Accurate prediction of survival days (SD) is vital for planning treatments in glioma patients, as type‐IV tumors typically have a poor prognosis and meager survival rates. SD prediction is challenging and heavily dependent on the extracted feature sets. Additionally, comprehending the behavior of complex machine learning models is a vital yet challenging aspect, particularly to integrate such models into the medical domain responsibly. Therefore, this study develops a robust feature set and an ensemble‐based regressor model to predict patients' SD accurately. We aim to understand how these features behave and contribute to predicting SD. To accomplish this, we employed post‐hoc interpretable techniques, precisely Shapley Additive exPlanations (SHAP), Partial Dependence Plots (PDP), and Accumulated Local Effects (ALE) plots. Furthermore, we introduced an investigation to establish a direct connection between radiomic features and their biological significance to enhance the interpretability of radiomic features. The best SD scores on the BraTS2020 training set are 0.504 for accuracy, 59927.38 mean squared error (MSE), 20101.86 median squared error (medianSE), and 0.725 Spearman ranking coefficient (SRC). The validation set's accuracy is 0.586, MSE is 76529.43, medianSE is 41402.78, and SRC is 0.52. The proposed predictor model exhibited superior performance compared with leading contemporary approaches across multiple performance metrics.

InterpretML is an open-source Python package which exposes machine learning interpretability algorithms to practitioners and researchers. InterpretML exposes two types of interpretability - glassbox models, which are machine learning models designed for interpretability (ex: linear models, rule lists, generalized additive models), and blackbox explainability techniques for explaining existing systems (ex: Partial Dependence, LIME). The package enables practitioners to easily compare interpretability algorithms by exposing multiple methods under a unified API, and by having a built-in, extensible visualization platform. InterpretML also includes the first implementation of the Explainable Boosting Machine, a powerful, interpretable, glassbox model that can be as accurate as many blackbox models. The MIT licensed source code can be downloaded from github.com/microsoft/interpret.

No abstract available

To boost the application of machine learning (ML) techniques for credit scoring models, the blackbox problem should be addressed. The primary aim of this paper is to propose a measure based on counterfactuals to evaluate the interpretability of a ML credit scoring technique. Counterfactuals assist with understanding the model with regard to the classification decision boundaries and evaluate model robustness. The second contribution is the development of a data perturbation technique to generate a stress scenario. We apply these two proposals to a dataset on UK unsecured personal loans to compare logistic regression and stochastic gradient boosting (SBG). We show that training a blackbox model (SGB) as conditioned on our data perturbation technique can provide insight into model performance under stressed scenarios. The empirical results show that our interpretability measure is able to capture the classification decision boundary, unlike AUC and the classification accuracy widely used in the banking sector.

Convolutional neural networks are showing promise in the automatic diagnosis of thoracic pathologies on chest x-rays. Their black-box nature has sparked many recent works to explain the prediction via input feature attribution methods (aka saliency methods). However, input feature attribution methods merely identify the importance of input regions for the prediction and lack semantic interpretation of model behavior. In this work, we first identify the semantics associated with internal units (feature maps) of the network. We proceed to investigate the following questions;Does a regression model that is only trained with COVID-19 severity scores implicitly learn visual patterns associated with thoracic pathologies? Does a network that is trained on weakly labeled data (e.g. healthy, unhealthy) implicitly learn pathologies? Moreover, we investigate the effect of pretraining and data imbalance on the interpretability of learned features. In addition to the analysis, we propose semantic attribution to semantically explain each prediction. We present our findings using publicly available chest pathologies (CheXpert [5], NIH ChestX-ray8 [25]) and COVID-19 datasets (BrixIA [20], and COVID-19 chest X-ray segmentation dataset [4]). The Code (https://github.com/CAMP-eXplain-AI/CheXplain-Dissection ) is publicly available. © 2021, Springer Nature Switzerland AG.

In this paper, we introduce a machine learning (ML)-based approach for Chronic Kidney Disease (CKD) and diabetes prediction and perform feature effect analysis through SHAP (SHapley Additive exPlanations). We utilize two publicly available clinical datasets and build five ML classifier models for the analysis. Among all models, CatBoost provides the best performance for both CKD and diabetes prediction. Our CatBoost model has an accuracy of 0.95, mean 10-fold cross-validation of 0.96, ROC-AUC score of 0.99, precision of 0.96, recall of 0.96, and F1-score of 0.96 for CKD. The CatBoost model shows an accuracy of 0.99, mean 10-fold cross-validation of 0.97, ROC-AUC score of 1.0, precision of 1.0, recall of 0.98, and F1-score of 0.99 for diabetes. We perform comprehensive feature effect analysis by computing SHAP values. This gives insights into the contribution of every feature to the model's predictions. The achieved results highlight that CatBoost is a robust choice for accurate and reliable predictions of CKD and diabetes. The findings of this study contribute to the corresponding field of ML approaches for medical diagnosis and illustrate the significance of feature effect analysis in understanding model predictions. With excellent results, our research has the potential to enhance the clinical decision-making process and improve patient outcomes.

To interprete the importance of clinical features and genotypes for warfarin daily dose prediction, we developed a post-hoc interpretable framework based on an ensemble predictive model. This framework includes permutation importance for global interpretation and local interpretable model-agnostic explanation (LIME) and shapley additive explanations (SHAP) for local explanation. The permutation importance globally ranks the importance of features on the whole data set. This can guide us to build a predictive model with less variables and the complexity of final predictive model can be reduced. LIME and SHAP together explain how the predictive model give the predicted dosage for specific samples. This help clinicians prescribe accurate doses to patients using more effective clinical variables. Results showed that both the permutation importance and SHAP demonstrated that VKORC1, age, serum creatinine (SCr), left atrium (LA) size, CYP2C9 and weight were the most important features on the whole data set. In specific samples, both SHAP and LIME discovered that in Chinese patients, wild-type VKORC1-AA, mutant-type CYP2C9*3, age over 60, abnormal LA size, SCr within the normal range, and using amiodarone definitely required dosage reduction, whereas mutant-type VKORC1-AG/GG, small age, SCr out of normal range, normal LA size, diabetes and heavy weight required dosage enhancementt.

Background There is considerable geographic heterogeneity in obesity prevalence across counties in the United States. Machine learning algorithms accurately predict geographic variation in obesity prevalence, but the models are often uninterpretable and viewed as a black-box. Objective The goal of this study is to extract knowledge from machine learning models for county-level variation in obesity prevalence. Methods This study shows the application of explainable artificial intelligence methods to machine learning models of cross-sectional obesity prevalence data collected from 3,142 counties in the United States. County-level features from 7 broad categories: health outcomes, health behaviors, clinical care, social and economic factors, physical environment, demographics, and severe housing conditions. Explainable methods applied to random forest prediction models include feature importance, accumulated local effects, global surrogate decision tree, and local interpretable model-agnostic explanations. Results The results show that machine learning models explained 79% of the variance in obesity prevalence, with physical inactivity, diabetes, and smoking prevalence being the most important factors in predicting obesity prevalence. Conclusions Interpretable machine learning models of health behaviors and outcomes provide substantial insight into obesity prevalence variation across counties in the United States.

Machine learning models (MLMs) have been increasingly used to forecast water pollution. However, the "black box" characteristic for understanding mechanism processes still limits the applicability of MLMs for water quality management in hydro-projects under complex and frequently artificial regulation. This study proposes an interpretable machine learning framework for water quality prediction coupled with a hydrodynamic (flow discharge) scenario-based Random Forest (RF) model with multiple model-agnostic techniques and quantifies global, local, and joint interpretations (i.e., partial dependence, individual conditional expectation, and accumulated local effects) of environmental factor implications. The framework was applied and verified to predict the permanganate index (CODMn) under different flow discharge regulation scenarios in the Middle Route of the South-to-North Water Diversion Project of China (MRSNWDPC). A total of 4664 sampling cases data matrices, including water quality, meteorological, and hydrological indicators from eight national stations along the main canal of the MRSNWDPC, were collected from May 2019 to December 2020. The results showed that the RF models were effective in forecasting CODMn in all flow discharge scenarios, with a mean square error, coefficient of determination, and mean absolute error of 0.006-0.026, 0.481-0.792, and 0.069-0.104, respectively, in the testing dataset. A global interpretation indicated that dissolved oxygen, flow discharge, and surface pressure are the three most important variables of CODMn. Local and joint interpretations indicated that the RF-based prediction model provides a basic understanding of the physical mechanisms of environmental systems. The proposed framework can effectively learn the fundamental environmental implications of water quality variations and provide reliable prediction performance, highlighting the importance of model interpretability for trustworthy machine learning applications in water management projects. This study provides scientific references for applying advanced data-driven MLMs to water quality forecasting and a reliable methodological framework for water quality management and similar hydro-projects.

We examine the main effects of ICT penetration and the shadow economy on sovereign credit ratings and the cost of debt, along with possible second-order effects between the two variables, on a dataset of 65 countries from 2001 to 2016. The paper presents a range of machine-learning approaches, including bagging, random forests, gradient-boosting machines, and recurrent neural networks. Furthermore, following recent trends in the emerging field of interpretable ML, based on model-agnostic methods such as feature importance and accumulated local effects, we attempt to explain which factors drive the predictions of the so-called ML black box models. We show that policies facilitating the penetration and use of ICT and aiming to curb the shadow economy may exert an asymmetric impact on sovereign ratings and the cost of debt depending on their present magnitudes, not only independently but also in interaction.

Model-agnostic tools for the post-hoc interpretation of machine-learning models struggle to summarize the joint effects of strongly dependent features in high-dimensional feature spaces, which play an important role in semantic image classification, for example in remote sensing of landcover. This contribution proposes a novel approach that interprets machine-learning models through the lens of feature-space transformations. It can be used to enhance unconditional as well as conditional post-hoc diagnostic tools including partial-dependence plots, accumulated local effects (ALE) plots, permutation feature importance, or Shapley additive explanations (SHAP). While the approach can also be applied to nonlinear transformations, linear ones are particularly appealing, especially principal component analysis (PCA) and a proposed partial orthogonalization technique. Moreover, structured PCA and model diagnostics along user-defined synthetic features offer opportunities for representing domain knowledge. The new approach is implemented in the R package wiml , which can be combined with existing explainable machine-learning packages. A case study on remote-sensing landcover classification with 46 features is used to demonstrate the potential of the proposed approach for model interpretation by domain experts. It is most useful in situations where groups of feature are linearly dependent and PCA can provide meaningful multivariate data summaries.

The black-box nature of machine learning models hinders the deployment of some high-accuracy medical diagnosis algorithms. It is risky to put one’s life in the hands of models that medical researchers do not fully understand or trust. However, through model interpretation, black-box models can promptly reveal significant biomarkers that medical practitioners may have overlooked due to the surge of infected patients in the COVID-19 pandemic. This research leverages a database of 92 patients with confirmed SARS-CoV-2 laboratory tests between 18th January 2020 and 5th March 2020, in Zhuhai, China, to identify biomarkers indicative of infection severity prediction. Through the interpretation of four machine learning models, decision tree, random forests, gradient boosted trees, and neural networks using permutation feature importance, partial dependence plot, individual conditional expectation, accumulated local effects, local interpretable model-agnostic explanations, and Shapley additive explanation, we identify an increase in N-terminal pro-brain natriuretic peptide, C-reaction protein, and lactic dehydrogenase, a decrease in lymphocyte is associated with severe infection and an increased risk of death, which is consistent with recent medical research on COVID-19 and other research using dedicated models. We further validate our methods on a large open dataset with 5644 confirmed patients from the Hospital Israelita Albert Einstein, at São Paulo, Brazil from Kaggle, and unveil leukocytes, eosinophils, and platelets as three indicative biomarkers for COVID-19.

Academic research and the financial industry have recently paid great attention to Machine Learning algorithms due to their power to solve complex learning tasks. In the field of firms' default prediction, however, the lack of interpretability has prevented the extensive adoption of the black-box type of models. To overcome this drawback and maintain the high performances of black-boxes, this paper relies on a model-agnostic approach. Accumulated Local Effects and Shapley values are used to shape the predictors' impact on the likelihood of default and rank them according to their contribution to the model outcome. Prediction is achieved by two Machine Learning algorithms (eXtreme Gradient Boosting and FeedForward Neural Network) compared with three standard discriminant models. Results show that our analysis of the Italian Small and Medium Enterprises manufacturing industry benefits from the overall highest classification power by the eXtreme Gradient Boosting algorithm without giving up a rich interpretation framework.

A main drawback of eXplainable Artificial Intelligence (XAI) approaches is the feature independence assumption, hindering the study of potential variable dependencies. This leads to approximating black box behaviors by analyzing the effects on randomly generated feature values that may rarely occur in the original samples. This paper addresses this issue by integrating causal knowledge in an XAI method to enhance transparency and enable users to assess the quality of the generated explanations. Specifically, we propose a novel extension to a widely used local and model-agnostic explainer, which encodes explicit causal relationships within the data surrounding the instance being explained. Extensive experiments show that our approach overcomes the original method in terms of faithfully replicating the black-box model's mechanism and the consistency and reliability of the generated explanations.

This paper studies the problem of estimating the contributions of features to the prediction of a specific instance by a machine learning model and the overall contribution of a feature to the model. The causal effect of a feature (variable) on the predicted outcome reflects the contribution of the feature to a prediction very well. A challenge is that most existing causal effects cannot be estimated from data without a known causal graph. In this paper, we define an explanatory causal effect based on a hypothetical ideal experiment. The definition brings several benefits to model agnostic explanations. First, explanations are transparent and have causal meanings. Second, the explanatory causal effect estimation can be data driven. Third, the causal effects provide both a local explanation for a specific prediction and a global explanation showing the overall importance of a feature in a predictive model. We further propose a method using individual and combined variables based on explanatory causal effects for explanations. We show the definition and the method work with experiments on some real-world data sets.

The interpretation of feature importance in machine learning models is challenging when features are dependent. Permutation feature importance (PFI) ignores such dependencies, which can cause misleading interpretations due to extrapolation. A possible remedy is more advanced conditional PFI approaches that enable the assessment of feature importance conditional on all other features. Due to this shift in perspective and in order to enable correct interpretations, it is beneficial if the conditioning is transparent and comprehensible. In this paper, we propose a new sampling mechanism for the conditional distribution based on permutations in conditional subgroups. As these subgroups are constructed using tree-based methods such as transformation trees, the conditioning becomes inherently interpretable. This not only provides a simple and effective estimator of conditional PFI, but also local PFI estimates within the subgroups. In addition, we apply the conditional subgroups approach to partial dependence plots, a popular method for describing feature effects that can also suffer from extrapolation when features are dependent and interactions are present in the model. In simulations and a real-world application, we demonstrate the advantages of the conditional subgroup approach over existing methods: It allows to compute conditional PFI that is more true to the data than existing proposals and enables a fine-grained interpretation of feature effects and importance within the conditional subgroups.

Machine learning algorithms commonly outperform the traditional hedonic models in property valuation; however, it is hard to capture the inner workings of these complex models due to their black-box nature. To address the opaqueness of ML models, this study applies model-agnostic interpretability methods at the both global and local levels for a Random Forest model which provided better prediction accuracy than Support Vector Machines and eXtreme Gradient Boosting algorithms in predicting residential property prices. The results of this study suggest that interpretable ML methods can bring transparency to opaque ML models, and visualize feature effects and interactions in property valuation models.

Accurate electricity demand forecasts play a key role in sustainable power systems. To enable better decision-making especially for demand flexibility of the end-user, it is necessary to provide not only accurate but also understandable and actionable forecasts. To provide accurate forecasts Global Forecasting Models (GFM) that are trained across time series have shown superior results in many demand forecasting competitions and real-world applications recently, compared with univariate forecasting approaches. We aim to fill the gap between the accuracy and the interpretability in global forecasting approaches. In order to explain the global model forecasts, we propose Local Interpretable Model-agnostic Rule-based Explanations for Forecasting (LIMREF), which is a local explainer framework that produces k-optimal impact rules for a particular forecast, considering the global forecasting model as a black-box model, in a model-agnostic way. It provides different types of rules which explain the forecast of the global model and the counterfactual rules, which provide actionable insights for potential changes to obtain different outputs for given instances. We conduct experiments using a large-scale electricity demand dataset with exogenous features such as temperature and calendar effects. Here, we evaluate the quality of the explanations produced by the LIMREF framework in terms of both qualitative and quantitative aspects such as accuracy, fidelity and comprehensibility, and benchmark those against other local explainers.

Model-agnostic explainable AI tools explain their predictions by means of ’local’ feature contributions. We empirically investigate two potential improvements over current approaches. The first one is to always present feature contributions in terms of the contribution to the outcome that is perceived as positive by the user (“positive framing”). The second one is to add “semantic labeling”, that explains the directionality of each feature contribution (“this feature leads to +5% eligibility”), reducing additional cognitive processing steps. In a user study, participants evaluated the understandability of explanations for different framing and labeling conditions for loan applications and music recommendations. We found that positive framing improves understandability even when the prediction is negative. Additionally, adding semantic labels eliminates any framing effects on understandability, with positive labels outperforming negative labels. We implemented our suggestions in a package ArgueView[11].

Local Interpretable Model-agnostic Explanations (LIME) is an interpretable method used to explain the predictions of machine learning models. It generates perturbed samples around an instance and fits a simple surrogate model, such as linear regression, to approximate the local behavior of the black-box model. This paper introduces Additive Local Interpretable Model (ALIME), a LIME variant based on generalized additive models (GAMs), where feature effects are modeled by penalized splines (P-splines), providing flexibility for capturing nonlinear relationships. Experimental results show that ALIME outperforms the original LIME method in terms of local fidelity. In addition, the shape functions generated by ALIME can clearly capture local feature contributions, providing insight into the relationship between features and model outputs, and enhancing overall interpretability.

Saliency methods provide post-hoc model interpretation by attributing input features to the model outputs. Current methods mainly achieve this using a single input sample, thereby failing to answer input-independent inquiries about the model. We also show that input-specific saliency mapping is intrinsically susceptible to misleading feature attribution. Current attempts to use `general' input features for model interpretation assume access to a dataset containing those features, which biases the interpretation. Addressing the gap, we introduce a new perspective of input-agnostic saliency mapping that computationally estimates the high-level features attributed by the model to its outputs. These features are geometrically correlated, and are computed by accumulating model's gradient information with respect to an unrestricted data distribution. To compute these features, we nudge independent data points over the model loss surface towards the local minima associated by a human-understandable concept, e.g., class label for classifiers. With a systematic projection, scaling and refinement process, this information is transformed into an interpretable visualization without compromising its model-fidelity. The visualization serves as a stand-alone qualitative interpretation. With an extensive evaluation, we not only demonstrate successful visualizations for a variety of concepts for large-scale models, but also showcase an interesting utility of this new form of saliency mapping by identifying backdoor signatures in compromised classifiers.

The proposed work evaluates the performance of five machine learning algorithms XGBoost, RandomForest, GradientBoosting, Support Vector Machine (SVM), and Logistic Regression in predicting the effects of small interfering RNA (siRNA) on cellular viability and gene expression. Model performance was assessed using accuracy, precision, recall, F1-score, and feature importance, with interpretability facilitated by Shapley Additive Explanations (SHAP) and Local Interpretable Model-agnostic Explanations (LIME). XGBoost emerged as the most effective model, achieving 75% accuracy with robust identification of Class 0, though its precision for Class 1 remained suboptimal. GradientBoosting exhibited comparable accuracy but did not surpass XGBoost. RandomForest and Logistic Regression underperformed significantly, each achieving only 25% accuracy, while SVM demonstrated moderate performance with 50% accuracy, reflecting its limitations in handling non-linearities. SHAP and LIME analyses consistently identified GAPDH, NC siRNA, and ACP6 siRNA as critical features influencing model predictions. Despite XGBoost's strong performance, challenges such as class imbalance and limited interpretability were evident. Future work will prioritize hyperparameter optimization, advanced feature engineering, and techniques like SMOTE to address class imbalance. These findings underscore the potential of machine learning in evaluating siRNA-mediated effects while highlighting areas for methodological refinement to enhance predictive accuracy and interpretability in gene expression analysis.

Hyperkalemia is a major complication of chronic kidney disease (CKD). However, owing to the absence of specific symptoms in its early stages, hyperkalemia frequently remains undiagnosed. This study aimed to develop a machine learning model for predicting the risk of early hyperkalemia in patients with CKD. By conducting a comparative analysis of six machine learning methods, CatBoost demonstrated superiority across various evaluation metrics. Further evaluation using confusion matrix and decision curve analysis (DCA) confirmed its high classification accuracy and substantial clinical utility. Meanwhile, through multiple interpretability analyses based on SHAP and Local Interpretable Model-agnostic Explanations (LIME) techniques, we precisely quantify the contributions and positive or negative effects of risk factors for hyperkalemia.

Abstract Machine learning models have been widely used to obtain prediction in various domains; however, most of such models are black boxes owing to the high complexity. The lack of transparency of machine learning models hampers their applications because the practitioners do not understand the internal mechanism of these models. This study proposes a model-agnostic method, based on the high dimensional model representation (HDMR), to interpret supervised learning models by determining the local feature contribution. Compared to the existing methods, which only assign a single value to the feature contribution and do not consider the feature dependence, the HDMR-based feature contribution can be decomposed into individual and combined contribution, and it can take feature dependence into account. Certain agnostic and specific methods to measure the HDMR-based feature contributions are developed and categorized as pertaining to either feature independence or dependence. Experiments are performed to demonstrate the effects of the HDMR-based feature contributions, and compare the performance of several estimation methods.

Obesity is a significant health concern linked to severe medical conditions. Obesity increases the risk of diabetes, thyroid issues, cardiac disease, liver tumors, and stroke. Early prediction of obesity risk is essential for improving public health and well-being. To address this, an optimized explain-able CatBoost model, referred to as OptiX-CatBoost, has been developed. Initially, various machine learning (ML) models are trained using the publicly available dhaka obesity dataset. Among these, the categorical boosting (CatBoost) model emerged as the top performer and is selected for optimization. Bayesian optimization is employed to fine-tune key hyperparameters of the CatBoost model, enhancing its performance. The experimental results demonstrated that the OptiX-CatBoost model outperformed existing methods, achieving an accuracy of 95.60%, a precision of 94.60%, a recall of 93.12%, an F1-score of 98.15%, a Jaccard score of 96.14%, and a Kappa score of 87.14%. Additionally, the predictions made by the proposed model are further analyzed using explainable artificial intelligence (XAI) techniques. Healthcare professionals can evaluate the predictions of the proposed model and learn more about the different factors driving obesity by using XAI techniques such as partial dependence graphs, permutation significance, and LIME (Local Interpretable Model-agnostic Explanations). This dual capability of accurate prediction and interpretability renders OptiX-CatBoost a valuable tool for early obesity risk assessment, ultimately aiding in the development of better public health strategies and interventions.

Flooding can damage pavement infrastructure significantly, causing both immediate and long-term structural and functional issues. This research investigates how flooding events affect pavement deterioration, specifically focusing on measuring pavement roughness by the International Roughness Index (IRI). To quantify these effects, we utilized 20 years of pavement condition data from TxDOT’s PMIS database, which is integrated with flood event data, including duration and spatial extent. Statistical analyses were performed to compare IRI values before and after flooding and to calculate the deterioration rates influenced by flood exposure. Moreover, we applied explainable artificial intelligence (XAI) techniques, such as Shapley Additive Explanations (SHAP) and Local Interpretable Model-Agnostic Explanations (LIME), to assess the impact of flooding on pavement performance. The results demonstrate that flood-affected pavements experience a more rapid increase in roughness compared to non-flooded sections. These findings emphasize the need for proactive flood mitigation strategies, including improved drainage systems, flood-resistant materials, and preventative maintenance, to enhance pavement resilience in vulnerable regions.

No abstract available

No abstract available

No abstract available

In this paper we propose and study a class of simple, nonparametric, yet interpretable measures of conditional dependence between two random variables $Y$ and $Z$ given a third variable $X$, all taking values in general topological spaces. The population version of any of these measures captures the strength of conditional dependence and it is 0 if and only if $Y$ and $Z$ are conditionally independent given $X$, and 1 if and only if $Y$ is a measurable function of $Z$ and $X$. Thus, our measure -- which we call kernel partial correlation (KPC) coefficient -- can be thought of as a nonparametric generalization of the partial correlation coefficient that possesses the above properties when $(X,Y,Z)$ is jointly normal. We describe two consistent methods of estimating KPC. Our first method utilizes the general framework of geometric graphs, including $K$-nearest neighbor graphs and minimum spanning trees. A sub-class of these estimators can be computed in near linear time and converges at a rate that automatically adapts to the intrinsic dimension(s) of the underlying distribution(s). Our second strategy involves direct estimation of conditional mean embeddings using cross-covariance operators in the reproducing kernel Hilbert spaces. Using these empirical measures we develop forward stepwise (high-dimensional) nonlinear variable selection algorithms. We show that our algorithm, using the graph-based estimator, yields a provably consistent model-free variable selection procedure, even in the high-dimensional regime when the number of covariates grows exponentially with the sample size, under suitable sparsity assumptions. Extensive simulation and real-data examples illustrate the superior performance of our methods compared to existing procedures. The recent conditional dependence measure proposed by Azadkia and Chatterjee (2019) can be viewed as a special case of our general framework.

Data-driven approaches have accelerated materials discovery, yet they often remain "black boxes" that prioritize performance over physical understanding. To bridge the gap between statistical correlation and physical causality, this study establishes an interpretable machine learning (IML) framework applied to the sputter deposition of Mo-doped In2O3 thin films. Unlike conventional predictive models, our approach uses XGBoost regression combined with feature-importance analysis to quantitatively decouple the entangled effects of deposition parameters. Crucially, the model autonomously discovered─without explicit prior knowledge─that the carrier density is governed by oxygen partial pressure (PO2), while electron mobility is driven by crystallinity depending on the trade-off effects between adatom diffusion and high-energy particle bombardment. We experimentally validated these ML-derived hypotheses, identifying that the PO2 dependence stems from defect compensation by interstitial oxygen rather than simple oxygen vacancies and that the mobility peak corresponds to optimal crystallinity. This work demonstrates that IML can effectively "rediscover" governing physical laws from small experimental data sets, offering a scalable strategy to elucidate growth mechanisms in functional materials.

Accurate forecasting of blowing snow events is vital for improving numerical models of snow processes, yet traditional predictive methods often lack interpretability. This study leverages eXtreme Gradient Boosting (XGBoost) to detect blowing snow events using meteorological and snow flux monitoring data from three weather stations in the Alps. Through 5-fold cross-validation, the model achieved impressive performance metrics, with precision rates exceeding 0.94 for non-blowing snow events and 0.77-0.80 for blowing snow events. The SHAP framework was employed to analyze the relative importance of meteorological factors, revealing that maximum wind speed (WS-MAX), average wind speed (WS-AVG), air temperature (AT), and relative humidity (AH) are the most influential factors. Additionally, Partial dependence plots (PDP) demonstrated a linear correlation between increased WS-MAX and the probability of blowing snow, while WS-AVG showed diminishing returns beyond 10 m/s. Notably, AT below -3°C strongly correlates with blowing snow occurrence, whereas AT above -3°C exhibits a negative relationship. Relative humidity plays a significant role, with values exceeding 60% stabilizing the probability of blowing snow, peaking near 100%. This research contributes to drifting snow event dynamics by integrating explainable artificial intelligence techniques (XAI), thereby improving model interpretability and supporting data-driven decision-making in meteorological applications.

Explainable Artificial Intelligence (XAI) plays a crucial role in high-stakes decision-making by ensuring that machine learning models provide clear and trustworthy explanations. However, many existing interpretability methods, such as SHAP and Partial Dependence Plot (PDP), struggle to differentiate between correlation and causality. To overcome this challenge, we introduce a causality-aware surrogate modeling framework that improves the global interpretability of complex models. Our approach combines Probability of Sufficiency (PS), Probability of Necessity (PN), and Probability of Causality (PoC) with decision tree-based rule extraction to identify rules and features that have a direct causal impact on the target outcome. Experiments on the German Credit dataset reveal that certain rules exhibit strong sufficiency while showing weak necessity, highlighting key causal factors in loan approval decisions. Among these, savings and duration stand out as critical features for counterfactual reasoning. By ensuring that extracted rules capture true causal relationships rather than misleading correlations, our method enhances model transparency, trustworthiness, and counterfactual fundamentals.

Microplastic pollution is one of the challenges facing humanity, and the transport of microplastics in soils is a major limitation of traditional methods due to heterogeneity, complex particle-organic matter interactions, and inconsistent sampling protocols. To overcome these limitations, we present an integrated, mechanistically informed approach to soil Microplastics dynamics that combines experimental data with a machine learning model based on mixed physics, advanced statistical dependency analysis, and interpretability techniques. The framework, which uses TabNet for predictive modeling, is calibrated against an experimental dataset and reinforced with first-principles PDEs to ensure physical consistency. Statistical methods using Spearman's rho, Kendall's tau,distance correlation, HSIC, copula-based modeling, and Granger causality are employed, while interpretability is enhanced through SHAP, partial dependence plots, symbolic metamodeling, Double ML, and TCAV. The results show that density solution is one of the most influential parameters because it effectively acts as a latent and composite variable that integrates the interactions of all other inputs into a single, dominant indicator. Secondary factors, including land use (≈0.9-0.93), size range (≈0.77-0.86), sampling depth (≈0.73-0.81), and SOM operations (≈0.64-0.72), exert significant but context-dependent influence. Statistical dependency analyses further demonstrate nonlinear interactions, with Granger causality emphasizing the temporal and causal importance of density solution, land use, and size range.

In this study, a comprehensive framework combining Pearson correlation analysis, Q-learning-based feature selection, dimensionality reduction techniques, and ensemble learning was developed to assess water potability. Initially, Pearson correlation coefficients were used to evaluate the linear relationship between individual water quality indicators and drinkability, laying the groundwork for feature relevance assessment. To further enhance model performance, a Q-learning algorithm was introduced to formulate feature se-lection as a Markov Decision Process, automatically identifying an optimal subset of predictors. Subsequently, PCA and t-SNE were employed to reduce dimensionality and visualize sample clustering patterns in low-dimensional space. Random Forest and Logistic Regression models were then constructed to predict potability, and their decision boundaries and predictive responses were interpreted through Partial Dependence Plots. The results show that Solids, Conductivity, and Trihalomethanes have strong impacts on water safety assessment, and the integration of reinforcement learning significantly improves feature screening efficiency and model robustness. This hybrid approach provides valuable insights and a scalable strategy for future water quality monitoring.

Liquid rocket engines occasionally experience abnormal phenomena with unclear mechanisms, causing difficulty in design improvements. To address the above issue, a data mining method that combines ante hoc explainability, post hoc explainability, and prediction accuracy is proposed. For ante hoc explainability, a feature selection method driven by data, models, and domain knowledge is established. Global sensitivity analysis of a physical model combined with expert knowledge and data correlation is utilized to establish the correlations between different types of parameters. Then a two-stage optimization approach is proposed to obtain the best feature subset and train the prediction model. For the post hoc explainability, the partial dependence plot (PDP) and SHapley Additive exPlanations (SHAP) analysis are used to discover complex patterns between input features and the dependent variable. The effectiveness of the hybrid feature selection method and its applicability under different noise combinations are validated using synthesized data from a high-fidelity simulation model of a pressurization system. Then the analysis of the causes of a large vibration phenomenon in an active engine shows that the prediction model has good accuracy, and the feature selection results have a clear mechanism and align with domain knowledge, providing both accuracy and interpretability. The proposed method shows significant potential for data mining in complex aerospace products.

This study develops a data-driven predictive framework integrating hybrid feature selection, interpretable machine learning, and uncertainty quantification to forecast Olympic medal performance among elite nations. Focusing on the top ten countries from Paris 2024, the analysis employs a three-stage feature selection procedure combining Spearman correlation screening, random forest embedded importance, and the Caterpillar Fungus Optimizer (CFO) to identify stable long-term predictors. A novel test variable, rank, capturing historical competitive strength, and a refined continuous host-effect indicator derived from gravity-type trade models are introduced. Two complementary modeling strategies—a two-way fixed-effects econometric model and a CFO-optimized random forest—are implemented and validated. SHAP, LIME, and partial dependence plots enhance model interpretability, revealing nonlinear mechanisms underlying medal outcomes. Kernel density estimation generates probabilistic interval forecasts for Los Angeles 2028. Results demonstrate that historical performance and event-specific characteristics dominate medal predictions, while macroeconomic factors (GDP, population) and conventional host status contribute marginally once related variables are controlled. Consistent variable rankings across models and close alignment between 2028 projections and 2024 outcomes validate the framework’s robustness and practical applicability for sports policy and resource allocation decisions.

Artificial Intelligence (AI) and Machine Learning (ML) approaches that could learn from large data sources have been identified as useful tools to support clinicians in their decisional process; AI and ML implementations have had a rapid acceleration during the recent COVID-19 pandemic. However, many ML classifiers are “black box” to the final user, since their underlying reasoning process is often obscure. Additionally, the performance of such models suffers from poor generalization ability in the presence of dataset shifts. Here, we present a comparison between an explainable-by-design (“white box”) model (Bayesian Network (BN)) versus a black box model (Random Forest), both studied with the aim of supporting clinicians of Policlinico San Matteo University Hospital in Pavia (Italy) during the triage of COVID-19 patients. Our aim is to evaluate whether the BN predictive performances are comparable with those of a widely used but less explainable ML model such as Random Forest and to test the generalization ability of the ML models across different waves of the pandemic.

Brain Tumours are highly complex, particularly when it comes to their initial and accurate diagnosis, as this determines patient prognosis. Conventional methods rely on MRI and CT scans and employ generic machine learning techniques, which are heavily dependent on feature extraction and require human intervention. These methods may fail in complex cases and do not produce human-interpretable results, making it difficult for clinicians to trust the model’s predictions. Such limitations prolong the diagnostic process and can negatively impact the quality of treatment. The advent of deep learning has made it a powerful tool for complex image analysis tasks, such as detecting brain Tumours, by learning advanced patterns from images. However, deep learning models are often considered “black box” systems, where the reasoning behind predictions remains unclear. To address this issue, the present study applies Explainable AI (XAI) alongside deep learning for accurate and interpretable brain Tumour prediction. XAI enhances model interpretability by identifying key features such as Tumour size, location, and texture, which are crucial for clinicians. This helps build their confidence in the model and enables them to make better-informed decisions. In this research, a deep learning model integrated with XAI is proposed to develop an interpretable framework for brain Tumour prediction. The model is trained on an extensive dataset comprising imaging and clinical data and demonstrates high AUC while leveraging XAI for model explainability and feature selection. The study findings indicate that this approach improves predictive performance, achieving an accuracy of 92.98% and a miss rate of 7.02%. Additionally, interpretability tools such as LIME and Grad-CAM provide clinicians with a clearer understanding of the decision-making process, supporting diagnosis and treatment. This model represents a significant advancement in brain Tumour prediction, with the potential to enhance patient outcomes and contribute to the field of neuro-oncology.

No abstract available

Introduction Cardiovascular diseases (CVDs) are complex and affect a large part of the world’s population; early accurate and timely prediction is also complicated. Typically, predicting CVDs involves using statistical models and other forms of standard machine learning. Although these methods offer some level of prediction, their black-box nature severely hinders the ability of the healthcare professional to trust and use the predictions. The following are some of the challenges that Explainable Artificial Intelligence (XAI) may solve since it can give an understanding of the decision-making system of AI to build confidence and increase usability. Methods This research introduced an intelligent forecasting system for cardiovascular events using XAI and addressed the limitations of traditional methods. This proposed system incorporates advanced machine learning algorithms integrated with XAI to examine a dataset comprising 308,737 patient records with features including age, BMI, blood pressure, cholesterol levels, and lifestyle factors. This dataset was sourced from the Kaggle Cardiovascular Disease dataset. Results Incorporating XAI offers an understandable explanation so that the healthcare professional can understand and make the AI-driven prediction trustworthy enough to improve the decision-making of treatment and care delivery for the patients. The simulation results of the proposed system provide better results than those of the previously published research works in terms of 91.94% accuracy and 8.06% miss rate. Discussion This proposed system makes it clear that XAI has the potential to significantly improve cardiovascular healthcare by enhancing transparency, reliability, and the quality of patient care.

Diabetic retinopathy is a leading cause of vision complications and partially sighted which pose considerable diagnostic difficulties because of its diverse and varying symptoms. Some of them include the fact that the disease displays a non-uniform pattern, where patients present different symptoms; the requirement of highly qualified specialists to interpret the images of the fundus; the risk of errors in the interpretation of images or their inconsistency; and the absence of clear morphological signs often makes early diagnosis unlikely. Traditional diagnosis mostly rely on the expert interpretation of retinal images, which can lead to bias and inaccuracy; highlighting the need for improved diagnostic methods. Although traditional Artificial Intelligence (AI) methods enhance the diagnostic capabilities remarkably, their black box nature and information opacity restrict healthcare providers to comprehend the reasoning framework of the AI to build trust and optimize its usage in practice. Explainable AI (XAI) is an emerging approach that addresses the black-box problem by improving the interpretability of models, which allows users to understand the logic behind certain decisions. This research proposed a diagnosis model for detecting diabetic retinopathy using XAI approaches that increases the interpretability of the models to help clinicians understand the reasons behind the decisions. The proposed model is used to enhance diagnostic accuracy, offer comprehensible, and concise insights regarding the diagnostics. The convergence history plots of the proposed model validate the learning process to achieve 94% better diagnostic accuracy than traditional methods while improving interpretability and applicability in healthcare settings, indicating improvement in accuracy and loss reduction.

Deep learning (DL) models have demonstrated high accuracy in climate science applications but suffer from "black-box" opacity, hindering their adoption in environmental decision-making. This research bridges this gap by integrating Explainable AI (XAI) techniques with DL models to enhance transparency in climate predictions. Using a hybrid Convolutional Neural Network-Long Short-Term Memory (CNN-LSTM) architecture, we forecast regional temperature anomalies and interpret outputs via SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-agnostic Explanations). Our methodology is validated on ERA5 reanalysis data (1980–2025), achieving a prediction RMSE of 0.86°C. XAI analysis reveals that oceanic heat fluxes and atmospheric pressure patterns are critical drivers of anomalies. The framework empowers policymakers with actionable insights, ensuring DL models are both accurate and trustworthy for climate action.

The implementation of AI assisted cancer detection systems in clinical environments has faced numerous hurdles, mainly because of the restricted explainability of their elemental mechanisms, even though such detection systems have proven to be highly effective. Medical practitioners are skeptical about adopting AI assisted diagnoses as due to the latter’s inability to be transparent about decision making processes. In this respect, explainable artificial intelligence (XAI) has emerged to provide explanations for model predictions, thereby overcoming the computational black box problem associated with AI systems. In this particular research, the focal point has been the exploration of the Shapley additive explanations (SHAP) and local interpretable model-agnostic explanations (LIME) approaches which enable model prediction explanations. This study used an ensemble model consisting of three convolutional neural networks(CNN): InceptionV3, InceptionResNetV2 and VGG16, which was based on averaging techniques and by combining their respective predictions. These models were trained on the Kvasir dataset, which consists of pathological findings related to gastrointestinal cancer. An accuracy of 96.89% and F1-scores of 96.877% were attained by our ensemble model. Following the training of the ensemble model, we employed SHAP and LIME to analyze images from the three classes, aiming to provide explanations regarding the deterministic features influencing the model’s predictions. The results obtained from this analysis demonstrated a positive and encouraging advancement in the exploration of XAI approaches, specifically in the context of gastrointestinal cancer detection within the healthcare domain.

Opening the black box or Pandora's Box?

With the adoption of machine learning into routine clinical practice comes the need for Explainable AI methods tailored to medical applications. Shapley values have sparked wide interest for locally explaining models. Here, we demonstrate their interpretation strongly depends on both the summary statistic and the estimator for it, which in turn define what we identify as an 'anchor point'. We show that the convention of using a mean anchor point may generate misleading interpretations for survival analysis and introduce median-SHAP, a method for explaining black-box models predicting individual survival times.

Advances in machine learning and deep learning have assisted progress in stock market predictions, which have presented unique opportunities for investors and traders to benefit from predictions. However, understanding how these models work is crucial to trusting their predictions. This research applies three prominent Explainable AI (XAI) techniques - SHAP, LIME, and Permutation Importance - to three distinct forecasting models: Long Short-Term Memory (LSTM), Convolutional Neural Network (CNN), and Decision Tree. This work fills a gap in the literature by comparing XAI methods across a combination of “black-box” and “white-box” models. Two key research questions guide this research: the performance of these XAI methods across the chosen models and the consistency between global and local explanations across these models. Findings reveal interesting model-specific behaviors, like CNN's emphasis on slightly older data and LSTM's focus on immediate past data. In the S&P 500 dataset context, features such as ‘Close’ and ‘Adj Close’ prices emerged as significant across models, while ‘Volume’ remained insignificant. This study offers a broader perspective on applying XAI in financial time series forecasting and helps enhance trust and comprehension among stakeholders relying on these model predictions.

No abstract available

Simple Summary In the era of personalized medicine, Artificial Intelligence (AI) has emerged as a powerful tool with growing applications in the field of gynaecologic oncology. However, AI applications are encountered with several challenges derived from their “black-box” nature, which limits their adoption by clinicians. Surgical decision-making at cytoreductive surgery for epithelial ovarian cancer (EOC) is a complex matter, and an accurate prediction of surgical effort is required to ensure the good health and care of patients. We combined high-performance AI modeling with an eXplainable Artificial Intelligence (XAI) framework to explain feature effects and interactions associated with specific threshold surgical effort using data from a single public institution. We revealed features not routinely measured in the clinical practice, including human factors that could be responsible for the variation in the surgical effort. Selective decreased surgical effort may be associated with the surgeon’s age. The use of XAI frameworks can provide actionable information for surgeons to improve patient outcomes in gynaecologic oncology. Abstract (1) Background: Surgical cytoreduction for epithelial ovarian cancer (EOC) is a complex procedure. Encompassed within the performance skills to achieve surgical precision, intra-operative surgical decision-making remains a core feature. The use of eXplainable Artificial Intelligence (XAI) could potentially interpret the influence of human factors on the surgical effort for the cytoreductive outcome in question; (2) Methods: The retrospective cohort study evaluated 560 consecutive EOC patients who underwent cytoreductive surgery between January 2014 and December 2019 in a single public institution. The eXtreme Gradient Boosting (XGBoost) and Deep Neural Network (DNN) algorithms were employed to develop the predictive model, including patient- and operation-specific features, and novel features reflecting human factors in surgical heuristics. The precision, recall, F1 score, and area under curve (AUC) were compared between both training algorithms. The SHapley Additive exPlanations (SHAP) framework was used to provide global and local explainability for the predictive model; (3) Results: A surgical complexity score (SCS) cut-off value of five was calculated using a Receiver Operator Characteristic (ROC) curve, above which the probability of incomplete cytoreduction was more likely (area under the curve [AUC] = 0.644; 95% confidence interval [CI] = 0.598–0.69; sensitivity and specificity 34.1%, 86.5%, respectively; p = 0.000). The XGBoost outperformed the DNN assessment for the prediction of the above threshold surgical effort outcome (AUC = 0.77; 95% [CI] 0.69–0.85; p < 0.05 vs. AUC 0.739; 95% [CI] 0.655–0.823; p < 0.95). We identified “turning points” that demonstrated a clear preference towards above the given cut-off level of surgical effort; in consultant surgeons with <12 years of experience, age <53 years old, who, when attempting primary cytoreductive surgery, recorded the presence of ascites, an Intraoperative Mapping of Ovarian Cancer score >4, and a Peritoneal Carcinomatosis Index >7, in a surgical environment with the optimization of infrastructural support. (4) Conclusions: Using XAI, we explain how intra-operative decisions may consider human factors during EOC cytoreduction alongside factual knowledge, to maximize the magnitude of the selected trade-off in effort. XAI techniques are critical for a better understanding of Artificial Intelligence frameworks, and to enhance their incorporation in medical applications.

This paper presents a neural network-based classifier to predict whether a person is at risk of developing chronic kidney disease (CKD). The model is trained with the demographic data and medical care information of two population groups: on the one hand, people diagnosed with CKD in Colombia during 2018, and on the other, a sample of people without a diagnosis of this disease. Once the model is trained and evaluation metrics for classification algorithms are applied, the model achieves 95% accuracy in the test data set, making its application for disease prognosis feasible. However, despite the demonstrated efficiency of the neural networks to predict CKD, this machine-learning paradigm is opaque to the expert regarding the explanation of the outcome. Current research on eXplainable AI proposes the use of twin systems, where a black-box machine-learning method is complemented by another white-box method that provides explanations about the predicted values. Case-Based Reasoning (CBR) has proved to be an ideal complement as this paradigm is able to find explanatory cases for an explanation-by-example justification of a neural network’s prediction. In this paper, we apply and validate a NN-CBR twin system for the explanation of CKD predictions. As a result of this research, 3,494,516 people were identified as being at risk of developing CKD in Colombia, or 7% of the total population.

BACKGROUND AND OBJECTIVE In everyday clinical practice, medical decision is currently based on clinical guidelines which are often static and rigid, and do not account for population variability, while individualized, patient-oriented decision and/or treatment are the paradigm change necessary to enter into the era of precision medicine. Most of the limitations of a guideline-based system could be overcome through the adoption of Clinical Decision Support Systems (CDSSs) based on Artificial Intelligence (AI) algorithms. However, the black-box nature of AI algorithms has hampered a large adoption of AI-based CDSSs in clinical practice. In this study, an innovative AI-based method to compress AI-based prediction models into explainable, model-agnostic, and reduced decision support systems (NEAR) with application to healthcare is presented and validated. METHODS NEAR is based on the Shapley Additive Explanations framework and can be applied to complex input models to obtain the contributions of each input feature to the output. Technically, the simplified NEAR models approximate contributions from input features using a custom library and merge them to determine the final output. Finally, NEAR estimates the confidence error associated with the single input feature contributing to the final score, making the result more interpretable. Here, NEAR is evaluated on a clinical real-world use case, the mortality prediction in patients who experienced Acute Coronary Syndrome (ACS), applying three different Machine Learning/Deep Learning models as implementation examples. RESULTS NEAR, when applied to the ACS use case, exhibits performances like the ones of the AI-based model from which it is derived, as in the case of the Adaptive Boosting classifier, whose Area Under the Curve is not statistically different from the NEAR one, even the model's simplification. Moreover, NEAR comes with intrinsic explainability and modularity, as it can be tested on the developed web application platform (https://neardashboard.pythonanywhere.com/). CONCLUSIONS An explainable and reliable CDSS tailored to single-patient analysis has been developed. The proposed AI-based system has the potential to be used alongside the clinical guidelines currently employed in the medical setting making them more personalized and dynamic and assisting doctors in taking their everyday clinical decisions.

Primary malignancies in adult brains are globally fatal. Computer vision, especially recent developments in artificial intelligence (AI), have created opportunities to automatically characterize and diagnose tumor lesions in the brain. AI approaches have provided scores of unprecedented accuracy in different image analysis tasks, including differentiating tumor-containing brains from healthy brains. AI models, however, perform as a black box, concealing the rational interpretations that are an essential step towards translating AI imaging tools into clinical routine. An explainable AI approach aims to visualize the high-level features of trained models or integrate into the training process. This study aims to evaluate the performance of selected deep-learning algorithms on localizing tumor lesions and distinguishing the lesion from healthy regions in magnetic resonance imaging contrasts. Despite a significant correlation between classification and lesion localization accuracy (R = 0.46, p = 0.005), the known AI algorithms, examined in this study, classify some tumor brains based on other non-relevant features. The results suggest that explainable AI approaches can develop an intuition for model interpretability and may play an important role in the performance evaluation of deep learning models. Developing explainable AI approaches will be an essential tool to improve human–machine interactions and assist in the selection of optimal training methods.

Fairness is steadily becoming a crucial requirement of Machine Learning (ML) systems. A particularly important notion is subgroup fairness, i.e., fairness in subgroups of individuals that are defined by more than one attributes. Identifying bias in subgroups can become both computationally challenging, as well as problematic with respect to comprehensibility and intuitiveness of the finding to end users. In this work we focus on the latter aspects; we propose an explainability method tailored to identifying potential bias in subgroups and visualizing the findings in a user friendly manner to end users. In particular, we extend the ALE plots explainability method, proposing FALE (Fairness aware Accumulated Local Effects) plots, a method for measuring the change in fairness for an affected population corresponding to different values of a feature (attribute). We envision FALE to function as an efficient, user friendly, comprehensible and reliable first-stage tool for identifying subgroups with potential bias issues.

Global feature effect methods, such as partial dependence plots, provide an intelligible visualization of the expected marginal feature effect. However, such global feature effect methods can be misleading, as they do not represent local feature effects of single observations well when feature interactions are present. We formally introduce generalized additive decomposition of global effects (GADGET), which is a new framework based on recursive partitioning to find interpretable regions in the feature space such that the interaction-related heterogeneity of local feature effects is minimized. We provide a mathematical foundation of the framework and show that it is applicable to the most popular methods to visualize marginal feature effects, namely partial dependence, accumulated local effects, and Shapley additive explanations (SHAP) dependence. Furthermore, we introduce and validate a new permutation-based interaction detection procedure that is applicable to any feature effect method that fits into our proposed framework. We empirically evaluate the theoretical characteristics of the proposed methods based on various feature effect methods in different experimental settings. Moreover, we apply our introduced methodology to three real-world examples to showcase their usefulness.

Digital collaborative education plays a pivotal role in digital education research and significantly contributes to enhancing teaching quality. Furthermore, it provides a new impetus for family–school–community collaboration in talent development. Nevertheless, the key drivers and predictive models of digital collaborative education remain underexplored. To address this gap, this study adopts the perspective of teachers' digital literacy, focusing on primary and secondary school teachers as research subjects. Employing machine learning methods such as gradient boosting regression trees (GBRT) and random forest, we identify the key factors influencing digital collaborative education and develop predictive models. The SHapley Additive exPlanations (SHAP) framework is applied to conduct holistic, heterogeneous, and individual‐level explanatory analyses, whereas accumulated local effects (ALE) plots are used for single‐feature explanation. The results indicate that random forest outperforms other models in predicting digital collaborative education. Digital academic assessment, digital instructional implementation, digital teaching design, and digital instructional research and innovation are the four most important feature variables in predicting the effectiveness of digital collaborative education, with digital application emerging as the strongest predictor, followed by professional development. These key features exhibit heterogeneity in predicting digital collaborative education across gender, age, educational background, and teaching experience, demonstrating nonlinear relationships. The findings provide empirical support for advancing digital collaborative education and offer valuable insights for enhancing teachers' professional development.

Accurate prediction of the bearing capacity of double shear-bolted connections in structural steel is essential for ensuring safety and efficiency in structural design. This study explores the application of ten machine learning algorithms to enhance prediction accuracy while addressing the interpretability challenges often associated with such models. Models were tuned with 10-fold crossvalidation and assessed using RMSE, R 2 and a20 accuracy index. A comprehensive sensitivity analysis evaluates the influence of input parameters, while advanced interpretability techniques, such as partial dependence plots, accumulated local effects, and Shapley additive explanations, are employed alongside parametric studies to elucidate the decision-making processes of the models. These methods facilitate the identification of critical variables that influence bearing capacity predictions at both local and global scales. The study demonstrates that machine learning can be a trustworthy and data-driven complement to conventional mechanics-based approaches, when coupled with rigorous interpretability, advancing both safety and efficiency in steelconnection design. The findings highlight the potential of interpretable machine learning approaches to not only improve predictive precision but also provide actionable insights into complex model behaviours, ultimately advancing structural engineering practices and promoting data-driven design methodologies.

Effector is a Python package for interpreting machine learning (ML) models that are trained on tabular data through global and regional feature effects. Global effects, like Partial Dependence Plot (PDP) and Accumulated Local Effects (ALE), are widely used for explaining tabular ML models due to their simplicity -- each feature's average influence on the prediction is summarized by a single 1D plot. However, when features are interacting, global effects can be misleading. Regional effects address this by partitioning the input space into disjoint subregions with minimal interactions within each and computing a separate regional effect per subspace. Regional effects are then visualized by a set of 1D plots per feature. Effector provides efficient implementations of state-of-the-art global and regional feature effects methods under a unified API. The package integrates seamlessly with major ML libraries like scikit-learn and PyTorch. It is designed to be modular and extensible, and comes with comprehensive documentation and tutorials. Effector is an open-source project publicly available on Github at https://github.com/givasile/effector.

Despite the urgent need for ambitious national climate policies to reduce carbon emissions, their implementation lacks stringency. This lack of policy stringency is driven by a complex combination of a country’s numerous politico-economic, institutional and socio-economic characteristics. While extant studies aim at estimating causal effects between a selection of such characteristics and policy stringency, we examine the importance of a comprehensive set of predictors that underlie such empirical models. For this purpose, we employ machine-learning methods on a data set covering 22 potential predictors of policy stringency for 95 countries. Conditional random forests suggest that the most important predictors of policy stringency are of institutional nature: freedom (of press, media, associations, and elections), governmental effectiveness, and control of corruption. Further, accumulated local effects plots suggest that the relationship between some predictors, e.g. freedom or education, and policy stringency is highly non-linear.

We study the robustness of global post-hoc explanations for predictive models trained on tabular data. Effects of predictor features in black-box supervised learning are an essential diagnostic tool for model debugging and scientific discovery in applied sciences. However, how vulnerable they are to data and model perturbations remains an open research question. We introduce several theoretical bounds for evaluating the robustness of partial dependence plots and accumulated local effects. Our experimental results with synthetic and real-world datasets quantify the gap between the best and worst-case scenarios of (mis)interpreting machine learning predictions globally.

Due to the wealth of exposome data from longitudinal cohort studies that is currently available, the need for methods to adequately analyze these data is growing. We propose an approach in which machine learning is used to identify longitudinal exposome-related predictors of health, and illustrate its potential through an application. Our application involves studying the relation between exposome and self-perceived health based on the 30-year running Doetinchem Cohort Study. Random Forest (RF) was used to identify the strongest predictors due to its favorable prediction performance in prior research. The relation between predictors and outcome was visualized with partial dependence and accumulated local effects plots. To facilitate interpretation, exposures were summarized by expressing them as the average exposure and average trend over time. The RF model’s ability to discriminate poor from good self-perceived health was acceptable (Area-Under-the-Curve = 0.707). Nine exposures from different exposome-related domains were largely responsible for the model’s performance, while 87 exposures seemed to contribute little to the performance. Our approach demonstrates that ML can be interpreted more than widely believed, and can be applied to identify important longitudinal predictors of health over the life course in studies with repeated measures of exposure. The approach is context-independent and broadly applicable.

Interpretable machine learning tools including LIME (Local Interpretable Model-agnostic Explanations) and ALE (Accumulated Local Effects) are incorporated into a transformer-based deep learning model, named SolarFlareNet, to interpret the predictions made by the model.  SolarFlareNet is implemented into an operational flare forecasting system to predict whether an active region on the surface of the Sun would produce a >=M class flare within the next 24 hours. LIME determines the ranking of the features used by SolarFlareNet. 2D ALE plots identify the interaction effects of two features on the predictions. Together, these tools help scientists better understand which features are crucial for SolarFlareNet to make its predictions. Experiments show that the tools can explain the internal workings of SolarFlareNet while maintaining its accuracy.

Background Self-perceived general health (SPGH) is a general health indicator commonly used in epidemiological research and is associated with a wide range of exposures from different domains. However, most studies on SPGH only investigated a limited set of exposures and did not take the entire external exposome into account. We aimed to develop predictive models for SPGH based on exposome datasets using machine learning techniques and identify the most important predictors of poor SPGH status. Methods Random forest (RF) was used on two datasets based on personal characteristics from the 2012 and 2016 editions of the Dutch national health survey, enriched with environmental and neighborhood characteristics. Model performance was determined using the area under the curve (AUC) score. The most important predictors were identified using a variable importance procedure and individual effects of exposures using partial dependence and accumulated local effect plots. The final 2012 dataset contained information on 199,840 individuals and 81 variables, whereas the final 2016 dataset had 244,557 individuals with 91 variables. Results Our RF models had overall good predictive performance (2012: AUC = 0.864 (CI: 0.852–0.876); 2016: AUC = 0.890 (CI: 0.883–0.896)) and the most important predictors were “Control of own life”, “Physical activity”, “Loneliness” and “Making ends meet”. Subjects who felt insufficiently in control of their own life, scored high on the De Jong-Gierveld loneliness scale or had difficulty in making ends meet were more likely to have poor SPGH status, whereas increased physical activity per week reduced the probability of poor SPGH. We observed associations between some neighborhood and environmental characteristics, but these variables did not contribute to the overall predictive strength of the models. Conclusions This study identified that within an external exposome dataset, the most important predictors for SPGH status are related to mental wellbeing, physical exercise, loneliness, and financial status.

squares (PRESS) and based on it coefficient of prediction Q2. The measures for a binary outcome models based on confusion matrix include accuracy, precision, recall, sensitivity and specificity, Receiver Operating Characteristic (ROC) curve and the area under the curve (AUC), Gini coefficient and lift. Similar measures and Chi-square statistics are used for a categorical dependent variable. Chapter 16 describes permutationbased methods for evaluation of an explanatory-variable’s importance with help of the random forest and generalized boosted regression techniques. Chapter 17 describes the partialdependence (PD) plots, aka PD profiles, introduced in the context of gradient boosting machine (GBM) and implemented in many data-science oriented packages. Clustered, contrastive, and grouped PD profiles are discussed. Chapter 18 continues with the local dependence and accumulated-local dependence profiles for more complicated models with interaction and correlated predictors, finding the univariate variable’s effects by the incremental calculus estimations. Chapter 19 describes presents residual-diagnostics plots for various predictive models. Chapter 20 summarizes the data-level global exploration by different methods. The Titanic and apartment prices datasets are used in examples of this part. Part IV of Use-cases, in its Chapter 21 applies and compares all the considered approaches to a dataset of the Fédération Internationale de Football Association (FIFA). The characteristics of 16,924 soccer players on 24 variables are used to build a predictive model for evaluation of a player value, producing various their features, for instance, Shapley value for Cristiano Ronaldo. Chapter 22 states that all the examples and R and Python packages are reproducible. They are available at the GitHub repository https://github.com/pbiecek/ema, and Explanatory Model Analysis (drwhy.ai) as well. Bibliography on a dozen pages with the most recent sources and a detail index finalize the book. The book presents a valuable collection of methods for models’ exploration and diagnostics for various machine learning algorithms. It can be useful in the data and computer science courses for students and instructors, as well as for researchers and practitioners who need to analyze and interpret their statistical and machine learning models both of glass-box and blackbox kind. The book also serves as a great primary for applications of the R and Python software and their packages/libraries, so it is valuable in solving various problems of statistical prediction in various fields. Some additional sources on the considered topics can be found in the references (Lipovetsky 2014, 2020a,b, 2021a,b,c, 2022a,b).

The present study explored the interrelations between a broad set of appraisal ratings and five physiological signals, including facial EMG, electrodermal activity, and heart rate variability, that were assessed in 157 participants watching 10 emotionally charged videos. A total of 134 features were extracted from the physiological data, and a benchmark comparing different kinds of machine learning algorithms was conducted to test how well the appraisal dimensions can be predicted from these features. For 13 out of 21 appraisals, a robust positive R2 was attained, indicating that the dimensions are actually related to the considered physiological channels. The highest R2 (.407) was reached for the appraisal dimension intrinsic pleasantness. Moreover, the comparison of linear and nonlinear algorithms and the inspection of the links between the appraisals and single physiological features using accumulated local effects plots indicates that the relationship between physiology and appraisals is nonlinear. By constructing different importance measures for the assessed physiological channels, we showed that for the 13 predictable appraisals, the five channels explained different amounts of variance and that only a few blocks incrementally explained variance beyond the other physiological channels.

This paper examines objective purchasing information for inherently seasonal self-medication product groups using transaction-level loyalty card records. Predictive models are applied to predict future monthly self-medication purchasing. Analyses are undertaken at the lower super output area level, allowing the exploration of ˜300 retail, social, demographic and environmental predictors of purchasing. The study uses a tree ensemble predictive algorithm, applying XGBoost using one year of historical training data to predict future purchase patterns. The study compares static and dynamic retraining approaches. Feature importance rank comparison and accumulated local effects plots are used to ascertain insights of the influence of different features. Clear purchasing seasonality is observed for both outcomes, reflecting the climatic drivers of the associated minor ailments. Although dynamic models perform best, where previous year behaviour differs greatly, predictions had higher error rates. Important features are consistent across models (eg previous sales, temperature, seasonality). Feature importance ranking had the greatest difference where seasons changed. Accumulated local effects plots highlight specific ranges of predictors influencing self-medication purchasing. Loyalty card records offer promise for monitoring the prevalence of minor ailments and reveal insights about the seasonality and drivers of over-the-counter medicine purchasing in England.

Atmospheric fine particles (PM2.5) have been found to be harmful to the environment and human health. Recently, remote sensing technology and machine learning models have been used to monitor PM2.5 concentrations. Partial dependence plots (PDP) were used to explore the meteorology mechanisms between predictor variables and PM2.5 concentration in the “black box” models. However, there are two key shortcomings in the original PDP. (1) it calculates the marginal effect of feature(s) on the predicted outcome of a machine learning model, therefore some local effects might be hidden. (2) it requires that the feature(s) for which the partial dependence is computed are not correlated with other features, otherwise the estimated feature effect has a great bias. In this study, the original PDP’s shortcomings were analyzed. Results show the contradictory correlation between the temperature and the PM2.5 concentration that can be given by the original PDP. Furthermore, the spatiotemporal heterogeneity of PM2.5-AOD relationship cannot be displayed well by the original PDP. The drawbacks of the original PDP make it unsuitable for exploring large-area feature effects. To resolve the above issue, multi-way PDP is recommended, which can characterize how the PM2.5 concentrations changed with the temporal and spatial variations of major meteorological factors in China.

In high-stakes scientific contexts, explainable AI is crucial for deriving meaningful insights from complex tabular data. A formidable challenge is ensuring both rigorous statistical guarantees and clear interpretability in feature extraction. While traditional methods like principal component analysis are limited by linear assumptions, powerful neural network approaches often lack the transparency required in scientific domains. To address this gap, we introduce Spofe, a novel self-supervised learning pipeline that makes nonlinear feature interactions interpretable. Spofe marries the power of kernel principal components (KPCs) for capturing complex dependencies with a sparse, principled polynomial representation to achieve clear interpretability with statistical rigor. Our approach bridges data-driven complexity and statistical reliability via three stages. First, it generates self-supervised signals using KPCAs to model complex patterns. Second, it distills these signals into sparse polynomial functions for interpretability. Third, it constructs the Spofe statistic by aggregating knockoff feature-importance scores across self-supervised signals, and applies a data-adaptive threshold to identify significant polynomial interactions with rigorous false discovery rate control. Extensive experiments on diverse real-world datasets demonstrate the effectiveness of Spofe, achieving competitive or superior performance relative to other methods in feature selection for regression and classification tasks. In particular, applications on physics datasets highlight the ability of the proposed method to produce scientifically valid and interpretable explanations, reinforcing its practical utility and the critical role of explainability in AI for science.