SHAP - A Practical Guide to Explainable AI and Model Interpretability
Machine learning models increasingly drive decisions that affect people: loan approvals, medical triage, fraud flags, and churn interventions. When a model says "no," stakeholders reasonably ask why. SHAP (SHapley Additive exPlanations) gives a principled, consistent answer by attributing each prediction to the features that produced it.
This tutorial is a dedicated deep dive into SHAP: the theory behind it, the modern API, the right explainer for each model type, every core plot and how to read it, and the pitfalls that trip up teams in production.
Table of Contents
Why Interpretability Matters
A model that scores well on a held-out set is not automatically trustworthy. Interpretability addresses four concrete needs that show up in real projects.
- Trust and adoption. Domain experts adopt a model faster when they can see that it relies on sensible signals rather than spurious correlations. A credit analyst is more comfortable acting on a score when the drivers match their mental model.
- Debugging. Explanations expose leakage and shortcut learning. If a "future" timestamp column dominates a fraud model, the explanation surfaces it long before a postmortem does.
- Fairness. By inspecting how protected or proxy attributes contribute to predictions, teams can detect and quantify undesirable behavior.
- Regulation. Frameworks such as the EU's GDPR right-to-explanation, the EU AI Act, and financial supervisory guidance increasingly expect documented, reproducible reasoning behind automated decisions.
SHAP does not make a model fair or correct on its own. It is a measurement tool that tells you, faithfully, what the model is doing so you can decide whether that behavior is acceptable.
Global vs Local Explanations
Interpretability questions come in two flavors, and SHAP answers both with the same underlying quantity.
- Local explanation: "Why did the model produce this prediction for this customer?" The answer is the per-feature contribution for a single row.
- Global explanation: "Which features matter most across the whole dataset, and in which direction?" The answer is obtained by aggregating local explanations over many rows.
A useful property of SHAP is that the global view is literally a summary of local views. Mean absolute SHAP value per feature gives global importance; the sign and spread of per-row values describe direction and heterogeneity. There is no separate, inconsistent "global importance" metric to reconcile.
The Intuition Behind Shapley Values
Shapley values come from cooperative game theory (Lloyd Shapley, 1953). Imagine a game where several players cooperate to produce a payout, and you need to split that payout fairly. The Shapley value of a player is their average marginal contribution across every possible order in which players could join the coalition.
Map this onto a prediction: the "players" are the input features, the "payout" is the model's output for a given row, and a feature's Shapley value is how much, on average, including that feature changes the prediction as features are added in every possible order.
The reason this matters is that Shapley values are the unique attribution method satisfying a set of fairness axioms simultaneously:
- Efficiency: the feature contributions sum exactly to the difference between the prediction and a baseline.
- Symmetry: two features that contribute identically receive equal credit.
- Dummy: a feature that never changes the output gets zero.
- Additivity: contributions combine consistently across ensembles of models.
Exact Shapley computation considers all feature subsets, which is exponential. SHAP's contribution is a set of efficient algorithms that compute or approximate these values for specific model classes, most notably an exact polynomial-time algorithm for tree ensembles.
What a SHAP Value Means
A SHAP value is an additive contribution measured against a base value. For a single prediction:
prediction = basevalue + sum(shapvaluei for every feature i)
The basevalue (also called the expected value) is the average model output over the background dataset. Each shapvaluei is the signed amount that feature i pushed the prediction above or below that baseline. Positive values push the output up; negative values push it down.
This additive decomposition is what makes SHAP both interpretable and verifiable: you can always add the contributions back to the base value and recover the exact model output. The units match the model output. For a regression model the SHAP values are in target units; for a classifier they are typically in log-odds (margin) space unless you explicitly explain probabilities.
Installation
SHAP is published on PyPI and works with NumPy, pandas, scikit-learn, XGBoost, LightGBM, and the major deep learning frameworks.
pip install shap
For this tutorial we also use a few standard libraries:
pip install scikit-learn xgboost pandas matplotlib
SHAP renders interactive force plots using JavaScript. In a Jupyter notebook, initialize the JS once per session:
import shap
shap.initjs()
Static plots based on Matplotlib work without initjs() and are preferable for saving figures to disk or embedding in reports.
The Unified API and Specialized Explainers
Modern SHAP centers on one entry point, shap.Explainer, which inspects the model and the data and chooses an appropriate algorithm automatically.
import shap
explainer = shap.Explainer(model, masker)
explanation = explainer(X)
The call returns an Explanation object. This is the recommended container: it bundles the .values (SHAP values), .basevalues, and .data (the feature values that were explained) together, plus feature names. Most modern plotting functions accept an Explanation directly, so you rarely manipulate raw arrays.
The masker controls how features are "removed" when computing contributions. For tabular data the masker is usually a background dataset (it defines the baseline distribution); for text it is a tokenizer; for images it is a region-masking strategy.
Under the hood, shap.Explainer dispatches to a specialized explainer. You can also instantiate these directly when you want explicit control:
TreeExplainer- fast and exact for tree ensembles: XGBoost, LightGBM, CatBoost, scikit-learnRandomForestand gradient boosting. This is the workhorse for tabular ML and should be your default for tree models.LinearExplainer- for linear and logistic regression. It accounts for feature correlations in the background data and is essentially closed-form.KernelExplainer- model-agnostic. It works for any model that exposes a predict function but uses weighted local linear regression and is slow, scaling poorly with feature count. Use it only when no faster explainer applies.DeepExplainer/GradientExplainer- for neural networks (TensorFlow/Keras, PyTorch).DeepExplaineradapts DeepLIFT;GradientExplaineruses expected gradients. Both exploit backpropagation for efficiency.PartitionExplainerand maskers - for structured inputs like text and images, where features are correlated and hierarchically grouped. The unifiedshap.Explaineruses this automatically with a text or image masker.
A practical rule: match the explainer to the model. Use TreeExplainer for trees, LinearExplainer for linear models, the deep explainers for nets, and fall back to KernelExplainer only as a last resort.
A Complete Tabular Example
We will use a single coherent example throughout: predicting customer churn from a tabular dataset. To keep the tutorial self-contained and reproducible, we generate a realistic synthetic dataset with named, interpretable features.
import numpy as np
import pandas as pd
from sklearn.modelselection import traintestsplit
import xgboost as xgb
rng = np.random.defaultrng(42)
n = 4000
tenuremonths = rng.integers(1, 72, n)
monthlycharges = rng.normal(70, 25, n).clip(15, 150)
supporttickets = rng.poisson(1.2, n)
isfiber = rng.integers(0, 2, n)
hascontract = rng.integers(0, 2, n)
Construct a churn signal: short tenure, high charges, many tickets,
and month-to-month (no contract) increase churn risk.
logit = (
-0.04 tenuremonths
- 0.015 monthlycharges
- 0.35 supporttickets
- 1.2 hascontract
- 0.3 isfiber
- 0.5
)
prob = 1 / (1 + np.exp(-logit))
churn = (rng.random(n) < prob).astype(int)
X = pd.DataFrame({
"tenuremonths": tenuremonths,
"monthlycharges": monthlycharges,
"supporttickets": supporttickets,
"isfiber": isfiber,
"hascontract": hascontract,
})
y = pd.Series(churn, name="churn")
Xtrain, Xtest, ytrain, ytest = traintestsplit(
X, y, testsize=0.25, randomstate=42, stratify=y
)
Train an XGBoost classifier:
model = xgb.XGBClassifier(
nestimators=300,
maxdepth=4,
learningrate=0.05,
subsample=0.9,
colsamplebytree=0.9,
evalmetric="logloss",
randomstate=42,
)
model.fit(Xtrain, ytrain)
Build the explainer and compute SHAP values. For tree models, shap.Explainer automatically selects TreeExplainer:
import shap
explainer = shap.Explainer(model, Xtrain)
shapvalues = explainer(Xtest)
print(type(shapvalues)) # explanation.Explanation'>
print(shapvalues.shape) # (ntest, nfeatures)
print(shapvalues.basevalues[0])
Verify the additivity property, which is a good sanity check that the explainer is wired correctly. SHAP values for a binary XGBoost classifier are in margin (log-odds) space, so they reconstruct the raw margin, not the probability:
margin = model.predict(Xtest, outputmargin=True)
reconstructed = shap
values.basevalues + shapvalues.values.sum(axis=1)
print(np.allclose(margin, reconstructed, atol=1e-4)) # True
If you prefer to attribute probabilities directly, pass a probability-space model output and a background dataset:
probexplainer = shap.Explainer(
model.predict
proba, Xtrain, featurenames=X.columns
)
This uses the model-agnostic path and is slower; sample the background.
For tree models, staying in margin space is the fast, exact, and recommended default.
Reading the Core Plots
Each plot answers a different question. Knowing which to reach for is most of the skill.
Beeswarm Summary Plot - Global Importance and Direction
The beeswarm plot is the single most informative global view. Each dot is one row's SHAP value for one feature; color encodes the feature value (red high, blue low).
shap.plots.beeswarm(shapvalues)
How to read it: features are ranked top to bottom by mean absolute SHAP value (importance). For each feature, look at the horizontal spread and the color gradient. In our churn model, tenuremonths typically shows red dots (high tenure) on the left (negative SHAP, lower churn) and blue dots (low tenure) on the right - confirming that short tenure raises churn risk, in the direction we designed.
Bar Plot - Mean Absolute SHAP
For a clean importance ranking without direction, use the bar plot. The bar length is the mean of the absolute SHAP values across all explained rows.
shap.plots.bar(shapvalues)
Use the bar plot in slide decks and executive summaries; use the beeswarm when direction and interaction matter.
Waterfall - A Single Prediction
The waterfall plot decomposes one prediction. It starts at the base value and adds each feature's contribution until it reaches the model output for that row.
shap.plots.waterfall(shapvalues[0])
This is the plot to show when someone asks "why did this customer get flagged?" Each bar is a feature's signed push; positive bars increase churn risk, negative bars reduce it, and they sum to the final margin.
Force Plot - A Compact Single Prediction
The force plot conveys the same single-row decomposition as the waterfall but as a horizontal pushed-balance visualization.
shap.initjs()
shap.plots.force(shapvalues[0])
Force plots are interactive in notebooks and can be stacked for many rows to give a "global force" overview. For static reports, the waterfall is usually clearer.
Dependence / Scatter - Feature Effect and Interaction
The scatter plot shows how a feature's SHAP value changes with its value, revealing the shape of the learned effect and, through coloring, interactions with a second feature.
shap.plots.scatter(shapvalues[:, "monthlycharges"],
color=shap
values[:, "hascontract"])
A rising trend means higher charges increase churn contribution. If the cloud splits into two colored bands, the effect of monthlycharges depends on whether the customer has a contract - an interaction.
Decision Plot - Cumulative Path
The decision plot draws the cumulative SHAP path for one or many rows, from the base value to the output, line by line across features.
shap.decisionplot(
shap
values.basevalues[0],
shap
values.values[:20],
featurenames=list(X.columns),
)
It is useful for comparing several similar cases at once and for spotting where predictions diverge.
Heatmap - Instance-Level Patterns
The heatmap plot lays out many instances side by side, with features on the vertical axis and rows on the horizontal axis, coloring each cell by its SHAP value. A line on top traces the model output per row.
shap.plots.heatmap(shapvalues[:200])
It is effective for spotting clusters of customers that the model treats similarly and for seeing which features drive distinct segments. Order the rows by similarity (the default hierarchical clustering) to make the blocks visible.
Aggregating Local into Global Views
Because the global view is just an aggregation of local SHAP values, you can compute it directly with NumPy when you need a custom report rather than a built-in plot. This is handy for logging importance over time or for feeding a monitoring dashboard.
import numpy as np
Mean absolute SHAP per feature = global importance ranking.
globalimportance = (
pd.Series(np.abs(shapvalues.values).mean(axis=0), index=X.columns)
.sortvalues(ascending=False)
)
print(globalimportance)
Signed mean SHAP per feature = average directional push.
meansigned = pd.Series(shapvalues.values.mean(axis=0), index=X.columns)
print(meansigned)
The absolute-mean ranking matches the bar plot exactly; the signed mean tells you whether a feature, on average, pushes predictions up or down. Logging globalimportance on each retrain gives a simple, model-agnostic drift signal: if the importance ranking reshuffles sharply between two training runs on similar data, something upstream changed.
Interaction Values
SHAP interaction values extend attribution from individual features to pairs of features, splitting each feature's contribution into a main effect and pairwise interaction effects. For tree models this is exact:
interaction = explainer.shapinteractionvalues(Xtest[:500])
print(interaction.shape) # (500, n
features, nfeatures)
The diagonal holds main effects; off-diagonal entries are symmetric interaction contributions. Interaction values are expensive (quadratic in features) so compute them on a sample. They answer questions like "does the effect of support tickets depend on contract type?" beyond what a colored scatter suggests.
Explaining a Regression Model
SHAP works identically for regression; the only difference is that values are in target units and additivity reconstructs the predicted value directly.
from sklearn.ensemble import RandomForestRegressor
Reuse the same feature matrix; create a continuous target: monthly revenue.
revenue = (
20
- 0.8 X["monthlycharges"]
- 1.5 X["tenuremonths"] 0.1
- rng.normal(0, 5, len(X))
)
Xrtrain, Xrtest, yrtrain, yrtest = traintestsplit(
X, revenue, testsize=0.25, randomstate=0
)
reg = RandomForestRegressor(nestimators=300, maxdepth=8, randomstate=0)
reg.fit(Xrtrain, yrtrain)
regexplainer = shap.Explainer(reg, Xrtrain)
regshap = regexplainer(Xrtest)
shap.plots.bar(regshap)
shap.plots.waterfall(regshap[0])
Because the base value plus contributions equals the predicted revenue, a waterfall here reads as "the average prediction is X dollars; this customer is +Y because of high charges and -Z because of short tenure."
Explaining Text and NLP Models
For transformer pipelines, shap.Explainer pairs a PartitionExplainer with a text masker built from the tokenizer, attributing the output to spans of tokens. This is a brief illustration.
from transformers import pipeline
import shap
classifier = pipeline(
"sentiment-analysis",
model="distilbert-base-uncased-finetuned-sst-2-english",
topk=None,
)
explainer = shap.Explainer(classifier)
shaptext = explainer(["The support team was slow but the product is excellent."])
shap.plots.text(shaptext[0])
The text plot highlights tokens that pushed the prediction toward each class, with intensity proportional to the SHAP value. Because tokens are correlated, the partition masker groups them hierarchically rather than removing them independently, which gives more sensible attributions for language.
Operationalizing SHAP in Production
Explanations are most valuable when they are part of the system rather than a one-off notebook. A few patterns make SHAP production-ready.
Compute explanations at scoring time for tree models, since TreeExplainer is fast enough to add negligible latency. Persist the explainer alongside the model so the background dataset and feature order are fixed.
import joblib
Save model and a fixed, representative background together.
joblib.dump(
{"model": model, "background": Xtrain.sample(200, randomstate=0)},
"churnmodelbundle.joblib",
)
At serving time, rebuild the explainer once and reuse it across requests.
bundle = joblib.load("churnmodelbundle.joblib")
servingexplainer = shap.Explainer(bundle["model"], bundle["background"])
def explainone(row: pd.DataFrame) -> dict:
expl = servingexplainer(row)
contributions = dict(zip(row.columns, expl.values[0].round(4)))
return {
"basevalue": float(expl.basevalues[0]),
"contributions": contributions,
}
Return the top contributors with each prediction in your API so downstream consumers can display "reasons" without re-running anything. For batch monitoring, aggregate stored SHAP values per feature over a rolling window and alert when the mean absolute contribution of a key feature shifts beyond a threshold; this catches data drift that accuracy metrics can miss until labels arrive.
Keep explanation generation versioned: log the model version, the background hash, and the SHAP library version with each batch so that any explanation can be reproduced exactly.
Common Pitfalls
- Correlated features. When two features are highly correlated, the model can use either, and SHAP may split or shift credit between them in ways that look unstable. Reduce redundancy before explaining, or interpret correlated groups together rather than individually.
- Attribution is not causation. A SHAP value tells you how a feature influenced the model's output, not what would happen in the world if you changed that feature. A feature can have a large SHAP value because it proxies something causal upstream. Do not read SHAP plots as causal effect estimates.
- Background / masker choice matters. The base value and the contributions are defined relative to the background dataset. A background of all-zeros, a random sample, or a class-balanced sample can yield different explanations. Use a representative sample of training data and document it. For
KernelExplainer, summarize the background withshap.sampleorshap.kmeansto control cost. - Performance on large data.
KernelExplainerand interaction values scale poorly. Explain a representative sample (a few hundred to a few thousand rows) rather than the full set; the global summaries converge quickly.TreeExplaineris fast enough to run on large test sets directly. - Probability vs margin space. For classifiers, raw SHAP values are usually in log-odds. Mixing margin-space values with probability-space intuition leads to misreading magnitudes. Decide which space you want and stay in it.
- Stale explanations. SHAP explains the model as trained. After retraining, regenerate explanations; do not reuse old plots against a new model.
Best Practices
- Default to
TreeExplainerfor tree ensembles andLinearExplainerfor linear models; reserveKernelExplainerfor genuinely black-box models. - Always pass and persist a representative background dataset, and record it alongside the explanations for reproducibility.
- Use the
Explanationobject and theshap.plots.*API rather than raw arrays; it carries feature names and base values for you. - Pair a global view (beeswarm or bar) with a few local views (waterfall) when presenting to stakeholders - one without the other is incomplete.
- Sanity-check additivity (
basevalues + values.sum(axis=1)equals the model output) whenever you set up a new pipeline. - Validate that top features are sensible domain signals; a surprising top feature is often leakage, not insight.
- Cache SHAP values for expensive explainers; they are deterministic for a fixed model and background.
Conclusion and Key Takeaways
SHAP turns an opaque model into an auditable one by attributing every prediction to its features, with contributions that add up exactly to the output minus a baseline. That single, theoretically grounded quantity powers both local explanations ("why this prediction?") and global ones ("what matters overall?").
Key takeaways:
- Interpretability supports trust, debugging, fairness, and regulatory needs; SHAP measures behavior faithfully but does not fix it.
- A SHAP value is an additive contribution relative to a base value, and the contributions reconstruct the model output exactly.
- Use the unified
shap.Explainer, but understand the specialized explainers -TreeExplainer(fast and exact for trees),LinearExplainer,KernelExplainer(slow, model-agnostic), the deep explainers, and partition-based explainers for text and images. - Reach for the right plot: beeswarm and bar for global, waterfall and force for local, scatter for feature effects and interactions, decision plots for cumulative paths.
- Mind the pitfalls: correlated features, attribution versus causation, background choice, performance, and margin-versus-probability space.
With these foundations you can integrate SHAP into model reviews, monitoring dashboards, and compliance documentation, and explain not just whether a model works, but why.