Verified for scikit-survival 0.28.0 on 2026-07-23. API statements below use official scikit-survival documentation; interpretation is anchored to the cited primary methodological literature.
Do not label one aggregate number "model accuracy" without its estimand, horizon, censoring estimator, and input type.
Shape (n_test,). Used by:
concordance_index_censoredconcordance_index_ipcwcumulative_dynamic_auc (same score at each requested time)Typical source:
risk = estimator.predict(X_test)
Confirm direction. Cox and ranking-only SVM predictions are higher-is-riskier. Predicted survival/log-time outputs are the opposite direction and must not be silently treated as risk.
Shape (n_test, n_times). Accepted by cumulative_dynamic_auc, where column
j is risk at times[j]. For a random survival forest:
functions = estimator.predict_cumulative_hazard_function(X_test)
risk_by_time = np.vstack([fn(times) for fn in functions])
Cumulative hazard is risk-oriented. Survival probability is not accepted by
cumulative_dynamic_auc; do not pass it without an explicitly justified
transformation.
Shape (n_test, n_times), values in [0, 1], non-increasing across columns.
Used by:
brier_scoreintegrated_brier_scorefunctions = estimator.predict_survival_function(X_test)
survival_probability = np.vstack([fn(times) for fn in functions])
Metric functions expect the numeric matrix, not a list of unevaluated
StepFunction objects and not a 1D risk score.
from sksurv.metrics import concordance_index_censored
harrell_c = concordance_index_censored(
y_test["event"],
y_test["time"],
risk,
)[0]
Harrell C is the fraction of comparable pairs whose risk ordering is concordant, with handling for tied risk and event times. It measures rank discrimination over the observed follow-up mix.
Important limitations:
There is no universal censoring-percentage cutoff at which Harrell C suddenly becomes invalid. Report censoring, compare sensitivity to IPCW concordance, and justify the estimand.
from sksurv.metrics import concordance_index_ipcw
uno_c = concordance_index_ipcw(
y_train,
y_test,
risk,
tau=tau,
)[0]
survival_train estimates the censoring distribution. Never pass pooled
train+test outcomes or y_test as the training distribution.
tau truncates the concordance target. Choose it before seeing model performance
and where the estimated training censoring survival is positive. Test follow-up
must be supported by training follow-up; otherwise scikit-survival raises a
ValueError.
The implementation uses Kaplan-Meier censoring weights and assumes censoring is random/independent of features. If censoring depends on covariates, this marginal weight model may be inadequate. IPCW is not a generic correction for informative censoring.
from sksurv.metrics import cumulative_dynamic_auc
auc, mean_auc = cumulative_dynamic_auc(
y_train,
y_test,
risk, # (n_test,) or (n_test, n_times)
times,
)
At each (t), cumulative cases have an observed event by (t), while dynamic controls remain event-free after (t). IPCW handles right censoring.
Requirements:
times is one-dimensional, unique, and strictly increasing;(n_test,) or (n_test, n_times);The returned mean_auc is not the arithmetic mean. It integrates
(\widehat{AUC}(t)) over the time range, weighted by the estimated survival
function.
The cumulative/dynamic definition is one time-dependent ROC estimand. State it; other incident/dynamic or competing-risk definitions answer different questions.
from sksurv.metrics import brier_score, integrated_brier_score
returned_times, brier = brier_score(
y_train,
y_test,
survival_probability,
times,
)
ibs = integrated_brier_score(
y_train,
y_test,
survival_probability,
times,
)
The time-dependent Brier score is an IPC-weighted squared error between predicted survival probability and event-free status at (t). Lower is better.
Requirements:
(n_test, n_times);IBS integrates Brier score over [times[0], times[-1]] with the implementation's
time weighting. It depends on the chosen interval; IBS values from different
time ranges are not directly comparable.
Compare against useful reference predictions such as a training-derived Kaplan-Meier survival curve. Do not estimate the reference curve on test outcomes.
Brier score responds to both discrimination and calibration. A good Brier score does not prove that predicted 20% risk corresponds to 20% observed risk in every horizon or subgroup.
For a prespecified horizon:
1 - S(t | x) on independent validation rows;scikit-survival 0.28 does not expose a dedicated calibration-curve API. Do not use
sklearn.calibration.calibration_curve naively on censored binary labels. If an
external method is used, document its censoring assumptions and train/validation
separation.
Any recalibration layer is another learned model. Fit it on a calibration split or inner resampling, then assess on independent data.
A pragmatic fold-specific grid:
import numpy as np
from sksurv.nonparametric import CensoringDistributionEstimator
test_time = y_test["time"]
train_time = y_train["time"]
lower = np.quantile(test_time, 0.10)
upper = min(
np.quantile(test_time, 0.80),
np.nextafter(train_time.max(), -np.inf),
)
times = np.linspace(lower, upper, 50)
if not (test_time.min() < times[0] < times[-1] < test_time.max()):
raise ValueError("grid is outside test follow-up")
if not test_time.max() < train_time.max():
raise ValueError("test follow-up exceeds training support")
censoring = CensoringDistributionEstimator().fit(y_train)
if np.any(censoring.predict_proba(times) <= 0):
raise ValueError("training censoring survival reaches zero on grid")
Quantiles are an operational example, not a scientific default. Prefer prespecified meaningful horizons, then verify fold support. Do not select a grid because it maximizes a metric.
The bundled evaluator rejects unsupported grids and shape/type confusion:
python skills/scikit-survival/scripts/evaluate_survival_metrics.py \
--input predictions.npz \
--output metrics-summary.json
Its NPZ contract is:
train_event, train_timetest_event, test_timetimesrisksurvivalArchives are loaded with allow_pickle=False.
Survival estimators' default .score() is Harrell concordance. For other targets,
scikit-survival provides estimator wrappers:
as_concordance_index_ipcw_scorer(estimator, tau=None, tied_tol=...)as_cumulative_dynamic_auc_scorer(estimator, times, tied_tol=...)as_integrated_brier_score_scorer(estimator, times)Correct pattern:
from sklearn.model_selection import GridSearchCV
from sksurv.metrics import as_integrated_brier_score_scorer
wrapped = as_integrated_brier_score_scorer(
estimator,
times=inner_times,
)
search = GridSearchCV(
wrapped,
{"estimator__model__max_depth": [1, 2, 4]},
cv=inner_splits,
)
search.fit(X_outer_train, y_outer_train)
The wrapper:
GridSearchCV;estimator_;estimator__...;Do not write scoring=as_integrated_brier_score_scorer(times); the constructor
requires an estimator.
For tuned performance:
for each outer split:
outer_train, outer_valid
for each inner split within outer_train:
fit preprocessing + model + censoring-dependent scorer state
select hyperparameters
refit selected pipeline on all outer_train
evaluate once on outer_valid using outer_train censoring distribution
aggregate outer scores and uncertainty
Every fold needs its own valid tau/time grid. A single global grid derived from
all outcome times leaks outer-validation support information and may fail when an
outer training fold has shorter follow-up.
After protocol evaluation, tune on all development data and evaluate once on an untouched final test set. Do not report the best inner-CV score as generalization performance.
If events have causes:
State event coding and use methods designed for the cause-specific estimand. See
competing-risks.md.
tau and exact time grid/range;Checked 2026-07-23: