Real survival data arrives in many shapes. It might be a Pandas data frame you built from a CSV, a Polars data frame from a Parquet file, a table pulled straight from a database, or one of the datasets bundled with Greenwood for learning and testing. One of the design goals of this package is that none of this should matter: you bring the data in whatever form you already have, and Greenwood works with it directly. This page explains how, and lists the datasets that ship with the package.
Backend-agnostic by design
Greenwood does not have its own data frame type and does not force you to convert your data into one. Instead it is built on Narwhals, a lightweight compatibility layer that lets a library speak to many data frame backends through one interface. When you hand Greenwood a data frame or a column, it reads the values through Narwhals and runs its numerical work on the underlying arrays. Your original object is never mutated and never copied into a foreign format.
In practice this means you can pass data from any of the backends Narwhals supports. These are covered directly by the Greenwood test suite, which checks that they all produce identical estimates on the same data.
- Pandas, the most widely used data frame library, and the default for the bundled datasets
- Polars, a fast, multi-threaded, Arrow-based data frame library, well suited to large datasets, including lazy
LazyFrame queries
- PyArrow tables, the in-memory Arrow format that many file readers and databases produce
- DuckDB relations, so a SQL query over a Parquet file or a database table can be analyzed without first loading it into another library
With a lazy source (a Polars LazyFrame or a DuckDB relation), Greenwood selects only the columns a call names and collects them once. Other backends that Narwhals supports, such as Modin, cuDF, and Ibis, use the same interface and are expected to work the same way.
You do not have to rewrite your analysis when you switch data frame libraries. Code written for a Pandas workflow runs unchanged when the data arrives as Polars, because Greenwood reads both through the same interface. This also means results do not depend on the backend: the same data produces the same estimates regardless of where the frame came from. The test suite verifies this for Pandas, Polars, PyArrow, and DuckDB on identical inputs.
Naming columns with data=
The simplest way to feed a frame to Greenwood is to name its columns and pass the frame itself as data=. With covariates, write an R-style formula. The lung status is coded 1/2, which Surv() recognizes and passes straight through, just as R’s survival::Surv() does.
import greenwood as gw
# Load the bundled lung dataset as a Polars DataFrame
lung = gw.load_dataset("lung", backend="polars")
# Fit a Cox model with age, sex, and ECOG score as covariates
cox = gw.CoxPH().fit("Surv(time, status) ~ age + sex + ph.ecog", data=lung)
# Print the coefficient table and global test statistics
cox
CoxPH (Cox proportional hazards model, ties='efron')
coef exp(coef) se(coef) z p
age 0.01107 1.011 0.009267 1.194 0.2324
sex -0.5526 0.5754 0.1677 -3.294 0.0009861
ph.ecog 0.4637 1.59 0.1136 4.083 4.447e-05
n = 227, events = 164 (1 observation deleted due to missingness)
Likelihood ratio test = 30.5 on 3 df, p = 1.083e-06
The model reads every column the formula names from the frame, converts non-numeric covariates to indicator variables, and drops rows with a missing value in any of them, just as R’s modeling functions do. The printout says how many rows were dropped. Covariates can also be given as a list of column names with covariates=.
When you fit several models to one endpoint, or a model without covariates, describe the response once with gw.Outcome.surv(). It takes the arguments of Surv() written as column names and expressions, and each fit evaluates it against its own data=. The Survival data page covers formulas, Outcome, and the expressions they accept in detail.
# Describe the endpoint once, then fit from the frame
death = gw.Outcome.surv(time="time", event="status")
gw.KaplanMeier().fit(death, by="sex", data=lung)
KaplanMeier (Kaplan-Meier survival estimate)
n events median 0.95LCL 0.95UCL
sex=1 138 112 270 212 310
sex=2 90 53 426 348 550
gw.Surv() still takes values directly: a Pandas or Polars Series, a NumPy array, or an ordinary Python list. Use it when the values are already in hand rather than in a frame. Selecting and comparing columns yourself (lung["status"] == 2) is not portable across backends: it fails on lazy frames and DuckDB relations, and on a PyArrow table the comparison silently returns a single False. A formula or an Outcome avoids all of this.
Printing a fitted model shows a readable summary, but for programmatic use you read its results through tidy output frames. The tidy function returns one row per model term, and passing exponentiate=True reports hazard ratios rather than raw coefficients.
# Extract a tidy per-term table; exponentiate=True gives hazard ratios
gw.tidy(cox, exponentiate=True, format="polars")
PolarsRows3Columns7 |
|
|
|
|
|
|
|
|
| 0 |
age |
1.01112822772 |
0.0092674110137 |
1.19415924725 |
0.232415681 |
0.99292809717 |
1.02966196224 |
| 1 |
sex |
0.57544455619 |
0.167739053787 |
-3.29447664826 |
0.000986051372138 |
0.414213018549 |
0.799435127387 |
| 2 |
ph.ecog |
1.58999119005 |
0.113577266162 |
4.0829339448 |
4.44706665186e-05 |
1.27267517777 |
1.98642358129 |
These patterns (a formula, or an Outcome evaluated against data= at fit time) are the only interface you need for the entire package.
Choosing a backend
The load_dataset() function is itself dataframe-agnostic. With its default it returns a Polars data frame when Polars is installed, falls back to Pandas when only Pandas is available, and raises a clear error if neither is. When you want a specific type, name it with the backend= argument.
# Load the same dataset explicitly in each backend
lung_pd = gw.load_dataset("lung", backend="pandas")
lung_pl = gw.load_dataset("lung", backend="polars")
# Confirm each object belongs to a different library
type(lung_pd).__module__.split(".")[0], type(lung_pl).__module__.split(".")[0]
The backend makes no difference to any result. The same model fit on any supported frame returns the same coefficients, a property the test suite checks on every release. To show this, we fit one formula to the same data held five ways: pandas, Polars, a PyArrow table, a DuckDB relation, and a Polars LazyFrame. Because the formula names columns rather than using backend-specific expressions, the call is identical for all of them.
import duckdb
# The same data in five forms (a SQL query over a Parquet file works like the DuckDB relation)
frames = {
"pandas": lung_pd,
"polars": lung_pl,
"pyarrow": lung_pl.to_arrow(),
"duckdb": duckdb.from_arrow(lung_pl.to_arrow()),
"lazy": lung_pl.lazy(),
}
# The same call on each backend, and the coefficients agree exactly
for name, frame in frames.items():
cox = gw.CoxPH().fit("Surv(time, status) ~ age + sex", data=frame)
print(f"{name:8}", cox.coef_)
pandas [ 0.01704533 -0.51321852]
polars [ 0.01704533 -0.51321852]
pyarrow [ 0.01704533 -0.51321852]
duckdb [ 0.01704533 -0.51321852]
lazy [ 0.01704533 -0.51321852]
An Outcome works the same way, so the Kaplan-Meier curve below is identical whichever frame it reads. With a lazy source, only the columns the call names are read, and they are collected once.
# The Outcome defined above, evaluated against a DuckDB relation
gw.KaplanMeier().fit(death, by="sex", data=frames["duckdb"])
KaplanMeier (Kaplan-Meier survival estimate)
n events median 0.95LCL 0.95UCL
sex=1 138 112 270 212 310
sex=2 90 53 426 348 550
You can choose whichever backend fits your existing workflow. If your pipeline already uses Polars, stick with Polars. If it uses Pandas, stick with Pandas. The choice has no effect on any numerical result.
Greenwood does not read files itself, and it does not need to. Use your DataFrame library’s reader (pandas.read_csv(), polars.scan_parquet(), a database query, and so on) and pass the result as data=. Anything that produces a supported DataFrame is a valid source, including lazy queries.
Shaping raw study data for a response
Real study exports often arrive in a form that needs light transformation before the times and event indicators are ready. Three situations come up often: (1) the data stores calendar dates rather than durations, (2) the event indicator is encoded in a compound or string status column, and (3) a multi-endpoint DataFrame mixes rows that belong to different analyses. The first two are handled by expressions inside the response, so the frame itself needs no changes.
Computing durations from timestamps
The most common transformation is computing the elapsed time between two date columns. Write it as duration(start, end, unit=) in the response. The outcome column needs no recoding either, since a comparison says which value marks the event.
import polars as pl
# Build a simulated study export with enrollment dates, exit dates, and outcomes
study = pl.DataFrame({
"subject": [1, 2, 3, 4, 5, 6],
"enroll": ["2021-03-01", "2021-03-15", "2021-04-01",
"2021-04-20", "2021-05-01", "2021-06-01"],
"exit": ["2022-01-10", "2021-09-22", "2023-01-01",
"2021-12-05", "2022-08-14", "2022-03-30"],
"outcome": ["died", "censored", "censored", "died", "withdrew", "died"],
}).with_columns(pl.col("enroll", "exit").str.to_date())
# The time is the gap between the two dates in days, and only "died" is an event
visit = gw.Outcome.surv(time="duration(enroll, exit)", event="outcome == 'died'")
# Fit a Kaplan-Meier curve straight from the export
gw.KaplanMeier().fit(visit, data=study)
KaplanMeier (Kaplan-Meier survival estimate)
n events median 0.95LCL 0.95UCL
all 6 3 315 302 NA
A few things to notice in this example. First, the outcome column uses three values: "died" (the event), "censored" (lost to follow-up at the exit date), and "withdrew" (left the study early). With outcome == 'died', the event is recorded only for "died", and all other outcomes become censored observations. Second, the exit date serves the same role regardless of the reason for exit: it is always the last time at which the subject was observed, and the event column records what happened at that time. duration() takes a unit= of "days" (the default), "weeks", "months", or "years" (months and years use average lengths of 30.4375 and 365.25 days), and the date columns can be dates, datetimes, or ISO date strings. In a formula, the same response is Surv(duration(enroll, exit), outcome == 'died').
Like R, a response keeps missing times and statuses, and estimators drop those rows when they fit. With a formula or an Outcome, every row with a missing value in any column the model uses (response, covariates, or grouping) is dropped once, and the printout reports how many.
Encoding from compound status columns
Some datasets store multiple outcome levels in a single integer status column. pbc is a typical example: status is 0 (censored), 1 (transplant), or 2 (dead). The right encoding depends on the analysis you intend to run, and each choice is one comparison. Do not pass this column raw. Following R, a status whose largest value is 2 is read as a 1/2 coding, so the 0 rows would become missing.
# Load the pbc dataset, whose status column has three levels (0, 1, 2)
pbc = gw.load_dataset("pbc", backend="polars")
# All-cause analysis: anything other than 0 is an event
any_event = gw.Outcome.surv(time="time", event="status != 0")
# Death-only: transplant becomes censoring (patient still alive at last contact)
death_only = gw.Outcome.surv(time="time", event="status == 2")
# Compare event counts between the two endpoint definitions
print("all-cause: ", any_event.bind(pbc))
print("death-only:", death_only.bind(pbc))
all-cause: Surv(type=right, n=418, events=186)
death-only: Surv(type=right, n=418, events=161)
The event counts differ because 25 subjects reached transplant rather than death: the all-cause endpoint treats them as events, the death-only endpoint treats them as censored. Which is correct depends entirely on your research question. For a competing-risks analysis that keeps the two outcomes separate, R’s factor() turns the codes into states, with the first label meaning censored:
# Transplant and death as competing states
gw.Outcome.surv(
time="time",
event="factor(status, c(0, 1, 2), c('censor', 'transplant', 'death'))",
).bind(pbc)
Surv(type=mright, n=418, events=186, states=('transplant', 'death'))
See the competing risks page for the analysis itself.
Filtering multi-endpoint frames
Some study exports give each subject one row per potential endpoint. The colon dataset is structured this way: etype is 1 for recurrence and 2 for death, so the full frame has 1,858 rows for 929 patients. Mixing rows from different endpoints in a single response is meaningless, so always filter to one endpoint first, with your data frame library.
# Load the colon dataset; each patient contributes two rows, one per endpoint
colon = gw.load_dataset("colon", backend="polars")
# Keep only the recurrence-endpoint rows
recurrence = colon.filter(pl.col("etype") == 1)
# Bind the response to the filtered frame (929 patients, not 1858 rows)
gw.Outcome.surv(time="time", event="status").bind(recurrence)
Surv(type=right, n=929, events=468)
The same principle applies to mgus2, which stores its two competing endpoints in separate column pairs (ptime/pstat for progression, futime/death for death) rather than separate rows. For a single endpoint, pick the matching pair. For a competing-risks analysis, gw.Outcome.first_event() combines the pairs: the earliest observed event becomes the cause, ties go to the endpoint listed first, and subjects with no event are censored at their last follow-up time.
# Load the mgus2 competing-risks dataset
mgus2 = gw.load_dataset("mgus2", backend="polars")
# One (time, event) pair per endpoint, in priority order for ties
pcm_or_death = gw.Outcome.first_event(
endpoints={"pcm": ("ptime", "pstat"), "death": ("futime", "death")}
)
pcm_or_death.bind(mgus2)
Surv(type=mright, n=1384, events=975, states=('pcm', 'death'))
Tidy output frames
Every estimator and model returns its results as tidy data frames, through the to_frame() method and through the tidy() and glance() functions. Pass format= ("pandas", "polars", or "pyarrow") to choose the backend; when you leave it off, Greenwood auto-detects one (Polars if installed, otherwise Pandas, otherwise PyArrow). This makes the results easy to inspect, filter, join, or pass on to a plotting or table library.
# Load data and fit a Cox model with three covariates
lung = gw.load_dataset("lung", backend="polars")
cox = gw.CoxPH().fit("Surv(time, status) ~ age + sex + ph.ecog", data=lung)
# Retrieve the one-row model summary: n, events, log-likelihood, AIC
gw.glance(cox, format="polars")
PolarsRows1Columns10 |
|
|
|
|
|
|
|
|
|
frailty_lrt_statistic null |
|
| 0 |
227 |
164 |
-729.230121375 |
1464.46024275 |
30.5006687732 |
3 |
1.0828176992e-06 |
None |
None |
None |
The tidy and glance functions follow the same conventions as R’s broom package, so a fitted model always yields a per-term table (tidy()) and a one-row model summary (glance()). This is the part that lets Greenwood feed publication tables in Great Tables.
Datasets bundled with Greenwood
For learning, testing, and reproducible examples, Greenwood ships six classic survival datasets drawn from R’s survival package. You can load any of them by name with the load_dataset() function, and you can list them with available_datasets().
# List the names of all datasets bundled with Greenwood
gw.available_datasets()
['colon', 'e1684', 'lung', 'mgus2', 'ovarian', 'pbc', 'pbcseq', 'veteran']
The six datasets vary widely in size, covariate richness, and event structure. The sections below describe each one, identify which Greenwood models are a natural fit, and display the raw data in an interactive explorer so you can inspect columns and spot missing values before writing a single line of analysis code.
lung: NCCTG lung cancer
The lung dataset follows 228 patients enrolled in a North Central Cancer Treatment Group trial of advanced lung cancer. It is the most commonly-used type of example throughout the Greenwood documentation and the most commonly cited benchmark in the Python and R survival-analysis literature.
Columns: inst (institution code), time (survival days), status (1 = censored, 2 = dead), age, sex (1 = male, 2 = female), ph.ecog (ECOG performance score 0–4), ph.karno (Karnofsky score rated by physician), pat.karno (Karnofsky score rated by patient), meal.cal (calories consumed at meals), wt.loss (weight loss in the last six months).
Model fit: The modest size, clean right-censoring, and mix of continuous and ordinal covariates make lung suitable for virtually every model in Greenwood. Use it for Kaplan-Meier and Nelson-Aalen curves (overall or stratified by sex or ph.ecog), log-rank and related group-comparison tests, Cox proportional-hazards regression, the Royston-Parmar flexible parametric model, and AFT regression. It is the standard dataset used in the Surv construction examples throughout this guide.
status uses the R-style 1/2 convention: 1 = censored, 2 = dead. Following R, this coding is recognized, so the formula response Surv(time, status) (or gw.Outcome.surv(time="time", event="status")) works directly. Writing Surv(time, status == 2) gives the same response.
veteran: Veterans’ Administration lung cancer trial
veteran is a randomized clinical trial comparing two treatments in 137 male veterans with inoperable lung cancer. Its standout feature is the celltype column, which partitions patients into four histological groups (squamous, small cell, adeno, and large cell). This makes it the natural choice for demonstrating stratified Kaplan-Meier curves and multi-group comparison tests.
Columns: trt (1 = standard, 2 = test), celltype ("squamous"/"smallcell"/"adeno"/ "large"), time (survival days), status (0 = censored, 1 = dead), karno (Karnofsky performance score), diagtime (months from diagnosis to study entry), age, prior (prior therapy: 0 or 10).
Model fit: veteran is ideal for Kaplan-Meier curves stratified by trt or celltype, log-rank tests, pairwise log-rank, and trend tests. For regression, CoxPH accepts the celltype string column directly and converts it to treatment-coded dummies automatically, giving one hazard ratio per cell type relative to the reference level.
ovarian: Ovarian cancer survival
ovarian is a deliberately small dataset (26 patients) from a trial comparing two chemotherapy regimens for ovarian cancer. Its small size makes it ideal for teaching, for verifying that a method produces the right result on data you can inspect by eye, and for quick checks during development.
Columns: futime (follow-up days), fustat (0 = censored, 1 = dead), age, resid.ds (residual disease after surgery: 1 = no, 2 = yes), rx (treatment group: 1 or 2), ecog.ps (ECOG performance status).
Model fit: Because of its size, ovarian is best used with non-parametric estimators (KaplanMeier, NelsonAalen) and simple group comparisons (logrank_test()). Regression models can be fit but coefficient estimates will have wide confidence intervals. Use pbc or colon when you need a richer covariate structure.
pbc: Mayo Clinic primary biliary cholangitis
pbc records 418 patients enrolled in a Mayo Clinic randomized trial of D-penicillamine for primary biliary cholangitis, a chronic liver disease. With 20 columns including laboratory values, clinical signs, and histological staging, it is the reference dataset for high-dimensional regression in Greenwood.
Columns: id, time (days to death or transplant), status (0 = censored, 1 = transplant, 2 = dead), trt (1 = D-penicillamine, 2 = placebo), age, sex, ascites, hepato (hepatomegaly), spiders, edema, bili (serum bilirubin), chol (cholesterol), albumin, copper, alk.phos (alkaline phosphatase), ast, trig (triglycerides), platelet, protime (prothrombin time), stage (histologic stage 1–4).
Model fit: pbc is the standard choice for CoxPH with many covariates and for CoxNet (penalized Cox) when you want automatic variable selection or regularization. The Royston-Parmar flexible parametric model also fits well here. Because status has three levels, you can treat transplant and death as a combined endpoint (Surv(time, status != 0)) for a standard right-censored analysis, or separate them into competing events for AalenJohansen and FineGray.
status has three levels: 0 = censored, 1 = transplant, 2 = dead. Do not pass it raw. For an all-cause analysis use Surv(time, status != 0), and for death only use Surv(time, status == 2). For competing risks, use Surv(time, factor(status, 0:2, c('censor', 'transplant', 'death'))) and give the response to AalenJohansen or FineGray.
colon: Adjuvant therapy for colon cancer
colon is Greenwood’s largest dataset at 1858 rows. It comes from a three-arm trial (observation, levamisole, and levamisole plus 5-FU) for adjuvant therapy after surgery for colon cancer. The key structural feature is the etype column: each patient contributes two rows, one for recurrence (etype = 1) and one for death (etype = 2), so the full frame contains roughly twice the number of subjects.
Columns: id, study, rx (Obs / Lev / Lev+5FU), sex, age, obstruct, perfor (bowel perforation), adhere, nodes (positive lymph nodes), status (0 = censored, 1 = event), differ (tumor differentiation), extent (depth of invasion), surg (short / long time from surgery to registration), node4 (nodes > 4), time, etype (1 = recurrence, 2 = death).
Model fit: Always filter to one endpoint before fitting: df.filter(pl.col("etype") == 1) for recurrence or == 2 for death. The recurrence endpoint is well suited to Kaplan-Meier curves and log-rank tests comparing the three treatment arms; the large sample gives precise estimates. Filtering to etype == 2 and treating the recurrence event as a competing risk gives a natural setup for AalenJohansen and FineGray.
mgus2: Monoclonal gammopathy
mgus2 is Greenwood’s canonical competing-risks dataset. It follows 1,384 patients diagnosed with monoclonal gammopathy of undetermined significance (MGUS), a precursor plasma-cell condition, tracking two competing endpoints: progression to plasma-cell malignancy (pstat, time in ptime) and death without progression (death, time in futime). Because a patient who progresses cannot later die as a non-progressor, the two events compete; a Kaplan-Meier analysis would overestimate the probability of each event, so this dataset is the motivating example for cumulative incidence functions in this guide.
Columns: id, age, sex, dxyr (year of diagnosis), hgb (hemoglobin), creat (creatinine), mspike (M-spike size in g/dL), ptime (months to progression), pstat (0 = no progression, 1 = progressed), futime (months to death or censoring), death (0 = alive, 1 = dead).
Model fit: mgus2 is designed for competing-risks analysis. Use AalenJohansen for non-parametric cumulative incidence functions and FineGray for regression on the cause-specific subdistribution hazard. Build the competing-risks response with gw.Outcome.first_event(), which combines the two column pairs. The MultiState model can extend the analysis to a three-state illness-death structure (MGUS -> progression -> death). The competing-risks page of this guide builds its examples entirely on this dataset.
Several of these datasets follow R’s convention rather than the 0/1 convention for the status column. lung codes status as 1 = censored and 2 = dead. pbc uses three levels (0/1/2). Always inspect the column. A 1/2 coding passes straight through, as in R, but any other coding needs a comparison such as Surv(time, status == 2). See Survival data for the full rules.
Next steps
You can now bring survival data into Greenwood from whatever source you have, and you know what the bundled datasets contain.