Data sources and formats

Real survival data arrives in many shapes. It might be a pandas data frame you built from a CSV, a Polars data frame from a Parquet file, a table pulled straight from a database, or one of the datasets bundled with Greenwood for learning and testing. One of the design goals of this package is that none of this should matter: you bring the data in whatever form you already have, and Greenwood works with it directly. This page explains how, and lists the datasets that ship with the package.

Backend-agnostic by design

Greenwood does not have its own data frame type and does not force you to convert your data into one. Instead it is built on Narwhals, a lightweight compatibility layer that lets a library speak to many data frame backends through one interface. When you hand Greenwood a data frame or a column, it reads the values through Narwhals and runs its numerical work on the underlying arrays. Your original object is never mutated and never copied into a foreign format.

In practice this means you can pass data from any of the backends Narwhals supports. Three of these are covered directly by the Greenwood test suite, which checks that they all produce identical estimates on the same data.

pandas, the most widely used data frame library, and the default for the bundled datasets.
Polars, a fast, multi-threaded, Arrow-based data frame library, well suited to large datasets.
PyArrow tables, the in-memory Arrow format that many file readers and databases produce.

Other backends that Narwhals supports, such as Modin, cuDF, and DuckDB, use the same interface and are expected to work the same way, though the test suite pins parity on the three above.

What “backend-agnostic” buys you

You do not have to rewrite your analysis when you switch data frame libraries. Code written for a pandas workflow runs unchanged when the data arrives as Polars, because Greenwood reads both through the same interface. This also means results do not depend on the backend: the same data produces the same estimates regardless of where the frame came from. The test suite verifies this for pandas, Polars, and PyArrow on identical inputs.

Passing columns and passing frames

There are two ways to feed data to Greenwood, and both are always available.

The first is to pass individual columns to a Surv constructor. This is how you build the response. A column can be a pandas Series, a Polars Series, a NumPy array, or an ordinary Python list. Greenwood treats them all the same.

import greenwood as gw

lung = gw.load_dataset("lung", backend="polars")

y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))
y

Surv(type=right, n=228, events=165)

The second is to pass a whole data frame of covariates to a model. The model reads the columns it needs, converts non-numeric columns to indicator variables, and drops rows with missing values, just as R’s modeling functions do. We fit the model in its own cell, then set it aside and look at its results through the tidy functions below.

cox = gw.CoxPH().fit(y, lung[["age", "sex", "ph.ecog"]])

Printing a fitted model shows a readable summary, but for programmatic use you read its results through tidy output frames. The tidy function returns one row per model term, and passing exponentiate=True reports hazard ratios rather than raw coefficients.

gw.tidy(cox, exponentiate=True, format="polars")

	term str	estimate f64	std_error f64	statistic f64	p_value f64	conf_low f64	conf_high f64
PolarsRows3Columns7
0	age	1.01112822772	0.0092674110137	1.19415924725	0.232415681	0.99292809717	1.02966196224
1	sex	0.57544455619	0.167739053787	-3.29447664826	0.000986051372138	0.414213018549	0.799435127387
2	ph.ecog	1.58999119005	0.113577266162	4.0829339448	4.44706665186e-05	1.27267517777	1.98642358129

Choosing a backend

The load_dataset function is itself dataframe-agnostic. With its default it returns a Polars data frame when Polars is installed, falls back to pandas when only pandas is available, and raises a clear error if neither is. When you want a specific type, name it with the backend argument.

lung_pd = gw.load_dataset("lung", backend="pandas")
lung_pl = gw.load_dataset("lung", backend="polars")

type(lung_pd).__module__.split(".")[0], type(lung_pl).__module__.split(".")[0]

('pandas', 'polars')

The backend makes no difference to any result. The same model fit on the pandas frame and on the Polars frame returns the same coefficients, a property the test suite checks on every release. To show this, we first build a response from the Polars columns. The resulting Surv object looks the same as one built from pandas: it records the same times and events, because Greenwood read both through Narwhals.

y_pl = gw.Surv.right(lung_pl["time"], event=(lung_pl["status"] == 2))
y_pl

Surv(type=right, n=228, events=165)

Fitting a Cox model on the Polars-backed response and covariates, then viewing the result with to_frame(), gives the same coefficient table you would get from the Polars frame.

gw.CoxPH().fit(y_pl, lung_pl[["age", "sex"]]).to_frame(format="polars")

	term str	estimate f64	std_error f64	statistic f64	p_value f64	conf_low f64	conf_high f64
PolarsRows2Columns7
0	age	0.0170453318454	0.00922327347697	1.8480783301	0.0645910121295	-0.00103195198901	0.0351226156798
1	sex	-0.513218517108	0.167457962356	-3.06476031291	0.00217844504671	-0.84143009225	-0.185006941967

Reading from files and databases

Greenwood does not read files itself, and it does not need to. Use your data frame library’s reader (pandas.read_csv, polars.read_parquet, a database query, and so on) to get a frame, then pass its columns to Surv exactly as above. Anything that produces a supported data frame is a valid source.

Tidy output frames

Every estimator and model returns its results as tidy data frames, through the to_frame() method and through the tidy and glance functions. Pass format= ("pandas", "polars", or "pyarrow") to choose the backend; when you leave it off, Greenwood auto-detects one (Polars if installed, otherwise pandas, otherwise PyArrow). This makes the results easy to inspect, filter, join, or pass on to a plotting or table library.

lung = gw.load_dataset("lung", backend="polars")
y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))

cox = gw.CoxPH().fit(y, lung[["age", "sex", "ph.ecog"]])

gw.glance(cox, format="polars")

	n i64	nevent i64	loglik f64	aic f64	lr_statistic f64	df i64	lr_p_value f64
PolarsRows1Columns7
0	227	164	-729.230121375	1464.46024275	30.5006687732	3	1.0828176992e-06

The tidy and glance functions follow the same conventions as R’s broom package, so a fitted model always yields a per-term table (tidy) and a one-row model summary (glance). This is the seam that lets Greenwood feed publication tables in Great Summaries and Great Tables.

Datasets bundled with Greenwood

For learning, testing, and reproducible examples, Greenwood ships several classic survival datasets drawn from R’s survival package. Load any of them by name with gw.load_dataset, and list them with gw.available_datasets().

gw.available_datasets()

['colon', 'lung', 'mgus2', 'ovarian', 'pbc', 'veteran']

Each one targets a different situation. The lung dataset (228 subjects), from the North Central Cancer Treatment Group, is the standard right-censored example used throughout this guide. veteran (137 subjects) is a randomized lung cancer trial that includes a cell-type covariate, which makes it a good fit for group comparisons. ovarian (26 subjects) is a small trial that is convenient for quick checks and teaching. pbc (418 subjects) comes from the Mayo Clinic trial in primary biliary cholangitis and carries many covariates for regression. colon (1858 subjects) records adjuvant therapy for colon cancer and has both recurrence and death endpoints. Finally, mgus2 (1384 subjects) follows patients with monoclonal gammopathy and provides competing risks of progression and death, which the competing-risks and multi-state chapters build on.

Event coding in the bundled data

Several of these datasets, lung among them, code the status column as 1 for censored and 2 for the event, following R’s convention rather than the 0/1 convention. Convert the column explicitly when you build the response, as in the examples above with event=(lung["status"] == 2). See Survival data for why Greenwood does not guess at this coding.

Next steps

You can now bring survival data into Greenwood from whatever source you have, and you know what the bundled datasets contain.

Estimating survival with Kaplan-Meier puts this data to work with the first and most important estimator.
Survival data and the Surv object covers the response type in more detail if you skipped ahead.
The Quick start is a fast end-to-end tour if you prefer to skim first.