# Data sources and formats
Real survival data arrives in many shapes. It might be a pandas data frame you built from a CSV, a Polars data frame from a Parquet file, a table pulled straight from a database, or one of the datasets bundled with Greenwood for learning and testing. One of the design goals of this package is that none of this should matter: you bring the data in whatever form you already have, and Greenwood works with it directly. This page explains how, and lists the datasets that ship with the package.
# Backend-agnostic by design
Greenwood does not have its own data frame type and does not force you to convert your data into one. Instead it is built on [Narwhals](https://narwhals-dev.github.io/narwhals/), a lightweight compatibility layer that lets a library speak to many data frame backends through one interface. When you hand Greenwood a data frame or a column, it reads the values through Narwhals and runs its numerical work on the underlying arrays. Your original object is never mutated and never copied into a foreign format.
In practice this means you can pass data from any of the backends Narwhals supports. Three of these are covered directly by the Greenwood test suite, which checks that they all produce identical estimates on the same data.
- **pandas**, the most widely used data frame library, and the default for the bundled datasets.
- **Polars**, a fast, multi-threaded, Arrow-based data frame library, well suited to large datasets.
- **PyArrow** tables, the in-memory Arrow format that many file readers and databases produce.
Other backends that Narwhals supports, such as Modin, cuDF, and DuckDB, use the same interface and are expected to work the same way, though the test suite pins parity on the three above.
> **Note: What "backend-agnostic" buys you**
>
> You do not have to rewrite your analysis when you switch data frame libraries. Code written for a pandas workflow runs unchanged when the data arrives as Polars, because Greenwood reads both through the same interface. This also means results do not depend on the backend: the same data produces the same estimates regardless of where the frame came from. The test suite verifies this for pandas, Polars, and PyArrow on identical inputs.
# Passing columns and passing frames
There are two ways to feed data to Greenwood, and both are always available.
The first is to pass individual columns to a [Surv](../reference/Surv.md#greenwood.Surv) constructor. This is how you build the response. A column can be a pandas `Series`, a Polars `Series`, a NumPy array, or an ordinary Python list. Greenwood treats them all the same.
``` python
import greenwood as gw
lung = gw.load_dataset("lung", backend="polars")
y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))
y
```
Surv(type=right, n=228, events=165)
The second is to pass a whole data frame of covariates to a model. The model reads the columns it needs, converts non-numeric columns to indicator variables, and drops rows with missing values, just as R's modeling functions do. We fit the model in its own cell, then set it aside and look at its results through the tidy functions below.
``` python
cox = gw.CoxPH().fit(y, lung[["age", "sex", "ph.ecog"]])
```
Printing a fitted model shows a readable summary, but for programmatic use you read its results through tidy output frames. The `tidy` function returns one row per model term, and passing `exponentiate=True` reports hazard ratios rather than raw coefficients.
``` python
gw.tidy(cox, exponentiate=True, format="polars")
```
PolarsRows3Columns7
|
|
term
str
|
estimate
f64
|
std_error
f64
|
statistic
f64
|
p_value
f64
|
conf_low
f64
|
conf_high
f64
|
| 0 |
age |
1.01112822772 |
0.0092674110137 |
1.19415924725 |
0.232415681 |
0.99292809717 |
1.02966196224 |
| 1 |
sex |
0.57544455619 |
0.167739053787 |
-3.29447664826 |
0.000986051372138 |
0.414213018549 |
0.799435127387 |
| 2 |
ph.ecog |
1.58999119005 |
0.113577266162 |
4.0829339448 |
4.44706665186e-05 |
1.27267517777 |
1.98642358129 |
# Choosing a backend
The `load_dataset` function is itself dataframe-agnostic. With its default it returns a Polars data frame when Polars is installed, falls back to pandas when only pandas is available, and raises a clear error if neither is. When you want a specific type, name it with the `backend` argument.
``` python
lung_pd = gw.load_dataset("lung", backend="pandas")
lung_pl = gw.load_dataset("lung", backend="polars")
type(lung_pd).__module__.split(".")[0], type(lung_pl).__module__.split(".")[0]
```
('pandas', 'polars')
The backend makes no difference to any result. The same model fit on the pandas frame and on the Polars frame returns the same coefficients, a property the test suite checks on every release. To show this, we first build a response from the Polars columns. The resulting [Surv](../reference/Surv.md#greenwood.Surv) object looks the same as one built from pandas: it records the same times and events, because Greenwood read both through Narwhals.
``` python
y_pl = gw.Surv.right(lung_pl["time"], event=(lung_pl["status"] == 2))
y_pl
```
Surv(type=right, n=228, events=165)
Fitting a Cox model on the Polars-backed response and covariates, then viewing the result with [to_frame()](../reference/AFT.md#greenwood.AFT.to_frame), gives the same coefficient table you would get from the Polars frame.
``` python
gw.CoxPH().fit(y_pl, lung_pl[["age", "sex"]]).to_frame(format="polars")
```
PolarsRows2Columns7
|
|
term
str
|
estimate
f64
|
std_error
f64
|
statistic
f64
|
p_value
f64
|
conf_low
f64
|
conf_high
f64
|
| 0 |
age |
0.0170453318454 |
0.00922327347697 |
1.8480783301 |
0.0645910121295 |
-0.00103195198901 |
0.0351226156798 |
| 1 |
sex |
-0.513218517108 |
0.167457962356 |
-3.06476031291 |
0.00217844504671 |
-0.84143009225 |
-0.185006941967 |
> **Tip: Reading from files and databases**
>
> Greenwood does not read files itself, and it does not need to. Use your data frame library's reader (`pandas.read_csv`, `polars.read_parquet`, a database query, and so on) to get a frame, then pass its columns to [Surv](../reference/Surv.md#greenwood.Surv) exactly as above. Anything that produces a supported data frame is a valid source.
# Tidy output frames
Every estimator and model returns its results as tidy data frames, through the [to_frame()](../reference/AFT.md#greenwood.AFT.to_frame) method and through the `tidy` and `glance` functions. Pass `format=` (`"pandas"`, `"polars"`, or `"pyarrow"`) to choose the backend; when you leave it off, Greenwood auto-detects one (Polars if installed, otherwise pandas, otherwise PyArrow). This makes the results easy to inspect, filter, join, or pass on to a plotting or table library.
``` python
lung = gw.load_dataset("lung", backend="polars")
y = gw.Surv.right(lung["time"], event=(lung["status"] == 2))
cox = gw.CoxPH().fit(y, lung[["age", "sex", "ph.ecog"]])
gw.glance(cox, format="polars")
```
PolarsRows1Columns7
|
|
n
i64
|
nevent
i64
|
loglik
f64
|
aic
f64
|
lr_statistic
f64
|
df
i64
|
lr_p_value
f64
|
| 0 |
227 |
164 |
-729.230121375 |
1464.46024275 |
30.5006687732 |
3 |
1.0828176992e-06 |
The tidy and glance functions follow the same conventions as R's broom package, so a fitted model always yields a per-term table (`tidy`) and a one-row model summary (`glance`). This is the seam that lets Greenwood feed publication tables in Great Summaries and Great Tables.
# Datasets bundled with Greenwood
For learning, testing, and reproducible examples, Greenwood ships several classic survival datasets drawn from R's `survival` package. Load any of them by name with `gw.load_dataset`, and list them with `gw.available_datasets()`.
``` python
gw.available_datasets()
```
['colon', 'lung', 'mgus2', 'ovarian', 'pbc', 'veteran']
Each one targets a different situation. The `lung` dataset (228 subjects), from the North Central Cancer Treatment Group, is the standard right-censored example used throughout this guide. `veteran` (137 subjects) is a randomized lung cancer trial that includes a cell-type covariate, which makes it a good fit for group comparisons. `ovarian` (26 subjects) is a small trial that is convenient for quick checks and teaching. `pbc` (418 subjects) comes from the Mayo Clinic trial in primary biliary cholangitis and carries many covariates for regression. `colon` (1858 subjects) records adjuvant therapy for colon cancer and has both recurrence and death endpoints. Finally, `mgus2` (1384 subjects) follows patients with monoclonal gammopathy and provides competing risks of progression and death, which the competing-risks and multi-state chapters build on.
> **Warning: Event coding in the bundled data**
>
> Several of these datasets, `lung` among them, code the status column as `1` for censored and `2` for the event, following R's convention rather than the `0`/`1` convention. Convert the column explicitly when you build the response, as in the examples above with `event=(lung["status"] == 2)`. See [Survival data](survival-data.md) for why Greenwood does not guess at this coding.
# Next steps
You can now bring survival data into Greenwood from whatever source you have, and you know what the bundled datasets contain.
- [Estimating survival with Kaplan-Meier](kaplan-meier.md) puts this data to work with the first and most important estimator.
- [Survival data and the Surv object](survival-data.md) covers the response type in more detail if you skipped ahead.
- The [Quick start](quick-start.md) is a fast end-to-end tour if you prefer to skim first.