DataFrame Backend (pandas or Polars)#

LiveF1 builds every table of a session’s data lake, from the raw Bronze tables to the Silver and Gold tables, with one DataFrame library: pandas (the default) or Polars. Both backends produce the same tables with the same columns and values. Polars is faster and uses less memory.

Installing Polars#

pandas is always installed. Polars is an optional dependency:

pip install "livef1[polars]"

Choosing the backend#

Set the backend with livef1.configure() before loading data:

import livef1

livef1.configure(dataframe_backend="polars")

session = livef1.get_session(2026, "British", "Race")
session.generate()

session.laps                 # polars.DataFrame
session.carTelemetry         # polars.DataFrame
session.get_data("CarData.z")  # polars.DataFrame (Bronze)

dataframe_backend accepts "pandas" (default) or "polars". It is only set through livef1.configure(): get_session() and Session do not take a backend argument. An unknown name raises ValueError, and choosing "polars" without Polars installed raises ImportError with the install command.

What the backend changes#

With the Polars backend, nothing in the data lake is pandas and nothing is converted between the two libraries:

  • Bronze tables are built directly from the parsed livetiming records as polars.DataFrame, with the same column names, number types and timestamp parsing as the pandas tables.

  • Silver default tables (laps, carTelemetry, raceControlMessages) are built by Polars implementations of the pandas silver functions (livef1.data_processing.silver_functions_polars).

  • Gold and custom tables receive Polars source tables (see Custom tables).

  • session.get_data(), session.get_laps(), session.get_car_telemetry() and the table attributes such as session.laps return Polars DataFrames.

When the backend is applied#

A session takes the configured backend when it builds its first table (usually when it loads its first Bronze topic, or when generate() runs) and keeps it from then on, so all of its tables share one format. Changing configure() later only affects sessions that have not loaded any data yet:

livef1.configure(dataframe_backend="polars")
race = livef1.get_session(2026, "British", "Race")
race.get_data("TimingData")       # race is now a Polars session

livef1.configure(dataframe_backend="pandas")
quali = livef1.get_session(2026, "British", "Qualifying")
quali.generate()                  # quali uses pandas

race.dataframe_backend            # "polars"
quali.dataframe_backend           # "pandas"

session.dataframe_backend is read-only.

Custom tables#

Tables registered with create_silver_table and create_gold_table receive their source tables in the session’s backend, and must return a DataFrame of the same library. Returning the other library’s DataFrame raises TypeError; LiveF1 does not convert it. Write the table function with the API of the backend you configure:

Example: a custom Silver table with the Polars backend#
import livef1
import polars as pl

livef1.configure(dataframe_backend="polars")
session = livef1.get_session(2026, "British", "Race")

@session.create_silver_table(table_name="FastestLaps", source_tables=["laps"])
def fastest_laps(laps):
    return (
        laps.filter(~pl.col("IsDeleted"))
        .group_by("DriverNo")
        .agg(pl.col("LapTime").min())
        .sort("LapTime")
    )

session.generate()
print(session.FastestLaps.head(3))

The pandas examples in Registering Custom Tables work unchanged with the default pandas backend.

Differences between the backends#

The values are the same; a few representation details follow each library:

  • Types: pandas categorical columns (for example DriverNo, Compound, CarStatus) are plain strings in Polars, SessionKey is an integer, and time columns use Polars Duration and Datetime. Polars DataFrames have no index.

  • Driver column: the Driver column of laps holds LiveF1 Driver objects in both backends (a Polars Object column).

  • Track regions: the Circuit model keeps its track_regions as a pandas DataFrame because a circuit is shared by sessions of either backend. The track_regions Bronze table of a session follows the session’s backend.

  • TrackRegion boundaries: a telemetry sample whose Distance falls exactly on a corner boundary can, very rarely, be labelled with the neighbouring region because of floating-point rounding (2 of about 840,000 samples in the 2026 British Grand Prix).

Performance#

Measured on the 2026 British Grand Prix race on an Apple Silicon laptop, with the raw data already downloaded:

Step

pandas

Polars

Speed-up

Build Bronze tables from parsed records

2.0 s

1.3 s

1.6x

laps

1.4 s

0.12 s

11x

carTelemetry

2.2 s

0.65 s

3.4x

All Silver tables

3.7 s

0.8 s

4.6x

Size of the Bronze tables

184 MB

121 MB

Size of the Silver tables

112 MB

87 MB

Peak extra memory while generating

45-97 MB

58-61 MB

To compare the two backends on any session, run the comparison script from the repository:

python examples/polars_silver_compare.py 2026 British Race

It builds the session with each backend, checks that the tables match, and prints the timings and memory use.