laketower

Name	laketower JSON
Version	0.6.3 JSON
	download
home_page	None
Summary	Oversee your lakehouse
upload_time	2025-10-19 21:48:55
maintainer	None
docs_url	None
author	None
requires_python	<3.14,>=3.10
license	Apache-2.0
keywords	data delta-lake lakehouse sql
VCS
bugtrack_url
requirements	No requirements were recorded.
Travis-CI	No Travis.
coveralls test coverage	No coveralls.

            # 🗼 Laketower

> Oversee your lakehouse

[![PyPI](https://img.shields.io/pypi/v/laketower.svg)](https://pypi.org/project/laketower/)
[![Python Versions](https://img.shields.io/pypi/pyversions/laketower?logo=python&logoColor=white)](https://pypi.org/project/laketower/)
[![CI/CD](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml)
[![License](https://img.shields.io/github/license/datalpia/laketower)](https://github.com/datalpia/laketower/blob/main/LICENSE)

Utility application to explore and manage tables in your data lakehouse, especially tailored for data pipelines local development.

## Features

- Delta Lake table format support
- Remote tables support (S3, ADLS)
- Inspect table metadata
- Inspect table schema
- Inspect table history
- Get table statistics
- Import data into a table from CSV files
- View table content with a simple query builder
- Query all registered tables with DuckDB SQL dialect
- Execute saved queries
- Export query results to CSV files
- Static and versionable YAML configuration
- Web application
- CLI application

## Installation

Using `pip` (or any other Python package manager):

```bash
pip install laketower
```

Using `uvx`:

```bash
uvx laketower
```

## Usage

### Configuration

Laketower configuration is based on a static YAML configuration file allowing to:

- List all tables to be registered

Format:

```yaml
tables:
  - name: <table_name>
    uri: <local path to table>
    format: {delta}

queries:
  - name: <query_name>
    title: <Query name>
    description: <Query description>
    parameters:
      <param_name_1>:
        default: <default_value>
    sql: <sql expression>
```

Current limitations:

- `tables.uri`:
    - Local paths are supported (`./path/to/table`, `/abs/path/to/table`, `file:///abs/path/to/table`)
    - Remote paths to S3 (`s3://<bucket>/<path>`) and ADLS (`abfss://<container>/<path>`)
- `tables.format`: only `delta` is allowed

Example from the provided demo:

```yaml
tables:
  - name: sample_table
    uri: demo/sample_table
    format: delta
  - name: weather
    uri: demo/weather
    format: delta

queries:
  - name: all_data
    title: All data
    sql: |
      select
        sample_table.*,
        weather.*
      from
        sample_table,
        weather
      limit 10
  - name: daily_avg_temperature
    title: Daily average temperature
    sql: |
      select
        date_trunc('day', time) as day,
        round(avg(temperature_2m)) as avg_temperature
      from
        weather
      group by
        day
      order by
        day asc
```

Support for environment variables substitution is also supported within the YAML
configuration using a object containing a single key `env` with the name of the
environment variable to be injected. The value of the variable can contain JSON
and will be decoded in a best effort manner (default to string value). For instance:

```yaml
# export TABLE_URI=path/to/table

tables:
  - name: sample_table
    uri:
      env: TABLE_URI
    format: delta
```

#### Remote S3 Tables

Configuring S3 tables (AWS, MinIO, Cloudflare R2):

```yaml
tables:
  - name: delta_table_s3
    uri: s3://<bucket>/path/to/table
    format: delta
    connection:
      s3:
        s3_access_key_id: access-key-id
        s3_secret_access_key: secret-access-key
        s3_region: s3-region
        s3_endpoint_url: http://s3.domain.com
        s3_allow_http: false
```

Depending on your object storage location and configuration, one might have to
set part or all the available `connection.s3` parameters. The only required ones
are `s3_access_key_id` and `s3_secret_access_key`.

Also as a security best practice, it is best not to write secrets directly in
static configuration files, so one can use environment variables to all dynamic substitution,
e.g.

```yaml
tables:
  - name: delta_table_s3
    uri: s3://<bucket>/path/to/table
    format: delta
    connection:
      s3:
        s3_access_key_id: access-key-id
        s3_secret_access_key:
          env: S3_SECRET_ACCESS_KEY
        s3_region: s3-region
        s3_endpoint_url: http://s3.domain.com
        s3_allow_http: false
```

#### Remote ADLS Tables

Configuring Azure ADLS tables:

```yaml
tables:
  - name: delta_table_adls
    uri: abfss://<container>/path/to/table
    format: delta
    connection:
      adls:
        adls_account_name: adls-account-name
        adls_access_key: adls-access-key
        adls_sas_key: adls-sas-key
        adls_tenant_id: adls-tenant-id
        adls_client_id: adls-client-id
        adls_client_secret: adls-client-secret
        azure_msi_endpoint: https://msi.azure.com
        use_azure_cli: false
```

Depending on your object storage location and configuration, one might have to
set part or all the available `connection.adls` parameters. The only required one
is `adls_account_name`.

Also as a security best practice, it is best not to write secrets directly in
static configuration files, so one can use environment variables to all dynamic substitution,
e.g.

```yaml
tables:
  - name: delta_table_adls
    uri: abfss://<container>/path/to/table
    format: delta
    connection:
      adls:
        adls_account_name: adls-account-name
        adls_access_key:
          env: ADLS_ACCESS_KEY
```

### Web Application

The easiest way to get started is to launch the Laketower web application:

```bash
$ laketower -c demo/laketower.yml web
```

#### Screenshots

![Laketower UI - Tables Overview](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_overview.png)
![Laketower UI - Tables View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_view.png)
![Laketower UI - Tables Statistics](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_statistics.png)
![Laketower UI - Tables History](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_history.png)
![Laketower UI - Tables Import](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_import.png)
![Laketower UI - Tables Query](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_query.png)
![Laketower UI - Queries View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/queries_view.png)

### CLI

Laketower provides a CLI interface:

```bash
$ laketower --help

usage: laketower [-h] [--version] [--config CONFIG] {web,config,tables,queries} ...

options:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  --config, -c CONFIG   Path to the Laketower YAML configuration file (default: laketower.yml)

commands:
  {web,config,tables,queries}
    web                 Launch the web application
    config              Work with configuration
    tables              Work with tables
    queries             Work with queries
```

By default, a YAML configuration file named `laketower.yml` will be looked for.
A custom path can be specified with the `-c` / `--config` argument.

#### Validate YAML configuration

```bash
$ laketower -c demo/laketower.yml config validate

╭────────────────────────╮
│ Configuration is valid │
╰────────────────────────╯
Config(
    tables=[
        ConfigTable(name='sample_table', uri='demo/sample_table', table_format=<TableFormats.delta: 'delta'>),
        ConfigTable(name='weather', uri='demo/weather', table_format=<TableFormats.delta: 'delta'>)
    ]
)
```

#### List all registered tables

```bash
$ laketower -c demo/laketower.yml tables list

tables
├── sample_table
│   ├── format: delta
│   └── uri: demo/sample_table
└── weather
    ├── format: delta
    └── uri: demo/weather
```

#### Display a given table metadata

```bash
$ laketower -c demo/laketower.yml tables metadata sample_table

sample_table
├── name: Demo table
├── description: A sample demo Delta table
├── format: delta
├── uri: /Users/romain/Documents/dev/datalpia/laketower/demo/sample_table/
├── id: c1cb1cf0-1f3f-47b5-a660-3cc800edd341
├── version: 3
├── created at: 2025-02-05 22:27:39.579000+00:00
├── partitions:
└── configuration: {}
```

#### Display a given table schema

```bash
$ laketower -c demo/laketower.yml tables schema weather

weather
├── time: timestamp[us, tz=UTC]
├── city: string
├── temperature_2m: float
├── relative_humidity_2m: float
└── wind_speed_10m: float
```

#### Display a given table history

```bash
$ uv run laketower -c demo/laketower.yml tables history weather

weather
├── version: 2
│   ├── timestamp: 2025-02-05 22:27:46.425000+00:00
│   ├── client version: delta-rs.0.23.1
│   ├── operation: WRITE
│   ├── operation parameters
│   │   └── mode: Append
│   └── operation metrics
│       ├── execution_time_ms: 4
│       ├── num_added_files: 1
│       ├── num_added_rows: 168
│       ├── num_partitions: 0
│       └── num_removed_files: 0
├── version: 1
│   ├── timestamp: 2025-02-05 22:27:45.666000+00:00
│   ├── client version: delta-rs.0.23.1
│   ├── operation: WRITE
│   ├── operation parameters
│   │   └── mode: Append
│   └── operation metrics
│       ├── execution_time_ms: 4
│       ├── num_added_files: 1
│       ├── num_added_rows: 408
│       ├── num_partitions: 0
│       └── num_removed_files: 0
└── version: 0
    ├── timestamp: 2025-02-05 22:27:39.722000+00:00
    ├── client version: delta-rs.0.23.1
    ├── operation: CREATE TABLE
    ├── operation parameters
    │   ├── metadata: {"configuration":{},"createdTime":1738794459722,"description":"Historical and forecast weather data from
    │   │   open-meteo.com","format":{"options":{},"provider":"parquet"},"id":"a9615fb1-25cc-4546-a0fe-1cb534c514b2","name":"Weather","partitionCol
    │   │   umns":[],"schemaString":"{\"type\":\"struct\",\"fields\":[{\"name\":\"time\",\"type\":\"timestamp\",\"nullable\":true,\"metadata\":{}},
    │   │   {\"name\":\"city\",\"type\":\"string\",\"nullable\":true,\"metadata\":{}},{\"name\":\"temperature_2m\",\"type\":\"float\",\"nullable\":
    │   │   true,\"metadata\":{}},{\"name\":\"relative_humidity_2m\",\"type\":\"float\",\"nullable\":true,\"metadata\":{}},{\"name\":\"wind_speed_1
    │   │   0m\",\"type\":\"float\",\"nullable\":true,\"metadata\":{}}]}"}
    │   ├── protocol: {"minReaderVersion":1,"minWriterVersion":2}
    │   ├── mode: ErrorIfExists
    │   └── location: file:///Users/romain/Documents/dev/datalpia/laketower/demo/weather
    └── operation metrics
```

#### Get statistics of a given table

Get basic statistics on all columns of a given table:

```bash
$ laketower -c demo/laketower.yml tables statistics weather

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ column_name          ┃ count ┃ avg                ┃ std                ┃ min                    ┃ max                    ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩
│ time                 │ 576   │ None               │ None               │ 2025-01-26 01:00:00+01 │ 2025-02-12 00:00:00+01 │
│ city                 │ 576   │ None               │ None               │ Grenoble               │ Grenoble               │
│ temperature_2m       │ 576   │ 5.2623263956047595 │ 3.326529069892729  │ 0.0                    │ 15.1                   │
│ relative_humidity_2m │ 576   │ 78.76909722222223  │ 15.701802163559918 │ 29.0                   │ 100.0                  │
│ wind_speed_10m       │ 576   │ 7.535763886032833  │ 10.00898058743763  │ 0.0                    │ 42.4                   │
└──────────────────────┴───────┴────────────────────┴────────────────────┴────────────────────────┴────────────────────────┘
```

Specifying a table version yields according results:

```bash
$ laketower -c demo/laketower.yml tables statistics --version 0 weather

┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┓
┃ column_name          ┃ count ┃ avg  ┃ std  ┃ min  ┃ max  ┃
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━┩
│ time                 │ 0     │ None │ None │ None │ None │
│ city                 │ 0     │ None │ None │ None │ None │
│ temperature_2m       │ 0     │ None │ None │ None │ None │
│ relative_humidity_2m │ 0     │ None │ None │ None │ None │
│ wind_speed_10m       │ 0     │ None │ None │ None │ None │
└──────────────────────┴───────┴──────┴──────┴──────┴──────┘
```

#### Import data into a given table

Import a CSV dataset into a table in append mode:

```bash
$ laketower -c demo/laketower.yml tables import weather --file data.csv --mode append --format csv --delimiter ',' --encoding 'utf-8'
```

`--mode` argument can be one of:
- `append`: append rows to the table (default)
- `overwrite`: replace all rows with the ones from the input file

`--format` argument can be one of:
- `csv`: CSV file format (default)

`--delimiter` argument can be:
- Any single character (only valid for CSV file format)
- Default is _comma_ (`','`)

`--encoding` argument can be:
- Any [standard Python encoding](https://docs.python.org/3/library/codecs.html#standard-encodings),
- Default is `'utf-8'`

#### View a given table

Using a simple query builder, the content of a table can be displayed.
Optional arguments:

- `--cols <col1> <col2>`: select which columns to display
- `--sort-asc <col>`: sort by a column name in ascending order
- `--sort-desc <col>`: sort by a column name in descending order
- `--limit <num>` (default 10): limit the number of rows
- `--version`: time-travel to table revision number

```bash
$ laketower -c demo/laketower.yml tables view weather

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ time                      ┃ city     ┃ temperature_2m     ┃ relative_humidity_2m ┃ wind_speed_10m    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ 2025-02-05 01:00:00+01:00 │ Grenoble │ 2.0                │ 84.0                 │ 4.0               │
│ 2025-02-05 02:00:00+01:00 │ Grenoble │ 2.0999999046325684 │ 83.0                 │ 1.5               │
│ 2025-02-05 03:00:00+01:00 │ Grenoble │ 1.600000023841858  │ 86.0                 │ 1.100000023841858 │
│ 2025-02-05 04:00:00+01:00 │ Grenoble │ 1.899999976158142  │ 80.0                 │ 4.199999809265137 │
│ 2025-02-05 05:00:00+01:00 │ Grenoble │ 1.899999976158142  │ 81.0                 │ 3.299999952316284 │
│ 2025-02-05 06:00:00+01:00 │ Grenoble │ 1.399999976158142  │ 88.0                 │ 4.300000190734863 │
│ 2025-02-05 07:00:00+01:00 │ Grenoble │ 1.7000000476837158 │ 87.0                 │ 5.5               │
│ 2025-02-05 08:00:00+01:00 │ Grenoble │ 1.5                │ 82.0                 │ 4.699999809265137 │
│ 2025-02-05 09:00:00+01:00 │ Grenoble │ 1.899999976158142  │ 80.0                 │ 2.200000047683716 │
│ 2025-02-05 10:00:00+01:00 │ Grenoble │ 2.9000000953674316 │ 80.0                 │ 0.800000011920929 │
└───────────────────────────┴──────────┴────────────────────┴──────────────────────┴───────────────────┘
```

```bash
$ laketower -c demo/laketower.yml tables view weather --cols time city temperature_2m --limit 5 --sort-desc time

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┓
┃ time                      ┃ city     ┃ temperature_2m    ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━┩
│ 2025-02-12 00:00:00+01:00 │ Grenoble │ 5.099999904632568 │
│ 2025-02-12 00:00:00+01:00 │ Grenoble │ 5.099999904632568 │
│ 2025-02-11 23:00:00+01:00 │ Grenoble │ 4.900000095367432 │
│ 2025-02-11 23:00:00+01:00 │ Grenoble │ 4.900000095367432 │
│ 2025-02-11 22:00:00+01:00 │ Grenoble │ 4.900000095367432 │
└───────────────────────────┴──────────┴───────────────────┘
```

```bash
$ laketower -c demo/laketower.yml tables view weather --version 1

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ time                      ┃ city     ┃ temperature_2m    ┃ relative_humidity_2m ┃ wind_speed_10m     ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ 2025-01-26 01:00:00+01:00 │ Grenoble │ 7.0               │ 87.0                 │ 8.899999618530273  │
│ 2025-01-26 02:00:00+01:00 │ Grenoble │ 6.099999904632568 │ 87.0                 │ 6.199999809265137  │
│ 2025-01-26 03:00:00+01:00 │ Grenoble │ 6.0               │ 86.0                 │ 2.700000047683716  │
│ 2025-01-26 04:00:00+01:00 │ Grenoble │ 6.099999904632568 │ 82.0                 │ 3.0999999046325684 │
│ 2025-01-26 05:00:00+01:00 │ Grenoble │ 5.5               │ 87.0                 │ 3.299999952316284  │
│ 2025-01-26 06:00:00+01:00 │ Grenoble │ 5.199999809265137 │ 91.0                 │ 2.200000047683716  │
│ 2025-01-26 07:00:00+01:00 │ Grenoble │ 4.800000190734863 │ 86.0                 │ 3.0                │
│ 2025-01-26 08:00:00+01:00 │ Grenoble │ 4.900000095367432 │ 83.0                 │ 1.100000023841858  │
│ 2025-01-26 09:00:00+01:00 │ Grenoble │ 4.0               │ 92.0                 │ 3.0999999046325684 │
│ 2025-01-26 10:00:00+01:00 │ Grenoble │ 5.0               │ 86.0                 │ 6.400000095367432  │
└───────────────────────────┴──────────┴───────────────────┴──────────────────────┴────────────────────┘
```

#### Query all registered tables

Query any registered tables using DuckDB SQL dialect!

```bash
$ laketower -c demo/laketower.yml tables query "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3"

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ day                       ┃ mean_temperature   ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ 2025-02-12 00:00:00+01:00 │ 5.099999904632568  │
│ 2025-02-11 00:00:00+01:00 │ 4.833333373069763  │
│ 2025-02-10 00:00:00+01:00 │ 2.1083333243926368 │
└───────────────────────────┴────────────────────┘
```

Use named parameters within a giving query (note: escape `$` prefixes properly!):

```bash
$ laketower -c demo/laketower.yml tables query "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather where day between \$start_date and \$end_date group by day order by day desc" -p start_date 2025-01-29 -p end_date 2025-01-31

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┓
┃ day                       ┃ mean_temperature   ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━┩
│ 2025-01-31 00:00:00+01:00 │ 5.683333257834117  │
│ 2025-01-30 00:00:00+01:00 │ 8.900000015894571  │
│ 2025-01-29 00:00:00+01:00 │ 7.770833313465118  │
└───────────────────────────┴────────────────────┘
```

Export query results to CSV:

```bash
$ laketower -c demo/laketower.yml tables query --output results.csv "select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3"

Query results written to: results.csv
```

#### List saved queries

```bash
$ laketower -c demo/laketower.yml queries list

queries
├── all_data
└── daily_avg_temperature
```

#### Execute saved queries

```bash
$ laketower -c demo/laketower.yml queries view daily_avg_temperature

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ day                       ┃ avg_temperature ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 2025-01-26 00:00:00+01:00 │ 8.0             │
│ 2025-01-27 00:00:00+01:00 │ 13.0            │
│ 2025-01-28 00:00:00+01:00 │ 7.0             │
│ 2025-01-29 00:00:00+01:00 │ 8.0             │
│ 2025-01-30 00:00:00+01:00 │ 9.0             │
│ 2025-01-31 00:00:00+01:00 │ 6.0             │
│ 2025-02-01 00:00:00+01:00 │ 4.0             │
│ 2025-02-02 00:00:00+01:00 │ 4.0             │
│ 2025-02-03 00:00:00+01:00 │ 4.0             │
│ 2025-02-04 00:00:00+01:00 │ 3.0             │
│ 2025-02-05 00:00:00+01:00 │ 3.0             │
│ 2025-02-06 00:00:00+01:00 │ 2.0             │
│ 2025-02-07 00:00:00+01:00 │ 6.0             │
│ 2025-02-08 00:00:00+01:00 │ 7.0             │
│ 2025-02-09 00:00:00+01:00 │ 5.0             │
│ 2025-02-10 00:00:00+01:00 │ 2.0             │
│ 2025-02-11 00:00:00+01:00 │ 5.0             │
│ 2025-02-12 00:00:00+01:00 │ 5.0             │
└───────────────────────────┴─────────────────┘
```

Executing a predefined query with parameters (here `start_date` and `end_date`):

```bash
$ laketower -c demo/laketower.yml queries view daily_avg_temperature_params -p start_date 2025-02-01 -p end_date 2025-02-05

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓
┃ day                       ┃ avg_temperature ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩
│ 2025-02-01 00:00:00+01:00 │ 4.0             │
│ 2025-02-02 00:00:00+01:00 │ 4.0             │
│ 2025-02-03 00:00:00+01:00 │ 4.0             │
│ 2025-02-04 00:00:00+01:00 │ 3.0             │
│ 2025-02-05 00:00:00+01:00 │ 3.0             │
└───────────────────────────┴─────────────────┘
```

## License

Licensed under [Apache License 2.0](LICENSE)

Copyright (c) 2025 - present Romain Clement / Datalpia

            

Raw data

            {
    "_id": null,
    "home_page": null,
    "name": "laketower",
    "maintainer": null,
    "docs_url": null,
    "requires_python": "<3.14,>=3.10",
    "maintainer_email": null,
    "keywords": "data, delta-lake, lakehouse, sql",
    "author": null,
    "author_email": "Romain Clement <git@romain-clement.net>",
    "download_url": "https://files.pythonhosted.org/packages/1c/df/9467252ac394236ed9f5b27e49c4c9517d8e3bdc0592997d74978eed1c47/laketower-0.6.3.tar.gz",
    "platform": null,
    "description": "# \ud83d\uddfc Laketower\n\n> Oversee your lakehouse\n\n[![PyPI](https://img.shields.io/pypi/v/laketower.svg)](https://pypi.org/project/laketower/)\n[![Python Versions](https://img.shields.io/pypi/pyversions/laketower?logo=python&logoColor=white)](https://pypi.org/project/laketower/)\n[![CI/CD](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml/badge.svg)](https://github.com/datalpia/laketower/actions/workflows/ci-cd.yml)\n[![License](https://img.shields.io/github/license/datalpia/laketower)](https://github.com/datalpia/laketower/blob/main/LICENSE)\n\nUtility application to explore and manage tables in your data lakehouse, especially tailored for data pipelines local development.\n\n## Features\n\n- Delta Lake table format support\n- Remote tables support (S3, ADLS)\n- Inspect table metadata\n- Inspect table schema\n- Inspect table history\n- Get table statistics\n- Import data into a table from CSV files\n- View table content with a simple query builder\n- Query all registered tables with DuckDB SQL dialect\n- Execute saved queries\n- Export query results to CSV files\n- Static and versionable YAML configuration\n- Web application\n- CLI application\n\n## Installation\n\nUsing `pip` (or any other Python package manager):\n\n```bash\npip install laketower\n```\n\nUsing `uvx`:\n\n```bash\nuvx laketower\n```\n\n## Usage\n\n### Configuration\n\nLaketower configuration is based on a static YAML configuration file allowing to:\n\n- List all tables to be registered\n\nFormat:\n\n```yaml\ntables:\n  - name: <table_name>\n    uri: <local path to table>\n    format: {delta}\n\nqueries:\n  - name: <query_name>\n    title: <Query name>\n    description: <Query description>\n    parameters:\n      <param_name_1>:\n        default: <default_value>\n    sql: <sql expression>\n```\n\nCurrent limitations:\n\n- `tables.uri`:\n    - Local paths are supported (`./path/to/table`, `/abs/path/to/table`, `file:///abs/path/to/table`)\n    - Remote paths to S3 (`s3://<bucket>/<path>`) and ADLS (`abfss://<container>/<path>`)\n- `tables.format`: only `delta` is allowed\n\nExample from the provided demo:\n\n```yaml\ntables:\n  - name: sample_table\n    uri: demo/sample_table\n    format: delta\n  - name: weather\n    uri: demo/weather\n    format: delta\n\nqueries:\n  - name: all_data\n    title: All data\n    sql: |\n      select\n        sample_table.*,\n        weather.*\n      from\n        sample_table,\n        weather\n      limit 10\n  - name: daily_avg_temperature\n    title: Daily average temperature\n    sql: |\n      select\n        date_trunc('day', time) as day,\n        round(avg(temperature_2m)) as avg_temperature\n      from\n        weather\n      group by\n        day\n      order by\n        day asc\n```\n\nSupport for environment variables substitution is also supported within the YAML\nconfiguration using a object containing a single key `env` with the name of the\nenvironment variable to be injected. The value of the variable can contain JSON\nand will be decoded in a best effort manner (default to string value). For instance:\n\n```yaml\n# export TABLE_URI=path/to/table\n\ntables:\n  - name: sample_table\n    uri:\n      env: TABLE_URI\n    format: delta\n```\n\n#### Remote S3 Tables\n\nConfiguring S3 tables (AWS, MinIO, Cloudflare R2):\n\n```yaml\ntables:\n  - name: delta_table_s3\n    uri: s3://<bucket>/path/to/table\n    format: delta\n    connection:\n      s3:\n        s3_access_key_id: access-key-id\n        s3_secret_access_key: secret-access-key\n        s3_region: s3-region\n        s3_endpoint_url: http://s3.domain.com\n        s3_allow_http: false\n```\n\nDepending on your object storage location and configuration, one might have to\nset part or all the available `connection.s3` parameters. The only required ones\nare `s3_access_key_id` and `s3_secret_access_key`.\n\nAlso as a security best practice, it is best not to write secrets directly in\nstatic configuration files, so one can use environment variables to all dynamic substitution,\ne.g.\n\n```yaml\ntables:\n  - name: delta_table_s3\n    uri: s3://<bucket>/path/to/table\n    format: delta\n    connection:\n      s3:\n        s3_access_key_id: access-key-id\n        s3_secret_access_key:\n          env: S3_SECRET_ACCESS_KEY\n        s3_region: s3-region\n        s3_endpoint_url: http://s3.domain.com\n        s3_allow_http: false\n```\n\n#### Remote ADLS Tables\n\nConfiguring Azure ADLS tables:\n\n```yaml\ntables:\n  - name: delta_table_adls\n    uri: abfss://<container>/path/to/table\n    format: delta\n    connection:\n      adls:\n        adls_account_name: adls-account-name\n        adls_access_key: adls-access-key\n        adls_sas_key: adls-sas-key\n        adls_tenant_id: adls-tenant-id\n        adls_client_id: adls-client-id\n        adls_client_secret: adls-client-secret\n        azure_msi_endpoint: https://msi.azure.com\n        use_azure_cli: false\n```\n\nDepending on your object storage location and configuration, one might have to\nset part or all the available `connection.adls` parameters. The only required one\nis `adls_account_name`.\n\nAlso as a security best practice, it is best not to write secrets directly in\nstatic configuration files, so one can use environment variables to all dynamic substitution,\ne.g.\n\n```yaml\ntables:\n  - name: delta_table_adls\n    uri: abfss://<container>/path/to/table\n    format: delta\n    connection:\n      adls:\n        adls_account_name: adls-account-name\n        adls_access_key:\n          env: ADLS_ACCESS_KEY\n```\n\n### Web Application\n\nThe easiest way to get started is to launch the Laketower web application:\n\n```bash\n$ laketower -c demo/laketower.yml web\n```\n\n#### Screenshots\n\n![Laketower UI - Tables Overview](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_overview.png)\n![Laketower UI - Tables View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_view.png)\n![Laketower UI - Tables Statistics](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_statistics.png)\n![Laketower UI - Tables History](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_history.png)\n![Laketower UI - Tables Import](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_import.png)\n![Laketower UI - Tables Query](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/tables_query.png)\n![Laketower UI - Queries View](https://raw.githubusercontent.com/datalpia/laketower/refs/heads/main/docs/static/queries_view.png)\n\n### CLI\n\nLaketower provides a CLI interface:\n\n```bash\n$ laketower --help\n\nusage: laketower [-h] [--version] [--config CONFIG] {web,config,tables,queries} ...\n\noptions:\n  -h, --help            show this help message and exit\n  --version             show program's version number and exit\n  --config, -c CONFIG   Path to the Laketower YAML configuration file (default: laketower.yml)\n\ncommands:\n  {web,config,tables,queries}\n    web                 Launch the web application\n    config              Work with configuration\n    tables              Work with tables\n    queries             Work with queries\n```\n\nBy default, a YAML configuration file named `laketower.yml` will be looked for.\nA custom path can be specified with the `-c` / `--config` argument.\n\n#### Validate YAML configuration\n\n```bash\n$ laketower -c demo/laketower.yml config validate\n\n\u256d\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256e\n\u2502 Configuration is valid \u2502\n\u2570\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u256f\nConfig(\n    tables=[\n        ConfigTable(name='sample_table', uri='demo/sample_table', table_format=<TableFormats.delta: 'delta'>),\n        ConfigTable(name='weather', uri='demo/weather', table_format=<TableFormats.delta: 'delta'>)\n    ]\n)\n```\n\n#### List all registered tables\n\n```bash\n$ laketower -c demo/laketower.yml tables list\n\ntables\n\u251c\u2500\u2500 sample_table\n\u2502   \u251c\u2500\u2500 format: delta\n\u2502   \u2514\u2500\u2500 uri: demo/sample_table\n\u2514\u2500\u2500 weather\n    \u251c\u2500\u2500 format: delta\n    \u2514\u2500\u2500 uri: demo/weather\n```\n\n#### Display a given table metadata\n\n```bash\n$ laketower -c demo/laketower.yml tables metadata sample_table\n\nsample_table\n\u251c\u2500\u2500 name: Demo table\n\u251c\u2500\u2500 description: A sample demo Delta table\n\u251c\u2500\u2500 format: delta\n\u251c\u2500\u2500 uri: /Users/romain/Documents/dev/datalpia/laketower/demo/sample_table/\n\u251c\u2500\u2500 id: c1cb1cf0-1f3f-47b5-a660-3cc800edd341\n\u251c\u2500\u2500 version: 3\n\u251c\u2500\u2500 created at: 2025-02-05 22:27:39.579000+00:00\n\u251c\u2500\u2500 partitions:\n\u2514\u2500\u2500 configuration: {}\n```\n\n#### Display a given table schema\n\n```bash\n$ laketower -c demo/laketower.yml tables schema weather\n\nweather\n\u251c\u2500\u2500 time: timestamp[us, tz=UTC]\n\u251c\u2500\u2500 city: string\n\u251c\u2500\u2500 temperature_2m: float\n\u251c\u2500\u2500 relative_humidity_2m: float\n\u2514\u2500\u2500 wind_speed_10m: float\n```\n\n#### Display a given table history\n\n```bash\n$ uv run laketower -c demo/laketower.yml tables history weather\n\nweather\n\u251c\u2500\u2500 version: 2\n\u2502   \u251c\u2500\u2500 timestamp: 2025-02-05 22:27:46.425000+00:00\n\u2502   \u251c\u2500\u2500 client version: delta-rs.0.23.1\n\u2502   \u251c\u2500\u2500 operation: WRITE\n\u2502   \u251c\u2500\u2500 operation parameters\n\u2502   \u2502   \u2514\u2500\u2500 mode: Append\n\u2502   \u2514\u2500\u2500 operation metrics\n\u2502       \u251c\u2500\u2500 execution_time_ms: 4\n\u2502       \u251c\u2500\u2500 num_added_files: 1\n\u2502       \u251c\u2500\u2500 num_added_rows: 168\n\u2502       \u251c\u2500\u2500 num_partitions: 0\n\u2502       \u2514\u2500\u2500 num_removed_files: 0\n\u251c\u2500\u2500 version: 1\n\u2502   \u251c\u2500\u2500 timestamp: 2025-02-05 22:27:45.666000+00:00\n\u2502   \u251c\u2500\u2500 client version: delta-rs.0.23.1\n\u2502   \u251c\u2500\u2500 operation: WRITE\n\u2502   \u251c\u2500\u2500 operation parameters\n\u2502   \u2502   \u2514\u2500\u2500 mode: Append\n\u2502   \u2514\u2500\u2500 operation metrics\n\u2502       \u251c\u2500\u2500 execution_time_ms: 4\n\u2502       \u251c\u2500\u2500 num_added_files: 1\n\u2502       \u251c\u2500\u2500 num_added_rows: 408\n\u2502       \u251c\u2500\u2500 num_partitions: 0\n\u2502       \u2514\u2500\u2500 num_removed_files: 0\n\u2514\u2500\u2500 version: 0\n    \u251c\u2500\u2500 timestamp: 2025-02-05 22:27:39.722000+00:00\n    \u251c\u2500\u2500 client version: delta-rs.0.23.1\n    \u251c\u2500\u2500 operation: CREATE TABLE\n    \u251c\u2500\u2500 operation parameters\n    \u2502   \u251c\u2500\u2500 metadata: {\"configuration\":{},\"createdTime\":1738794459722,\"description\":\"Historical and forecast weather data from\n    \u2502   \u2502   open-meteo.com\",\"format\":{\"options\":{},\"provider\":\"parquet\"},\"id\":\"a9615fb1-25cc-4546-a0fe-1cb534c514b2\",\"name\":\"Weather\",\"partitionCol\n    \u2502   \u2502   umns\":[],\"schemaString\":\"{\\\"type\\\":\\\"struct\\\",\\\"fields\\\":[{\\\"name\\\":\\\"time\\\",\\\"type\\\":\\\"timestamp\\\",\\\"nullable\\\":true,\\\"metadata\\\":{}},\n    \u2502   \u2502   {\\\"name\\\":\\\"city\\\",\\\"type\\\":\\\"string\\\",\\\"nullable\\\":true,\\\"metadata\\\":{}},{\\\"name\\\":\\\"temperature_2m\\\",\\\"type\\\":\\\"float\\\",\\\"nullable\\\":\n    \u2502   \u2502   true,\\\"metadata\\\":{}},{\\\"name\\\":\\\"relative_humidity_2m\\\",\\\"type\\\":\\\"float\\\",\\\"nullable\\\":true,\\\"metadata\\\":{}},{\\\"name\\\":\\\"wind_speed_1\n    \u2502   \u2502   0m\\\",\\\"type\\\":\\\"float\\\",\\\"nullable\\\":true,\\\"metadata\\\":{}}]}\"}\n    \u2502   \u251c\u2500\u2500 protocol: {\"minReaderVersion\":1,\"minWriterVersion\":2}\n    \u2502   \u251c\u2500\u2500 mode: ErrorIfExists\n    \u2502   \u2514\u2500\u2500 location: file:///Users/romain/Documents/dev/datalpia/laketower/demo/weather\n    \u2514\u2500\u2500 operation metrics\n```\n\n#### Get statistics of a given table\n\nGet basic statistics on all columns of a given table:\n\n```bash\n$ laketower -c demo/laketower.yml tables statistics weather\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 column_name          \u2503 count \u2503 avg                \u2503 std                \u2503 min                    \u2503 max                    \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 time                 \u2502 576   \u2502 None               \u2502 None               \u2502 2025-01-26 01:00:00+01 \u2502 2025-02-12 00:00:00+01 \u2502\n\u2502 city                 \u2502 576   \u2502 None               \u2502 None               \u2502 Grenoble               \u2502 Grenoble               \u2502\n\u2502 temperature_2m       \u2502 576   \u2502 5.2623263956047595 \u2502 3.326529069892729  \u2502 0.0                    \u2502 15.1                   \u2502\n\u2502 relative_humidity_2m \u2502 576   \u2502 78.76909722222223  \u2502 15.701802163559918 \u2502 29.0                   \u2502 100.0                  \u2502\n\u2502 wind_speed_10m       \u2502 576   \u2502 7.535763886032833  \u2502 10.00898058743763  \u2502 0.0                    \u2502 42.4                   \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nSpecifying a table version yields according results:\n\n```bash\n$ laketower -c demo/laketower.yml tables statistics --version 0 weather\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 column_name          \u2503 count \u2503 avg  \u2503 std  \u2503 min  \u2503 max  \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 time                 \u2502 0     \u2502 None \u2502 None \u2502 None \u2502 None \u2502\n\u2502 city                 \u2502 0     \u2502 None \u2502 None \u2502 None \u2502 None \u2502\n\u2502 temperature_2m       \u2502 0     \u2502 None \u2502 None \u2502 None \u2502 None \u2502\n\u2502 relative_humidity_2m \u2502 0     \u2502 None \u2502 None \u2502 None \u2502 None \u2502\n\u2502 wind_speed_10m       \u2502 0     \u2502 None \u2502 None \u2502 None \u2502 None \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### Import data into a given table\n\nImport a CSV dataset into a table in append mode:\n\n```bash\n$ laketower -c demo/laketower.yml tables import weather --file data.csv --mode append --format csv --delimiter ',' --encoding 'utf-8'\n```\n\n`--mode` argument can be one of:\n- `append`: append rows to the table (default)\n- `overwrite`: replace all rows with the ones from the input file\n\n`--format` argument can be one of:\n- `csv`: CSV file format (default)\n\n`--delimiter` argument can be:\n- Any single character (only valid for CSV file format)\n- Default is _comma_ (`','`)\n\n`--encoding` argument can be:\n- Any [standard Python encoding](https://docs.python.org/3/library/codecs.html#standard-encodings),\n- Default is `'utf-8'`\n\n#### View a given table\n\nUsing a simple query builder, the content of a table can be displayed.\nOptional arguments:\n\n- `--cols <col1> <col2>`: select which columns to display\n- `--sort-asc <col>`: sort by a column name in ascending order\n- `--sort-desc <col>`: sort by a column name in descending order\n- `--limit <num>` (default 10): limit the number of rows\n- `--version`: time-travel to table revision number\n\n```bash\n$ laketower -c demo/laketower.yml tables view weather\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 time                      \u2503 city     \u2503 temperature_2m     \u2503 relative_humidity_2m \u2503 wind_speed_10m    \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-02-05 01:00:00+01:00 \u2502 Grenoble \u2502 2.0                \u2502 84.0                 \u2502 4.0               \u2502\n\u2502 2025-02-05 02:00:00+01:00 \u2502 Grenoble \u2502 2.0999999046325684 \u2502 83.0                 \u2502 1.5               \u2502\n\u2502 2025-02-05 03:00:00+01:00 \u2502 Grenoble \u2502 1.600000023841858  \u2502 86.0                 \u2502 1.100000023841858 \u2502\n\u2502 2025-02-05 04:00:00+01:00 \u2502 Grenoble \u2502 1.899999976158142  \u2502 80.0                 \u2502 4.199999809265137 \u2502\n\u2502 2025-02-05 05:00:00+01:00 \u2502 Grenoble \u2502 1.899999976158142  \u2502 81.0                 \u2502 3.299999952316284 \u2502\n\u2502 2025-02-05 06:00:00+01:00 \u2502 Grenoble \u2502 1.399999976158142  \u2502 88.0                 \u2502 4.300000190734863 \u2502\n\u2502 2025-02-05 07:00:00+01:00 \u2502 Grenoble \u2502 1.7000000476837158 \u2502 87.0                 \u2502 5.5               \u2502\n\u2502 2025-02-05 08:00:00+01:00 \u2502 Grenoble \u2502 1.5                \u2502 82.0                 \u2502 4.699999809265137 \u2502\n\u2502 2025-02-05 09:00:00+01:00 \u2502 Grenoble \u2502 1.899999976158142  \u2502 80.0                 \u2502 2.200000047683716 \u2502\n\u2502 2025-02-05 10:00:00+01:00 \u2502 Grenoble \u2502 2.9000000953674316 \u2502 80.0                 \u2502 0.800000011920929 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n```bash\n$ laketower -c demo/laketower.yml tables view weather --cols time city temperature_2m --limit 5 --sort-desc time\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 time                      \u2503 city     \u2503 temperature_2m    \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-02-12 00:00:00+01:00 \u2502 Grenoble \u2502 5.099999904632568 \u2502\n\u2502 2025-02-12 00:00:00+01:00 \u2502 Grenoble \u2502 5.099999904632568 \u2502\n\u2502 2025-02-11 23:00:00+01:00 \u2502 Grenoble \u2502 4.900000095367432 \u2502\n\u2502 2025-02-11 23:00:00+01:00 \u2502 Grenoble \u2502 4.900000095367432 \u2502\n\u2502 2025-02-11 22:00:00+01:00 \u2502 Grenoble \u2502 4.900000095367432 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n```bash\n$ laketower -c demo/laketower.yml tables view weather --version 1\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 time                      \u2503 city     \u2503 temperature_2m    \u2503 relative_humidity_2m \u2503 wind_speed_10m     \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-01-26 01:00:00+01:00 \u2502 Grenoble \u2502 7.0               \u2502 87.0                 \u2502 8.899999618530273  \u2502\n\u2502 2025-01-26 02:00:00+01:00 \u2502 Grenoble \u2502 6.099999904632568 \u2502 87.0                 \u2502 6.199999809265137  \u2502\n\u2502 2025-01-26 03:00:00+01:00 \u2502 Grenoble \u2502 6.0               \u2502 86.0                 \u2502 2.700000047683716  \u2502\n\u2502 2025-01-26 04:00:00+01:00 \u2502 Grenoble \u2502 6.099999904632568 \u2502 82.0                 \u2502 3.0999999046325684 \u2502\n\u2502 2025-01-26 05:00:00+01:00 \u2502 Grenoble \u2502 5.5               \u2502 87.0                 \u2502 3.299999952316284  \u2502\n\u2502 2025-01-26 06:00:00+01:00 \u2502 Grenoble \u2502 5.199999809265137 \u2502 91.0                 \u2502 2.200000047683716  \u2502\n\u2502 2025-01-26 07:00:00+01:00 \u2502 Grenoble \u2502 4.800000190734863 \u2502 86.0                 \u2502 3.0                \u2502\n\u2502 2025-01-26 08:00:00+01:00 \u2502 Grenoble \u2502 4.900000095367432 \u2502 83.0                 \u2502 1.100000023841858  \u2502\n\u2502 2025-01-26 09:00:00+01:00 \u2502 Grenoble \u2502 4.0               \u2502 92.0                 \u2502 3.0999999046325684 \u2502\n\u2502 2025-01-26 10:00:00+01:00 \u2502 Grenoble \u2502 5.0               \u2502 86.0                 \u2502 6.400000095367432  \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n#### Query all registered tables\n\nQuery any registered tables using DuckDB SQL dialect!\n\n```bash\n$ laketower -c demo/laketower.yml tables query \"select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3\"\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 day                       \u2503 mean_temperature   \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-02-12 00:00:00+01:00 \u2502 5.099999904632568  \u2502\n\u2502 2025-02-11 00:00:00+01:00 \u2502 4.833333373069763  \u2502\n\u2502 2025-02-10 00:00:00+01:00 \u2502 2.1083333243926368 \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nUse named parameters within a giving query (note: escape `$` prefixes properly!):\n\n```bash\n$ laketower -c demo/laketower.yml tables query \"select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather where day between \\$start_date and \\$end_date group by day order by day desc\" -p start_date 2025-01-29 -p end_date 2025-01-31\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 day                       \u2503 mean_temperature   \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-01-31 00:00:00+01:00 \u2502 5.683333257834117  \u2502\n\u2502 2025-01-30 00:00:00+01:00 \u2502 8.900000015894571  \u2502\n\u2502 2025-01-29 00:00:00+01:00 \u2502 7.770833313465118  \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nExport query results to CSV:\n\n```bash\n$ laketower -c demo/laketower.yml tables query --output results.csv \"select date_trunc('day', time) as day, avg(temperature_2m) as mean_temperature from weather group by day order by day desc limit 3\"\n\nQuery results written to: results.csv\n```\n\n#### List saved queries\n\n```bash\n$ laketower -c demo/laketower.yml queries list\n\nqueries\n\u251c\u2500\u2500 all_data\n\u2514\u2500\u2500 daily_avg_temperature\n```\n\n#### Execute saved queries\n\n```bash\n$ laketower -c demo/laketower.yml queries view daily_avg_temperature\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 day                       \u2503 avg_temperature \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-01-26 00:00:00+01:00 \u2502 8.0             \u2502\n\u2502 2025-01-27 00:00:00+01:00 \u2502 13.0            \u2502\n\u2502 2025-01-28 00:00:00+01:00 \u2502 7.0             \u2502\n\u2502 2025-01-29 00:00:00+01:00 \u2502 8.0             \u2502\n\u2502 2025-01-30 00:00:00+01:00 \u2502 9.0             \u2502\n\u2502 2025-01-31 00:00:00+01:00 \u2502 6.0             \u2502\n\u2502 2025-02-01 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-02 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-03 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-04 00:00:00+01:00 \u2502 3.0             \u2502\n\u2502 2025-02-05 00:00:00+01:00 \u2502 3.0             \u2502\n\u2502 2025-02-06 00:00:00+01:00 \u2502 2.0             \u2502\n\u2502 2025-02-07 00:00:00+01:00 \u2502 6.0             \u2502\n\u2502 2025-02-08 00:00:00+01:00 \u2502 7.0             \u2502\n\u2502 2025-02-09 00:00:00+01:00 \u2502 5.0             \u2502\n\u2502 2025-02-10 00:00:00+01:00 \u2502 2.0             \u2502\n\u2502 2025-02-11 00:00:00+01:00 \u2502 5.0             \u2502\n\u2502 2025-02-12 00:00:00+01:00 \u2502 5.0             \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\nExecuting a predefined query with parameters (here `start_date` and `end_date`):\n\n```bash\n$ laketower -c demo/laketower.yml queries view daily_avg_temperature_params -p start_date 2025-02-01 -p end_date 2025-02-05\n\n\u250f\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2533\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2513\n\u2503 day                       \u2503 avg_temperature \u2503\n\u2521\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2547\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2501\u2529\n\u2502 2025-02-01 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-02 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-03 00:00:00+01:00 \u2502 4.0             \u2502\n\u2502 2025-02-04 00:00:00+01:00 \u2502 3.0             \u2502\n\u2502 2025-02-05 00:00:00+01:00 \u2502 3.0             \u2502\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\n```\n\n## License\n\nLicensed under [Apache License 2.0](LICENSE)\n\nCopyright (c) 2025 - present Romain Clement / Datalpia\n",
    "bugtrack_url": null,
    "license": "Apache-2.0",
    "summary": "Oversee your lakehouse",
    "version": "0.6.3",
    "project_urls": {
        "Changelog": "https://github.com/datalpia/laketower/blob/main/CHANGELOG.md",
        "Issues": "https://github.com/datalpia/laketower/issues",
        "Repository": "https://github.com/datalpia/laketower"
    },
    "split_keywords": [
        "data",
        " delta-lake",
        " lakehouse",
        " sql"
    ],
    "urls": [
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "b2cc888a35676d66c709173e1e7c641b4ad8b6a80bc6dd2de04e519c23f9fc0b",
                "md5": "9acbb6caba968fbd5052cb4264994db9",
                "sha256": "32d0b5e1bd2f1b35b5974c558e370adefbb07c105c72aad74c3ac544cc7a9c84"
            },
            "downloads": -1,
            "filename": "laketower-0.6.3-py3-none-any.whl",
            "has_sig": false,
            "md5_digest": "9acbb6caba968fbd5052cb4264994db9",
            "packagetype": "bdist_wheel",
            "python_version": "py3",
            "requires_python": "<3.14,>=3.10",
            "size": 709225,
            "upload_time": "2025-10-19T21:48:53",
            "upload_time_iso_8601": "2025-10-19T21:48:53.275452Z",
            "url": "https://files.pythonhosted.org/packages/b2/cc/888a35676d66c709173e1e7c641b4ad8b6a80bc6dd2de04e519c23f9fc0b/laketower-0.6.3-py3-none-any.whl",
            "yanked": false,
            "yanked_reason": null
        },
        {
            "comment_text": null,
            "digests": {
                "blake2b_256": "1cdf9467252ac394236ed9f5b27e49c4c9517d8e3bdc0592997d74978eed1c47",
                "md5": "45a3987dbe0e9ff6cb2c4c6543272d58",
                "sha256": "97e753284bb1e2bbafbcf5d78a797c36ca111fb8ec05e57e444376e8306f2245"
            },
            "downloads": -1,
            "filename": "laketower-0.6.3.tar.gz",
            "has_sig": false,
            "md5_digest": "45a3987dbe0e9ff6cb2c4c6543272d58",
            "packagetype": "sdist",
            "python_version": "source",
            "requires_python": "<3.14,>=3.10",
            "size": 2001088,
            "upload_time": "2025-10-19T21:48:55",
            "upload_time_iso_8601": "2025-10-19T21:48:55.019503Z",
            "url": "https://files.pythonhosted.org/packages/1c/df/9467252ac394236ed9f5b27e49c4c9517d8e3bdc0592997d74978eed1c47/laketower-0.6.3.tar.gz",
            "yanked": false,
            "yanked_reason": null
        }
    ],
    "upload_time": "2025-10-19 21:48:55",
    "github": true,
    "gitlab": false,
    "bitbucket": false,
    "codeberg": false,
    "github_user": "datalpia",
    "github_project": "laketower",
    "travis_ci": false,
    "coveralls": false,
    "github_actions": true,
    "lcname": "laketower"
}