This page is for engineering teams self-hosting their own Lightdash instance. If you want to get started with pre-aggregates, see the pre-aggregates reference.
Enabling pre-aggregates
Pre-aggregates materialize query results so that repeated queries are served from DuckDB instead of hitting your data warehouse. This requires NATS for async job processing and S3-compatible storage for materialized results.Prerequisites
- A valid Lightdash license key
- An S3-compatible bucket (AWS S3, GCS, MinIO, etc.)
Helm values
Setting these three values in your Helm values is the minimum required configuration:NATS_ENABLED, PRE_AGGREGATES_ENABLED, NATS_URL, and PRE_AGGREGATES_PARQUET_ENABLED from the flags above.
What gets deployed
| Component | Purpose |
|---|---|
| NATS JetStream | Message broker for async query jobs |
| Warehouse worker | Processes interactive queries from users |
| Pre-aggregate worker | Materializes pre-aggregates and processes DuckDB queries |
Scaling
The defaults are tuned for typical workloads. The main levers if you need to adjust:| Warehouse worker | Pre-aggregate worker | |
|---|---|---|
| CPU | 250m | 650m |
| Memory | 1.5Gi | 4Gi |
| Ephemeral storage | 9Gi | 9Gi |