Tier 1 — Evaluation / PoC
Follow the self-hosting guide (or docker compose for a local spin-up) for a minimum production setup suitable for evaluating Lightdash. Prerequisites- External PostgreSQL —
postgresql.enabled: falseeven for a PoC; a small managed instance is fine, no HA or backup requirements yet - S3-compatible object storage — required regardless; Lightdash fails to start without it
- Enterprise license key if you’re evaluating Enterprise features — needs outbound access to
https://api.keygen.sh
- Deploy with the self-hosting guide
- Pin an image version - pin
image.tagto the latest version - Set
SITE_URLto your finalhttps://URL before first boot - Generate a strong
LIGHTDASH_SECRETand store it somewhere durable - losing it means losing access to encrypted data -
SECURE_COOKIESandTRUST_PROXYboth"true"behind your TLS-terminating load balancer - Headless browser enabled (the chart default) - image previews, PDF exports, Slack unfurls
- HTTPS end-to-end via your ingress or load balancer
- AI Analyst — model provider key +
pgvectorin Postgres
Tier 2 — Scalable production deployment
None of these steps are required but are recommended for running Lightdash beyond a PoC.Operations
- Choose upgrade strategy — pinned image tag, at least monthly cadence
Workers and scaling
- Data apps — sandboxes runtime, apps bucket, and a separate preview origin
- Dedicated workers -
scheduler.enabled: trueto enable scalable scheduled deliveries - NATS + warehouse workers -
nats.enabled: trueandwarehouseNatsWorker.enabled: truefor handling large volumes of warehouse queries - ≥ 2 backend replicas with pod anti-affinity and a pod disruption budget
- Size resource requests
Infrastructure dependencies
- Postgres HA, backups and point-in-time recovery on the external database from Tier 1, plus the
pgvectorextension if you use Enterprise AI features - Bucket strategy and lifecycle rules - a dedicated bucket per purpose, with a 1-day lifecycle delete on the results bucket
- SMTP email
- Email deliverability - transactional provider, SPF/DKIM on the sender domain
- Load balancer settings - timeout ≥ 300s, health check on
/api/v1/health, request body limit ≥LIGHTDASH_MAX_PAYLOAD
Security and authentication
- SSO (Okta / Azure AD / Google / generic OIDC) with password authentication disabled
- Account linking and personal access token policy
- CSP enforcement and explicit CORS
Tier 3 — Optional features and observability
Enterprise features
- Pre-aggregations — pre-aggregate NATS workers + a dedicated S3 bucket
- SCIM provisioning from your IdP
- Results caching, embedding, service accounts, custom roles
Integrations (enable what you use)
- Slack — scheduled deliveries, unfurls, AI agents in Slack
- GitHub or GitLab — dbt repo write-back
- Google Sheets sync
Observability
- Prometheus metrics enabled, and scraped on every pod
- Structured JSON logging shipped to your log platform
- Alerts on HTTP error rate/latency, queue depth, and Postgres pool saturation
Architecture: what you’re deploying
| Component | Chart value | What it does |
|---|---|---|
| Backend | (always on) | API + UI. Scale horizontally, 2+ replicas |
| Scheduler worker | scheduler.enabled | Scheduled deliveries, Slack/email sends, exports, syncs |
| NATS | nats.enabled | JetStream message bus for async query execution |
| Warehouse NATS worker | warehouseNatsWorker.enabled | Executes warehouse queries + streams results to S3 |
| Pre-aggregate NATS worker | preAggregateNatsWorker.enabled | Builds pre-aggregated materializations (Enterprise) |
| Headless browser | browserless-chrome.enabled | Chromium pool for screenshots/PDFs |
| Migration job | migrationJob.enabled | Pre-upgrade Helm hook that runs database migrations exactly once |
| PostgreSQL | external (postgresql.enabled: false) | Application state. Don’t use the bundled subchart, even for a PoC |
| S3 bucket(s) | external | Query results, downloads, pre-agg materializations, data apps |
configMap.*— non-sensitive env vars, applied to backend and all workerssecrets.*— sensitive env vars, rendered into a Kubernetes Secret (or bring your own viaexistingSecret)extraEnv/schedulerExtraEnv— raw env entries, includingvalueFrom.secretKeyRef
Core configuration
SITE_URLis required for invite emails, OAuth redirect URIs, Slack unfurls, and scheduled-delivery links. Changing it later means reconfiguring every OAuth integration.LIGHTDASH_SECRETmust be changed and stored securely. It signs session cookies and encrypts data at rest in Postgres (warehouse credentials, tokens). You must set thisSECURE_COOKIES+TRUST_PROXYare both"false"by default — production behind HTTPS needs both"true"(see Secure Lightdash with HTTPS).
- Session length is controlled by
COOKIES_MAX_AGE_HOURS(unset = session cookie). Set it to your security policy, e.g."24". - If backend pods can’t reach
SITE_URLfrom inside the cluster (hairpin/NAT issues), setINTERNAL_LIGHTDASH_HOSTto the in-cluster service URL — the headless browser and internal calls use it.
Secrets management
secrets.* values live in plaintext in your Helm values. For enterprise deployments, prefer
existingSecret populated by External Secrets Operator / a CSI driver from your secret
manager, and keep Helm values (and git) free of credentials.
Headless browser
Enabled by default in the chart — keep it on, and tune it like Lightdash Cloud does:HEADLESS_BROWSER_HOST / HEADLESS_BROWSER_PORT. Remember: the browser
renders dashboards by calling SITE_URL — it must be able to resolve and reach that URL
from inside the cluster (use INTERNAL_LIGHTDASH_HOST if it can’t). For timeout/retry
troubleshooting and internal-HTTPS setups, see
enable headless browser.
Upgrades and operations
Upgrade mechanics are in update Lightdash and the version policy in versioning. Operational best practice:- Pin
image.tagand upgrade deliberately. There is no LTS tag. Lightdash versioning: patch = routine, minor = check release notes (may be backwards-incompatible), major = read the upgrade guide - Upgrade cadence: at least monthly. Lightdash ships continuously; falling many minor versions behind makes the eventual migration jump riskier
- Upgrade path: bump
image.tag→helm upgrade→ themigrationJobhook migrates the database → rolling restart. Roll back the image only if the release notes say the migrations are backwards-compatible; otherwise restore the database backup - Keep a UAT instance that mirrors production config, and rehearse upgrades there against a refreshed copy of the production database
- DR runbook: you need three things to rebuild an instance from scratch — the Postgres
backup, the
LIGHTDASH_SECRET, and your Helm values. Test that you can
Scheduler worker
By default the backend runs scheduled jobs in-process. In production, run a dedicated scheduler worker so a heavy dashboard export can’t starve the API:scheduler.enabled: true the chart automatically sets SCHEDULER_ENABLED: "false" on
the backend — don’t set it yourself.
Tuning: SCHEDULER_JOB_TIMEOUT (default 600000 ms), SCHEDULER_SCREENSHOT_TIMEOUT, and
scheduler.tasks.include / scheduler.tasks.exclude if you want to shard task types across
worker groups.
For async warehouse queries, see the NATS workers overview
and warehouse workers — including the critical
rule: never enable nats.enabled without warehouseNatsWorker.enabled.
Sizing and availability
Baseline per-component requests for a standard instance (see also recommended resources):| Component | CPU | Memory | Ephemeral | Replicas |
|---|---|---|---|---|
| Backend | 500m–1 | 1.5–4 Gi | 1–2 Gi | 2+ |
| Scheduler worker | 500m | 1425 Mi | 1 Gi | 1 |
| Warehouse NATS worker | 500m | 1.5 Gi | 9 Gi | 1 |
| Pre-agg NATS worker | 650m | 4 Gi | 9 Gi | 1 |
| Browserless | 2 | 4 Gi | 1 Gi | 1 |
| NATS | 100m | 256 Mi–1 Gi | — | 1 |
PostgreSQL
Setup is covered in Configure Lightdash to use an external database. This section is what to run on the database side, matching Lightdash Cloud:- Managed Postgres (Cloud SQL / RDS / Azure Database) — version 14+ recommended (12 is the documented minimum)
- Extensions:
uuid-ossp(required) andpgvector, required for Enterprise AI features (AI Analyst embeddings, verified answers) - HA: regional/multi-AZ primary. Lightdash Cloud runs regional HA primaries with a cross-region replica for disaster recovery
- Backups: daily automated backups, point-in-time recovery enabled, ≥ 7 days of transaction logs, 7–31 retained backups
- SSD storage with auto-resize
- Sizing: 2 vCPU / 8 GB RAM is a solid single-org baseline; 4 vCPU / 16 GB+ for large orgs or heavy scheduler usage
- Connection budget: every backend/worker pod opens its own pool. Set
PGMAXCONNECTIONS: "50"andPGMINCONNECTIONS: "0"inconfigMap, then make sure Postgresmax_connectionscoverspods × PGMAXCONNECTIONSwith headroom - Query insights / slow-query logging enabled on the database side
- TLS to the database:
pre-install,pre-upgrade hook and the backend then starts without migrating:
knex_migrations_lock table.)
Object storage
Setup (env vars, per-provider instructions, IAM roles) is covered in Configure Lightdash to use external object storage. Bucket strategy is where enterprise deployments differ from the basic setup — this is how Lightdash Cloud sets up storage:- Dedicated bucket per purpose, each with a scoped credential that can only touch its
own bucket:
- results/cache bucket — add a lifecycle rule deleting objects after 1 day (results are ephemeral; this keeps the bucket small and cheap)
- pre-aggregations bucket (if using pre-aggs) — lifecycle delete after ~30 days
- data-apps bucket (if using data apps) — persistent, no delete lifecycle
- Block all public access on every bucket
- Prefer IAM roles (IRSA on EKS, Workload Identity on GKE) over static keys — but note signed URLs are then capped at 7 days
- If users share download links that must outlive the signed-URL TTL
(
S3_EXPIRATION_TIME, default 3 days), enablePERSISTENT_DOWNLOAD_URLS_ENABLED: "true"
Email deliverability
Env vars are in the SMTP reference. Best practice on top:- Use a transactional provider (SES, Postmark, SendGrid) — Lightdash Cloud sends through Postmark
- Set up SPF/DKIM for the sender domain so scheduled deliveries don’t land in spam
- Internal open relay without auth: set
EMAIL_SMTP_USE_AUTH: "false"
Load balancer and networking
Ingress and TLS setup is covered in Secure Lightdash with HTTPS. Match what Lightdash Cloud configures at the load balancer:- Redirect HTTP → HTTPS, minimum TLS 1.2
- Backend/LB timeout ≥ 300s — long-running exports and queries will be killed by the common 30–60s defaults
- Health check on
GET /api/v1/health - Request body size limit at the ingress ≥ your
LIGHTDASH_MAX_PAYLOAD(dbt manifests for big projects are tens of MB) - Optional: IP allowlisting / WAF / rate limiting at the load balancer
- On GKE specifically, the chart can create a
BackendConfigfor you (backendConfig.create: true,backendConfig.spec) — this is how timeouts, CDN and Cloud Armor policies are attached on Lightdash Cloud
Authentication policy
Per-provider SSO setup (Okta, Azure AD, Google, generic OIDC — env vars and redirect URIs) is covered in use SSO login for self-hosted Lightdash. Enterprise deployments should be SSO-only:- PAT policy:
PAT_ALLOWED_ORG_ROLES,PAT_MAX_EXPIRATION_TIME_IN_DAYS, orDISABLE_PAT - Keep
ALLOW_MULTIPLE_ORGS: "false"(default) for a single-company instance
Security hardening
- CSP enforcement:
LIGHTDASH_CSP_REPORT_ONLY: "false"(default is report-only; enforce in production), plusLIGHTDASH_CSP_ALLOWED_DOMAINSfor any extra origins you load from - CORS: leave disabled unless embedding; if embedding,
LIGHTDASH_CORS_ENABLED: "true"with an explicitLIGHTDASH_CORS_ALLOWED_DOMAINSlist — never* - Egress policy: Lightdash needs your warehouse, S3, SMTP,
api.keygen.sh(license), your IdP, and any AI provider endpoints — everything else can be blocked - NetworkPolicies: the chart only ships one for NATS (keep
nats.networkPolicy.enabled: true, the default); add your own default-deny + allow rules for backend ↔ postgres/browserless/S3 if your cluster uses them -
podSecurityContext/securityContext: the chart sets none by default — addrunAsNonRoot, drop capabilities per your Pod Security Standards baseline - Soft delete for content recovery:
SOFT_DELETE_ENABLED: "true"(+SOFT_DELETE_RETENTION_DAYS, default 30)
Enterprise features
License key setup and validation is covered in enterprise license keys — remember the key is validated againsthttps://api.keygen.sh on every server start, so allowlist that
domain in your egress policy.
Enterprise feature flags
AI Analyst
- Azure OpenAI:
AZURE_AI_API_KEY,AZURE_AI_ENDPOINT,AZURE_AI_API_VERSION,AZURE_AI_DEPLOYMENT_NAME; Bedrock:BEDROCK_REGION+ credentials - Routing through an internal LLM gateway (LiteLLM etc.):
AI_DEFAULT_PROVIDER: openai+OPENAI_BASE_URL - Guardrails:
AI_COPILOT_MAX_QUERY_LIMIT(default 500),AI_COPILOT_ALLOWED_PROJECT_UUIDto pilot on one project first - MCP details: Configure MCP for Lightdash
Data apps
Sandbox providers (E2B, AWS Lambda MicroVMs, Azure) and their security model are covered in sandboxes; model configuration in self-hosting data apps. Production checklist on top:*.lightdash.app next
to *.lightdash.cloud) so untrusted app content never shares an origin with your Lightdash
session cookies, and restrict that hostname to /api/apps/* paths at the ingress.
Observability
The metric catalogue, alerting guidance, and Prometheus server setup are covered in Prometheus metrics; log configuration in configure logging. Run what Lightdash Cloud runs on every instance:- Scrape port 9090 on all pods labelled
app.kubernetes.io/name=lightdash(the chart doesn’t ship a ServiceMonitor/PodMonitoring — create one; Lightdash Cloud does exactly this, 30s interval) - Metrics worth alerting on: HTTP p95/error rate (
http_server_request_duration_seconds), queue depth / scheduler job failures, pg pool saturation, event-loop lag - If you run NATS, its Prometheus exporter is on port 7777
(
nats.promExporter.enabled: true)