Scheduler

Enable the scheduler in self-hosted Lightdash

Lightdash

Lightdash instantly turns your dbt project into a full-stack BI platform.

Welcome to Lightdash

You can easily make changes in dbt and see them updated in your Lightdash project.

dbt Projects

Below is everything you need to know about the Lightdash Slack integration.

Slack

Slack integration

Syncs let you send your chart results to a Google Sheet in your Google Drive on a schedule.

Google Sheets

Lightdash allows you to embed your dashboards using expirable URLs and tokens (embed URLs from now on). This is a great way to enable self-serve analytics in your own application and platform by leveraging the insights you've got in Lightdash and making them available to your own users.

Embedding quickstart

How to embed content

You can embed Lightdash content in your website or application securely using Lightdash embedding.

Embedding reference

Embedding Overview

Reference documentation for Lightdash React SDK, used for Embedding. 

React SDK

Self-Host Lightdash

Lightdash is open-source and licensed under the MIT license. This means that you are free to self-host Lightdash on your own infrastructure but without any warranty or support from the Lightdash team.

Lightdash Cloud vs. Self-Hosted

Docker compose

Self-host Lightdash using docker compose

Restack

Self-Host Lightdash with Restack

Updating Lightdash

Updating Lightdash to the latest version

FAQs

There are a few ways you can get in touch with the team. Here's which one to use depending on your situation:

Contact Us

Learn what dbt's package-lock.yml file does, why it matters for your data projects, and how to manage it effectively.

Guide to package-lock.yml in dbt

Learn how to fix dbt 1.10 deprecation warnings for `meta` and `tags` properties, and migrate your project automatically with MetaMove.

Migrate to dbt 1.10

Migrating to dbt 1.10: Fix Meta and Tags Configuration Warnings

Introduction

Lightdash API

Welcome to Lightdash! We're going to try to get you developing, exploring or just browsing your data in no time.

Overview

Getting started

This tutorial is for people who are new to Lightdash and would like to learn how to set up a new project. We'll show you how to get your dbt project Lightdash-ready, how to explore your data in Lightdash, and how to add metrics to your project.

Setting up a new project

Create your first project in Lightdash

Update your project connection

An intro to metrics and dimensions

Now that you've connected your dbt project to Lightdash, it's time to start exploring your data.

Using Tables and the Explore view

How to create metrics

You can only share things with people you've invited to your Lightdash project. Users with **Admin** access to your project can invite users.

Inviting teammates to your Lightdash project

Share insights with your team

This tutorial is for people who are new to Lightdash and would like to develop their existing Lightdash project. We'll go over how to use Tables and the Explore view, how to develop Tables in your project, adding your first metrics, and some more advanced configurations for your Lightdash project's content.

Developing in Lightdash

Exploring your content

How to create dimensions

This tutorial is for people who are new to Lightdash and would like to explore the tables, charts and dashboards in their Lightdash project. We'll go over how to use Tables and the Explore view, how to share your insights, and how to build dashboards.

 Learning to explore data in Lightdash

It's time to share your hard work with the world!

Creating Dashboards

AI Analyst

Adding and managing Tables

Learn how to add joins to your YAML files to connect different models to each other.

How to join tables

The Lightdash CLI is the recommended way to develop your dbt + Lightdash project. It makes development faster and easier, as well as giving you options for building more powerful automation to manage your Lightdash instance. Here are some guides to get you started!

The Lightdash CLI

The Lightdash CLI is the recommended way to develop your dbt + Lightdash project. It makes development faster and easier, as well as giving you options for building more powerful automation to manage your Lightdash instance.

How to install the Lightdash CLI

Authenticating your CLI (login)

Upgrading your Lightdash CLI

The beauty of Lightdash is that we're pretty well synced with your dbt project. So, in Lightdash, Tables actually come from dbt models that have been defined in your dbt project's `.yml` files.

Autogenerate Lightdash-ready .yml files for your models

Preview projects are temporary Lightdash projects where you can safely experiment with your metrics, dimensions and charts without affecting your production project.

Lightdash Preview

Sync your changes with Lightdash deploy

If you've connected Lightdash to GitHub, you can setup a `github action` and get Lightdash to compile your project and test your changes whenever you open a pull request.

Test your changes with Lightdash compile

You can check to see if your changes break any of the content in your project using `lightdash validate`. This command will run the validator which will return a list of content in your project that is broken and details about the errors.

Lightdash Validate

Formatting your fields

Using Filters

Use our SQL templates to get started with your table calculations!

SQL templates

The percent change gives you the percent change in a value compared to the value in the row above it.

Percent change from previous

Percent of previous value

The percent of the column total will give you the percentage of each value in the column out of the total sum of the values in the column.

Percent of column total

The percent of the group/pivot total will give you the percentage of each value in a group out of the total sum of the values for that group.

Percent of group/pivot total

Ranking values in a column is when you set every value as a number either higher than or lower than the other values. The lowest value (rank = 1) indicates the value with the first rank in your set of values.

Rank in column

Running totals give you the sum of a value + all of the previous values in that column.

Running total

Rolling windows, or lookbacks, calculate metrics like rolling averages using the current row combined with N previous rows. This can be useful for smoothing out volatile time series data to better understand long term trends.

Rolling window

Interacting with your dashboards

Filtering a Dashboard in the URL

Pinning

Add the Slack integration

Use the Slack integration

Scheduled deliveries let you send a dashboard or saved chart, and their results to people via email or Slack - even to people who don't have an account in your Lightdash instance.

Scheduled deliveries

Alerts enable you to receive notifications via email or Slack when the value of a saved chart meets your specified conditions.

Alerts

How to set up Lightdash YAML validation for VS Code

You can roll back to previous versions of charts in Lightdash. In case you make a mistake or simply want to go back to a previous version of a chart you built, you can easily revert back to previous versions of your charts. It's a great way to keep track of changes and revert back to previous versions if needed.

Chart Version History

With Lightdash, you can have a single organization (e.g. Lightdash), and multiple projects within that organization (e.g. Production, Jaffle Shop, etc.)

Creating multiple projects

You can update the default colours used in your organization's charts so that everything you build is on brand.

Customizing the appearance of your project

Promoting content enables you to copy content from one project to another. It's especially useful when working with preview or development projects. You can promote charts created in your development projects to production, so it's easy to test and make changes without the fear of breaking things in production.

How to promote content

Spotlight is a suite of features designed to put Metrics front and center. It simplifies data exploration, making it easy for business users to focus on metrics without dealing with complex tables or time dimensions.

Spotlight

Reference docs are broken down into the categories below.

Lightdash Reference Docs

Below are detailed reference docs about data development in Lightdash. That includes the YAML in Lightdash Semantic Layer and anywhere you need to use SQL.

Lightdash Development Reference Docs

Dimensions are the columns in your table. They are the "attributes" of your data. For example, `user_id` in your users table is a dimension.

Dimensions

Dimensions reference

Metrics

Metrics reference

You can run queries against the tables in your Lightdash project.

Tables

Tables reference

Joins let you connect different models to each other so that you can explore more than one model at the same time in Lightdash and see how different parts of your data relate to each other.

Joins

Joins reference

**Preview projects** are temporary Lightdash projects where you can safely experiment with your metrics, dimensions and charts without affecting your production project. 

Preview Projects

Broken content is not useful content. We've built tools in Lightdash to help validate your content so you can keep it useful and we'll notify you when it isn't.

Validator

Validating your content

dbt Write-Back allows you to develop your models and Metrics in Lightdash and write back these changes to your dbt project. Writing back to dbt will create a new branch and pull request in your project so you can save and manage these changes.

dbt Write-Back

You can download your charts and dashboards as code, make changes, then upload your changes back to your Lightdash project using the CLI. Content as code is also useful for creating new Lightdash projects where you want the content to be copied from an existing project and added to your new project.

Dashboards as code

When writing SQL in your Lightdash models, you can use the following references to template you SQL and make it more reusable

SQL Variables

Caching can reduce the overall number of queries that Lightdash executes against your data warehouse. Learn more about the types of caching and how to enable caching in this doc.

Caching

Below are detailed reference docs for all the ways you can explore and visualize data in Lightdash.

Data Exploration Reference Docs

Results tables can be useful for finding specific numbers or checking out your data, but it's usually easier to see trends and make sense of your data using charts.

Chart types

Custom charts are available in Beta for those that want additional customization and an extended library of charts.  The charts are powered by [vega-lite](https://vega.github.io/vega-lite/examples/) - the link will take you to some example galleries of what you can build using this powerful charting library.  This includes heat maps, bubble plots, box plots and more.

Custom Charts

Filters appear at the top of the Explore view and allow users to change the data being pulled in.

Filters reference

The fields that you see in your `dimensions` and `metrics` are created by the people maintaining your Lightdash project. But, you can create new custom dimensions or metrics for quick calculations and binning on the fly. On this page, we'll explain what custom dimensions and metrics are and how to use them.

Using custom fields

Table calculations make it easy to create metrics on the fly (for example, aggregating the data in your table over some window, or getting a running total of a column).

Table Calculations

Once you've used Lightdash for a while, your team will probably end up with a lot of saved charts and dashboards. Spaces let you organize your saved charts and dashboards.

Spaces

The Lightdash CLI is the recommended way to develop your Lightdash project. It makes development faster and easier, as well as giving you options for building more powerful automation to manage your Lightdash projects.

Lightdash CLI reference

Reference docs for all integrations available on Lightdash.

Lightdash Integration Reference Docs

dbt Semantic Layer

Slack integration reference

Below are reference docs for all admin functions and features in Lightdash.

Lightdash Admin References

To understand who is accessing content or running queries in your Lightdash projects, we've built usage analytics to help you track this.

Usage analytics

Roles and permissions

Groups are a way to manage access to multiple users at the same time. A user can belong to multiple groups.

Groups

User attributes provide a way to customise Lightdash charts, dashboards, and other behaviour depending on the user that is logged in. For example, you could only show your sales team members data that are relevant to their region.

User Attributes

SCIM Integration

Below are reference docs for all user functions and features in Lightdash.

Lightdash User References

Personal access tokens

Slack Integration

Configure a slack integration for self-hosted Lightdash

Use External Database

Configure Lightdash to use an external database

External storage

Configure Lightdash to use external storage

Logging

Configure logging for Lightdash

SMTP for emails

Configure SMTP for Lightdash email notifications

Standalone scheduler worker

Configure a standalone scheduler worker for self-hosted Lightdash

We use a chrome headless browser to generate images from your charts and dashboards so we can send them via email or Slack.

Add headless browser on self-hosting

Environment variables

In this guide we will show you how you can [upload to Google Sheets in scheduled deliveries](/references/google-sheets) and upload results from Google Sheets on `charts` on your on self-hosted Lightdash server.

Configure a Google Sheets integration for self-hosted Lightdash

The recommended resource requests for each Lightdash component are as follows

Resource recommendations

To run Lightdash in production it is highly recommended to secure it with HTTPS. This is usually done using your cloud provider's load balancer or ingress controller. Optionally you could use a proxy service like \[nginx\]([https://www.nginx.com/](https://www.nginx.com/)) or [traefik](https://traefik.io/).

Introduction

Explore and analyze

Build your semantic layer

Workspace and user management

Integrations

Embedding and SDKs

Self-hosting and deployment

Contact

Enable the scheduler in self-hosted Lightdash