Adding Tables to your Lightdash project
Tables are the things that you see in the
Tables list when you open Lightdash. They're the foundation of your data exploration, the backbone to figuring out your unknowns, the art behind your charts...basically, they're pretty important.
So, we wanted to go through how to actually add these Tables to your Lightdash project, and once they're in there, how you can configure them to make the most of them.
How do I add tables to my Lightdash project?
The beauty of Lightdash is that we're pretty well synced with your dbt project. So, in Lightdash, Tables actually come from dbt models that have been defined in your dbt project's
If your dbt model has been defined in a .yml file, it will appear in Lightdash as a Table.
Not sure what a .yml file is? Make sure to check out dbt's docs to learn more about building .yml files for your dbt project.
For example, if we had a file called
users.yml in dbt that looked something like the one below, we'd see a Table called
users in Lightdash with the dimensions:
- name: users
- name: user_id
- name: created_at_date
- name: first_purchase_date
We recommend structuring your dbt project with one .yml file per model (or .sql file).
We've found that this makes it easier to navigate through your .yml files and easier to manage your dbt models, especially as your project becomes bigger.
Here's an example of our dbt project at Lightdash too see what that looks like in practice:
- We have one .sql file per model (these are the files where all of our models' business logic sits)
- We have one .yml file per model (these are the files where all of your Tables' configuration sits)
But, in my dbt project, I have a single schema.yml file. Not one for each model. Will that still work?
Yep! We realize that schema files come in all shapes and sizes.
Some people prefer to write the .yml files for all of their models in a single .yml file at the directory level, and that's totally fine - it will still work with Lightdash.
But, like we said just above, if you're trying to decide how to setup your dbt project, we'd recommend having one .yml file per model.
How to create
There are a few ways to do this - some easier than others.
1. The good old fashioned way: totally manually
You can create .yml files for your models by just creating a file in your dbt model's directory, then typing out all of the columns into it.
You'll need to make sure that your .yml file follows the formatting rules for a dbt
2. The much easier way: using a schema generator
If you've ever written a full .yml file manually, you won't be surprised that someone has gone ahead and automated the process.
There are a couple of schema generators out there, but the one we've ended up using the most was the
dbt-invoke package. You can read more about how to install + use it here.
How to add Tables to Lightdash
Once you have your .yml files all ready to go and synced to the branch that your Lightdash project is connected to, you can just hit
refresh dbt and your Tables should pop up in your project. (Not sure about how to sync dbt + Lightdash? Check out our docs on it here.)
They'll be tucked away under
Tables in the navigation bar.
Configuring which Tables appear in Lightdash
Sometimes, there are models in our dbt project with .yml files that we might not want to appear in Lightdash (
base tables, I'm looking at you 👀). So, we've made it possible for you to configure which Tables you want to appear in Lightdash.
To get to your Table Configuration settings, just:
Settingsin the navigation bar
- Click on
project managementin the sidebar
- Click on the
Settingsbutton for the project's tables you want to configure
- Once you're in your
Project settings, click on the
Tables configurationin the sidebar.
Now that you're in the right place, let's get to the juicy stuff. You have three options for configuring the Tables that pop up in Lightdash:
- Show entire project: I hope this one isn't too much of a surprise. If you select this option, it shows all of the models with .yml files in your dbt project in Lightdash.
- Show models with any of these tags: This option depends on dbt tags. If you're not familiar with dbt tags, we'd recommend checking them out (they're really handy for managing models!). If you already have a specific model tag (or tags) you want to limit Lightdash to using, this is where you can add them in. For example, all of our production models have the tag
prod, so we've configured our Tables using that tag.
- Show models in this list: If you're not keen on using tags then you can manually select the models you want to include as Tables in your Lightdash project using this option.
🔥 Hot tip: if you want to manage the models you're using in Lightdash in your dbt project, then you can use dbt's tags to tag all of your models with
lightdash then setup your Tables Configuration to limit tables with the tag
lightdash (see option 2 above).
So, for the models you want to include in Lightdash, you'd just add the tag to your model's .yml file:
- name: model_name
Or, to your model's
.sql file in the config block:
Then, you'll set your Table Configuration:
Showing hidden tables in Lightdash
You can still access + explore all of the Tables from your dbt project (with .yml files), even if you've filtered them from appearing in the sidebar. You can access these hidden table by just clicking on the
Show hidden tables toggle in the Tables sidebar.
Changing your Table's labels, adding joins, and more!
Once you're happy with which Tables are showing up in Lightdash, you can add configurations to your Tables like:
- Changing how the Table name appears in Lightdash (using the
- Joining your Table to other Tables (using the
All of these configurations and more are outlined in the Tables reference doc here.
Next up: adding metrics + exploring your Tables
Now that you have some of your Tables in Lightdash, it's time to enrich them with some metrics and learn how to explore them.
We'd recommend checking out these docs next: