Skip to main content
Experimental External sources are available to design partners behind the external-sources feature flag, and the flow, limits and wording on this page may change between releases. Contact Lightdash support to enable it for your organization. Self-hosted instances also need an Enterprise license and the setup below. What Experimental means.
An external source brings data that lives outside your warehouse into a project: an uploaded CSV, or a tab of a Google Sheet. Lightdash reads it once, stores a typed copy, and turns it into a table you can explore, chart, put on dashboards and join to your warehouse data. Nothing changes in your dbt project. Use it for small, hand-maintained data that never made it into the warehouse, such as monthly targets, a mapping of regions to owners, or a list of accounts to watch.
The Channel targets table from a Google Sheet, with Target month and Sum of Revenue target (USD) selected and the results by monthThe Channel targets table from a Google Sheet, with Target month and Sum of Revenue target (USD) selected and the results by month

A Google Sheet of monthly targets, explored like any other table

​
Before you start

  • Editor role or above. Editors, developers and admins can add, refresh, rename and delete external sources. Everyone else in the project sees the tables, and the charts built on them follow normal space permissions.
  • A header row. The first row names the columns. Every following row is data.
  • For Google Sheets, access to the sheet. The first time you connect a sheet, Lightdash asks your Google account for access to Google Sheets and to the Drive files you pick.
  • Size limits. Up to 100 MB per file, 1,000,000 rows per table and 5 GB per organization.
Everyone in the project can see an imported table, whatever the Google Sheet’s own sharing settings. Lightdash reads the sheet with the Google access of the person who connected it and stores a copy. User attributes and row-level filters don’t apply to external sources, so only import data that every project member may see.

​
Upload a CSV

1

Open Add data

In the Tables sidebar, select Add data.
The Tables sidebar with the Add data button next to its headingThe Tables sidebar with the Add data button next to its heading
2

Drop your file

Drop a CSV on the upload area, or click it to pick a file.
The Upload a CSV dialog with a drop area and a link to connect a Google Sheet insteadThe Upload a CSV dialog with a drop area and a link to connect a Google Sheet instead
3

Name the table and its columns

Give the table a name and check the type detected for each column. Rename any column whose name reads oddly, then select Create table. Lightdash opens the new table as soon as it’s ready.
The CSV preview with the table name sales_targets and editable names for each column, with the month column renamed to TargetThe CSV preview with the table name sales_targets and editable names for each column, with the month column renamed to Target

​
Connect a Google Sheet

1

Paste the sheet link

Select Add data, then Connect a Google Sheet instead. Paste the sheet’s URL from your browser and name the table.Lightdash reads the tab the link points to. Open the tab you want before copying the URL, so the link ends in #gid=…. A link without a tab reads the first tab.
The Connect a Google Sheet dialog with a sheet URL and the table name channel_targetsThe Connect a Google Sheet dialog with a sheet URL and the table name channel_targets
2

Connect your Google account

Select Connect sheet. If you haven’t connected Google before, select Connect Google account, approve access in the pop-up, then select Connect sheet again.
3

Review the table

Once the import finishes, rename the table or any column, then select Open table.
The Review your table step listing seven imported columns with editable names, types and an Open table buttonThe Review your table step listing seven imported columns with editable names, types and an Open table button

​
What the table contains

Each column becomes a dimension with a detected type: string, number, date, timestamp or boolean. Date columns also get day, week, month, quarter and year breakdowns. Lightdash adds a Total rows metric, plus a sum and an average for every number column. Column names are tidied into SQL-safe names, so Região becomes regiao and a column called month becomes _month. The name you see in the sidebar is the one you gave it while importing, or a readable version of the SQL name.

​
Join with warehouse data

Select Join with your warehouse data under the table name to combine it with an explore from your dbt project, such as targets next to actual orders by month. Lightdash suggests a join on a matching field. The result is a merge query, which you can save, add to dashboards and schedule like any other chart.
A merge query joining Channel targets Target month to Orders Order date month, with revenue targets next to total order amountsA merge query joining Channel targets Target month to Orders Order date month, with revenue targets next to total order amounts

Monthly targets from a Google Sheet joined to warehouse orders by month

The join belongs to the chart, so each chart that needs targets next to actuals sets it up again.

​
Keep the table up to date

An external source is a copy taken when you import it, and edits to the sheet or file don’t show up until you refresh it. The menu next to the table name holds the source actions:
  • Refresh from Google Sheets reads the sheet again. Column names you set are kept.
  • Replace file uploads a new version of a CSV.
  • Reconnect Google account makes your Google access the one used for future refreshes, for example when the person who connected the sheet leaves.
  • Rename table changes the table and column names. Saved charts keep working.
  • View source details previews the columns and the first rows.
  • Delete removes the table. Charts built on it stop working.
The source menu next to the Channel targets table name, listing Merge another query, Refresh from Google Sheets, Reconnect Google account, Rename table, View source details and DeleteThe source menu next to the Channel targets table name, listing Merge another query, Refresh from Google Sheets, Reconnect Google account, Rename table, View source details and Delete
The Rename table dialog with the table name and an editable name for every column, with channel renamed to Sales channelThe Rename table dialog with the table name and an editable name for every column, with channel renamed to Sales channel

Renaming a column from Rename table

​
Self-hosted setup

External sources run on the same DuckDB engine and object storage as pre-aggregates, but they don’t need pre-aggregate workers or NATS. You need:
  • An Enterprise license. Set LIGHTDASH_LICENSE_KEY. See enterprise features.
  • A bucket for the imported data. Set the PRE_AGGREGATE_RESULTS_S3_* variables listed under pre-aggregates. Lightdash stores each table under external-sources/ in that bucket.
  • The feature flag. Add external-sources to LIGHTDASH_ENABLE_FEATURE_FLAGS. See feature flags.
  • For Google Sheets, a Google OAuth client. Follow configure a Google Sheets integration. Enable the Google Sheets API and the Google Drive API on the same Google Cloud project, and add <SITE_URL>/api/v1/oauth/redirect/google to the client’s authorized redirect URIs.
To change the size limits, set EXTERNAL_SOURCES_MAX_FILE_SIZE_BYTES, EXTERNAL_SOURCES_MAX_ROWS or EXTERNAL_SOURCES_MAX_ORGANIZATION_BYTES.