Live data sources

Live data sources let you analyze the live data in your tools, with or without a warehouse. Sign in to a tool and you get a ready-made set of dimensions and metrics to explore, chart and ask AI agents about. Each query goes to the tool’s API when it runs, so results are as current as the tool and its data isn’t loaded into your warehouse. Speed, history and rate limits depend on each tool. A project can use live data sources on their own, with no warehouse connected.

​

Connect your tools

Project admins add a source by signing in to the tool or entering an API key. There’s no sync to schedule or dbt model to write. Each integration page lists its dimensions, metrics, supported features and limits. To ask for another tool, click Request next to it in the Add a live data source dialog. If it isn’t listed, contact Lightdash support.

​

Integration levels

An integration’s level describes how much its API can do when Lightdash queries it live. The higher the level, the more questions work. Each integration page states its level and the features it supports.

​

Access and security

A source uses one shared connection, so everyone who can query it sees the same data. User attributes don’t apply to live data sources, so you can’t limit rows or columns per person. To limit who can query a source, connect it in a separate project that only those people can access. To keep sensitive data, such as properties that hold email addresses, away from people and agents, turn off In use for that dimension or metric.

​

Limits

​

Rate limits

Each source caps queries at once, calls per minute and calls per hour, and everyone who queries it shares those caps. Each person’s first open of a chart makes its own call, so a popular dashboard can reach them. Scheduled deliveries send results to people without each of them querying the source. A query waits up to 15 seconds for a free slot. Over a cap, it fails with a message saying when to retry. Calls also count toward your API quota with the tool.

​

Date ranges and rows

A query with no filter on the source’s date dimension reads a default range, such as the last 28 days. Each integration also caps the rows a query returns and how long it waits for an answer.

​

Unavailable features

These aren’t available on any live data source:

• View underlying data
• SQL runner and compiled SQL
• Embedded dashboards and charts
• Funnels, retention and user paths
• Pivot tables and charts grouped by a dimension
• Data apps

Other features, such as table calculations and custom metrics, vary by integration. A query that uses an unsupported feature fails with a message saying what to remove, even when the control for it still appears.

​

Caching

Results caching doesn’t apply to live data sources, so its project setting, duration and sharing between people have no effect. Integrations have their own processing delays, listed on each integration page.

• Repeat queries. If you run the same query again within 60 seconds, you get your earlier result. It doesn’t call the integration or count toward its rate limits. This reuse is per person, so the same query from someone else calls the integration.
• Scheduled deliveries. Deliveries fetch new results unless Use cached results when available is on.
• Clearing the cache. Clearing cached results for a project, chart, dashboard or explore, on demand or on a schedule, also clears live data source results.

​

Choose a data source

You can’t join a live data source to other tables in Explore.