Upgrading dbt - Lightdash

Lightdash reads your dbt project, so a dbt upgrade can change how your semantic layer is declared. Two changes matter: where meta and tags live as of 1.10, and the Fusion engine that ships as dbt Core v2.

Moving to dbt 1.10

When you upgrade to dbt 1.10, you’ll likely encounter deprecation warnings that look like this:

Warning: while parsing model config: Ignore unexpected key "meta"
Warning: while parsing model config: Ignore unexpected key "tags"

These warnings indicate that dbt is changing how meta and tags should be structured in your YAML files. Instead of manually updating potentially hundreds of files, you can use MetaMove - a CLI tool specifically designed to automate this migration.

What changed

Starting in dbt 1.10, you will receive deprecation warnings for dbt code that will become invalid in the future, including custom inputs like unrecognized resource properties and configurations. Some properties are moving to configs, including meta and tags. The key change affects how you define meta and tags properties across your dbt project: Before (dbt < 1.10):

models:
  - name: my_model
    meta:
      owner: "Data Team"
      priority: "high"
    tags: ["core", "customer"]
    columns:
      - name: id
        meta:
          is_primary_key: true
        tags: ["identifier"]

After (dbt 1.10+):

models:
  - name: my_model
    config:
      meta:
        owner: "Data Team"
        priority: "high"
      tags: ["core", "customer"]
    columns:
      - name: id
        config:
          meta:
            is_primary_key: true
          tags: ["identifier"]

Why it matters

Previously, dbt allowed you to configure inputs largely unconstrained. Without a set of strictly defined inputs, it becomes challenging to validate your project’s configuration, creating unintended issues such as silently ignoring misspelled properties and configurations. This change provides several benefits:

Better Validation: dbt can now catch misspelled properties and configurations
Consistent Structure: All custom attributes must be nested under the meta config
Future-Proofing: Prevents conflicts when dbt introduces new reserved properties

Automated migration with MetaMove

Rather than manually updating your YAML files, use MetaMove - a CLI tool built specifically for this migration. MetaMove automatically transforms your files while preserving comments and formatting.

Installation

Install MetaMove using pipx (recommended):

pip install pipx  # if you don't have it
pipx install metamove

Basic usage

Transform a single YAML file:

metamove models/my_model.yml

Transform multiple files:

metamove models/* seeds/* snapshots/*

Transform all YAML files in your dbt project:

metamove models/**/* seeds/**/* snapshots/**/*

Safe migration process

MetaMove follows a safe-by-default approach:

Test First: By default, transformed files are saved to a transformed directory:
metamove models/* # saves to ./transformed/
Review Changes: Compare the original and transformed files to ensure accuracy.
Apply In-Place: Once confident, transform files in place:
metamove models/* -i

Advanced options

Specify a custom output directory:

metamove models/* -o my_transformed_models

The tool automatically processes only .yml and .yaml files, so wildcards work safely:

# Transform specific model directories
metamove models/marts/* models/staging/*

# Transform all nested directories
metamove models/**/*

What MetaMove handles

MetaMove intelligently processes:

Nested Structures: meta and tags at any nesting level, including inside columns
Existing Config Blocks: Merges new values into existing config sections
All YAML Types: Dictionaries, lists, and scalar values
Formatting Preservation: Maintains your comments and whitespace
Complex Cases: Handles edge cases following dbt precedence rules

Migrating by hand

If you prefer to migrate manually or need to understand the changes, here are the key transformations:

Model-level changes

Before:

models:
  - name: customer_metrics
    description: "Customer analytics model"
    meta:
      owner: "analytics_team"
      sla: "daily"
    tags: ["core", "customer", "metrics"]

After:

models:
  - name: customer_metrics
    description: "Customer analytics model"
    config:
      meta:
        owner: "analytics_team"
        sla: "daily"
      tags: ["core", "customer", "metrics"]

Column-level changes

Before:

models:
  - name: customers
    columns:
      - name: customer_id
        description: "Unique customer identifier"
        meta:
          is_primary_key: true
          data_type: "uuid"
        tags: ["pii", "identifier"]

After:

models:
  - name: customers
    columns:
      - name: customer_id
        description: "Unique customer identifier"
        config:
          meta:
            is_primary_key: true
            data_type: "uuid"
          tags: ["pii", "identifier"]

Source-level changes

Before:

sources:
  - name: raw_data
    schema: staging
    meta:
      loader: "fivetran"
      freshness_sla: "1 hour"
    tags: ["external", "raw"]
    tables:
      - name: users
        meta:
          row_count: 50000
        tags: ["pii"]

After:

sources:
  - name: raw_data
    schema: staging
    config:
      meta:
        loader: "fivetran"
        freshness_sla: "1 hour"
      tags: ["external", "raw"]
    tables:
      - name: users
        config:
          meta:
            row_count: 50000
          tags: ["pii"]

Other 1.10 changes to consider

While migrating meta and tags, be aware of other properties moving to configs:

freshness (for sources)
docs
group
access

These properties should now be set under the config block following the same pattern as meta and tags.

Handling warnings

If you’re using --warn-error flags that promote warnings to errors, you may need to adjust your configuration during migration:

# In dbt_project.yml
flags:
  warn_error_options:
    warn: ["Deprecations"]  # Keep deprecation warnings as warnings
    error: ["other_warning_types"]  # Promote other warnings to errors

This allows you to continue working while addressing deprecation warnings gradually.

Testing the migration

After migrating your files:

Run dbt parse: Ensure your project parses without errors:
dbt parse
Check for Warnings: Run a simple command to verify warnings are resolved:
dbt compile --select my_model
Run Tests: Execute your test suite to ensure functionality is preserved:
dbt test

Migrating safely

Backup First: Always backup your project before running any migration tool
Test in Development: Run the migration in a development branch first
Review Changes: Use git diff to review all changes before committing
Migrate Incrementally: For large projects, consider migrating one directory at a time
Update CI/CD: Ensure your deployment processes work with the new structure

Troubleshooting

”Config block already exists” errors

If you have existing config blocks, MetaMove will merge the new properties. However, if you encounter conflicts:

Review the specific file causing issues
Manually resolve conflicts between existing and new config properties
Re-run MetaMove on the corrected file

Unexpected YAML structure

If your YAML files have unusual structures that MetaMove doesn’t handle:

Note the problematic files in the tool output
Manually migrate these files using the patterns shown above
Use MetaMove for the remaining standard files

Performance on large projects

For projects with hundreds of YAML files:

Run MetaMove on subdirectories rather than the entire project at once
Use the -o flag to organize transformed files by directory
Process files in batches to make review manageable

Moving to dbt Core v2 (Fusion)

The Fusion engine is a rewrite of dbt in Rust, with faster builds, native SQL understanding, and real-time validation. It is not a separate product: Fusion ships as dbt Core v2, and it is open source like the Core releases before it. The proprietary parts of dbt live in dbt Cloud, which you reach by logging in from the CLI — the engine itself is not gated. Your SQL and YAML syntax is unchanged. What differs is enforcement: Fusion validates more strictly than earlier Core releases, so a project carrying unresolved deprecation warnings will fail on it.

Before you upgrade

Move to dbt 1.10 and resolve the meta and tags warnings, using MetaMove above.
Run your project and clear any remaining deprecation warnings. dbt-autofix resolves many of them automatically.
Upgrade, and run your project once to surface anything left.

Check the dbt Fusion documentation for engine-specific behavior and any features still landing.

Lightdash works with dbt Core v2. If your project builds on Fusion, Lightdash compiles it the same way it compiles earlier versions — see dbt projects for connection settings.

​Moving to dbt 1.10

​What changed

​Why it matters

​Automated migration with MetaMove

​Installation

​Basic usage

​Safe migration process

​Advanced options

​What MetaMove handles

​Migrating by hand

​Model-level changes

​Column-level changes

​Source-level changes

​Other 1.10 changes to consider

​Handling warnings

​Testing the migration

​Migrating safely

​Troubleshooting

​”Config block already exists” errors

​Unexpected YAML structure

​Performance on large projects

​Moving to dbt Core v2 (Fusion)

​Before you upgrade

Moving to dbt 1.10

What changed

Why it matters

Automated migration with MetaMove

Installation

Basic usage

Safe migration process

Advanced options

What MetaMove handles

Migrating by hand

Model-level changes

Column-level changes

Source-level changes

Other 1.10 changes to consider

Handling warnings

Testing the migration

Migrating safely

Troubleshooting

”Config block already exists” errors

Unexpected YAML structure

Performance on large projects

Moving to dbt Core v2 (Fusion)

Before you upgrade