terragrunt dependency output resolution is slow

[lucalooz]

Describe the bug

When fetching outputs of a dependency, if these are not already initialized, Terragrunt creates a temporary dir and does a "fast" terraform init -get=false to be able to do a terraform output -json. After each run the temporary dir is deleted.

By default this can be very slow because it would initialize the provider for each dependency at every run.

Enabling the Provider Cache Server doesn't give any benefit because it's injected "too late" in the cli context when running the action, so the parsing / resolution phase already happened.

At the same time relying on the Terraform Plugin Cache is unsafe if there are multiple dependencies because they can be resolved concurrently. The old terragrunt code that applied an init mutex was removed when the Provider Cache Server was introduced.

I've investigated this after I prepared multiple components and my coworker did a checkout and needed to work just on 1 component. On his machine each terragrunt plan took 30 seconds just in the resolution phase.

Possible fixes

I think that the first improvement, without changing the dependency resolution code, would be to enable the Provider Cache Server also at parsing time.

Another one would be, if the dependency isn't already initialized, to try to make some sort of "partial" terragrunt init so that future runs can leverage it. I suppose it needs some work to avoid side effects or broken .terragrunt-cache.

Currently I've chosen to short-circuit all of this because we've the remote state on S3, so with direnv I've an a .envrc with:

export TG_PROVIDER_CACHE=1
# Speedup dependency resolution by fetching outputs directly (works only with S3 remote) without relying on "terraform init -get=false && terraform output"
export TG_DEPENDENCY_FETCH_OUTPUT_FROM_STATE=true
# Speedup dependency resolution skipping inputs. Limitation: you can only use .outputs and not .inputs under a "dependency"
export TG_STRICT_CONTROL=skip-dependencies-inputs

Steps To Reproduce

1. Create components A, B, C
2. C depends both on A and B using their outputs
3. A and B are never initialized
4. Run terragrunt --log-level=debug plan for C
5. Inspect logs and timestamps, e.g:

12:37:48.723 DEBUG  [../A] Running command: terraform init -get=false
12:37:48.723 DEBUG  [../B] Running command: terraform init -get=false
12:37:57.341 DEBUG  [../A] Running command: terraform output -json
12:37:57.341 DEBUG  [../B] Running command: terraform output -json

Expected behavior

Dependency resolution shouldn't be slowed down this much by default if a coworker that didn't initialize everything wants to interact just with one component.

Versions

• Terragrunt version: 0.73.13
• Environment details: macOS 15.3