"LoadError: Constant memory is stored (or returned) to a differentiable variable" #2306
Comments
|
on the autodiff line https://github.com/swilliamson7/data_assimilation_2/blob/b605f3dbe4670e9b79d9e9b1c2cf9d4b62c0138d/barotropic_gyre/cp_enzyme_error.jl#L239 can you try doing (that's what the error message suggests trying) |
|
I tried that, and got seemingly the same error message: julia> include("cp_enzyme_error.jl")
m = 130
n = 130
[ Info: Revolve: Number of checkpoints: 82
[ Info: Revolve: Number of steps: 6733
[ Info: Prediction:
[ Info: Forward steps : 16629
[ Info: Overhead factor : 2.469775731471855
[ Info: Size per checkpoint: 6.442 MiB
[ Info: Revolve: First Uturn
[ Info: Size of total storage: 528.203 MiB
ERROR: LoadError: Constant memory is stored (or returned) to a differentiable variable.
As a result, Enzyme cannot provably ensure correctness and throws this error.
This might be due to the use of a constant variable as temporary storage for active memory (https://enzyme.mit.edu/julia/stable/faq/#Runtime-Activity).
If Enzyme should be able to prove this use non-differentable, open an issue!
To work around this issue, either:
a) rewrite this variable to not be conditionally active (fastest, but requires a code change), or
b) set the Enzyme mode to turn on runtime activity (e.g. autodiff(set_runtime_activity(Reverse), ...) ). This will maintain correctness, but may slightly reduce performance.
Mismatched activity for: store {} addrspace(10)* %.fca.0.12.extract, {} addrspace(10)* addrspace(10)* %.innerparm.sroa.13.0..sroa_cast, align 8, !dbg !14, !alias.scope !27, !noalias !31 const val: %.fca.0.12.extract = extractvalue { { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)* } %0, 0, 12, !dbg !16, !enzyme_type !23, !enzymejl_byref_MUT_REF !0, !enzymejl_source_type_Matrix\7BFloat32\7D !0
value=Unknown object of type Matrix{Float32}
llvalue= %.fca.0.12.extract = extractvalue { { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, { i64, i64, {} addrspace(10)*, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, i64, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)*, {} addrspace(10)* }, {} addrspace(10)* } %0, 0, 12, !dbg !16, !enzyme_type !23, !enzymejl_byref_MUT_REF !0, !enzymejl_source_type_Matrix\7BFloat32\7D !0
Stacktrace:
[1] getproperty
@ ./Base.jl:37
[2] getproperty
@ ./Base.jl:0
Stacktrace:
[1] getproperty
@ ./Base.jl:37 [inlined]
[2] getproperty
@ ./Base.jl:0 [inlined]
[3] augmented_julia_getproperty_11560_inner_1wrap
@ ./Base.jl:0
[4] macro expansion
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:5340 [inlined]
[5] enzyme_call
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4878 [inlined]
[6] AugmentedForwardThunk
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4814 [inlined]
[7] runtime_generic_augfwd(activity::Type{…}, runtimeActivity::Val{…}, width::Val{…}, ModifiedBetween::Val{…}, RT::Val{…}, f::typeof(getproperty), df::Nothing, primal_1::ShallowWaters.DiagnosticVars{…}, shadow_1_1::Nothing, primal_2::Symbol, shadow_2_1::Nothing)
@ Enzyme.Compiler ~/.julia/packages/Enzyme/QsaeA/src/rules/jitrules.jl:424
[8] macro expansion
@ ~/.julia/packages/UnPack/EkESO/src/UnPack.jl:100 [inlined]
[9] macro expansion
@ ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:55 [inlined]
[10] #20
@ ~/.julia/packages/Checkpointing/uBrnJ/src/Checkpointing.jl:182 [inlined]
[11] diffejulia__20_13266_inner_1wrap
@ ~/.julia/packages/Checkpointing/uBrnJ/src/Checkpointing.jl:0
[12] macro expansion
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:5340 [inlined]
[13] enzyme_call
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4878 [inlined]
[14] CombinedAdjointThunk
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4750 [inlined]
[15] autodiff
@ ~/.julia/packages/Enzyme/QsaeA/src/Enzyme.jl:503 [inlined]
[16] autodiff
@ ~/.julia/packages/Enzyme/QsaeA/src/Enzyme.jl:544 [inlined]
[17] rev_checkpoint_struct_for
@ ~/.julia/packages/Checkpointing/uBrnJ/src/Schemes/Revolve.jl:460
[18] macro expansion
@ ~/.julia/packages/Checkpointing/uBrnJ/src/Checkpointing.jl:177 [inlined]
[19] exp1_cpintegrate
@ ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:49 [inlined]
[20] exp1_cpintegrate
@ ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:0 [inlined]
[21] diffejulia_exp1_cpintegrate_12926_inner_1wrap
@ ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:0
[22] macro expansion
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:5340 [inlined]
[23] enzyme_call
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4878 [inlined]
[24] CombinedAdjointThunk
@ ~/.julia/packages/Enzyme/QsaeA/src/compiler.jl:4750 [inlined]
[25] autodiff
@ ~/.julia/packages/Enzyme/QsaeA/src/Enzyme.jl:503 [inlined]
[26] autodiff(::ReverseMode{…}, ::Const{…}, ::Duplicated{…}, ::Const{…}, ::Duplicated{…}, ::Duplicated{…})
@ Enzyme ~/.julia/packages/Enzyme/QsaeA/src/Enzyme.jl:544
[27] autodiff
@ ~/.julia/packages/Enzyme/QsaeA/src/Enzyme.jl:516 [inlined]
[28] exp1_gradient_eval(G::Vector{Float64}, param_guess::Vector{Float32}, data::Matrix{Float32}, data_spots::StepRange{Int64, Int64}, data_steps::StepRange{Int64, Int64}, Ndays::Int64)
@ Main ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:239
[29] exp1_initialcond()
@ Main ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:327
[30] top-level scope
@ ~/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:341
[31] include(fname::String)
@ Base.MainInclude ./client.jl:494
[32] top-level scope
@ REPL[8]:1
in expression starting at /Users/swilliamson/Documents/GitHub/data_assimilation_2/barotropic_gyre/cp_enzyme_error.jl:341
Some type information was truncated. Use `show(err)` to see complete types. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Not sure what to call this bug, but I hit it when I try and differentiate my code using both Checkpointing.jl and Enzyme.jl, when I remove Checkpointing.jl it seems to go away. The bug itself is
and to reproduce it you can run this script, but you'll need ShallowWaters.jl in the same folder (just cloning the standard ShallowWaters.jl should work)
The text was updated successfully, but these errors were encountered: