cmd/link: can't dynamically import static CFString constants using the internal linker

[qmuntal]

Go version

go version devel go1.24-2b3b5305c6 Fri Jan 24 12:35:21 2025 +0100 darwin/arm64

Output of go env in your module/workspace:

AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/Users/qmuntaldiaz/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/qmuntaldiaz/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/6_/bk27w0c510ld88800_hfj6j00000gn/T/go-build3109590806=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/Users/qmuntaldiaz/code/golang-go/src/go.mod'
GOMODCACHE='/Users/qmuntaldiaz/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPATH='/Users/qmuntaldiaz/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/Users/qmuntaldiaz/code/golang-go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/qmuntaldiaz/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/Users/qmuntaldiaz/code/golang-go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='devel go1.24-2b3b5305c6 Fri Jan 24 12:35:21 2025 +0100'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

Execute the following code with go run -ldflags=-linkmode=internal ..

package main

// #cgo LDFLAGS: -framework CoreFoundation
// #include <CoreFoundation/CFString.h>
import "C"
import "unsafe"

//go:cgo_import_dynamic main._kCFErrorUnderlyingErrorKey kCFErrorUnderlyingErrorKey "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation"

var _kCFErrorUnderlyingErrorKey byte
var kCFErrorUnderlyingErrorKey = C.CFStringRef(unsafe.Pointer(*(*uintptr)(unsafe.Pointer(&__kCFErrorUnderlyingErrorKey))))

func main() {
	out := make([]byte, C.CFStringGetLength(kCFErrorUnderlyingErrorKey))
	var used C.CFIndex
	C.CFStringGetBytes(kCFErrorUnderlyingErrorKey, C.CFRange{0, C.CFIndex(len(out))},
		C.kCFStringEncodingUTF8, 0, 0,
		(*C.UInt8)(&out[0]), C.CFIndex(len(out)), &used)

	want := "NSUnderlyingError"
	got := string(out[:used])
	if got != want {
		println("got", got, "want", want)
		panic("FAIL")
	}
}

Note that I'm using some C functions (C.CFStringGetLength and C.CFStringGetBytes ) to make the reproducer self-contained. What I really do is define all external functions using cgo_import_dynamic, which doesn't modify the failure described here.

@golang/compiler

What did you see happen?

main.init: program too large, address relocation distance = -4295397376

What did you expect to see?

The execution succeeds.

[gopherbot]

Change https://go.dev/cl/645235 mentions this issue: cmd/link: improve support for dynamic import variables on Darwin

[gabyhelp]

Related Issues

• cmd/link: cgo_import_dynamic with internal linking provides wrong symbol #61220 (closed)
• cmd/compile: internal compiler error: missing import reader for cc.noder.Position #49016 (closed)
• cmd/go/internal/work: CGO builds don't link all required libraries when pkg-config files use Requires.private #70209 (closed)
• cmd/link: "pc-relative relocation address too big" for program with huge static data #63202 (closed)
• cmd/link: external linking does not link dynamic libraries without cgo #42459 (closed)
• cmd/compile: ICE compiling rangefunc for < 1.23 in combination with Cgo #67497 (closed)
• cmd/link: cgo fails to link on macos 13 beta (arm64) #53372 (closed)
• cmd/link: using -fuzz with test that links with cgo on darwin causes linker failure #65169 (closed)
• cmd/ld: unsupported relocation for dynamic symbol #9967 (closed)
• Fail to call c++ static lib. #39455 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[qmuntal]

This is another instance of #61220. @cherrymui @ianlancetaylor you mentioned in there that cgo_import_dynamic is not supported in end user code, but the fix for this issue (see CL 645235) is small and straightforward. It would have even helped the Go runtime in the past:

go/src/crypto/x509/internal/macos/security.go

Lines 72 to 80 in e81f715

	// Dictionary keys are defined as build-time strings with CFSTR, but the Go
	// linker's internal linking mode can't handle CFSTR relocations. Create our
	// own dynamic strings instead and just never release them.
	//
	// Note that this might be the only thing that can break over time if
	// these values change, as the ABI arguably requires using the strings
	// pointed to by the symbols, not values that happen to be equal to them.
	var SecTrustSettingsResultKey = StringToCFString("kSecTrustSettingsResult")

(The types in macos/security.go are now unused and will be soon removed as part of CL 643277).

[cherrymui]

I think we support data relocations (e.g. R_ADDR) to a dynamic import variable. So this is only a problem if you reference the variable (or address of it) directly from a function (including compiler-generated init function)?

If you do something like

var addr = &__kCFErrorUnderlyingErrorKey
var kCFErrorUnderlyingErrorKey = C.CFStringRef(...(addr))

Would that work?

Since cgo_import_dynamic is internal, we can ensure that we only use data relocations to them.

//go:linkname __kCFErrorUnderlyingErrorKey _kCFErrorUnderlyingErrorKey

I'm also not sure I understand the linkname, and the different number of underscores...

[qmuntal]

Would that work?

It does compile, but the variable doesn't point to the right location. I will investigate a bit more.

I'm also not sure I understand the linkname, and the different number of underscores...

I also don't really understand why that's necessary. I took the linkname and the pointer mangling from the cgo generated code, which does something similar. Edit: I've removed the linkname from the reproducer, it is not necessary.

[qmuntal]

I've simplified my reproducer to the following code:

package main

// #cgo LDFLAGS: -framework CoreFoundation
// #include <CoreFoundation/CFString.h>
import "C"

//go:cgo_import_dynamic main.kCFErrorUnderlyingErrorKey kCFErrorUnderlyingErrorKey "/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation"

var kCFErrorUnderlyingErrorKey C.CFStringRef

func main() {
	C.CFShow(C.CFTypeRef(kCFErrorUnderlyingErrorKey))
}

With CL 645235 I managed to make it work if using var kCFErrorUnderlyingErrorKey *C.CFStringRef and dereferencing the pointer in the C.CFShow call. I'm having a hard time trying to instruct the linker to automatically dereference the GOT pointer...