cmd/cue: be more consistent and flexible with how we represent optional CUE fields in gengotypes

[mvdan]

Currently, CUE optional fields which generate as struct types use pointers, e.g. StructField *Struct, so that we can represent a missing field as nil.

We do the opposite for basic types, e.g. StringField string, because using a pointer from the Go side would require shenanigans like x := "value"; v.StringField = &x. Moreover, in many cases, it's OK to represent a missing field as the zero value of a basic type; it's relatively rare to need to separate "missing field" from "field set to a zero value".

However, this current mechanism is inconsistent, which causes surprise to any new users of cue exp gengotypes. Moreover, even though pointers are less painful with structs, as one can do e.g. v.StructField = &Struct{...}, one may still need nested nil checks to safely use the structured data, e.g. if v.StructField != nil && v.StructField.Sub1 != nil && v.StructField.Sub1.Sub2 != nil { ... }. So pointers can lead to underwhelming Go UX for any kind of type.

As such, we should not use pointers by default for any type. This leads to a lossy representation of optional fields in Go, but we assume that that's fine for the majority of users.

We would then pair this with an option, such as @go(,optional=pointer), which would switch a field, or an entire definition, or an entire file, or an entire package, over to using pointers to represent optional fields. Pointer-like types such as maps and slices would continue to omit an extra pointer, as it's unnecessary - they are already nilable. This opt-in mode would ensure optional fields map to Go types correctly, ensuring no information is lost, but at the cost of UX as described above.

One open question is what to do about json:",omitempty". I suggest we continue to generate that for all CUE optional fields, whether or not the type is nilable. Even in the default mode, I still think using omitempty is best because:

• It keeps consistency, as it's always there, no matter the Go type or presence of optional=pointer
• For slices and maps, omitempty will still work perfectly fine
• For structs, omitempty will not work (a struct with some fields is never empty), but then it's just a harmless reminder
• For basic types like string or int, omitempty will work based on the zero value, which conflates missing versus set to the zero value - but that correctly represents the lossy translation to the Go types, and is still better than nothing

cc @jmgilman @davidmdm who raised these issues on Slack

[mvdan]

I also note that @go(,optional=pointer) leaves room for other strategies to represent CUE optional fields in Go. For example, if Go gained an "optional" or "maybe" standard generic type akin to https://pkg.go.dev/database/sql#Null, then we could offer @go(,optional=generic). And likewise for other useful strategies that users might request.