Question: capabilities, externref, and the C language

[hoodmane]

Back in 2019 in #109 @sbc100 and @sunfishcode had this interaction:

Are you suggesting that one of our goals should be to able to create and enforce such [capability]
boundaries within a single module?

Yes :-). This is part of what capability-based API design [...] means

I am curious whether my understanding of the state of affairs is correct:

1. This is still a goal?
2. But it is not currently implemented?
3. And the reason it is not currently implemented is because the webassembly frontend in C cannot express externrefs?

More specifically, the key idea of capabilities as I understand it is that they are supposed to be unforgeable even within wasm modules. So that if a particular wasm function does not have access to the capability either as a global or as an argument, it cannot access it at all even if other wasm functions in the same module can access the capability.

But in the case of file descriptors, currently wasi-libc does typedef uint32_t __wasi_fd_t; meaning that a file descriptor is just a number and a function only has to guess the number in order to access the backing stream. In @sunfishcode's terms, this is a ghost. The only way to get around this is to have typedef externref __wasi_fd_t; but there is currently no way to express this in valid C code.

Is this state of affairs correct? Is it documented anywhere? And is anyone working on adding externref support to C? Based on my understanding, this seems pretty important to the success of wasi as a project. Although using the webassembly module as the barrier for capabilities works fine and does not require externref.

[sbc100]

Your summary of the current state of affairs sounds reasonable to me.

Remember that there are many languages that can/will target WASI, and not all of them have the same limitations that C/C++ does. The only work I know of that is happening with adding such support to the C/C++ frontend is the work being done by igalia here: https://github.com/Igalia/ref-cpp. Some of that work has landed already, but IIUC there followup work to make it really usable has been put on hold. Somebody with time/energy to invest in it you could potentially try to push it forward.

Also, even if all the work in https://github.com/Igalia/ref-cpp is completed it would not help with the wasi-libc problem that you are refering too since we cannot change the type signature of the core C library functions open/close/read/write without breaking existing code. Existing code expects open to return an integer and not a reference-typed object as proposed in https://github.com/Igalia/ref-cpp#language-extension.

[hoodmane]

we cannot change the type signature of the core C library functions open/close/read/write without breaking existing code

Interesting, so it's not just an ABI break but an actual API break because people write

int fd = open(...)

it's a bit too bad that the file descriptors weren't typed as a newtype of some sort so that it would be possible to only break API and not ABI. Of course that would have negative consequences for porting pre-existing posix code but it seems to be the worst of both worlds to break posix but not get actual "fine grained capabilities"...

Perhaps one could use a compiler flag to choose between a future externref-based "fine grained capabilities" API and the current i32-based "capabilities scoped to the webassembly module boundary" state of affairs.

[sunfishcode]

Even if file descriptors in POSIX had been some form of newtype, I expect we'd still be in this position, because externref can't be stored anywhere that can have its address taken. You can't memcpy it, and so on. So yes, where we're at today is, at the level of just the component model, the unit of isolation is the instance.

However, in source languages which support user-defined newtypes, it is possible to wrap up a handle index that you'll get from the component model canonical ABI in a newtype, and then the source-language-level type system can enforce finer-grained capabilities. For example, in Rust File is just such a newtype around an integer index, and it's even considered unsafe to convert from a raw integer value into a File. So the result is effectively a fine-grained capability model enforced at the source-language level, rather than at the wasm level. And WASI's handles help support this.