TIP 001: Deterministic Logs

[carsonfarmer]

The following is proposed text, to be added to the Textile Threads Whitepaper. It describes a proposed mechanism for deriving deterministic Log IDs and associated keys. It addresses #44 and #40, plus some other ongoing work. It is presented here for discussion and approval before being added to the document, and then creating an implementation proposal on https://github.com/textileio/go-textile-threads. References, citations, and other links to be added.

Log Identity

Like IPFS Peers, a hash of a public key is used as a Log identifier (Log ID). Unlike IPFS Peers, this key is actually derived from a random Seed (or source), rather than used directly. In practice, this seed is derived from a Peer's private key, but it could also potentially come from an external identity such as those provided by Keybase.io, 3Box.io, and others.

Identity Key
: The Identity Key is an asymmetric key-pair that is used to derive the Log ID (hash of the public key), and subsequent child Writer Key(s).

Writer Key
: The Writer Key is an asymmetric key-pair that is used to sign Events added to a Log. It is derived from its owner's Log Identity Key.

Additionally, each Log requires at least one asymmetric Writer Key-pair that determines ownership and identity. The Writer Key(s) is/are used to sign each Event added to the Log, so downstream consumers can verify the Log's authenticity. To make authentication easier for external peers, some means of deterministically deriving the public component of a Writer Key is needed. This is done using a Hierarchical Deterministic Key (HDK) derivation framework, as defined in BIP32. Details of how BIP32 implements HDKs can be found here and here. For the purposes of this paper, it is sufficient to understand that nested child keys can be derived from a single parent key. To protect against the potential data exposure from a leaked child key, a hardened key derivation should be used at the upper levels of the hierarchy, as outlined in BIP44 (and by proxy, BIP43). The hierarchy should be setup as in fig. X.

INSERT DERIVATION PATH DIAGRAM HERE

The levels of the derivation path in fig. X correspond to the components of the path given in:

m / Purpose' / Account' / ...Path / Log / Writer

where each component in the path corresponds to a different level of the key-pair hierarchy. Tiers marked with a ' are levels that are "hardened" (see ref for details). This is a variation on the standard BIP44 format, inspired by 3Box and EIP1775. In the case of Threads, Purpose is set to 7478746 (which is the hex representation of txtl, but could also be other values). The Account level can be used to manage different Textile accounts or even identities. By default, only the 0 Account is used, and this is should be hardened to protect the user. An external identity can be used in place of the Account level (including a Libp2p host key).

The ...Path component is actually a series of levels derived from some source of random bytes. In practice, the Random Component (see sec from Whitepaper on Thread IDs) of the Thread ID is decomposed into n 8-bit levels (l) to produce a path such as:

...Path = l0 / l1 / ... / ln

All levels in this ...Path component, as well as the subsequent Log level are left unhardened to allow deterministic creation of child public keys from the "extended" (see below) public keys. An advantage of this HDK approach is that new child public keys can be created exclusively from the child's extended key, such that other peers can derive addresses without knowing any private keys. However, extended keys need to be handled with care to avoid private key leakage, as such, no private keys from the ...Path component are used, to avoid leaking them to external peers.

In practice, the Log level can default to 0, with the option of using other indices to produce multiple Logs per Thread. It is the (extended) public key from this Log component that is used to derive the actual Log ID (we use the hash of the public key). Any number of Writer Key(s) can then be derived under a given Log. From this, Log public keys can be deterministically derived by external peers, simplifying validation on remote clients when processing Log Events.

Putting the above path description together, we arrive at a derivation path that may look something like:

 m / 7478746' / 0' / 0 / 1 / 2 / 3 / ... / 30 / 31 / 0 / 0

Using the Thread's Random Component to derive the ...Path component means that any Thread Peer is able to deterministically derive another Peer's Log ID (and public component of their Writer Key) using only their Seed/Peer ID and the given Thread ID. This HDK framework also opens up the possibility of using BIP39 mnemonics for re-creating the root of a HDK hierarchy, and by association, a Peer's (set of) key-pairs.

The actual path implementation may vary from implementation to implementation, as long as Log siblings can be deterministically derived from the same parent level. The path derivation above should however, provide sufficient randomness to prevent accidental leackage of secrets should an account Seed be drawn from elsewhere.

Note
: Extended public keys are the 256-bit public key and a 256-bit "chain code", see ref for details

[burdiyan]

What's the status of this TIP? I'm really interested in this kind of discussion, as this is similar to what I'm trying to achieve currently.

[carsonfarmer]

Hi @burdiyan, the status of this TIP will likely be updated at the end of next week. Our team is planning to have a high and low level discussion offline to start. We’ll report back here with the results of that discussion, with the goal of opening the discussion up more broadly to the wider community. If you have some specific use cases or requests, we’d love to hear them. If possible, feel free to share here. Or ping me on Slack if it is not publicly shareable. In the mean time, it is likely that the actual mechanics of this will live ‘outside’ of the core Threads protocol, but we will provide tooling and libraries to do this type of key derivation for input into Threads. The idea there being, Threads and Logs should be agnostic to where a key comes from, and instead just support generic interfaces that can be satisfied by something like the above, or even just simple IPFS peer ids, 3IDs, or whatever.
Either way, looking forward to a productive discussion in this thread!

[burdiyan]

@carsonfarmer Is there any progress on this?

I noticed something today that might be related to this topic.

In the paper it’s described how logs have their own identity and their own multiaddress with log protocol. But I couldn’t find anything about it in go-threads. Am I missing something?

[carsonfarmer]

This TIP has recently become less relevant due to (in part) this one: textileio/go-threads#229. Having said that, we will be making some updates to the thread/log identification design soon. The TL;DR here is that Logs now have a LogKey that defines their 'address' (in addition to the thread with which they belong). That piece is already in go-threads, so you can see how LogKey is used to get an idea of how this is working now.