Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: Spiffe Utils #11522

Closed
wants to merge 16 commits into from
Closed

core: Spiffe Utils #11522

wants to merge 16 commits into from

Conversation

erm-g
Copy link
Contributor

@erm-g erm-g commented Sep 13, 2024

No description provided.

Copy link
Contributor

@matthewstevenson88 matthewstevenson88 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for putting this together @erm-g! I took an initial pass and left some comments. Please give me a ping when it's ready for another look. :)

checkArgument(checkNotNull(certChain, "certChain").length > 0, "CertChain can't be empty");
Collection<List<?>> subjectAltNames = certChain[0].getSubjectAlternativeNames();
if (subjectAltNames != null) {
for (List<?> altName : subjectAltNames) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please ensure that there is a unique URI SAN.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve


private SpiffeUtil() {}

public static Optional<SpiffeId> extractSpiffeId(X509Certificate[] certChain)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a Javadoc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve

private static final Integer URI_SAN_TYPE = 6;
private static final String USE_PARAMETER_VALUE = "x509-svid";

private SpiffeUtil() {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Please move to the bottom of the class.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped it completely, similar to #11490 (comment)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve


public static class TrustBundle {

private final Map<String, Long> sequenceNumbers;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What are the sequence numbers? Could you explain why they're relevant here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In short it's a number that indicates the bundle version for a particular TB, however it's not mandatory -
https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE_Trust_Domain_and_Bundle.md#411-sequence-number

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for explaining. Are we going to use the sequence numbers for anything? If not, should we drop them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC the ideas behind the spec, we can use it for logging (because it indicates a kind of version of that particular TD entry in the file) but I don't think we'll be making any decisions based on these numbers (like 'Hey let's reject this entry because current sequence number is less than the previous one).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context. If they're optional and we don't need to use them for anything, can we omit them to simplify the code?

}
}

public static class TrustBundle {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a Javadoc for this class and its methods.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK to resolve

for (Map<String, ?> keyNode : keysNode) {
String kid = JsonUtil.getString(keyNode, "kid");
if (kid != null && !kid.equals("")) {
log.log(Level.SEVERE, String.format("'kid' parameter must not be set but value '%s' "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and throughout this method: since this is a library, should we throw an exception rather than logging?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still need to construct a map even if we can't process some of the entries, i.e json like

domain: google.com
cert:correctBytes
domain: google.com.test
cert:corruptedBytes

should still produce a valid entry for google.com. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, good question. I was thinking that we should be strict for the start and fail if anything is malformed and, if we need to change in the future, we can do so without breaking users. However, if we start off allowing malformed entries, then we can't disallow them in the future without a breaking change. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, ok, we made a similar decision for CRL (a bad/corrupted entry doesn't influence loading other entries - https://github.com/grpc/proposal/pull/382/files#diff-42c23f30b346a23a489989e1fcd1cd3773f62208c99d36b9a246005b2395aa6aR107)
In SPIIFE case it's very similar reasoning - imagine a situation when there is a bunch of providers of trust domains data, who sends the info to distributor. Distributor only merges the stuff together to create the resulting json file and then sends it to the machines. In that case, a mistake by single TD provider crashes all the others.


private final Map<String, Long> sequenceNumbers;

private final Map<String, List<X509Certificate>> trustBundleMap;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Prefer ImmutableMap and ImmutableList.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

return Optional.absent();
}

public static TrustBundle loadTrustBundleFromFile(String trustBundleFile) throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you point me to the spec you're using as a reference for this method?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! What's confusing me is the following - the code and the examples seem to suggest that the file has the following format:

{
  "trust_domains": {
      "trust_domain": {
          // JWK set
      }
   }
}

But the spec only seems to talk about the structure of the JWK set. Where is the rest of the structure specified?

Apologies in advance if I'm missing something obvious. :)

Copy link
Contributor Author

@erm-g erm-g Sep 23, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I got your question - IIUC it's just spread across that md. For example, use and 'kid' parameter - https://github.com/spiffe/spiffe/blob/main/standards/X509-SVID.md#61-publishing-spiffe-bundle-elements

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved in offline discussion, feel free to resolve.

for (String trustDomainName : trustDomainsNode.keySet()) {
Map<String, ?> domainNode = JsonUtil.getObject(trustDomainsNode, trustDomainName);
if (domainNode == null || domainNode.size() == 0) {
trustBundleMap.put(trustDomainName, Collections.emptyList());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this allowed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From connection status perspective it's the same. However, from monitoring/debugging perspective I think it's useful to differentiate between 'Hey, the file we loaded doesn't contain the domain' and 'Hey, the the domain doesn't have the correct root to validate this conn'.
WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'm surprised that the spec allows an empty key or empty value, but agree with you that it doesn't matter from an authentication point of view.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

trustDomainName));
break;
}
String rawCert = JsonUtil.getString(keyNode, "x5c");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this DER-encoded or PEM-encoded?

https://github.com/spiffe/spiffe/blob/main/standards/SPIFFE_Trust_Domain_and_Bundle.md#appendix-a-spiffe-bundle-example suggests the former, but all of our discussions have been slanted to the latter.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

if (subjectAltNames != null) {
boolean spiffeFound = false;
for (List<?> altName : subjectAltNames) {
if (URI_SAN_TYPE.equals(altName.get(0))) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should we check that altName.length > 0?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch - the JDK I have actually defines these altNames as ArrayList(2), but there is no guarantee other JDKs have the same impl details. Added 2 checks.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

throws CertificateParsingException {
checkArgument(checkNotNull(certChain, "certChain").length > 0, "CertChain can't be empty");
Collection<List<?>> subjectAltNames = certChain[0].getSubjectAlternativeNames();
if (subjectAltNames != null) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Readability suggestion: To minimize nesting, consider returning early here, e.g.

if (subjectAltNames == null) {
  return Optional.absent();
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this - if I do it, then I'll need an extra check for the last loop
if no spiffe, then return Optional.absent()
Let's ask Eric?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed as discussed offline

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

checkArgument(checkNotNull(certChain, "certChain").length > 0, "CertChain can't be empty");
Collection<List<?>> subjectAltNames = certChain[0].getSubjectAlternativeNames();
if (subjectAltNames != null) {
boolean spiffeFound = false;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT of storing the SPIFFE ID instead of a boolean, so we don't need to iterate over the list twice?

For example,

Optional<String> uriSan = Optional.absent();
for (List<?> altName : subjectAltNames) {
  if (altName.length > 0 && altName.get(0).equals(URI_SAN_TYPE) {
    if (uriSan.absent()) {
      uriSan = Optional.of(altName.get(0));
    } else {
      throw exception
    }
  }
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you add SPIFFE parsing logic to the same loop, then the following might occur:

  1. SPIFFE found
  2. SPIFFE parsing failed
  3. Exception about SPIFFE parsing thrown
    But if there is another SPIFFE, then we want to throw Duplicate SPIFFE exception, but not the parsing one.

So the second loop is needed for your approach as well (if we store it in the first loop, we still need to iterate over collection one more time to 're-find '), unless we want to play with indices.

What I like about the current approach is the readability - it clearly follows the spec

  1. check if there are duplicates
  2. if none, do the parsing

WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed as discussed offline

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.


public static class TrustBundle {

private final Map<String, Long> sequenceNumbers;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for explaining. Are we going to use the sequence numbers for anything? If not, should we drop them?

private static final String USE_PARAMETER_VALUE = "x509-svid";

/**
* Parses a leaf certificate from the chain to extract unique SPIFFE ID. In case of success,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested edit: "Returns the SPIFFE ID from the leaf certificate, if present."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on the fence here - a reader already sees that the method returns Optional<SpiffeId>, so my intent here is to provide a little bit more details about what we;re returning.
WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed as discussed offline

for (String trustDomainName : trustDomainsNode.keySet()) {
Map<String, ?> domainNode = JsonUtil.getObject(trustDomainsNode, trustDomainName);
if (domainNode == null || domainNode.size() == 0) {
trustBundleMap.put(trustDomainName, Collections.emptyList());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense. I'm surprised that the spec allows an empty key or empty value, but agree with you that it doesn't matter from an authentication point of view.

"e": "AQAB"
},
{
"kty": "RSA",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add the x5c key so that we can confirm in our tests that multiple certs for the same trust domain get properly captured in the spiffe bundle?

for (Map<String, ?> keyNode : keysNode) {
String kid = JsonUtil.getString(keyNode, "kid");
if (kid != null && !kid.equals("")) {
log.log(Level.SEVERE, String.format("'kid' parameter must not be set but value '%s' "
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see, good question. I was thinking that we should be strict for the start and fail if anything is malformed and, if we need to change in the future, we can do so without breaking users. However, if we start off allowing malformed entries, then we can't disallow them in the future without a breaking change. WDYT?

try {
Collection<? extends Certificate> certs = CertificateFactory.getInstance("X509")
.generateCertificates(stream);
if (certs.size() > 0) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this be > 1? If yes, should we take those into account? What does the spec say about this case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it can be >1 (it's not an exception). IIUC we can simply ignore it - https://datatracker.ietf.org/doc/html/rfc7517#section-4.7
WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taking back my previous comment - it must be exactly one
Changed accordingly

String trustDomainName) {
List<X509Certificate> result = new ArrayList<>();
for (Map<String, ?> keyNode : keysNode) {
String kid = JsonUtil.getString(keyNode, "kid");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optional: To make the code more readable, consider moving the checks on the "kid" and "use" claims into a helper function.

return Optional.absent();
}
String uri = null;
for (List<?> altName : subjectAltNames) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment before this for loop to add some explanation, e.g. "Search for the unique URI SAN."


public static class TrustBundle {

private final Map<String, Long> sequenceNumbers;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the context. If they're optional and we don't need to use them for anything, can we omit them to simplify the code?

}
}

public static class TrustBundle {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK to resolve

}
}

public static class TrustBundle {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK to resolve


private final ImmutableMap<String, ImmutableList<X509Certificate>> bundleMap;

public SpiffeBundle(Map<String, Long> sequenceNumbers,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Prefer using a static factory (and a private constructor) instead in case we need to do work in the constructor in the future (e.g. validation of the inputs).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class is a part of non-public API and the intent is to use it as a simple 'value' class, similar to https://github.com/grpc/grpc-java/blob/master/core/src/main/java/io/grpc/internal/SpiffeUtil.java#L103. If we need to add some validation, I'd prefer to do it in a caller code instead.
But definitely constructor should be private, fixed


private final ImmutableMap<String, Long> sequenceNumbers;

private final ImmutableMap<String, ImmutableList<X509Certificate>> bundleMap;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Should it be ImmutableSet rather than ImmutableList?

Rationale: There is no inherent order to the trusted certs in this list. If we ultimately need to feed this in to an API that only accepts a List, I'm fine keeping it tho.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a really interesting question. JDK API uses arrays, and gRPC Java uses lists -

void updateTrustedRoots(List<X509Certificate> trustedRoots);

I suspect the issue with Sets is that X509Certificate itself is an abstract class, so developers can override equals/hashcode and it might lead to bugs.
So I'd just prefer to leave it a List similar with surrounding classes

return Optional.absent();
}

public static TrustBundle loadTrustBundleFromFile(String trustBundleFile) throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK to resolve

Path path = Paths.get(checkNotNull(trustBundleFile, "trustBundleFile"));
String json = new String(Files.readAllBytes(path), StandardCharsets.UTF_8);
Object jsonObject = JsonParser.parse(json);
if (!(jsonObject instanceof Map)) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve, I don't have a strong preference

return Optional.absent();
}

public static TrustBundle loadTrustBundleFromFile(String trustBundleFile) throws IOException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

}
@SuppressWarnings("unchecked")
Map<String, ?> root = (Map<String, ?>)jsonObject;
Map<String, ?> trustDomainsNode = JsonUtil.getObject(root, "trust_domains");
Copy link
Contributor

@matthewstevenson88 matthewstevenson88 Sep 26, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Can trust_domains be a constant? Similarly for the other string literals below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a similar thought, but other json parsing examples in the repo never use constants for such cases, for example https://github.com/grpc/grpc-java/blob/master/core/src/main/java/io/grpc/internal/ServiceConfigUtil.java


private final Map<String, Long> sequenceNumbers;

private final Map<String, List<X509Certificate>> trustBundleMap;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok to resolve.

{
"kty": "RSA",
"use": "x509-svid",
"x5c": "-----BEGIN CERTIFICATE-----
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Per offline discussions, we need to remove the block headers. Similarly throughout.

public void extractSpiffeIdSuccessTest() throws CertificateParsingException {
Optional<SpiffeUtil2.SpiffeId> spiffeId = SpiffeUtil2.extractSpiffeId(spiffeCert);
assertEquals("foo.bar.com", spiffeId.get().getTrustDomain());
assertEquals("client/workload/1", spiffeId.get().getPath());
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Shouldn't the path be prefixed with "/"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Correct, this is a deficiency of this dummy impl. It's fixed after merging with real parsing library at https://github.com/grpc/grpc-java/pull/11575/files#diff-fdbe5016403200b9fc7adab29732582366f2494986933098d86fddd1081d6a0bR241


private X509Certificate[] spiffeCert;
private X509Certificate[] spiffeMultiCert;
private X509Certificate[] serverCert0;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: Why is there a 0 suffix? Can this be omitted?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's an existing cert from other tests I'm reusing (they have server0 and server1), so I just keep the name aligned with other tests - https://github.com/grpc/grpc-java/blob/master/testing/src/main/resources/certs/server0.pem



private X509Certificate[] spiffeCert;
private X509Certificate[] spiffeMultiCert;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "SPIFFE multi cert" is a bit misleading, could we rename it (and the file) to "multipleUriSanCert"?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in #11575

}

@Test
public void extractSpiffeIdParameterValidityTest() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a test where we fail to extract a SPIFFE ID from a cert without any URI SANs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@erm-g erm-g closed this Sep 30, 2024
@erm-g
Copy link
Contributor Author

erm-g commented Sep 30, 2024

Closed in favor if #11575

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants