Difference from javax.xml.crypto.dsig and xml-crypto signature value #486

sibelius · 2025-02-04T12:56:59Z

This is the SignedInfo canonilized

<ds:SignedInfo xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:CanonicalizationMethod><ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"></ds:SignatureMethod><ds:Reference URI="key-info-id"><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod><ds:DigestValue>aUK42Eje/aj9EEEZn1wHoSsQMnPW0J8GpDP124YLE1w=</ds:DigestValue></ds:Reference><ds:Reference URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"></ds:Transform><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod><ds:DigestValue>3lnGNn7gACFoylCvc8RXyyg3etLlVjfuxHYcOZhTL+c=</ds:DigestValue></ds:Reference><ds:Reference><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"></ds:Transform></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"></ds:DigestMethod><ds:DigestValue>aIc0+7BXP+Ee09zng0JE21rkzgVlYaa6qMGHIBQKvTc=</ds:DigestValue></ds:Reference></ds:SignedInfo>

it generates this hash in node

QOyatmgHiGguYkBeNFUJU3b/448S7BnkNA79II01aBrsoP0AlKXAc6wAxh5B9vIRddQrMkOAiC6o8qGF2NVDd7dyNZZ5PeHyMnV88c+MbX4v68w+uVaK4nG2fFucA6Uyvl2UciohGW9bcRk3O6ImF/0tIxyuiZETEYDAd5VYTcLvWgFS2yDMFF+8FadhcFaBB4y79++YK4fCgbyUzgywfoozr+5Q7vyVxVXbhD+rb5e74v/Jg/nkS0fUmQWT2WFt9ZeSYAc/+HgMT3zlADnP6DJHgoPd/Ptmz2SnaUnolAzhFOPFL9zDG0EfWfGBJekjts/EKuq4qC6mCfwdtF/mng==

and it generates this hash in java/kotlin

XQJ8s0LlWL75DwXJ3LPZy6e8JOvkq6ccpQssQVIkr66CWGfgz2WVC0qz6s7duHhbSZxicpgYGQME&#xD;
cwAfrX4rNAEpI3+NcIS7c2Mue2X3BNovfk6mBzB5bSci4bVwB2vWK0lErnjPXHAviYq2lM6YdV50&#xD;
V6hiNwLNIqsLsv7k4ECkY9H0aD693K7yc1n3KT3jrEtql4uouOZbpV1LbFr+vWs0A6aMbJOm7zFz&#xD;
f26gBtgjDlqRIEBgyItG+O4EBKV4DCcxPXvAS4Qx/RXiIz2wUH6EcXvdw40+VmgwgBkWHvCkCQKM&#xD;
KIjEgqyv5jv8pIlPYdttYJq3Ip+vL03Sbtkb5A==

do you have an idea why java signature has #xD;\n ?

The text was updated successfully, but these errors were encountered:

srd90 · 2025-02-04T16:36:23Z

do you have an idea why java signature has #xD;\n ?

Yes. At least I have an idea. Dunno about others.

BTW. You considered this to be a bug at xml-crypto due to reporting bug issue instead of discussion post.

Whats your xml-crypto version and node version and steps to reproduce bug.

(What was your exact JDK version)

srd90 · 2025-02-05T19:59:16Z

In case your JDK version was some of those affected by following:

https://bugs.openjdk.org/browse/JDK-8264194
[BUG] Keycloak document signature is rejected due to the extra characters() at the end of the line node-saml#242 (comment)
- also follow link from aforementioned comment to keycloak side issue and read through comments

sibelius · 2025-02-05T20:04:29Z

Here is a Java implementation https://github.com/aws-samples/pix-proxy-samples/blob/fa20042d19d4c0b78f6898f3053b4cd0729938d0/proxy/core/src/main/java/com/amazon/aws/pix/core/xml/Iso20022XmlSigner.java#L25

srd90 · 2025-02-05T22:01:58Z

I suggested #486 (comment) as answer to your question's #xD; part.

About splitting every 76th character (the \n part of your question)...dunno if I read spec correctly but I get a feeling that oneliner and splitted at every 76th character are both ok.

https://www.w3.org/TR/2002/REC-xmldsig-core-20020212/#sec-SignatureValue

https://www.w3.org/TR/xmlschema-2/#base64Binary

Quick internet search provided following thread regarding xmlsig's base64 stuff:
https://lists.w3.org/Archives/Public/w3c-ietf-xmldsig/2001AprJun/thread.html#msg183

There are also other threads about same stuff (with same subject) at that mailing list with emails from xmlsig spec contributors.

About link you provided to pix-proxy-samples...it does not reveal anything about version of implementation of javax.xml.crypto.dsig.*. Output of java -version of the env that you used to run that example would have. Eitherway...are you saying xml-crypto has a bug or aren't you? If some java sw emits s due whatever reason are those handled by C14N correctly? Provide testcase which pin points actual issue with content that one can feed to whatever xmlsig tool (like xmlsec1) to gather "second opinions" (i.e. provide some signed xml instead of random snippet). You can use e.g. xmlsec1 also by yourself to check whether it validates some signed xml correctly and if it does but xml-crypto does not then there might be issue at xml-crypto side (in that case provide also exact commands how you validated document with some other tool (e.g. with xmlsec1). Consider also checking producer side (in this case pix-proxy-samples) and consider reading speculations from earlier indirectly referenced keycloak issue.

sibelius · 2025-02-05T22:44:27Z

The Java implementation passes our internal tool validation but node xml-crypto does not

We don't have access to the internal tool validation of the signature

I just wanna produce the same signature value both in Java and node

Im trying to understand why they differ and how to make both the same

srd90 · 2025-02-05T22:58:56Z

Give example of signed xml document (along with certificate of key that was used to sign) that passes shell script which uses xmlsec1 --verify ... (construct shell script so that it has everything needed to verify signature. I.e. certificate and signed xml document which shell script passes to xmlsec1) and example of JS code which uses xml-crypto (and xml-crypto fails to validate that same xml document).
and/or give example of JS which uses xml-crypto to sign some XML so that xmlsec1 AND your internal tool fails to validate it (if xmlsec1 validates it correctly but your internal tool fails then you should debug your internal tool)

Having such material gives something to work with. Without such material continuation of this discussion is pointless.

srd90 · 2025-02-05T23:12:19Z

Here is example how xmlsec1 was used at one case as a reference implementation: #212 (comment)

Note: that case was related to signing (and more specifically to find out lack of implicit C14N during signing and how adding it explicitly lined digest values and thus signatures between different tools).

Sidenote: if you would have posted full signed document (signed by xml-crypto) one could see whether there is chance that your root cause is same.

sibelius · 2025-02-05T23:45:37Z

Here is the spec https://www.bcb.gov.br/content/estabilidadefinanceira/cedsfn/Manual_de_Seguranca_PIX.pdf

It is in Portuguese (Brazil), you could easily translate it

It has some specific references style

You can also check some examples in the unit tests of the aws repo

sibelius · 2025-02-06T00:22:12Z

input xml https://github.com/aws-samples/pix-proxy-samples/blob/fa20042/proxy/core/src/test/resources/xml/pacs.008_CONTA_1_msg.xml

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<Envelope xmlns="https://www.bcb.gov.br/pi/pacs.008/1.4">
    <AppHdr>
        <Fr>
            <FIId>
                <FinInstnId>
                    <Othr>
                        <Id>00038166</Id>
                    </Othr>
                </FinInstnId>
            </FIId>
        </Fr>
        <To>
            <FIId>
                <FinInstnId>
                    <Othr>
                        <Id>99999010</Id>
                    </Othr>
                </FinInstnId>
            </FIId>
        </To>
        <BizMsgIdr>M0003816612345678901234567890123</BizMsgIdr>
        <MsgDefIdr>pacs.008.spi.1.4</MsgDefIdr>
        <CreDt>2020-01-01T08:30:12.000Z</CreDt>
        <Sgntr/>
    </AppHdr>
    <Document>
        <FIToFICstmrCdtTrf>
            <GrpHdr>
                <MsgId>M0003816612345678901234567890123</MsgId>
                <CreDtTm>2020-01-01T08:30:12.000Z</CreDtTm>
                <NbOfTxs>1</NbOfTxs>
                <SttlmInf>
                    <SttlmMtd>CLRG</SttlmMtd>
                </SttlmInf>
                <PmtTpInf>
                    <InstrPrty>HIGH</InstrPrty>
                </PmtTpInf>
            </GrpHdr>
            <CdtTrfTxInf>
                <PmtId>
                    <EndToEndId>E9999901012341234123412345678900</EndToEndId>
                    <TxId>90000</TxId>
                </PmtId>
                <IntrBkSttlmAmt Ccy="BRL">1000.00</IntrBkSttlmAmt>
                <AccptncDtTm>2020-01-01T08:30:00.000Z</AccptncDtTm>
                <ChrgBr>SLEV</ChrgBr>
                <Dbtr>
                    <Nm>Fulano da Silva</Nm>
                    <Id>
                        <PrvtId>
                            <Othr>
                                <Id>70000000000</Id>
                            </Othr>
                        </PrvtId>
                    </Id>
                </Dbtr>
                <DbtrAcct>
                    <Id>
                        <Othr>
                            <Id>500000</Id>
                            <Issr>3000</Issr>
                        </Othr>
                    </Id>
                    <Tp>
                        <Cd>CACC</Cd>
                    </Tp>
                </DbtrAcct>
                <DbtrAgt>
                    <FinInstnId>
                        <ClrSysMmbId>
                            <MmbId>10000000</MmbId>
                        </ClrSysMmbId>
                    </FinInstnId>
                </DbtrAgt>
                <CdtrAgt>
                    <FinInstnId>
                        <ClrSysMmbId>
                            <MmbId>20000000</MmbId>
                        </ClrSysMmbId>
                    </FinInstnId>
                </CdtrAgt>
                <Cdtr>
                    <Id>
                        <PrvtId>
                            <Othr>
                                <Id>80000000000</Id>
                            </Othr>
                        </PrvtId>
                    </Id>
                </Cdtr>
                <CdtrAcct>
                    <Id>
                        <Othr>
                            <Id>600000</Id>
                            <Issr>4000</Issr>
                        </Othr>
                    </Id>
                    <Tp>
                        <Cd>SVGS</Cd>
                    </Tp>
                </CdtrAcct>
                <RmtInf>
                    <Ustrd>Campo livre [0]</Ustrd>
                </RmtInf>
            </CdtTrfTxInf>
        </FIToFICstmrCdtTrf>
    </Document>
</Envelope>

signed

<?xml version="1.0" encoding="UTF-8" standalone="no"?><Envelope xmlns="https://www.bcb.gov.br/pi/pacs.008/1.4">
    <AppHdr>
        <Fr>
            <FIId>
                <FinInstnId>
                    <Othr>
                        <Id>00038166</Id>
                    </Othr>
                </FinInstnId>
            </FIId>
        </Fr>
        <To>
            <FIId>
                <FinInstnId>
                    <Othr>
                        <Id>99999010</Id>
                    </Othr>
                </FinInstnId>
            </FIId>
        </To>
        <BizMsgIdr>M0003816612345678901234567890123</BizMsgIdr>
        <MsgDefIdr>pacs.008.spi.1.4</MsgDefIdr>
        <CreDt>2020-01-01T08:30:12.000Z</CreDt>
        
    <Sgntr><ds:Signature xmlns:ds="http://www.w3.org/2000/09/xmldsig#"><ds:SignedInfo><ds:CanonicalizationMethod Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/><ds:SignatureMethod Algorithm="http://www.w3.org/2001/04/xmldsig-more#rsa-sha256"/><ds:Reference URI="#9115a29c-1246-4de2-8d33-d3c8549efac3"><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/><ds:DigestValue>Kmg8b7mLCJtFj1co5CvLBusbRQcpq2rurd87Fy1VmMA=</ds:DigestValue></ds:Reference><ds:Reference URI=""><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2000/09/xmldsig#enveloped-signature"/><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/><ds:DigestValue>0mtmawoDDPLwAAAkSYdy0JmPf5oS55rHZjsXOASzY4c=</ds:DigestValue></ds:Reference><ds:Reference><ds:Transforms><ds:Transform Algorithm="http://www.w3.org/2001/10/xml-exc-c14n#"/></ds:Transforms><ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/><ds:DigestValue>UP6Uy9SYzwVsO+djbyusrydi2/If22U4bEHZt/NJhH0=</ds:DigestValue></ds:Reference></ds:SignedInfo><ds:SignatureValue>IhddKqaN3zNXNN30h6+7RYL6tdAQZmf0UqPnrBwlL5K/cjS6J8ph3DVGODQDWZHNHXqReOiHaAzT&#13;
svR5SnBYek2/Z0rmzs1zuf3pYOKb5n2K/EnG6uLot/lWILd9oOYBJcN5eYRNF70sKcEy5pE0TuEJ&#13;
y1yh0EQKWSOACuESLNH+FH71sFpBXJSgOSj6FVQ0utZc1A4YdvPZZCsayOmSVHqwBgRSGHG05k4b&#13;
C7rX5xhXT4ezMEqk4XXTVZD42j6LFcB09Hvq5RNOqrspRlguztOwl23M6iAtsbmCfgDbG2w0B8HQ&#13;
yFw8maLMVGbfz2WONs6gRasdaEKVnEe1rhnKwg==</ds:SignatureValue><ds:KeyInfo Id="9115a29c-1246-4de2-8d33-d3c8549efac3"><ds:X509Data><ds:X509IssuerSerial><ds:X509IssuerName>CN=client.pix.aws.com,OU=PIX,O=AWS,L=Sao Paulo,ST=SP,C=BR</ds:X509IssuerName><ds:X509SerialNumber>2004099543</ds:X509SerialNumber></ds:X509IssuerSerial></ds:X509Data></ds:KeyInfo></ds:Signature></Sgntr></AppHdr>
    <Document>
        <FIToFICstmrCdtTrf>
            <GrpHdr>
                <MsgId>M0003816612345678901234567890123</MsgId>
                <CreDtTm>2020-01-01T08:30:12.000Z</CreDtTm>
                <NbOfTxs>1</NbOfTxs>
                <SttlmInf>
                    <SttlmMtd>CLRG</SttlmMtd>
                </SttlmInf>
                <PmtTpInf>
                    <InstrPrty>HIGH</InstrPrty>
                </PmtTpInf>
            </GrpHdr>
            <CdtTrfTxInf>
                <PmtId>
                    <EndToEndId>E9999901012341234123412345678900</EndToEndId>
                    <TxId>90000</TxId>
                </PmtId>
                <IntrBkSttlmAmt Ccy="BRL">1000.00</IntrBkSttlmAmt>
                <AccptncDtTm>2020-01-01T08:30:00.000Z</AccptncDtTm>
                <ChrgBr>SLEV</ChrgBr>
                <Dbtr>
                    <Nm>Fulano da Silva</Nm>
                    <Id>
                        <PrvtId>
                            <Othr>
                                <Id>70000000000</Id>
                            </Othr>
                        </PrvtId>
                    </Id>
                </Dbtr>
                <DbtrAcct>
                    <Id>
                        <Othr>
                            <Id>500000</Id>
                            <Issr>3000</Issr>
                        </Othr>
                    </Id>
                    <Tp>
                        <Cd>CACC</Cd>
                    </Tp>
                </DbtrAcct>
                <DbtrAgt>
                    <FinInstnId>
                        <ClrSysMmbId>
                            <MmbId>10000000</MmbId>
                        </ClrSysMmbId>
                    </FinInstnId>
                </DbtrAgt>
                <CdtrAgt>
                    <FinInstnId>
                        <ClrSysMmbId>
                            <MmbId>20000000</MmbId>
                        </ClrSysMmbId>
                    </FinInstnId>
                </CdtrAgt>
                <Cdtr>
                    <Id>
                        <PrvtId>
                            <Othr>
                                <Id>80000000000</Id>
                            </Othr>
                        </PrvtId>
                    </Id>
                </Cdtr>
                <CdtrAcct>
                    <Id>
                        <Othr>
                            <Id>600000</Id>
                            <Issr>4000</Issr>
                        </Othr>
                    </Id>
                    <Tp>
                        <Cd>SVGS</Cd>
                    </Tp>
                </CdtrAcct>
                <RmtInf>
                    <Ustrd>Campo livre [0]</Ustrd>
                </RmtInf>
            </CdtTrfTxInf>
        </FIToFICstmrCdtTrf>
    </Document>
</Envelope>

srd90 · 2025-02-06T00:47:33Z

Now....how is that related to xml-crypto?

Are you saying that xml-crypto fails to verify signature? Does xmlsec1 verify that signature?

Or

Are you saying that if xml-crypto would be used to sign that document then output of xml-crypto cannot be verified by xmlsec1? Provide JS file which signs that document with xml-crypto so that we can see how you tried to do that and output also. Also keep in mind that xml-crypto is unable to sign keyinfo (as discussed at #481 ) so you have to modify your example (which is signing keyinfo also) accordingly (I gather that you are debugging another aspect).

And

where is the certificate / public key which can be used for signature verification.

sibelius · 2025-02-06T01:05:03Z

It is all in the aws repo

srd90 · 2025-02-06T01:14:20Z

It is all in the aws repo

Good. Then there chance that you hand out direct link. No-one else except youself is getting any money to run around to gather material or debugging this on behalf of you.

btw. I copy pasted your example signed xml to https://tools.chilkat.io/xmlDsigVerify.cshtml

It said:

Signature is Invalid
Number of Reference Digests = 3
Reference 1 digest is valid.
Reference 2 digest is invalid because the computed digest differs from the digest in the XML.
Reference 3 digest is invalid because the computed digest differs from the digest in the XML.

Note: there is no way chilkat could say that signature is valid because cert is not available.

But now you have another track which is: explain yourself why unrelated (from xml-crypto pov) tool is unable to calculate same digest values as your reference implementation. Consider consulting also xmlsec1 tool's opinion. Lets continue after you have sorted things out between your reference implementation and xmlsec1 and chilkat. And keep in mind that no matter what xml-crypto is currently unable to sign keyinfo.

sibelius · 2025-02-06T01:24:54Z

I could made xml-crypto to calculate the same digest

I have a custom implementation to sign KeyInfo, not generic enough for this package yet

The digest value are correct and the same as Java implementation

However the signature value has this difference

srd90 · 2025-02-06T01:49:18Z

Show full output (signed xml document) of your xml-crypto fork and your java sw. Use same input (private key and input xml) for both cases.
I am especially interest to see what transformations you have added for references when signing with xml-crypto (keep in mind that it does not apply implicitly that one C14N transformation - in case there wasn't any C14N algorithm at transformation list - during signing as other tools apply which means that due to that issue actual values of signature shall be different unless you add it explicitly ... yes it would pop up to list of transformations but it does not matter it just means that you made it explicit)
If actual values of signature matches but java implementation split signature to multiple lines and adds that #xD; (while xml-crypto writes oneliner) then tell us what xmlsec1 thinks about java and xml-crypto versions when you use it to verify signatures.

srd90 · 2025-02-06T02:08:34Z

tell us what xmlsec1 says about your java and xml-crypto versions of signed xml even if they have different signature (from actual value point of view or from "layout" point of view) but otherwise 1:1 content. Is either of your implementations generating correct signature from xmlsec1 point of view.

srd90 · 2025-02-17T17:53:06Z

Adding more context to this issue for future generations who might land here via some search engine.

Based on content of this comment: #486 (comment)

This issue seems to be at the end of the day "same" as:

Signing elements inside <Signature> #204

or at least very close to that in terms of how signature should be constructed (if we think how this issue might be related to xml-crypto).

Author of #204 was kind enought to describe expected signature content (without having to use any translator to translate portuguese document):

Where URI='' is the signature from the AppHdr without the Signature element, URI='uuid' is the signature from the KeyInfo(A x509 certificate) and ds:Reference is the signature from the Document element.

Regarding Reference which URI attribute is omitted spec https://www.w3.org/TR/2008/REC-xmldsig-core-20080610/#sec-URI says:

If the URI attribute is omitted altogether, the receiving application is expected to know the identity of the object. For example, a lightweight data protocol might omit this attribute given the identity of the object is part of the application context. This attribute may be omitted from at most one Reference in any particular SignedInfo, or Manifest.

So (IMHO) it seems that at the end of the day one has to construct signature without the help of xml-crypto because at least I can't figure out how xml-crypto would be made aware which part of the document should be handled when reference without URI is added.

This second opinion backs aforementioned interpretation: https://stackoverflow.com/questions/15522098/java-xmldsig-reference-with-no-uri/18526152#18526152

About those extra xml entities (which started this whole discussion)...its something java stack related.

srd90 mentioned this issue Feb 17, 2025

Signing elements inside <Signature> #204

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Difference from javax.xml.crypto.dsig and xml-crypto signature value #486

Difference from javax.xml.crypto.dsig and xml-crypto signature value #486

sibelius commented Feb 4, 2025

srd90 commented Feb 4, 2025 •

edited

Loading

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025 •

edited

Loading

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025

srd90 commented Feb 5, 2025

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025 •

edited

Loading

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025

srd90 commented Feb 6, 2025

srd90 commented Feb 17, 2025

Difference from javax.xml.crypto.dsig and xml-crypto signature value #486

Difference from javax.xml.crypto.dsig and xml-crypto signature value #486

Comments

sibelius commented Feb 4, 2025

srd90 commented Feb 4, 2025 • edited Loading

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025 • edited Loading

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025

srd90 commented Feb 5, 2025

srd90 commented Feb 5, 2025

sibelius commented Feb 5, 2025

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025 • edited Loading

sibelius commented Feb 6, 2025

srd90 commented Feb 6, 2025

srd90 commented Feb 6, 2025

srd90 commented Feb 17, 2025

srd90 commented Feb 4, 2025 •

edited

Loading

sibelius commented Feb 5, 2025 •

edited

Loading

srd90 commented Feb 6, 2025 •

edited

Loading