Configuration files not properly parsed

[cylldby]

Hello,

I'm experimenting with this package to get a hdfs client in a Docker container with minimal size.

For this I install it from source in an alpine container (no problem on that part).
Kerberos authentication is ensured, and a token is accessible through the KRB5CCNAME variable.

The HADOOP_CONF_DIR is set to a directory where are placed a core-site.xml and a hdfs-site.xml config files.
This should be enough to have the client access the hdfs but trying to use it gives the following error:

Couldn't find a namenode to connect to. You should specify hdfs://: in your paths. Alternatively, set HADOOP_NAMENODE or HADOOP_CONF_DIR in your environment.

After testing a bit it appears this is due to the contents of the configuration files, as for instance core-site.xml is of the form:

<configuration xmlns:xi="http://www.w3.org/2001/XInclude">
    <xi:include href="http://<somedomain>/<somedirectory>/core-site.xml" />

when I replace the contents of the config files by the contents of the referenced files, I get a different error:

Couldn't connect to namenode: no available namenodes: dial tcp: lookup ${dfs.namenode.hostname.nn4}: no such host

It appears that the variable is not substituted at read time.

Does this qualify as a bug ?
Are there any possible workaround ?

Thank you for your support !

[cylldby]

This is very likely related to this

[colinmarc]

It seems like the issue is that the golang xml parser doesn’t support XInclude. An easy workaround is to just not use that directive.

I’m going to close this as I don’t really see a reason to support that feature - please feel free to tell me if it’s important to you, though!

[cylldby]

Agreed! Thank you for looking into this.

The workaround is easy to setup so no reason to add this in the feature request list. Proper parsing will come naturally when go/xml supports it :)
I could test your tool by just replacing the config file by the file it referred to and everything ran smoothly.

Amazing tool ! Super easy to containerize, it allows for a very tiny container, extremely fast and easy to use.