Fix transliterator fallback loading

[robertbastian]

#5468

[sffc]

I left some commentary

[sffc]

Suggestion (optional): invert load_with_override to also return Option<Result>

[sffc]

Following the logic here:

1. dep.strip_prefix returns Option<...>
2. .and_then now resolves to Option<Result<...>>. Will be None if either strip_prefix failed or if load_special did not match anything. OK so far.
3. .or_else is skipped if load_special returned Some. Otherwise, it returns Option<Result<...>>, with a None value if there was no override. Still OK.
4. .unwrap_or_else returns the Result<...> or else calls load_rbt and propagates the result from that.

Previously, the behavior was:

1. dep.strip_prefix returns Option<...>
2. .map resolves to Option<Result<...>>. Will be None only if strip_prefix failed.
3. …

So the difference is that we never previously called load_with_override or load_rbt if the ID started with the string x-. As far as I can tell, the x- gets added here:

icu4x/components/experimental/src/transliterate/compile/pass2.rs

Line 397 in 287e3ff

unparsed.replace_range(0..0, "x-");

Note that load_with_override already fails unconditionally on x- because it needs the ID to be valid BCP-47. However, load_rbt just takes the ID and stuffs it into the DataMarkerAttributes.

Basically, I think this means that a string referencing a transliterator ID that doesn't have a BCP-47 alias, such as "::Lower" (which appears to become "x-any-lower"), could now be returned by a data provider that responds to requests for "und/x-any-lower" (using "/" as the data marker delimiter). So, that is the nature of the behavior change.

Without this change, it is required to always add a BCP-47 alias mappings for any transliterator IDs that are not in the built-in set.

[sffc]

OK. Were you comfortable with the observations from the comments I left above?

[robertbastian]

Yes