<regex>: Simplify regex_traits<_Elem>::translate(_Elem)

[muellerj2]

Simplifies regex_traits<_Elem>::translate(_Elem) to just return its only argument. In #5204 (comment), I voiced my suspicion that the current implementation essentially does just that in a very complicated and expensive way. I verified this now by running the following program that tested 902 locales available on my machine (no output to stdout produced by the program):

#include <iostream>
#include <locale>
#include <regex>
#include <string>
#include <vector>

#include <windows.h>

using namespace std;

BOOL add_locale(LPWSTR str, DWORD flags, LPARAM param) {
    vector<string>& vec = *reinterpret_cast<vector<string>*>(param);
    int size            = WideCharToMultiByte(CP_ACP, 0, str, -1, nullptr, 0, nullptr, nullptr);
    if (size) {
        vector<char> narrow(static_cast<size_t>(size), '\0');
        if (WideCharToMultiByte(CP_ACP, 0, str, -1, narrow.data(), size, nullptr, nullptr)) {
            vec.push_back(narrow.data());
        }
    }
    return TRUE;
}

int main() {
    vector<string> locales;
    locales.push_back("C");
    EnumSystemLocalesEx(&add_locale, 0, reinterpret_cast<LPARAM>(&locales), nullptr);

    for (const string& loc_name : locales) {
        const locale loc(loc_name);
        regex_traits<char> traits;
        regex_traits<wchar_t> wtraits;
        traits.imbue(loc);
        wtraits.imbue(loc);

        for (unsigned int i = 0; i <= 0xff; ++i) {
            try {
                const char x = (char) (unsigned char) i;
                if (traits.translate(x) != x) {
                    cout << loc_name << i << '\n';
                }
            } catch (const length_error&) {
            }
        }

        for (unsigned int i = 0; i <= 0xffff; ++i) {
            try {
                const wchar_t x = (wchar_t) i;
                if (wtraits.translate(x) != x) {
                    cout << loc_name << i << '\n';
                }
            } catch (const length_error&) {
            }
        }
    }
    return 0;
}

Even so, this PR still introduces a minor behavior change: The previous implementation can throw length_error("string too long") when it is passed a char that isn't a valid character in the locale's encoding (e.g., 0x80 in locales using UTF-8 encoding). But I think that the old behavior is undesirable anyway, as it makes the regex engine always fail with an exception in regex_constants::collate mode when a locale using UTF-8 encoding is imbued and the regex engine is applied to strings containing non-ASCII characters.

[frederick-vs-ja]

Aha, it seems that we took too much time to look into this.
[re.traits]/4 simply specifies that an implementation-provided regex_traits<C>::translate just returns the argument. See also https://en.cppreference.com/w/cpp/regex/regex_traits/translate.

[muellerj2]

I put so much work in this mainly because I was worried about mix-and-match scenarios, not because I consider the old implementation correct. Think of the scenario that the implementations actually commonly produce different results and regex parser and matcher pick up different implementations or some other strange combination. Then this change had the potential to actually degrade the regex engine in such a mix-and-match scenario and lead to regex bugs that are difficult to understand and reproduce.

[StephanTLavavej]

Thanks for the careful analysis! 😻

[StephanTLavavej]

This has gotta be a perf win.

[StephanTLavavej]

I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed.

[StephanTLavavej]

Who knew machine translation was so easy! 🤖 😹 🤪