failed to print error code `:s` fail on windows due to encoding error

[trim21]

#include <exception>
#include <filesystem>
#include <iostream>

#include "fmt/ostream.h"
#include "fmt/std.h"

namespace fs = std::filesystem;

int main() {
    const fs::path from{"/none1/a"}, to{"/none2/b"};

    try {
        fs::copy_file(from, to);
    } catch (fs::filesystem_error const &ex) {
        std::cout << "what():  " << ex.what() << '\n'
                  << "path1(): " << ex.path1() << '\n'
                  << "path2(): " << ex.path2() << '\n'
                  << "code().value():    " << ex.code().value() << '\n'
                  << "code().message():  " << ex.code().message() << '\n'
                  << "code().category(): " << ex.code().category().name() << '\n';

        try {
            fmt::println("{:s}", ex.code());
        } catch (std::exception &e) {
            fmt::println("fmt error: {}", e.what());
        }
    }

    fmt::println("exit");
    return 0;
}

[vitaut]

What error do you get and what encodings do you use?

[trim21]

it's error code 3 ERROR_PATH_NOT_FOUND https://learn.microsoft.com/en-us/windows/win32/debug/system-error-codes--0-499-

the ex.code().message() is "系统找不到指定的路径。" in gbk, cfb5cdb3d5d2b2bbb5bdd6b8b6a8b5c4c2b7beb6a1a3 in hex

[trim21]

I think we may need MultiByteToWideChar and WideCharToMultiByte from windows.h to convert locale encoding to utf8 when handling error code on windows ...

[sunmy2019]

Can you elaborate more on the output and the error?

Also an error here:

    try {
      fmt::println("{:s}", ex.code());
      fmt::println("{}", ex.code().message());
-   } catch (std::exception e) {
+   } catch (std::exception& e) { 
      fmt::println("fmt error: {}", e.what());
    }

[trim21]

OK, sorry, I didn't provide a clear description about this bug.

the root case of this bug is, the error message in ex.code().message() is not always utf-8 on windows, so when fmt try to print a error code, it cause a utf-8 encoding error. at fmt::println("fmt error: {}", e.what()); line it output "fmt error: invalid utf8"

base on my windows locale config, the error message may be gbk and not utf-8, which is "系统找不到指定的路径。" in gbk, cfb5cdb3d5d2b2bbb5bdd6b8b6a8b5c4c2b7beb6a1a3 in hex.

the fmt will throw a error "invalid utf8". And if I use fmt::format, it will give a string with mixed utf-8 and non utf-8 content.

[sunmy2019]

The question could be rephrased as:

Formatter of std::error_code does not handle non-UTF-8 encoding with s spec.

Maybe we can add a wrapper to convert ec.message() to UTF-8.

[trim21]

yeah, this should work?

#include <windows.h>

std::string ConvertToUTF8(const std::string &input_string) {
    if (input_string.empty()) {
        return std::string();
    }

    int wide_char_len = MultiByteToWideChar(CP_ACP, 0, input_string.c_str(), -1, nullptr, 0);
    if (wide_char_len == 0) {
        return input_string;
    }

    std::vector<wchar_t> wide_char_buffer(wide_char_len);
    if (MultiByteToWideChar(CP_ACP, 0, input_string.c_str(), -1, wide_char_buffer.data(),
                            wide_char_len) == 0) {
        return input_string;
    }

    int utf8_len =
        WideCharToMultiByte(CP_UTF8, 0, wide_char_buffer.data(), -1, nullptr, 0, nullptr, nullptr);
    if (utf8_len == 0) {
        return input_string;
    }

    std::vector<char> utf8_buffer(utf8_len);
    if (WideCharToMultiByte(CP_UTF8, 0, wide_char_buffer.data(), -1, utf8_buffer.data(), utf8_len,
                            nullptr, nullptr) == 0) {
        return input_string;
    }

    return std::string(utf8_buffer.data());
}

[vitaut]

It's a known bug in the standard (mostly affecting Windows): https://cplusplus.github.io/LWG/issue4156.