[Bug]: Can't run executables with CJK characters in the path if the bottle language is set to them.

[Hairo]

Describe the bug

If the bottle language is set to chinese, japanese, korean (and possibly others) launching an executable with CJK characters in the path throws this error in the console:

wine: failed to open "/home/hairo/Games/test/\xe3\x81\xa7\xe3\x81\x99\xe3\x81\xa7\xe3\x81\x99.exe": c0000135": c0000135

And the executable doesn't run.

If the bottle language is set to english (i tried with spanish too) the executables run as expected.

To Reproduce

1. Set the bottle language to chinese, japanese or korean
2. Launch an executable with CJK characters in the path/name
3. Nothing happens (executable doesn't run)

Package

Flatpak from Flathub

Distribution

EndeavourOS

Troubleshooting Logs

Version: 2022.10.14.1
Display:
    X.org: true
    X.org (port): :0
    Wayland: false
Graphics:
    vendors:
        amd:
            vendor: amd
            envs:
                DRI_PRIME: '1'
            icd: /usr/lib/x86_64-linux-gnu/GL/vulkan/icd.d/radeon_icd.x86_64.json:/usr/lib/i386-linux-gnu/GL/vulkan/icd.d/radeon_icd.i686.json
    prime:
        integrated: null
        discrete: null
Kernel:
    Type: Linux
    Version: 6.0.2-zen1-1-zen
Distro:
    Name: GNOME
    Version: '43'
Disk:
    Total: 16787501056
    Free: 16787361792
RAM:
    MemTotal: "33.6\_GB"
    MemAvailable: "29.2\_GB"
Bottles_envs: null

Display Protocol

X11

Display Backend used by Bottles

X11, I don't know

Additional context

I thought it was a locale issue but i do have ja_JP.utf8 enabled.

[StoneMoe]

The \xe3\x81\xa7\xe3\x81\x99\xe3\x81\xa7\xe3\x81\x99 hex string in filename is "ですです"(Japanese) in UTF-8 encoding.

Launch program from Bottles will pass language code without encoding suffix via LC_ALL according to these code:

Bottles/bottles/backend/wine/winecommand.py

Lines 187 to 190 in d04d791

	# Language
	if config["Language"] != "sys":
	env.add("LC_ALL", config["Language"])

Bottles/bottles/backend/utils/manager.py

Lines 302 to 333 in d04d791

	locales = [
	'sys',
	'bg_BG',
	'cs_CZ',
	'da_DK',
	'de_DE',
	'el_GR',
	'en_US',
	'es_ES',
	'et_EE',
	'fi_FI',
	'fr_FR',
	'hr_HR',
	'hu_HU',
	'it_IT',
	'lt_LT',
	'lv_LV',
	'nl_NL',
	'no_NO',
	'pl_PL',
	'pt_PT',
	'ro_RO',
	'ru_RU',
	'sk_SK',
	'sl_SI',
	'sv_SE',
	'tr_TR',
	'zh_CN',
	'ja_JP',
	'zh_TW',
	'ko_KR'
	]

And system will choose the default one codeset for the specified language:

UTF-8 for es_ES(Spanish)
ISO-8859-1(aka latin-1) for en_US
GB18030 for zh_CN
EUC-JP for ja_JP

which usually is ASCII compatible(or not) for historic reason and be the first one in locale.gen AFAIK

> cat /etc/locale.gen | grep ja_JP
#ja_JP.EUC-JP EUC-JP  
#ja_JP.UTF-8 UTF-8  

this MAYBE the cause of this problem, and a new LC_CTYPE encoding setting next to Language setting MAYBE a good solution.

More: https://www.gnu.org/software/libunistring/manual/html_node/Locale-encodings.html

[latin-1]

For those looking for a workaround:

flatpak config --set extra-languages ja_JP.UTF-8
flatpak update

and set the environment variable in Bottles (rather than the language setting):

LANG=ja_JP.UTF-8

---

Note: Flatpak does not use the system's locale.gen, you must configure it manually. The .UTF-8 part appears to be required for the Bottles to function properly.

[Revival8697]

@latin-1 this doesn't seem to work on my end. For example, can you install this EXE?
https://tkool.jp/products/rtp/2000rtp.zip

[trx1138]

@latin-1 this doesn't seem to work on my end. For example, can you install this EXE? https://tkool.jp/products/rtp/2000rtp.zip

installation failure could be irrelevant with locale config.
try running explorer within your bottle and if it shows a cjk path in right encoding,
that means there's nothing wrong with your locale configuration.

if your cjk path is showing in broken encoding,
try running flatpak config --get extra-languages to check if it's configured properly.

[Revival8697]

So I just tested the latest release and I discovered that it was the runner mishandling Japanese characters.
The current default runner, soda-7.0-9 cannot handle the installation of this Japanese EXE and launching programs with Japanese characters in PATH/EXE.
While wine-ge-proton7-37 handles them fine. This is not a Flatpak nor a Bottles bug. The runner is at fault here.

CC: @Hairo @mirkobrombin

[Hairo]

I've never used soda runners so it was probably something to do with the wine version. Just tried and yes, wine-ge-proton7-37 works.

[kgraefe]

The \xe3\x81\xa7\xe3\x81\x99\xe3\x81\xa7\xe3\x81\x99 hex string in filename is "ですです"(Japanese) in UTF-8 encoding.

Launch program from Bottles will pass language code without encoding suffix via LC_ALL according to these code:

Bottles/bottles/backend/wine/winecommand.py

Lines 187 to 190 in d04d791

	# Language
	if config["Language"] != "sys":
	env.add("LC_ALL", config["Language"])

Bottles/bottles/backend/utils/manager.py

Lines 302 to 333 in d04d791

	locales = [
	'sys',
	'bg_BG',
	'cs_CZ',
	'da_DK',
	'de_DE',
	'el_GR',
	'en_US',
	'es_ES',
	'et_EE',
	'fi_FI',
	'fr_FR',
	'hr_HR',
	'hu_HU',
	'it_IT',
	'lt_LT',
	'lv_LV',
	'nl_NL',
	'no_NO',
	'pl_PL',
	'pt_PT',
	'ro_RO',
	'ru_RU',
	'sk_SK',
	'sl_SI',
	'sv_SE',
	'tr_TR',
	'zh_CN',
	'ja_JP',
	'zh_TW',
	'ko_KR'
	]

And system will choose the default one codeset for the specified language:

UTF-8 for es_ES(Spanish)
ISO-8859-1(aka latin-1) for en_US
GB18030 for zh_CN
EUC-JP for ja_JP

which usually is ASCII compatible(or not) for historic reason and be the first one in locale.gen AFAIK

> cat /etc/locale.gen | grep ja_JP
#ja_JP.EUC-JP EUC-JP  
#ja_JP.UTF-8 UTF-8  

this MAYBE the cause of this problem, and a new LC_CTYPE encoding setting next to Language setting MAYBE a good solution.

More: https://www.gnu.org/software/libunistring/manual/html_node/Locale-encodings.html

I just ran into this. For taking a screenshot for bottlesdevs/programs#339 I changed the bottle language to English instead of System (which is de_DE.UTF-8 on my system) and suddenly my program did not find its files anymore as their paths contained UTF-8 characters. So yes, there should be an option to set the encoding.

[github-actions]

This issue has been automatically closed as part of a cleanup process to help the team manage issues more effectively.

If the problem still persists, feel free to comment and we will gladly reopen it.