User Agent: Mozilla/5.0 (Linux; Android 14; Mobile; rv:127.0) Gecko/127.0 Firefox/127.0

Steps to reproduce:

Nightly and Beta versions of Android Firefox display simplified Chinese kanji fonts instead of Japanese kanji fonts on some sites (github, bugzilla, chatgpt, etc.).
This did not happen in previous versions or in other browsers.
I tried resetting "intl.accept_languages" in about:config and the problem went away until I quit those apps.

Zack, thanks for reporting this bug! Can you please share your Firefox's about:support information in this bug?

What is your Android system locale? What was the value of "intl.accept_languages" before you reset it?

(In reply to Chris Peterson [:cpeterson] from comment #1)

Zack, thanks for reporting this bug! Can you please share your Firefox's about:support information in this bug?

What is your Android system locale? What was the value of "intl.accept_languages" before you reset it?

My Android system locale is Japan. The value of "intl.accept_languages" before resetting is "ja-JP,en-US,zh-Hans-CN,zh-Hant-TW,zh-Hant-HK,zh-CN,zh-TW,zh-HK".

The bug has a release status flag that shows some version of Firefox is affected, thus it will be considered confirmed.

:boek next week is the last week of beta for Fx126. We have little time to investigate this.
Could this be triaged and prioritized for investigation?

My Android system locale is Japanese too. intl.accept_languages was ja-JP here, I tried adding zh-CN, but, none of the sites you mentioned (chatgpt, bugzilla, github) are shown to me in Japanese or Chinese. They're all in English...

:zack I see you have this addon for translating pages: TWP - Translate Web Pages. Would you mind disabling or uninstalling this addon and checking again? The settings for this addon has language selectors for translating webpages in it's settings and I'm wondering if this is actually causing your problems. Additionally, there is a mobile version of this addon called TWP - Translate for Mobile that may work better but I haven't tested.

Sorry, my standard system locale is Japanese, but other languages were also set. All my Android languages are Japanese, English, Simplified Chinese (Mainland China), Traditional Chinese (Taiwan) and Traditional Chinese (Hong Kong).
Kanji has slightly different character forms for the same character code kanji depending on the country and region (you may not know this difference unless you use Kanji on a regular basis).
This bug is that Kanji characters in other regions are displayed even though the language is set to Japanese.
This is a trivial problem, but as I always use Kanji, I feel quite uncomfortable.

I have uninstalled TWP and initialized Firefox, but this bug still occurs.

Can you give a link to a specific page that shows Japanese written with Chinese characters? A screenshot could be useful too.

FWIW, after adding simplified Chinese via the Android settings, I now have intl.accept_languages set to ja-JP,zh-Hans-CN,zh-CN, and e.g. Google or Wikipedia are still displayed in proper Japanese.

So does chatGPT if I talk to it in Japanese.

この画像はintl.accept_languagesをリセットする前のchatgptです。

And this is after the reset.

Ahhh, Sorry, I wrote the sentence before I translated it
"この画像はintl.accept_languagesをリセットする前のchatgptです。" -> "This image shows chatgpt before resetting intl.accept_languages."

I don't get this bug with google or wikipedia on my device.

Okay, I'm definitely able to reproduce, and it's even reproducible with Android in English

My STR:

• Go to about:config and ensure intl.accept_languages is reset.
• Open, Android settings app, select "System", then "Languages & input", then "Languages"
• Set things up so that you have the following languages:
  • English (United States)
  • 日本語 (日本)
  • 简体中文(中国)
• In Firefox, go to https://chat.openai.com
• As a prompt, type 変化、設置、街角、絵画、抱く、花火

I think the most visible difference to people not accustomed to Chinese/Japanese is in 抱く

If you go back to the Android settings and remove 简体中文(中国), returning to Firefox resets the display (without reloading the page or anything) and switches to the right font.

I can also confirm this doesn't affect Chrome.

I can also reproduce with Firefox 125.2.0 (Build #2016016039)

In that case, this bug isn't a regression in Firefox 126, though perhaps it's a regression in 125 or earlier.

I wonder if this is related to the (fairly recent) implementation of Android font visibility restrictions (anti-fingerprinting) in bug 1826412. Does turning off tracking protection affect the behavior?

(In reply to Jonathan Kew [:jfkthame] from comment #21)

Does turning off tracking protection affect the behavior?

Assuming I turned off the right thing, no.

The cause may be that Firefox has set only ja-JP as the locale code for Japanese in accept-language.

Firefox for Windows sets ja for Japanese locale code.

Other mobile browsers set both ja and ja-JP.

Adding ja to the beginning of the intl.accept_languages value produces the same result as resetting it.

(In reply to zack from comment #23)

The cause may be that Firefox has set only ja-JP as the locale code for Japanese in accept-language.

Oh, interesting -- yes, I suspect that could be the issue; the font preferences may not recognize that. If that's the issue, it should be straightforward to fix, I think. I'm out of the office today but will try to check on this when I'm back, if no-one else does it in the meantime.

14:50.30 INFO: No more integration revisions, bisection finished.
14:50.30 INFO: Last good revision: c55a609359a20d23e6534c7c188007d07bb1877b
14:50.30 INFO: First bad revision: 0f63ab5624960b1aa464bb2e3bd76c6cd196f186
14:50.30 INFO: Pushlog:
https://hg.mozilla.org/integration/autoland/pushloghtml?fromchange=c55a609359a20d23e6534c7c188007d07bb1877b&tochange=0f63ab5624960b1aa464bb2e3bd76c6cd196f186

Makoto, this bug is a regression from your Accept-Language change in bug 1847701 in Fx 125. Is this a website/webcompat bug? Or should we change how we format the locale tags in Accept-Language again?

I suspect (but have not yet confirmed) that this is happening because the GetFontPrefLangFor function called here only recognizes ja, not ja-JP, and we should fix that to make it smarter.

And ja is not the only locale with the problem:
https://searchfox.org/mozilla-central/source/gfx/thebes/gfxFontPrefLangList.h#1

It is worth noting that the claim in bug 1847701's commit message that Accept-language is "language-region format like Desktop" is actually not true for Japanese:
https://hg.mozilla.org/l10n-central/ja/file/eaeda87d1d4ecbe97a51beccde04b09e055a5157/toolkit/chrome/global/intl.properties#l23

Many locales put both "language-region" and "language", but many skip "language-region".

Hmm, before landing bug 1847701, accept-language was ja-JP if system locale is Japanese. Chinese was zh-Hans-TW (Taiwan) or zh-Hans-CN (China). But after this change, Chinese becomes zh-TW or zh-CN. It means that font selection in gfx matches with Chinese.

(Although I filed bug 1889845, we should normalize this value by Gecko side at future)

(In reply to Makoto Kato [:m_kato] from comment #31)

Hmm, before landing bug 1847701, accept-language was ja-JP if system locale is Japanese. Chinese was zh-Hans-TW (Taiwan) or zh-Hans-CN (China). But after this change, Chinese becomes zh-TW or zh-CN. It means that font selection in gfx matches with Chinese.

In the geckoview example app, it was indeed ja-JP,zh-Hans-CN with my current Android system settings in version 124. And indeed, if I change that to ja-JP,zh-CN manually, I get the same font selection problem. Seems like comment 27 might be spot on, and we never even matched Japanese before.

That said, we are also currently not using language-region on desktop.

This bruteforce approach fixed the issue for me: https://hg.mozilla.org/try/rev/f960adc00da5cea1c5dc9bca5a3c7a3e0903033b, so comment 27 is confirmed. Interestingly, logging the GetFontPrefLangFor function, I can see that it regularly returns Others on my desktop, where it's queried for e.g. ja-jp (lowercase, I haven't looked where it comes from, but it's not from accept_languages).

Original issue is that font selection doesn't match with Japanese when system
locale is Japanese. Before landing bug 1847701, all far east languages don't
always match. But after langing it, Chinese (Simplified and Traditional) is
macthed only.

Gfx doesn't want that Accept-Langues preference has a country code if
languguage is Japanese or Korean.

So we should return language code only for accept-language if language tag is
ja-JP or ko-* (North or South).

In principle, this also affects the other font prefs where we use a bare language subtag without region, if the accept-language list (or other source of lang tags) has the language-region form. These include Arabic, Greek, Hebrew, and Thai, as well as Japanese and Korean. In practice those cases are less likely to be noticed, as fallback behavior will likely still pick an appropriate font; there aren't the same region-dependent variants as with CJK. But we should really fix them all.

The patch in D208588 is a somewhat ugly hack, but should resolve this AFAICS. We could go the whole way and use Locale parsing and then extract the lang subtag, but that seems overkill given the very limited set of font pref lang codes we're dealing with; and it's my hope that we'll eventually be replacing all this with a more principled lang- and script-driven architecture anyway.

I withdraw my fix

Moving this to Core :: Graphics: Text, as it's really an issue in how gfx/thebes handles lang codes in relation to font prefs, and in principle it could affect other platforms as well, if people modify their accept-language to have <lang-region> codes for cases where gfx only expects the lang.

Set release status flags based on info from the regressing bug 1847701

Actually, this is really about font selection, rather than rendering; -> Layout.

Original Revision: https://phabricator.services.mozilla.com/D208588

Original Revision: https://phabricator.services.mozilla.com/D208588

beta Uplift Approval Request

• User impact if declined: Japanese users may see incorrect (Chinese) font; potentially other similar impacts
• Code covered by automated testing: yes
• Fix verified in Nightly: no
• Needs manual QE test: no
• Steps to reproduce for manual QE testing: n/a
• Risk associated with taking this patch: low
• Explanation of risk level: simple check for additional forms of language code
• String changes made/needed: none
• Is Android affected?: yes

release Uplift Approval Request

• User impact if declined: Japanese users may see incorrect (Chinese) font; potentially other similar impacts
• Code covered by automated testing: yes
• Fix verified in Nightly: no
• Needs manual QE test: no
• Steps to reproduce for manual QE testing: n/a
• Risk associated with taking this patch: low
• Explanation of risk level: simple check for more lang code forms
• String changes made/needed: none
• Is Android affected?: yes

https://hg.mozilla.org/mozilla-central/rev/943302be4233

Just for information:
You can see the differences between the glyphs more easily with the following letters than with the test HTML.
刃 直 骨 海 角 遙 入

Added to the 125.0.3 relnotes.

Fixed an issue which could cause Japanese users to see incorrect fonts in some situations