Select language automatically, based on system settings

[Lobo Demente]

Hi everyone!

These days I've been working on a translation of a Ren'Py game, and wondered if it would be possible to make the game set automatically the proper language for the user, as I knew this is a system setting.

Almost all computer systems have a language setting that uses a convention you've probably seen somewhere, like "en_US" or "fr_FR". That string contains the language preferred by the system and in wich country variation, if any: "en_US" means "english for the USA", "en_GB" means "english for Great Britain" and so on. These codes are specified by two ISO standards (look at the end of this post for more information).

Let's imagine I'm a german player who hardly knows english, and who has downloaded a game wich is available in english and german -amongst other languages maybe-. I can't set the language to german if the game speaks english by default. Well, this code should solve it:

Code: Select all

init -3 python:
    import locale

    # Keep in mind that the language is only set the first time the game is played!
    if persistent.lang is None:
        # First, obtain the full locale. getdefaultlocale() returns some other fields we don't need:
        miau = locale.getdefaultlocale()[0]

        # Now get the value for each field
        iso_lang = miau[0:2]
        iso_ctry = miau[3:5]
        
        # You must modify these lines in order to make available only the languages implemented:
        if iso_lang == "es":
            persistent.lang = "spanish"
        elif iso_lang == "de":
            persistent.lang = "german"
        else:
            persistent.lang = "english"

    lang = persistent.lang

This snippet is fully compatible with the "language chooser" I've seen somewhere. The only difference is that this one will try to set the language of the game by using the one the computer is configured for and setting it to english if it isn't available in the game. The player can change it later in the language chooser if desired.

For country variations:

Code: Select all

        if iso_lang == "es":
            if iso_ctry == "AR":
                persistent.lang = "argentinian"
            elif iso_ctry == "MX":
                persistent.lang = "mexican"
            else: 
                persistent.lang = "spanish"
        elif iso_lang == "de":
            persistent.lang = "german"
        else:
            persistent.lang = "english"

This one assumes there are three different variations of spanish: one for mexican people, another one for argentinian people, and a last one for every other spanish-speaking country. If the game is properly translated, the user will find it as if it were written directly in its own language and wouldn't need to configure anything.

Maybe a faster (and a more technical) approach would be to use the locale in the game instead of translating it (i.e. using "es" instead of "spanish"), but there's still some work to be done if we have any country variation.

This code has been tested under Windows only, but it should also work under Linux and Mac. Any brave tester around?

Standards ISO-639-1 or ISO-639-2 details two letter codes for every language (values for iso_lang in the examples above), and ISO-3166-1 details two-letter codes for every country (values for iso_ctry); you can use your favourite search engine to get a full list of both of them.

[TrickWithAKnife]

This is really interesting. Is it possible to get the player's current country? That could allow for even more personalisation.

[AxemRed]

This takes the system locale, not the operating system language. The two aren't necessarily the same and many VN fans have their locale set to ja_JP to be compatible with certain VNs.

[Lobo Demente]

It's true this code gets the system locale, but that's the most universal way of guessing what country the system is configured to be run. It's easy to test it under Windows by changing the format in the "regional configuration and language" dialog in the Control Panel and setting it to something different or exotic. That would make the Ren'Py program think the system is, for example, phisically located in Greece or Bali (thus trying to set the game language to the one used in this countries, if available), but that wouldn't change the language of the system itself. Most programs and games use the system locale in this exact same way because that's the idea behind its existence.

I still haven't found any case in wich the system locale doesn't match the system language by default, and it doesn't make sense to me unless the user is learning languages by "language immersion". Even if that's the case, the user should know well what he/she is doing, because even if the user is a polyglot, his/her system should be configured for the language he or she is more comfortable with, this is, its primary language, and that must be reflected in its default locale.

Anyways, as I stated in the post, this would only set the language in wich the interface is shown the first time is run ("try to match the system interface"), so the user can later change the language of the game without interfering with the system configuration or other games, nor having to manually change locales

For example, imagine I'm a german bilingual player (japanese-speaker too) and I get a game available in three flavours (english, japanese and german): the first time it's run in my computer, it will show itself as german by default, as probably my computer will have its locale set to "de_DE" or something simillar (remember: I can't read english, so this can be a handicap for me). If I want the game in japanese, I'll just go to Options -> Language and set it to japanese. Et voilà If you're from UK or USA, the game will appear directly in english, allowing you to change it later to german or japanese if you feel like it. A spanish player will probably see the game in english too.

The regular snippet I've seen for the language chooser will set the game to "english" by default and let the user change it later, but how to do it if english is not an option for me? That was the question behind the first post

[Lobo Demente]

This is really interesting. Is it possible to get the player's current country? That could allow for even more personalisation.

In fact it's possible to get to know much more information, like preferred date/time formats, thousands and/or decimal separators, currency symbol or even the character encoding that the system uses. I know for sure that this is true for every Windows and Linux/Unix system, and I assume MacOS systems too, but maybe some smaller devices can't give so much information.

If you want to be evil, Python is even able to change these settings too

[AxemRed]

Many VN fans have their locale set to Japanese in order to read translated Japanese VNs without fiddling with hacks like AppLocale/NTLEA. Most of them can't actually read Japanese, but the default codepage for C programs depends on the locale.

And then there's the preferred user interface language which is entirely separate from the locale in Windows.

[TrickWithAKnife]

It does happen. I bought my laptop in Japan, and the OS is installed in Japanese, but I'm not that great at reading Japanese. Drives me nuts when software installs and automatically uses Japanese as the default language. It can take a while to change the settings to English.

But there could be some situations where it might be nice, for example automatically using British or American writing conventions, or having the default system language as the first option when choosing the language from a list. It may not mean much to people in England or the US, but for me, seeing New Zealand at the top of the list was convenient and a kind of nice touch. People in small countries often get ignored.

[Lobo Demente]

Many VN fans have their locale set to Japanese in order to read translated Japanese VNs without fiddling with hacks like AppLocale/NTLEA. Most of them can't actually read Japanese, but the default codepage for C programs depends on the locale.

And then there's the preferred user interface language which is entirely separate from the locale in Windows.

I've never played a translated japanese game, so I've never fiddled with those hacks you tell me and I'm curious about it... you mean the game doesn't run if the locale is set to anything other than japanese?

Anyways, I insist on telling that the locale is the way of telling programs what language do you use in your computer. In fact, it seems that all those VN fans you speak of are telling the "un-japanesed" games that the computer is japanese in order to overpass some internal check. I'm spanish (this time it's not an example ), and I can change my locales to anything else, but my Windows/Linux interface will still remain spanish, that's true. But if I install a multilanguage program (try 7-zip, for example), it will set its interface automatically to match the locale I've set, so If my locale is russian in my spanish Windows when I install 7-zip, I'll get a russian 7-zip interface on my spanish Windows interface (just the same behaviour my snippet shows).

Locales aren't mandatory, but they are there in order the programs have a guide to know what does the user wants/likes as there's no other -standard- way of knowing these values, so playing with them without knowing what one's doing is a bit risky. In fact, a locale setting can even detail the code page used, and this affects directly to the keyboard mapping: that means that the key with a "\" drawn on it couldn't be able to write a "\" when pressed, but anything else (believe me, we spanish users have a lot -and I mean a lot- of different keyboard mappings and it's a mess when a system uses the wrong one).

Of course there are other methods of knowing the system configuration: Windows uses a publicly available API (written in C but available to almost any language) in order to expose its metrics, but that exact same library definition doesn't exists on Linux or Mac, so using it in Python (or Ren'Py) will restrict the game to a single platform, rendering it unusable under Linux and MacOS.

Think of changing the currency symbol. I could do it freely with some esoteric purpose in mind, but it doesn't make much sense as quantities wouldn't match with reality: I could see "$200" on my screen, being in fact "200€" with the changed syntax, but 200€ is, by now, more expensive than $200.

You can take a look at http://www.langtag.net/ (the first three "for users" links explains it very well) or search wikipedia.

[AxemRed]

The main problem is fopen uses the Locale's default codepage under Windows, making it impossible to open files with Japanese characters in their filenames unless your Locale is set to Japanese. Therefore you cannot trust the Locale's language setting as it may be something silly to be compatible with certain programs.

[Lobo Demente]

The main problem is fopen uses the Locale's default codepage under Windows, making it impossible to open files with Japanese characters in their filenames unless your Locale is set to Japanese. Therefore you cannot trust the Locale's language setting as it may be something silly to be compatible with certain programs.

Still can't get it. Windows' NTFS partitions (Windows 2000 or later, and before that, Windows NT) use UTF-16 to store filenames, nothing to do with locale or interface language, as it can store any charset or codepage at the same time. As far as I know, this is also true for ext3 and ReiserFS under Linux. In Windows I'm able to set up an IME for japanese or chinese input on my spanish-interfaced Windows and write japanese characters, even mixed in the same filename with roman characters... the problem then lies in the implementation of that fopen() function. I've opened flawlessly some mp3 czech, russian and japanese encoded filenames without needing to change anything.

We both are saying -more or less- the same 'cos by changing locales, you're telling that fopen-containing-library that your system is japanese though, in reality, it's not. That's it: locales tell all of your programs wich country they must be set for.

So: if you change it for anything else than your country/language, you must expect some strange behaviour, and you must know it's caused because of that change. With this in mind, if a Ren'Py program wich uses the auto-language snippet suddenly shows itself in japanese in your english interface, it's because you changed your locales and not because of the locales' reliability.

So: locales are reliable unless you fiddle with it. Imagine you change your car's adhesive (=locale) where it states what kind of gasoline (=language) it uses, because the new one looks cooler (=the language/locale a specific program needs)... don't expect the man at the gas station (=another program) to use the correct fuel unless you tell him, everytime, wich one (=the real locale/language) must be used. You can't say the cars' adhesives are innacurate because some people change'em

[AxemRed]

I'm not saying having an 'incorrect' locale is ideal, I'm just saying it's common. If you want to detect the user's preferred language under Windows, use the proper API.

[Lobo Demente]

I'm not saying having an 'incorrect' locale is ideal, I'm just saying it's common. If you want to detect the user's preferred language under Windows, use the proper API.

Ren'Py uses Python as base system, wich is multi-platform. If I use that API on Ren'Py, the game will not work under Linux or Mac.

[Lobo Demente]

In fact: http://msdn.microsoft.com/en-us/library/dd318693.aspx "The language identifier is used as part of the locale identifier".

[AxemRed]

They return a language code in the same format, but not necessarily the same language. On my system, they return different values.

[Lobo Demente]

I'll try that. But still there's no kernel32.dll or kernel32.lib files in Mac/Linux. Are you trying to tell me I should make a different version of my Ren'Py game for every known platform in order to not bother users who fake their locales?

I find it the same as pretending that everyone learn spanish in the country I'm visiting instead of me learning their language...