Right to left support

[mikedawson]

Hi All,

I just found renpy and I love what I see! So efficient... I am interested in helping out to get right to left (RTL) support in. Our company is doing the technical implementation of One Laptop Per Child (http://www.olpc.af). I want to use Renpy as a teaching tool; first to make a interview simulation game to help people learn interview skills. Where we lack people... Renpy could really help us to make 'advisors'...

I noticed:
http://lemmasoft.renai.us/forums/viewto ... bic#p22182

I'm not that familiar with Python GUI - I did mostly Java stuff for the last years... however I did make a couple code contributions to the Python offlineimap...

We did the entire localization of the One Laptop Per Child interface and can answer any questions on the language, and if required I can put some development work in as well. I look forward to hearing back and hopefully bringing Renpy to a very appreciative audience.

Thanks!

-Mike

[PyTom]

Assuming you're actually going to use this, I'm interested in modifying Ren'Py support these languages. So I think my first question is:

- Which languages/writing systems will you need to support?

My first insight is that this wouldn't be terribly hard to support, but we'd need to do the following:

1) For each word, decide if it should be rendered in RTL or LTR order. If RTL, reorder the characters in the word, and perform ligaturization as necessary. This is is the hard part, and the part that will require somewhat with experience in these languages to get right.

2) Break the words up into lines. Since the next step shouldn't change the length of any word, this step can be done with our existing (LTR-only) algorithm.

3) For each line, reorder it so that the words appear in the correct order.

If we can write a function for (1), then (2) and (3) should only be an hour or two's work. But I personally have no idea how to do (1) properly, so you'd have to write that function for me, and contribute it to Ren'Py.

You'd also need to figure out how to edit text inside Ren'Py scripts. But since this done for languages like C and Java, I don't think this will be the hard part.

[delta]

The two most common RTL scripts by far are Arabic and Hebrew. Determining when to switch between the two is easiest with a full implementation of Unicode's U+200E and U+200F marks, combined with a style property which states which direction is the default. This puts the responsibility for getting it right in the hands of the writer, but that is how it should be.

[PyTom]

Yes, but this is the Afghanistan OLPC project. According to Wikipedia, the two official languages are Dari and Pashtun. I don't know offhand if they use Arabic for writing, or if they have their own orthographies.

I assume Hebrew is right out.

[delta]

Not wanting to tread on anyone's toes, but if you're going to make a LTR text implementation, it's not a good idea to make it specifically for a very exotic case if much more widespread cases exist.

I assume Hebrew is right out.

Why? At least Hebrew is simple compared to Arabic, which has extremely complicated ligatures.

P.S. Both Dari and Pashtun use variants (supersets) of Arabic.
P.P.S. If you're already doing things with directionality, the CJK crowd may appreciate vertical text (and ruby text, for that matter).

[PyTom]

The joke (that you missed) is that I'm certain that the number of native Hebrew speakers in Afghanistan is small.

Vertical text would require its own text layout algorithms, as the underlying libraries (FreeType, in our case) assume horizontal text layout. We'd have to go character-by-character. It's one of those things that's something I'd like to add in the future, but it hasn't been a priority because it wouldn't be used by the bulk of our audience.

Ruby text is similar. A nice idea, and something we might eventually get, but not currently a priority.

My sense of priorities is generally influenced by what people will do with the proposed feature. RTL will be needed by OLPC, while I'm not sure vertical text or ruby would be used by anyone. (Certainly, we don't see people faking vertical text with images.)

[delta]

Of course it's better to make a feature that people actually will use. I guess my point is that this is a matter that has possibly far wider uses, so it makes more sense to completely abstract the text rendering system from directionality instead of including support for the very special case.

[mikedawson]

Hi,

I am most definitely seriously interested in this. Normally when the keyboard direction switches you will get a character called left to right mark or right to left mark to indicate a change of direction - see http://en.wikipedia.org/wiki/Bi-directional_text

Dari and Pashto are both primarily using Arabic letters with some additions. They are fully unicode supported, so as long as an appropriate font is around all is fine. Arabic is understood in some places and some words are from Arabic origins...

The difficult part is the way that the characters connect. If you are using FreeType it should take care of this transparently AFAIK, perhaps the best thing to do is to think of the word as a letter. I'm not (yet) familiar with Renpy internals.

As delta mentions as long as we use the RTL mark and LTR mark then whatever we do should work for all RTL languages - Hebrew, Hindi, Arabic, etc. whatever is unicode supported.

That would be enough to make it usable.

The thing that would be useful would be to define possibly for each fragment of text, speech, etc. if the base direction is right to left (e.g. the paragraph should start on the right) or left to right.

Hope that is useful - I will download the latest version tonight and try and make it open in Netbeans.

[PyTom]

I think I'd need a few examples before I could actually get started.

Could you write into a text file a few lines of Ren'Py script, and then also take a screenshot of what it _should_ look like?

[mikedawson]

Sure can do that - not a problem.

Just for quick reference here as well - if we take the word salam (hello / peace) سلام it is made up of

س ل ا م

Each letter can have a different form depending on if it is on it's own, connected only to the left, and connected only to the right. If you copy those letters and delete the spaces between them the letters will join exactly as they should.

What can also be helpful to compare is if you install the Persian (Iran/Farsi or Afghanistan/Dari) keyboard layout - then just at random start hitting the keys into notepad etc.

The rules about which letter can join with which other one are quite complex. Thus I would definitely recommend that we find a way that we leave the joining to the underlying library which will already support this.

If this is enough to get going let me know, otherwise I make that basic script and then I post that up and mock up a screenshot of how it should be.

Regards,

-Mike

[PyTom]

Hm... Currently, Ren'Py's font handling is based directly on Freetype. So I'm not sure there is an underlying library... unless I'm totally mistaken here, Freetype assumes that a library above it is going to be responsible for the joining.

(We access Freetype directly so we can do text effects, like outline fonts, that would be hard to do in something like Pango.)

I'd like to find a library that can handle the joining for us, but I'm totally unfamiliar with this area.

[PyTom]

I've thrown fribidi into the mix, so we now have basic support for RTL, including the Arabic substitution. It would really help if I could get some test cases, so we can determine if it's rendering correctly.

[delta]

http://ar.wikipedia.org/
http://he.wikipedia.org/

[PyTom]

Unfortunately, my text editing setup is ill-equipped to deal with RTL languages. So simply having text examples limits what I'm able to do. But thank you for your input.

[delta]

well, uh, serious question then... if text in the format the game will be written in is not what you want, what other "test cases" are there to give over the internet?