Unicode characters from U+1Fxxx group

[iichan_lolbot]

Two squares are shown instead of some unicode characters at RenPy 6.18.2, font is Symbola 7.19 ( http://users.teilar.gr/~g1951d/Symbola.zip ).
Unicode lines like u"ЙЦУКЕН℗" are displayed correctly.

But characters like "" (U+1F306) and "🛠" (U+1F6E0) are not displayed, 2 squares are viewed instead of each character.
It seems like renpy does not support U+1Fxxx unicode characters.

[PyTom]

I'll check into this at some point.

[iichan_lolbot]

Oh, look what I have found: http://stackoverflow.com/questions/1446 ... 2-or-ucs-4

RenPy 6.18.2.729 on windows: sys.maxunicode = 65536

[iichan_lolbot]

So, actually, len(u"") is 2, which is probably why it fails to load proper glygh during loops like "for c in s: ...".

[PyTom]

I don't think I can fix this anytime soon.

To be able to fix this, I'd have to recompile python on windows. I'm not set up to do that at the moment - and I don't think I could easily get set up to do it, at least to a degree of reliablity that I'm comfortable with. So I'm going to have to apologize, but I don't see an easy way to support characters on the astral planes. (I'm guessing this will be supported when Ren'Py moves to python 3.)

[iichan_lolbot]

>I'm guessing this will be supported when Ren'Py moves to python 3.

Huh, I thought it never will move to Python 3.
I think it's time for me to change "string % tuple" to "string.format()" at my projects.

[PyTom]

Why? % still works in python 3.

[iichan_lolbot]

Hmm, I wonder which exactly parts of unicode string processing are exactly limited to 2 bytes?
Obviously, stuff like len(u"")==2 is a problem for code that breaks a line into list of glyphs.
So, I hacked inside text.py and tried to replace list of glyphs with something like that just to check if there are any problems deeper:

Code: Select all

        if  len(glyphs) == 2:
            gl = renpy.text.textsupport.Glyph()
            gl.character = 129299L
            gl.x = 10
            gl.y = 20
            gl.ascent = 0
            gl.line_spacing = 5
            gl.width = 18.0
            gl.advance = 18.0
            gl.time = 0.0
            gl.hyperlink = 0
            glyphs = [gl]
        fo.draw(di.surface, xo, yo, color, glyphs, self.underline, self.strikethrough, black_color)

When I run RenPy 6.99.8.959 on Windows, I see unicode U+F913 character instead of U+1F913 that was set in gl.character = 129299L.

Which means to me that something is limiting character number to first 16 bits deeper during rendering.
I guess it's this code at ftfont.pyx:

Code: Select all

            index = FT_Get_Char_Index(face, <Py_UNICODE> glyph.character)
            cache = self.get_glyph(index) 

Looks like FT_UInt FT_Get_Char_Index(FT_Face face, FT_ULong charcode) supports 32bit charcode, but Py_UNICODE seems to default to 16bit on Windows.

I don't know how to rebuild ftfont.pyd, can you hint me how to do this on Windows?
Or, maybe this can be fixed by simply replacing <Py_UNICODE> with <FT_ULong> and you can provide me a build of ftfont.pyd with this hack if you have time for this?..