those obscure UnicodeDecodeErrors

[jack_norton]

I was wondering if there was a way to avoid those errors:

UnicodeDecodeError: 'utf8' codec can't decode byte 0x96 in position 1142021: invalid start byte

or at least make them more humanly readable? I mean, probably it's my fault if I used some weird uncommon character but still, I don't understand why the message couldn't be clearer?
Right now when that happens the only way I have to fix it is to manually delete parts of the text, in a sort of word-hunting to see which is the offending character. Often it's a simple "-" encoded by Word/other crap external text editor. Possible that there isn't an easier solution ?

[junna]

I encountered this while coding for the Adversity competition. Unfortunately, the bloody thing is never clear on where the problem is.
I found it's usually triggered by ', " and - symbols only.

So...I either converted my whole in Word to Unicode ANSI font before transferring to Renpy. Or used the Replace in Editra, copy the funky looking ', " and - (I ended up having a template of the problem parts) to find and type in the same symbol to be replaced.

EDIT: Adding another symbol, the '...' thing that Word sometimes autocorrect.

[jack_norton]

Yes but in my case is even more complex! Check the image:

In the new Editra (on bottom), the text appears clean, while in the good old SCiTE (the best editor IMHO, shown at the top) it shows the culprit: did you see that weird looking character ? I asked the writer and maybe the problem is that she used Openoffice and then copy/pasted the texts on the Ren'Py editor. Apparently the characters are normal, but then if you try to run the game...

[KittyKatStar]

OTL

There was no error when another Mac-user tested the script, so I guess it's the Mac to PC thing.

(I'm so sorry. ;_; Just give me the whip, I know what to do.)

We agreed to use .txt docs from now on and we'll see if it makes a difference.

[jack_norton]

Haha no sorry, no whip needed. In this case isn't your fault, but really some crazy behavior, I couldn't notice any problem myself until I had the idea to use the old SCiTE !
Hope this info will be useful if someone else has those issues.

[junna]

0_0 the heck is that symbol?

@_@ I'm sorry, I remembered I converted files a few times and the symbols still came up like this “ .
but in the end, I wrote half of the script in Notepad. T_T that was tough. but I didn't have to Replace anything, phew.

Then I prepared a template/table. Like so:

“ = "
” = "
‘ = '
’ = '
– = -
… = ...

so I just go through Replace one by one each time I cut and paste from Word. It's a bit time consuming using Replace but it solved my Word to Editra problem.

[nyaatrap]

I have encountered that UnicodeDecodeErrors million times... ┐(´д｀)┌
I just need to type everything on one editor, avoiding copy and paste from any others as possible. Writer type application only works well on just one language - in my case, Japanese.

[AxemRed]

<< SOLUTION: SAVE YOUR SCRIPTS AS UTF-8 >>

[jack_norton]

Hmm nope, I tried that, the error was still happening. I tried everything:
- Editra "reload with encoding" to reload the same script as UTF-8,
- opened the .rpy in notepad and saved it as UTF-8

And nothing worked. The only solution was to manually delete those weird characters

[junna]

Hmm nope, I tried that, the error was still happening. I tried everything:
- Editra "reload with encoding" to reload the same script as UTF-8,
- opened the .rpy in notepad and saved it as UTF-8

And nothing worked. The only solution was to manually delete those weird characters

I think he meant... save the Word script into UTF-8. I'm gonna go try that for a sec.

EDIT: didn't work...

[AxemRed]

- Editra "reload with encoding" to reload the same script as UTF-8

Doesn't make sense as the script is not UTF-8.

What does work: Open in notepad++, encoding -> convert to UTF-8 -> save

Edit: Resaving as UTF-8 using notepad.exe also works.

[jack_norton]

Hmm ok next time will try that (now I already fixed the errors manually). Thanks!