[SOLVED] Exec() not running in renpy

[Odysseus]

Hello, I would like renpy to execute some very specific, dynamically generated in-game python scripts outside the game's scope (I would rather it was done in-game, but that will need a virtual machine...).

The first part is to open up a file to write the script from input in the game. That I've managed to do.

The second part is to actually compile and execute. When using the following code, test.py is called up perfectly when run from the command line or directly from the window with a double click:

Code: Select all

filename = "test.py"
with open(filename, "rb") as source_file:
    code = compile(source_file.read(), filename, "exec")
exec(code)

When I adapt it to renpy as follows, though, nothing happens:

Code: Select all

            filename = os.path.join(renpy.config.gamedir, "texts\\test.py")
            with open(filename, "rb") as source_file:
                code = compile(source_file.read(), filename, "exec")
            exec(code)

I also tried this:

Code: Select all

            filename = "test.py"
            source_file = renpy.file(filename)
            code = compile(source_file.read(), filename, "exec")
            exec(code)

and this variant:

Code: Select all

            filepath = os.path.join(renpy.config.gamedir, "texts\\test.py")
            global_namespace = {
                "__file__": filepath,
                "__name__": "__main__",
            }
            with open(filepath, 'rb') as file:
                exec(compile(file.read(), filepath, 'exec'), global_namespace)

I mean, the game runs without a hitch. It just does nothing.

I'm running Windows 10 with Python installed. What's the catch? Is is a permission thing? A scope thing? I'm trying all options, but I might've missed something...

I really could use some help, since it's for a thesis I'm making on educational games and computer science. Thanks, guys.

[drKlauz]

Without knowing what you need to do it is hard to suggest best option.
If you just want to execute python code generated at run-time:

Code: Select all

$code_to_exec="print(123)"
...
$renpy.python.py_exec(code_to_exec) ## this will execute as normal code under python: block
#$renpy.python.py_exec(code_to_exec,hide=True) ## this will execute as under python hide: block

If you want to have actual player-editable files, like some game rules or saved characters then it would be bit harder, as in general you probably should check save locations, find if there is needed file there, then load it, then after you loaded all same files from all locations find most recently updated and exec this latest file. And saving should save same file to every renpy save location. Something like i guess.
If you need later and can't do it yourself pm me, will think something out.

[Odysseus]

The idea is the following:

1) Player writes simple program inside game (with prompts from the game for aid)
2) Game saves this program as a .py file (Already got that, fortunately)
3) Game executes this code
4) Game gets the result of the execution to continue

Simple example: I have a .txt that's all lower-case, and I want the player to write a python program that capitalizes after each full stop (like a CodeWars challenge). So the student/player writes the program, then it's saved, then executed, then the result comes in- all without the player needing to do anything other than type it and press "run".

The idea is to make an interactive game with programming challenges.

But I think what you proposed will work, I'll go try it out now! Thanks!

[Odysseus]

Update: After a bit of trial an error, I got this code:

Code: Select all

            filename = os.path.join(renpy.config.gamedir, "texts\\test.py")
            with open(filename, "rb") as source_file:
                code = renpy.python.py_compile(source_file.read(), filename, 'exec')
            renpy.python.py_exec(code)

But this time, it crashes thus:

Code: Select all

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "game/script.rpy", line 53, in script
    python:
  File "game/script.rpy", line 56, in <module>
    code = renpy.python.py_compile(source_file.read(), filename, 'exec')
ValueError: compile() arg 3 must be 'exec', 'eval' or 'single'

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "game/script.rpy", line 53, in script
    python:
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\ast.py", line 914, in execute
    renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2028, in py_exec_bytecode
    exec bytecode in globals, locals
  File "game/script.rpy", line 56, in <module>
    code = renpy.python.py_compile(source_file.read(), filename, 'exec')
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 663, in py_compile
    tree = compile(source, filename, py_mode, ast.PyCF_ONLY_AST | flags, 1)
ValueError: compile() arg 3 must be 'exec', 'eval' or 'single'

Windows-8-6.2.9200
Ren'Py 7.3.5.606
Test 1.0
Mon May 11 18:30:50 2020

I'm stumped, because arg 3 IS 'exec', and it still says "ValueError: compile() arg 3 must be 'exec', 'eval' or 'single'"!

Gah!

[trooper6]

Might this better be done in Python rather than Renpy?

[Odysseus]

I'm not really that good in Python to be able to make a proper game as easily as with RenPy. The idea is to have a very good game as basis, that will motivate students to play it, while at the same time actually learning Python by solving challenges.

[drKlauz]

You calling function with wrong order of arguments.

Code: Select all

def py_compile(source, mode, filename='<none>', lineno=1, ast_node=False, cache=True):

Should be

Code: Select all

code = renpy.python.py_compile(source_file.read(), 'exec', filename)

But then again, i don't think you need to actually compile code to exec it. If you check renpy/python.py you will see py_exec does it already internally:

Code: Select all

def py_exec(source, hide=False, store=None):

    if store is None:
        store = store_dicts["store"]

    if hide:
        locals = { }  # @ReservedAssignment
    else:
        locals = store  # @ReservedAssignment

    exec py_compile(source, 'exec') in store, locals

Only reason you may want to py_compile is to add filename into error messages.

P.S.: Also keep in mind, RenPy's Python and stand-along Python are tiny bit different. RenPy replace some classes to better handle rollback/save systems, plus some other stuff. As long as you stay on using basic programming constructs it will be same, but if you dive into lower level code you will see differences.
P.P.S.: Also currently RenPy uses Python 2.7 which is officially dead and buried, so it might be not exactly great idea to teach students using it, tho on general concepts they feel very similar, using practically identical code. There was some efforts to move to Python 3, but i'm not sure when we actually see this updated version, i guess it might appear along with proper 3d/shaders support.

[Odysseus]

I just found out it does internally. Still...

Code: Select all

            filename = os.path.join(renpy.config.gamedir, "texts\\test.py")
            with open(filename, "rb") as source_file:
                renpy.python.py_exec(source_file.read())

This does nothing.

...and...

Code: Select all

            filename = renpy.file("test.py")
            renpy.python.py_exec(filename)

This does:

Code: Select all

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "game/script.rpy", line 63, in script
    python:
  File "game/script.rpy", line 65, in <module>
    renpy.python.py_exec(filename)
SyntaxError: invalid syntax (<none>, line 1)

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "game/script.rpy", line 63, in script
    python:
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\ast.py", line 914, in execute
    renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2028, in py_exec_bytecode
    exec bytecode in globals, locals
  File "game/script.rpy", line 65, in <module>
    renpy.python.py_exec(filename)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2041, in py_exec
    exec py_compile(source, 'exec') in store, locals
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 692, in py_compile
    raise e
SyntaxError: invalid syntax (<none>, line 1)

Windows-8-6.2.9200
Ren'Py 7.3.5.606
Test 1.0
Mon May 11 20:34:57 2020

[Odysseus]

Ok, quick question: If Renpy uses Python 2.7, is there a chance that the little .py program I've written is somehow NOT compatible? I mean, bloody unlikely, since we're talking about the same stuff, but is it possible that's the problem?

[Odysseus]

And this:

Code: Select all

            filename = os.path.join(renpy.config.gamedir, "texts\\test.py")
            with open(filename, "rb") as source_file:
                code = renpy.python.py_compile(source_file.read(), 'exec', filename)
            renpy.python.py_exec(code)

does this:

Code: Select all

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "game/script.rpy", line 53, in script
    python:
  File "game/script.rpy", line 57, in <module>
    renpy.python.py_exec(code)
SyntaxError: invalid syntax (<none>, line 1)

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "game/script.rpy", line 53, in script
    python:
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\ast.py", line 914, in execute
    renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2028, in py_exec_bytecode
    exec bytecode in globals, locals
  File "game/script.rpy", line 57, in <module>
    renpy.python.py_exec(code)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2041, in py_exec
    exec py_compile(source, 'exec') in store, locals
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 692, in py_compile
    raise e
SyntaxError: invalid syntax (<none>, line 1)

Windows-8-6.2.9200
Ren'Py 7.3.5.606
Test 1.0
Mon May 11 20:37:52 2020

[drKlauz]

Ah, yes, i have seen this. There may be chance your py file is in utf8 encoding with BOM that little bastard - https://en.wikipedia.org/wiki/Byte_order_mark.
Try this:

Code: Select all

            filename = os.path.join(renpy.config.gamedir, "texts\\test.py")
            with open(filename, "r") as source_file:
                src=source_file.read()
                print(src)
                if src and src[0]==u'\ufeff':
                    src=src[1:]
            code = renpy.python.py_compile(src, 'exec', filename)
            renpy.python.py_exec(code)

If you see some garbage in console in front of your code then it is sneaky little bom.
Check renpy/loader.py to see how RenPy handle it:

Code: Select all

    def load_module(self, fullname):

        filename = self.translate(fullname, self.prefix)

        mod = sys.modules.setdefault(fullname, types.ModuleType(fullname))
        mod.__name__ = fullname
        mod.__file__ = filename
        mod.__loader__ = self

        if filename.endswith("__init__.py"):
            mod.__path__ = [ filename[:-len("__init__.py")] ]

        for encoding in [ "utf-8", "latin-1" ]:

            try:

                source = load(filename).read().decode(encoding)
                if source and source[0] == u'\ufeff':
                    source = source[1:]
                source = source.encode("raw_unicode_escape")

                source = source.replace("\r", "")

                code = compile(source, filename, 'exec', renpy.python.old_compile_flags, 1)
                break
            except:
                if encoding == "latin-1":
                    raise

        exec code in mod.__dict__

        return sys.modules[fullname]

[Odysseus]

Nope, it prints the contents of test.py.

And then I get this:

Code: Select all

I'm sorry, but an uncaught exception occurred.

While running game code:
  File "game/script.rpy", line 75, in script
    python:
  File "game/script.rpy", line 83, in <module>
    renpy.python.py_exec(code)
SyntaxError: invalid syntax (<none>, line 1)

-- Full Traceback ------------------------------------------------------------

Full traceback:
  File "game/script.rpy", line 75, in script
    python:
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\ast.py", line 914, in execute
    renpy.python.py_exec_bytecode(self.code.bytecode, self.hide, store=self.store)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2028, in py_exec_bytecode
    exec bytecode in globals, locals
  File "game/script.rpy", line 83, in <module>
    renpy.python.py_exec(code)
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 2041, in py_exec
    exec py_compile(source, 'exec') in store, locals
  File "C:\Users\Odysseas\Documents\renpy-7.3.5-sdk.7z\renpy-7.3.5-sdk\renpy\python.py", line 692, in py_compile
    raise e
SyntaxError: invalid syntax (<none>, line 1)

Windows-8-6.2.9200
Ren'Py 7.3.5.606
Test 1.0
Mon May 11 21:34:25 2020

[Odysseus]

By the way, thank you for helping me, I really appreciate it!

[Odysseus]

I GOT IT!

It would seem that in order for this (slightly retarder way I'm going) to work, I needed to:

A)ONLY exec, without compiling.
B)The .py program to run, EVEN IF EXTERNAL, still needs to use the renpy file handling functions etc., it just doesn't recognize other commands and renpy crashes.

THANK YOU. THANK YOU. THANK YOU.

This is a huge milestone for me. Now I can go on with my game! Thanks!

[drKlauz]

Heh, you welcome, good luck with your project