Page 1 of 1

[Solved?] How do I run ren'py script inside a user-defined statement?

Posted: Mon Jan 08, 2018 4:54 am
by PasswordIsntHunter2
This page says I can use block="script" to make a user-defined statement run ren'py code, but I don't think that's exactly what I want. It seems like this will execute the top-level block as ren'py script, but I want to execute some script inside other blocks that are within the top-level block.

What I want is a user-defined statement containing a block with some other statements, as well as some other labels, which themselves contain blocks. Sort of like the "menu" statement in ren'py. It might be easier to look at my code:

script.rpy:

Code: Select all

label start:

    $var = "start"

    "start" "[var]"

    test:
        #some stuff

        "Label":
            #ren'py script beneath the label (so, two subblock_lexers deep)
            $ var = "end"
            "test" "testing!!!"

    "end" "[var]"

    return
_test.rpy:

Code: Select all

python early:

    def parse_test(lex):
        # get block
        sub = lex.subblock_lexer()
        sub.advance()

        # do stuff with some other code

        label = sub.string()
        script = sub.subblock_lexer()

        return (label, script)



    def execute_test(o):
        script_lexer = o[1];
        # What do I put here to run the given ren'py script?



    def lint_test(o):
        script_lexer = o[1];
        # TODO: stuff



    renpy.register_statement("test", parse=parse_test, execute=execute_test, lint=lint_test, block=True)
So there's a say statement which prints out the value of var, I use the test command which contains some ren'py script, then I print out var again. Of course, the script doesn't actually do anything - the "Testing!!!" is never printed and the value doesn't change - because nothing is actually running the script that exists under "Label". It's captured in the lexer and passed to execute_test along with the value of the label string. This is more-or-less the same structure as the menuitems in ren'py's menu statement.

execute_test() has the lexer with the script passed into it. What can I do with this to actually run the code? Is there a ren'py Python function that will run ren'py script from a lexer? Do I need to add it to the AST somehow/somewhere? I just want the code to run and then go back to the script where the "test:" statement was.

Re: How do I run ren'py script inside a user-defined statement?

Posted: Mon Jan 08, 2018 8:53 am
by Remix
Have you tried it with proper label code rather than the "Label": ? (You're basically telling Ren'py to expect a Ren'py style node (or several) and then not even telling it what the node is - With your way you definitely would have to initialize a Label Node with the name and block then expressly call .execute on it which in my mind defeats the object of telling Ren'py to expect the block). Note: Even with proper Ren'py code I am not sure it would work nicely if at all - you would stll probably need to call a NodeType.execute() ( 0[1].execute() )

Without knowing why you want to include such a complex lexer node (Label) inside your own parser it is hard to advise further (I can almost see a reason to add something like a TextButton or some such inside CDS... just not anything more complex)
Sub-note: If you get Translation working smoothly in your CDS sub blocks, let me know ;)

Also note: Ren'Py reserves filenames beginning with an underscore or "00". So, perhaps the suggested 01filename.rpy might be a better choice

Re: How do I run ren'py script inside a user-defined statement?

Posted: Mon Jan 08, 2018 3:14 pm
by PasswordIsntHunter2
Thanks for the filename help, I'll do that.

The string isn't a ren'py label - I called it a "label" because I'm not sure what else I'd call it, but it's just a string (or list of strings and an integer and maybe some other stuff which I want my lexer to parse), which is the parent of a block of ren'py code. If there's a cleaner way to have my own lexer accept the strings but have ren'py run one of the script blocks, I'd love to hear it. What I'm trying to do is basically like a custom Menu statement, but Menu can't do everything I'd like to do.

What is NodeType.execute()? The object passed to execute_test() (the object in o[1]) is a Lexer returned from "sub.subblock_lexer()". How do I get a NodeType from this?

I just want to select from a number of subblocks of "labels" - not ren'py labels, but strings, like in the Menu statement - and run one of them. I'm looking for the ren'py code that handles the Menu and I just found the function parse_block() which seems to take a lexer containing the script and returns a result (an AST node?) which is then bundled with the rest of the menuitem in a tuple. But I still don't see where this node is executed - it gets returned as part of "rv", and my own function can't return anything (can it? The page on creator-defined statements isn't clear).

Re: How do I run ren'py script inside a user-defined statement?

Posted: Mon Jan 08, 2018 6:00 pm
by Remix
The parse_block or block=True setting is there so you can tell a lexer to expect a Ren'py node (basically anything you would normally write in a screen or label) and to parse it as such. That would then use renpy.screenlang.FunctionStatementParser which would return a nested set of renpy.ast Nodes (the NodeType I referenced before) which *presumably* could be ran using the .execute() method of each class.

To be honest, when I dabbled with Lexer stuff I stayed clear of the renpy.screenlang stuff and just compiled nested dictionaries which I then passed to a controller object - didn't need or use the Ren'py block=True bit.
~~~~~~ ignore this if using 6.99.13 or above ~~~~~~
If you do continue delving in that direction you will actually notice though that the menu: block doesn't solely use the FunctionStatementParser system as menu: is too complicated to easily integrate within it. Realistically menu: is one of the most complicated Ren'py blocks as it has to hook into so much of the underlying code in order to have the choices work with rollback/saves/prediction/auto-play etc
~~~~~~ seems menu: now does sit as a Statement for the Parser (either that or they removed the comment referencing that - still pretty complex code anyway) ~~~~~~

As a vaguely non answer, you might like to check out Event Handler with Lexer based controls. Around line 1770 of the 01event_handler.rpy file are some Lexer functions which handle parsing of various nested blocks used to pre-initialize labels as events within a handler.

If it doesn't yield any insight (obviously feel free to butcher any parts of the code to suit your own needs) maybe try explaining exactly what you are trying to achieve (as "like a custom Menu statement" doesn't give much info).

Re: How do I run ren'py script inside a user-defined statement?

Posted: Tue Jan 09, 2018 2:31 am
by PasswordIsntHunter2
I don't think your event handler is relevant to what I want to do. I see a lot of stuff about registering events which are toggled on and off? But I don't think it runs ren'py code inside a block at any point. The stuff at line 1770ish almost looks like it's parsing Python code or some custom syntax? But I'm not sure, there's a lot to sift through.

I did look at what ren'py does if you use block="script" when registering a statement. statements.py line 115 is "rv.code_block = renpy.parser.parse_block(l.subblock_lexer())", using the same parse_block function I found earlier. (It doesn't use screenlang.FunctionStatementParser... wouldn't that be for setting up a Screen?)

parse_block does return AST nodes, and I can actually run them properly (w00t!):

script.rpy:

Code: Select all

label start:
    $var = "start"
    "start" "[var]"

    test:
        #some stuff

        "Label":
            #ren'py script beneath the label (so, two subblock_lexers deep)
            $ var = "end"
            "test" "testing!!!"
            "test" "testing!!! 2"
            "test" "testing!!! 3"
            jump asdf



label asdf:
    "end" "[var]"

    return
01test.rpy:

Code: Select all

python early:

    def parse_test(lex):
        # get block
        sub = lex.subblock_lexer()
        sub.advance()

        # do stuff with some other code

        label = sub.string()
        script = renpy.parser.parse_block(sub.subblock_lexer())

        return (label, script)



    def execute_test(o):
        for stmt in o[1]:
            stmt.execute()



    def lint_test(o):
        test = o
        #TODO: stuff



    renpy.register_statement("test", parse=parse_test, execute=execute_test, lint=lint_test, block=True)
There are two problems with this approach. First, my code block has 3 say statements (the "Testing!!!" ones). If I save when the second or third are being displayed, when I load the game, it's back on "Testing!!!" (the first one) and I have to click through again. I think this is because ren'py has some kind of logic tree representing where you are in the game, and rather than cleanly adding to it, I'm just executing statements manually while I'm still "at" the test: statement.

The other problem probably has the same root cause: I've added a jump statement to return to the code surrounding the test: statement. This is because without it, "Testing!!! 3" is the last thing ren'py will display - after that it will go back to the main menu, rather than going back to where I was in the start: label (and executing the "end" Say). The menu: statement doesn't need to do this, so obviously there's still something I'm doing wrong here.

Re: How do I run ren'py script inside a user-defined statement?

Posted: Tue Jan 09, 2018 6:58 am
by Remix
PasswordIsntHunter2 wrote: Tue Jan 09, 2018 2:31 amThere are two problems with this approach. First, my code block has 3 say statements (the "Testing!!!" ones). If I save when the second or third are being displayed, when I load the game, it's back on "Testing!!!" (the first one) and I have to click through again. I think this is because ren'py has some kind of logic tree representing where you are in the game, and rather than cleanly adding to it, I'm just executing statements manually while I'm still "at" the test: statement.
Ren'py basically staggers a game by interactions (you might have seen renpy.restart_interaction() in several code examples here and there) which are all executed in their own context ( renpy.game.context() if I recall ) and I presume that your code is effectively creating just the one context where it should really be creating one per interactive line. No idea if this will help... perhaps:

Code: Select all

    def execute_test(o):
        for stmt in o[1]:
            renpy.invoke_in_new_context( stmt.execute() )
or

Code: Select all

    def execute_test(o):
        for stmt in o[1]:
            stmt.execute()
            renpy.checkpoint()
The other problem probably has the same root cause: I've added a jump statement to return to the code surrounding the test: statement. This is because without it, "Testing!!! 3" is the last thing ren'py will display - after that it will go back to the main menu, rather than going back to where I was in the start: label (and executing the "end" Say). The menu: statement doesn't need to do this, so obviously there's still something I'm doing wrong here.
Not entirely sure of why it wouldn't just continue from after the User Statement and I do not really have many possible ideas to get it to do so. The only one I can think of off hand is adding a renpy.restart_interaction after the execute_test loop or (and this is likely going to get a bit complex) traversing the Node stack and directly setting the next_node attribute of the final Node in your execute_test loop to be the first Node after the User Statement.

Well done on getting it working as far as you have anyway.

Re: How do I run ren'py script inside a user-defined statement?

Posted: Tue Jan 09, 2018 9:27 am
by PasswordIsntHunter2
Hmm, no progress so far - renpy.invoke_in_new_context() throws an error " 'NoneType' object is not callable" - I guess the invoke function would be trying to call what was returned from execute(), and execute() doesn't return anything? I might look through invoke_in_new_context() to see what it's trying to do.

Neither checkpoint() or restart_interaction() seem to change anything.

I saw "renpy.game.context().next_node = n" in ast.py so I tried this (just to get the statements onto the stack rather than calling execute()):

Code: Select all

    def execute_test(o):
        script = o[1]
        #renpy.say("Next", str(renpy.game.context().next_node))
        #renpy.say("Next", str(renpy.game.context().next_node.name))
        #renpy.say("Script", str(script[0]))

        renpy.game.context().next_node = script[0]
The old next_node object does seem to (correctly) be whatever I put right after the test: statement. In the example above it's the "asdf" label, but it can also be a Say (which shows up as a Translate object). script[0] is (correctly) a Python object representing the "$ var = "end"" command from script.rpy. So, hypothetically, I should set renpy.game.context().next_node to script[0] (which I'm doing above), then go through the list and set the last Node.next_node to what renpy.game.context().next_node used to be.

So everything seems to be in order but when I actually use this code, it does nothing at all - I see the "Start" Say from script.rpy, then the game immediately goes back to the main menu without even executing the "testing!!!" statements in o[1]. So the actual next node to run is not being set correctly.

I looked in execution.py and ast.py but I don't see what I need to do differently here. Every Node class's execute() function in ast.py uses a function next_node(n) which contains the single statement "renpy.game.context().next_node = n" - it looks like I'm doing it right. How do I cleanly and correctly modify the Node stack?

Re: How do I run ren'py script inside a user-defined statement?

Posted: Tue Jan 09, 2018 11:34 am
by Remix
What I meant was...
Your CDS covers the entirety of the test: block, even if it is creating new Nodes while parsing that. As such, once it finishes the final Node of your block, you would want the next_node to reference the Node after the test: block...

Code: Select all

label test_parse:
    "Start"
    test:
        #some stuff
        subnode here:
             "test" "testing!!! 3"
    "End"
So, the next_node there would want to reference "End" (unless you performed a Jump or somesuch and moved it elsewhere)...

Code: Select all

    def execute_test(o):
        for node in o:
            node.execute()
        renpy.game.context().next_node = # the reference of the "End" Node (probably taken somehow from reading the node stack)
Still just guessing here and, realistically not inclined to do any testing along this code direction as I still fail to see why you would want to incorporate so much Ren'py code within a CDS in the first place.

Re: How do I run ren'py script inside a user-defined statement?

Posted: Tue Jan 09, 2018 8:11 pm
by PasswordIsntHunter2
Yes, if I do this, it goes back to where it was at the end:

Code: Select all

    def execute_test(o):
        old_next = renpy.game.context().next_node

        for stmt in o[1]:
            stmt.execute()

        renpy.game.context().next_node = old_next
...but it still has a problem with saving/loading while inside the test: block, and probably some other issues I haven't encountered, which is why I was trying to properly put the test: block into the stack as next_node rather than just calling execute() on each one.

I tested some more and using next_node actually sort of works - I thought it wasn't before, because the first statement in my list just assigns a variable which I don't see happening - but it does run, and nothing after it does because the statements in the list aren't linked to each other. Their .next fields are all "None". So the variable assignment worked, and then the game quit without running any of the other Say statements because the next_node got set to None.

I really thought parse_block() would link them up for me since it returns the list of AST nodes. I could go through and do it manually but I want to look for where in the code this normally happens, to make sure I'm not missing anything.

Actually I just tried it with some "script[i].next = script[i+1]" and it works, but doesn't solve the save/load problem. So I'm going to look through ren'py's code some more to see where it links nodes up... maybe look for occurrences of checkpoint() as well, if that's a save/load thing. (I'm not at all familiar with how the savegames work)

[EDIT] renpy.ast.chain_block() is what I'm looking for - it recursively chains everything together.

Code: Select all

    def execute_test(o):
        renpy.ast.chain_block(o[1], renpy.game.context().next_node)
        renpy.game.context().next_node = o[1][0]
It still sets the savegames back to the start of the block, so that's what I'll look at next.

Re: How do I run ren'py script inside a user-defined statement?

Posted: Thu Jan 11, 2018 8:15 pm
by PasswordIsntHunter2
I might have gotten it working-ish - sorry for the bump, but I'm not sure if I'm doing it exactly right.

For loading/rollback (which are the same thing since a save is just the serialized log object) to work, you need Rollback objects in the log marked as checkpoints corresponding to each statement (AST Node) (for the longest time I thought this was what was wrong, that all my statements were being captured by a single Rollback object or something. But no, it was fine). But for rollback() to actually use these checkpoints, the Nodes also have to have names and entries in script.namemap. Since I was parsing them myself, this wasn't getting done.

After banging my head against a wall for a while I found the relevant functions: renpy.game.script.assign_names() will set Node.name on all the nodes (when printing them out, they have a slightly different format than "normal" nodes - the file path is relative rather than absolute - but I don't think that matters), and renpy.game.script.finish_load() will add them to namemap and do some other housekeeping (check for duplicate names, handle *.rpyc filenames etc).

This is my 01test.rpy now:

Code: Select all

python early:

    def parse_test(lex):
        sub = lex.subblock_lexer()
        sub.advance()

        # do stuff with some other code

        label = sub.string()
        script = renpy.parser.parse_block(sub.subblock_lexer())

        renpy.game.script.assign_names(script, lex.filename)
        renpy.game.script.finish_load(stmts=script, initcode=[])

        return (label, script)



    def execute_test(o):
        renpy.ast.chain_block(o[1], renpy.game.context().next_node)
        renpy.game.context().next_node = o[1][0]



    def lint_test(o):
        test = o
        #TODO: stuff



    renpy.register_statement("test", parse=parse_test, execute=execute_test, lint=lint_test, block=True)
Like I said, it seems to work fine, but just in case: these new functions (mainly finish_load) might break things or depend on things happening in the rest of the engine. Does anyone happen to know any unforeseen problems I might run into here? Is there anything else in ren'py that this code could break?

Re: [Solved?] How do I run ren'py script inside a user-defined statement?

Posted: Thu Jan 11, 2018 9:22 pm
by Remix
I would speculate that, as long as it is successfully handling any value changes that occur within the node (just a $ var += 1 sort of change) then it is likely doing fine on everything else. As you know, the saves are just a pickle of information so the load can pick up at the same place, so if it is pickling the user variable change ok then it is likely handling all the other stuff fine.

Well done on getting it working btw