TileEngine and UnitEngine: v1.0 released!

[chronoluminaire]

Wow. That's extremely helpful. Many thanks, Gau. I feel like I can take that outline and build on it now: I feel like the most difficult step has been done for me. Thank you!

Just out of interest, using the version from your earlier post with Ren'Py 6.6.2, the scrolling didn't work for me. Did it work for you in the earlier version? (I haven't tried the version you edited into your last post, since I didn't see it until just now.) It just stayed in the same position for each of the five comments from Eileen.

Still, extremely many thanks for "demystifying" custom displayables for me. I got partway through converting the TileEngine to use this display mechanism yesterday. I'm extremely grateful!

[Gau_Veldt]

Just out of interest, using the version from your earlier post with Ren'Py 6.6.2, the scrolling didn't work for me. Did it work for you in the earlier version? (I haven't tried the version you edited into your last post, since I didn't see it until just now.) It just stayed in the same position for each of the five comments from Eileen.

Yeah the first version uses integer offsets so when frame rate is really high between renders the thing truncates to 0 when dividing to get the number of pixels to scroll (at really high fps the scroll amount may wind up being a fraction of a pixel) for the frame and no scrolling occurs. On a slower system it'll scroll unless only a small part of the map being shown has any actual image tiles (raises the frame rate) in which case it exhibits the same problem. The second version uses floating point for map offsets which can handle fractions of pixels and thus eliminates the problem.

I should mention I've been using 6.6.2 all along.

[Gau_Veldt]

One thing this code is not doing and it has to for saves and rollback to work is that the .visit() method needs to do something sane (see my comment about it being broken) by returning all the displayables used by the class, namely the Images objects. The values() method of the dictionary should do the trick. PyTom could probably help us out here with this detail.

[jack_norton]

Today had some time to test your engine. Very well done
I had a crash though, while using the keyboard to walk in the village in topview mode, this is the traceback text:

Code: Select all

I'm sorry, but an exception occured while executing your Ren'Py
script.

TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

While running game code:
 - script call at line 86 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/demos.rpy
 - script at line 38 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/villagedemos.rpy
 - python at line 88 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/villagedemos.rpy.
 - python at line 268 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 293 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 302 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 321 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 346 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 528 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.
 - python at line 595 of I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy.

-- Full Traceback ------------------------------------------------------------

  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\bootstrap.py", line 247, in bootstrap
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\main.py", line 309, in main
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\main.py", line 92, in run
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\execution.py", line 199, in run
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\ast.py", line 554, in execute
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\python.py", line 880, in py_exec_bytecode
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/villagedemos.rpy", line 88, in <module>
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 268, in StartGame
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 293, in StartOfTurn
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 302, in LoopGettingCommands
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 321, in GetCommand
  File "c:\- indie dev -\games\tycoongames.eu\renpy\renpy\curry.py", line 38, in __call__
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 346, in SetMode
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 528, in HighlightMoveableSquares
  File "I:\Creations\Tactics\renpy-6.6.0\Tactics/game/unitengine.rpy", line 595, in CalculateMovementCosts
TypeError: unsupported operand type(s) for +: 'NoneType' and 'int'

While running game code:

Ren'Py Version: Ren'Py 6.8.1a

[Key-chain]

Nice..

[zanaikin]

Okay, seeing as we've been making massive changes to the Tile/UnitEngine for our game, something interesting as come up:

This engine is a massive memory hog, to the point that it's. To be precise, it's the Show() function. It's as if it doesn't clear the memory each time it draws the full map or something.

A few data collected via Python Profiler. This testing is all done using the original Tile/UnitEngine in Isometric format before all our modifications:

20 units, 25x25 map, basic sprites (~350Bytes each), 4 movement directions
start: Mem: 36940 K / VM: 33612 K
1 action: Mem: 50892 K / VM: 47356 K
2 actions: Mem: 64906 K / VM: 61380 K
4 actions (one enemy killed): Mem: 113012 K; VM: 109548 K

20 units, 25x25 map, detailed sprites (~4KB each), 4 movement directions
start: 36488 K / 33164 K
1 action: 47404 K / 43916 K
2 actions: 64676 K / 61208 K
4 actions: 100892 K / 97416 K



I don't particularly get why this is seeing as it calls UI to clear the layer at the beginning of the Show() function.

At the start of TileEngine.show():
ui.layer(self.layer) # drops everything into "tileengine" layer
ui.clear()
# add stuff
# applies ui.close() to ui.fixed() and ui.side() first...
ui.close() # at the end of it all...

I tried another ui.close() after that in case something was missed, but that throws an "ui.close() called to close the last open layer or widget." error, I presume it's cause the opened layer is already closed, along with all the widgets inside it.

I'm not completely sure right now if this is a graphics handling issue or a parameter handling leak issue. But the problem is pretty obvious: that memory requirement is increasing far too fast to have any kind of semi-complicated gaming on this Tileengine.



So my first question is : Is there a good way to wipe all graphic layers (or all UI memory not necessary for the Renpy engine itself) ? As far as I can tell ui.clear() and renpy.scene() both cleans out single layers.

How does renpy handle graphics memory anyways? As far as I can tell it just stacks the old layers and put the new graphics on top.

Renpy.scene() actually gives some rather interesting results when used in the TileEngine.show()... it causes show() to create multiple repeated layers in the most obvious manner.

Help would be much appreciated. I've been scratching my head out over this problem for half a year.

[PyTom]

So my first question is : Is there a good way to wipe all graphic layers (or all UI memory not necessary for the Renpy engine itself) ? As far as I can tell ui.clear() and renpy.scene() both cleans out single layers.

renpy.scene and ui.clear are reasonable ways of doing this. Ren'Py will cache some additional data (like up to 8 screens worth of images), but it should eventually reach a stable state after which memory usage is controlled entirely by the complexity of the screen.

How does renpy handle graphics memory anyways? As far as I can tell it just stacks the old layers and put the new graphics on top.

Well, Ren'Py uses a software renderer, so it doesn't manage "graphics memory" at all. It doesn't clear the screen between frames, so you need a fullscreen image or else you'll see an after-image of a previous frame.

Again, it has a least-recently-used cache for images, so that consumes a bit of memory, but it's bounded. (Unless all the images are used at once.)

Renpy.scene() actually gives some rather interesting results when used in the TileEngine.show()... it causes show() to create multiple repeated layers in the most obvious manner.

I have no idea what you mean by this.

[zanaikin]

renpy.scene and ui.clear are reasonable ways of doing this. Ren'Py will cache some additional data (like up to 8 screens worth of images), but it should eventually reach a stable state after which memory usage is controlled entirely by the complexity of the screen.

Then would this probably be the reason why it doesn't work well for the TileEngine? Because a large amount of data is being crammed into a single layer and displayed all at once, if it saves 8 screens of that it's going to be quite hectic.

Renpy.scene() actually gives some rather interesting results when used in the TileEngine.show()... it causes show() to create multiple repeated layers in the most obvious manner.

I have no idea what you mean by this.

Here's a pic to explain:

That happens when I put "Renpy.scene()" to the top of the TileEngine.show() function, go into game, scroll up, and then scroll back down. You can see the repeated layers being show behind the first in the spaces that would normally be black (since isometric map isn't an upright square after all). Also look @ the side-scroll-bar which is completely messed up as a result.
So somehow renpy.scene() made it worse, and I can't even think of how it does that when it's suppose to clear layers and not keep layers.

[Jake]

That happens when I put "Renpy.scene()" to the top of the TileEngine.show() function, go into game, scroll up, and then scroll back down. You can see the repeated layers being show behind the first in the spaces that would normally be black (since isometric map isn't an upright square after all).

It seems you're misunderstanding quite how Ren'Py draws things. It starts out with a black canvas at the very beginning, sure, but it never clears that canvas. If you want a black background, you have to explicitly draw one yourself - try showing a Solid("#000") in between renpy.scene() and TileEngine.show().

Basically, all Ren'Py does is paint whatever you tell it to show - the stuff in the current scene - directly on top of whatever was there before. Normally, in typically VN use, this means starting by painting an opaque background first, then a load of sprites on top of that. But calling renpy.scene() only clears out the list of things to draw in the scene, it doesn't do anything at all to the screen itself, which will still contain whatever you last drew. So if the thing you draw immediately after a renpy.scene() doesn't fill the entire screen with opaque pixels, you will be able to see some of what was there before through the transparent gaps.

[zanaikin]

I do get how it's handling. After all, I'm not trying to draw black with renpy.scene here, I'm just trying to erase the last tileengine layer that was drawn, if it wasn't being erased already. The fact in the pic older layers are covering up the black is convincing me it's not erasing the layers but leaving them all there. Cause yes, I know the 'black background' is basically the bottom-of-the-pit here and "scene black" doesn't actually draw it.
Covering up something with black wouldn't do me any good anyways, it's a memory issue, not a display issue.

The places where the duplicates are happening in the picture is where black is suppose to be shown because there isn't suppose to be another image layer covering it [shrug].

On another hand, I'm actually tempted to believe from some of the data that this isn't actually a graphics issue because it feels like the layer is being cleared proper in the original code. Except there's no way the logic required to perform a single action would take up 32MB of memory + VM on a simple engine like this... I just can't see how that would be possible.

[Jake]

I'm just trying to erase the last tileengine layer that was drawn, if it wasn't being erased already. The fact in the pic older layers are covering up the black is convincing me it's not erasing the layers but leaving them all there.

It erases them from the draw list, but not from the screen.

The places where the duplicates are happening in the picture is where black is suppose to be shown because there isn't suppose to be another image layer covering it [shrug].

Are you actually explicitly drawing black, though? Because you haven't mentioned that you are at all. All renpy.scene() will do is remove things from the unrelated-to-graphics-memory draw list which you have explicitly shown with renpy.show() or the Ren'Pyscript commands 'show' or 'scene'. If you remove the TileEngine stuff from the draw list it doesn't remove it from the screen.

If you know any graphics programming, then think of it like this - Ren'Py doesn't clear the framebuffer before drawing the next frame, whatever you drew in the previous frame is still there and will still be displayed unless you explicitly cover it up with something. Black just happens to be what it starts with at the very beginning of the game.

You can produce the same effect with regular Ren'Py-script as well, for example:

Code: Select all

init:
    image black = Solid("#000")
    image eileen = "eileen_happy.png"
    
label start:
    
    scene black
    
    "Now it's black"
    
    # This following line is equivalent to calling 'renpy.scene()' on all layers - it clears out the draw list, but doesn't add a background on its own
    scene
    
    show eileen at left
    
    "Eileen is at the left"
    
    hide eileen
    
    "Eileen is hidden"
    
    scene transparent
    
    "It's a new scene"
    
    show eileen at right
    
    "Eileen is at the right"

The 'scene black' at the beginning is just to get a black background to start with, because otherwise we'd still have the menu there because Ren'Py won't erase it. Try the code, then take the 'scene black' out and see what happens.
You'll note, if you run it, that the left-most Eileen stays on-screen through a hide (which explicitly removes her sprite from the draw list), a scene (which removes all sprites from the draw list and will be calling renpy.scene()) and a show (which removes all drawables with the same image tag - i.e. 'eileen' - from the draw list). That's three separate ways that she could have been removed from the draw list the same way your TileEngine stuff is when you call renpy.scene(), and she's still on-screen because nothing has explicitly been drawn over her.
When you call the commands 'hide' or 'scene' or whatever Ren'Py repaints the scene starting with the background layer and working forwards, but in this case the background layer is entirely transparent so it doesn't paint anything and leaves whatever was there before on the screen. However, the renpy.scene() function doesn't cause the scene to repaint at all - it just clears out everything from the draw list in preparation for adding a new background and redrawing the scene. It's used by the 'scene' command in Ren'Pyscript, but it's not totally equivalent.





Now, I find it pretty hard to believe that you have some mysterious 'memory issue' that's causing this, because I find it pretty bizarre that it would be affecting only the background parts of your image and not any of the parts you've actually explicitly drawn. Are you sure you're definitely drawing a background at some point before calling TileEngine.show() again? If you are, can you put together a small demonstration of the error that doesn't require your whole script but still exhibits the problem?

[zanaikin]

Ah sweet, I guess my brain was still missing the connection to that particularly detail. Thanks a bunch.

Are you actually explicitly drawing black, though? Because you haven't mentioned that you are at all. All renpy.scene() will do is remove things from the unrelated-to-graphics-memory draw list which you have explicitly shown with renpy.show() or the Ren'Pyscript commands 'show' or 'scene'. If you remove the TileEngine stuff from the draw list it doesn't remove it from the screen.

So far as I can tell TileEngine.show() doesn't tell it to draw black. Which isn't something I need anyways. I guess if all renpy.scene() does is remove stuff from the draw list it's not quite what I'm looking for - which is more of how to remove a set of graphics that's already drawn than to prevent something from being drawn again.
It says on the documentation that ui.clear() is effectively the same as renpy.scene(). Does that mean this same deal of not removing something already drawn [& only removing it from the draw list] also apply to ui.clear() ?

If you know any graphics programming, then think of it like this - Ren'Py doesn't clear the framebuffer before drawing the next frame, whatever you drew in the previous frame is still there and will still be displayed unless you explicitly cover it up with something. Black just happens to be what it starts with at the very beginning of the game.

Yeah, so as I can tell renpy.show() doesn't work like the .draw() function typically found in video games that's updated every frame. This has actually been giving me a lot of headaches, not quite something I'm used to.

Now, I find it pretty hard to believe that you have some mysterious 'memory issue' that's causing this, because I find it pretty bizarre that it would be affecting only the background parts of your image and not any of the parts you've actually explicitly drawn. Are you sure you're definitely drawing a background at some point before calling TileEngine.show() again? If you are, can you put together a small demonstration of the error that doesn't require your whole script but still exhibits the problem?

Sorry for any misunderstandings but... the memory issue doesn't cause the duplicates. The duplicates only happened thus far when I tried to solve the memory issue using renpy.scene(). The thing here is that I didn't specify a layer, so it defaultedly erased the 'master' layer and as a result it showed all the older/past layer versions still there. This leads me to believe the TileEngine.show() was covering up the past/already-drawn tileengine layers with a black background... except I don't see where it's doing this at all. The only time the background is set [via renpy.show()] is during the TileEngine constructor. As a matter of fact so far as I can tell TileEngine.show() doesn't draw anything to any layer except the 'tileengine' layer.



So pytom mentioned that renpy automatically caches X sets of past graphics in the back, I think a question I'd state is:

Is there a way to manually remove one of the already drawn set of images that's currently being cached ?

As a matter of fact it be a real nice thing if there's a method to flush that graphics cache altogether [shrug]