Red eyes in the darkness 2nd dungeon crawler Alpha

[Ryue]

Hi
As a few of you may remember I’m working on an rpg / digital novel hybrid called Red Eyes In The Darkness.
At the beginning of this year I made an alpha test for the first version of the dungeon crawler engine
http://lemmasoft.renai.us/forums/viewto ... 45&t=19559 used in the game.
The tests back then showed me that the basic concept worked but at the same time that I had severe troubles in terms of flexibility and maintainability with the engine, thus I decided to do a complete rewrite. Due to an increased workload at my job I faced some time difficulties but now managed to finish the rewrite of the engine.
As it is a complete rewrite and I want to find bugs,… as good and fast as possible I’m doing the alpha test while adding additional features (some of them known already from the original version but which had to be removed as they are no longer possible in that way in the new version [as some of them were done wrongly
back then and thus worked but they need to be done differently now]) to it.

How will I do this?
[*] I’m going to release a version of the engine every 1-2 days
[*] During the time in between releases I will read through this thread and try to incorporate things given feedback on and also update this post in terms of known issues and features and planed stuff.

What do I need infowise?
[*] A feedback on all issues (be it bugs, performance issues, lag issues, placing issues, graphic issues, ….) found
[*] A feedback on wishes / enhancements that could be made
[*] ….
______________________________________________________________________________________________________________

Versions:
Linux:

RedEyesInTheDarknessPart1-1.0-linux.tar.bz2

(19.46 MiB) Downloaded 40 times

Mac:

RedEyesInTheDarknessPart1-1.0-mac.zip

(16.39 MiB) Downloaded 37 times

Win:

RedEyesInTheDarknessPart1-1.0-win.zip

(18.04 MiB) Downloaded 74 times

______________________________________________________________________________________________________________

Current Version: 0.1

Features:
[*] Moving through a map and not able to move through solid objects
[*] Minimap
[*] Event processing
[*] Fog is in (where it is set on a mapcell)

Plans:
[*] Putting optional day/night cycle in
[*] Putting dynamical lightsources in

Known issues:
[*] Currently only 1 background is ready (thus even inside the cave the blue ceiling is seen.

Mapediting:
For the purpose of this demo the map is in the open and can be edited (its a .csv file: The map being used is mytest.csv). Be sure that the map is enclosed by blockable tiles though. Every map is designed to have a maximum of 512 x 512 cells. Here a description of how the data of each cell is made up:

BBB.TTT.CCC.VVV.PPP.LLL.EEE

BBB: 3 digit number of the background that is to be displayed when this is the 1st cell that the player is looking into
TTT: 3 digit number of the tile that is to be displayed
CCC: 3 digit number of a tile. if this number is given then all tiles between TTT and CCC are displayed as animation (the current status of the animation
aka the current Frame aka the current tile number is saved inside the tile itself. if this here is 000 or lower than TTT then no animation is made)
VVV: 0-100: Represents % of visibility. The lower the faster visibility is reduced when looking into the direction of this field.
PPP: Special effects. For a list look further below.
LLL: Blocking. 0...non blocking, 1...blocking (complete), 2...blocking (except flying)
EEE: 3 digit number representing the event that is to be started whenever this tile is moved into.

Special effects (Be aware that NOT all are currently implemented see features):
001: Light fog - Permanent
002: Dense fog - Permanent
003: Light rain - Permanent
004: Hvy rain - Permanent


101: Light fog - 10% chance for it to occur during daytime.
102: Dense fog - 10% chance for it to occur during daytime.
103: Light rain - 10% chance to occur during day and night.
104: Hvy rain - 10% chance to occur during day and night.

201: Light fog - 20% chance for it to occur during daytime.
202: Dense fog - 20% chance for it to occur during daytime.
203: Light rain - 20% chance to occur during day and night.
204: Hvy rain - 20% chance to occur during day and night.

301: Light fog - 30% chance for it to occur during daytime.
302: Dense fog - 30% chance for it to occur during daytime.
303: Light rain - 30% chance to occur during day and night.
304: Hvy rain - 30% chance to occur during day and night.

[Ryue]

redid the download links (had a problem in the distribution that manifested itself only in distributions and not in the original launcher files)

[Ryue]

As I was asked why the tiles going straight forward in the middle have no side part as all others do I posted here a screenshot of exactly what happens when they have:

Sadly those sidetiles then would appear right in 1/3 of the sidetiles

[Ryue]

Added event processing for events number 001, 002, 003 (001 is only being triggered if you enter the mapposition it is in from the south only).

[Ryue]

Added fog at long last (that was a problematic feature to say the least)

[Saltome]

I hate to say this... but if you ask me, you should give up on this project.
And if not... Well I think it needs to be better optimized. Occasionally it would freeze up to load stuff, take between 20 and 300 mb of my ram. It took a while to release all the memory and close the process after quitting the program too.
Right now my ram is 450mb so that probably has an effect on it, but considering what this is right now, I don't think such a high requirement is worth it. Imagine what an astronomical memory requirement this is gonna have, once you have the actually useful things implemented.

[SundownKid]

I hate to say this... but if you ask me, you should give up on this project.
And if not... Well I think it needs to be better optimized. Occasionally it would freeze up to load stuff, take between 20 and 300 mb of my ram. It took a while to release all the memory and close the process after quitting the program too.
Right now my ram is 450mb so that probably has an effect on it, but considering what this is right now, I don't think such a high requirement is worth it. Imagine what an astronomical memory requirement this is gonna have, once you have the actually useful things implemented.

Actually using 300mb is not that uncommon these days. You seem to have an extremely low ram compared to the typical computer these days. Most "non-gaming" computers have something like 2-4 GB and gaming PCs usually have 8-16 GB.

For me it lagged a little bit when starting and quitting, but the game itself worked okay. However, there are several problems. First of all, when you turn there is no turn animation, so it's hard to get your bearings. Also, this is made worse by the fact that the scenery is tiled and looks the same, so if you are trying to navigate around an area of trees it becomes hard to know where you are.

I think you should check out Etrian Odyssey gameplay footage for an idea of how it's better if it actually showed you turning. Right now I feel like it's too hard to get my bearings in the area.

[Ryue]

Ah just now saw the posts oO

@Saltome what type of PC are you using? (450MB does not seem high). That aside 300 MB can be primarily because of the preloading of the images which is there to improve ingame performance.


@SundownKid lags at start/quit is taht always? (only noticed it once myself). Turning animation: I thought about it myself but couldn't come up with anything there that wouldn't require a real 3d engine instead of a pseudo one or prerendering all pssible combinations (which would end in way over 1 GB for even small games as you would need to prerender with all possible positions for all possible tiles and even with only 3-4 different tiles the number of possible combinations (tiles and positions) is almost astronomical. :/ thus a real calculation would need to be there instead of a pre rendering which is where I don't have any other idea than a real 3d engine).
Aside from etryan oddisey I don't know any other game which uses old school 3d crawlers and utilizes animated turns (bards tales, eye of the beholder, most of the visual novels nowadays which use crawlers but no real polygons,.... don't use animated turns as far as I'm aware). Would anything else help out there? (like an animated compass? [that one I could do without a problem])

[Saltome]

...An old one ...A broken one.
Used to have about 1gb, but so it happened that one of my 2 ram cards died recently.
Aside from that, 1.8gh, 80gb, no dedicated graphics card.

I wasn't planning to discuss this further, but I guess I might as well take it off my chest.
For starters I see what you are trying to do, and it's kinda cool, I've thought about doing similar things myself.
But I can see that this isn't just some afternoon code experiment. It's a long and demanding project, you are making slow and meager progress, and the worst part is that you are only struggling with problems which have already been solved. That's why modern 3d graphics were developed in the first place.
I have a few projects like that myself, but I would advice you to leave this on the back seat and focus on something that can have prompter, more practical effect on your skills. As opposed to struggling with a task you don't have the skill to overcome.

The other thing that's been bothering me about this, is your approach. You are basically making an engine, based on an engine, based on an interpreted language, based on a compiled language. I'm not some sort of programming genius, but such an amount of overlapping and reinterpretation can't be good for the efficiency of the final product.

Lastly, the problem with the demo. To be a little more specific, for the most part the memory requirement was within what I would expect for what it does. If I recall correctly it is triggered when I'm trying to move, but not at a specific time or place. It would start loading, increasing the amount of allocated memory and hanging temporarily. That in itself is not surprising, considering the limited ram it has to work with, in my case. But apparently it allocates a lot more space than it needs for the graphics that are required for it to run, the unpredictable occurrence of this effect is also concerning to me. If I had to make a wild guess, I'd say that something is causing unpredicted and maybe inaccurate reloading of graphics. Aside from the obvious possibility, an oversight in your code. I am also wondering about the way your engine is incorporated in renpy. As far as I know custom code regarding graphics can have unpredictable side effects when it's not properly incorporated. Such a scenario may also be the case.

[Ryue]

But I can see that this isn't just some afternoon code experiment. It's a long and demanding project, you are making slow and meager progress, and the worst part is that you are only struggling with problems which have already been solved. That's why modern 3d graphics were developed in the first place.
I have a few projects like that myself, but I would advice you to leave this on the back seat and focus on something that can have prompter, more practical effect on your skills. As opposed to struggling with a task you don't have the skill to overcome.

I have only one thing to that to say: Don't jump to conclusions when you don't have data or facts straight. I don't know how YOU do projects but I do them that way that I set a goal test it on my pc or workstation or whatever I'm using at that moment and then if the performance,... are ok for what I wanted to achieve on my system I let it be tested either by dedicated testteams or in the case of a hobbyproject by volunteers who have different systems and approaches so that I see where improvements can be made or should be made. Until such problems have been identified I go with the quickest and dirtiest solution and optimize then and only then where necessary to avoid overoptimization which brings nothing but frustration. So I would make the advice to also not jump to any conclussions to what I can or cannot do tnx there (also the programming itself of the project was WAY shorter than you believe the longest thing was to experiment with what renpy can do and what not).

So back to the topic: I know things have been solved differently already. Graphic cards as you mentioned them are used for full 3d games as they calculate each voxel, pixel, .... . Also some old games like eye of the beholder used methods that graphiccards now emulate (I was honestly surprised during my research when I saw WHAT they really did and how. Always thought they just used full tiles but no they really calculated positions of small parts of the tiles with different methods). The mainquestion is: Is that necessary and can renpy do that? I saw one renpy program which uses raytracing my own research didn't show a good method that I found for that (aside from 5 seconds per frame to DRAW the pixels on screen). So I thought is there another way for renpy to do it especially when it was also done with other visual novel engines? Yes there is, just by using what renpy itself has to offer and can do. So I concentrated on that what renpy CAN do instead of what it CAN'T do and found a (at least for me and I know that I'm different to quite a lot of ppl in what I find easy and what they find easy) easy solution for it. So the only struggling part was where I had to find out what renpy can do and what not (and how to debug it......and I still haven't found a good method to debug renpy projects :/ )

And also to say it openly: If I'm in over my head for something I'm honest enough to fully admit it. For example I have searched for someone to look over the code and clear it up in terms of comments or if I overlooked duplicate code or if some things can be done more efficient as I didn't had the time during january and into february to do it myself (found someone and the original recruitment post is also on these forums). Also if I'm not able to do something because of knowledge or skill I'm honest enough to admit it (I always have problems for example with events in C#......although I do really insane things in the end with them I somehow have to overcome a lock in my thought processes when it comes to them)

But after this excurse to the rest of the answer:

The other thing that's been bothering me about this, is your approach. You are basically making an engine, based on an engine, based on an interpreted language, based on a compiled language. I'm not some sort of programming genius, but such an amount of overlapping and reinterpretation can't be good for the efficiency of the final product................I am also wondering about the way your engine is incorporated in renpy. As far as I know custom code regarding graphics can have unpredictable side effects when it's not properly incorporated.

I'll put up an answer to both things here: I think you are having a wrong conclusion there on how I do things. I'm NOT programming a custom code which I use in renpy or anything the like. I'm using renpy code in combination with a few python features like lists (mostly for containing map infos and going through the map infos). Thus the whole "engine" is just pain plure
renpy code using the renpy engines advantages and abilities to display pictures on precalculated positions and using renpies Z-order to the fullest in order to create images that look like they are overlapping / behind each other.

Lastly, the problem with the demo. To be a little more specific, for the most part the memory requirement was within what I would expect for what it does. If I recall correctly it is triggered when I'm trying to move, but not at a specific time or place. It would start loading, increasing the amount of allocated memory and hanging temporarily. That in itself is not surprising, considering the limited ram it has to work with, in my case. But apparently it allocates a lot more space than it needs for the graphics that are required for it to run, the unpredictable occurrence of this effect is also concerning to me. If I had to make a wild guess, I'd say that something is causing unpredicted and maybe inaccurate reloading of graphics. Aside from the obvious possibility, an oversight in your code. I am also wondering about the way your engine is incorporated in renpy. As far as I know custom code regarding graphics can have unpredictable side effects when it's not properly incorporated. Such a scenario may also be the case.

Memory: Can you make a test with another renpy game there? I'm interested there what the basic consumption is on your pc after graphics are loaded and the first graphics are used (thus to find out what of that could be from my program and what is directly renpy memory usage there).

Aside from that I'll open a post in the questions part there. The code itself does only preload images at one location which is triggered at start. The only thing I can think of atm where increased memory DURING the game running occurs is due to shwoing the images. I'm not sure when the memory that is allocated because of that is freed again (ran into similar problems there in c# so not sure if python/renpy handles it better would think so but better to make sure (I myself didn't see that as I only looked at the % of my memory the program uses which was less than 25% so the test with your minimal memory system is quite good there to find out such issues). Thread

Hmm reading about python garbage collection seems to indicate my guess is correct. If that is the case it should be solvable quite easily.

[Saltome]

On a side note, when you run backwards into a wall the fog overlay resets it's position.
And you don't actually bump into the wall, which would make more sense than running into a wall face first. x)

On a second look I can say that there is no performance issue while I'm idling. However pretty much every time I do something it starts loading, and it loads a good while. Starting a new game takes 3 minutes to load.
The average memory usage hovers around 200mb.
As for fluctuations, well I guess they happen whenever there's a lengthy loading. Which means, a lot.

For comparison:
Seven Kingdoms: The princess problem.
Hovers around 100 without any performance issues.
And the download size is 15 times bigger than yours.

Slumberland.
Around 50. There are some performance issues though, it seems to stop to load resources occasionally. Particularly between scenes, sound familiar?
The file size is around 80mb.

Most other projects I've played trough don't have any performance issues to speak of.

[Ryue]

On a side note, when you run backwards into a wall the fog overlay resets it's position.
And you don't actually bump into the wall, which would make more sense than running into a wall face first. x)

Tnx will look into that

On a second look I can say that there is no performance issue while I'm idling. However pretty much every time I do something it starts loading, and it loads a good while. Starting a new game takes 3 minutes to load.
The average memory usage hovers around 200mb.
As for fluctuations, well I guess they happen whenever there's a lengthy loading. Which means, a lot.

Hmmmm if I take my own and other memory usages into account I would guess that the performance issues come from swapping (the minimum memory usage so far has been 500 MB (and up to almost 800 MB). with 100 less if I put in gc.collect() after each rendering). I'm currently testing but I guess taht the memory usage comes mostly from teh image buffers

[Saltome]

What happens if you use a bigger map?
Something doesn't add up. You have about 4 image files, they should not take this much memory.
And I have a nagging suspicion...
Are you utilizing the sprite manager?

[Ryue]

What happens if you use a bigger map?
Something doesn't add up. You have about 4 image files, they should not take this much memory.
And I have a nagging suspicion...
Are you utilizing the sprite manager?

Nope. Not using the sprite manager at all only plain old renpy.show

From what I gather from tests it looks like its mostly a caching thing. After running 10 times around the whole area it comes down to 506MB top.
The map itself is small bitewise so its the image displaying itself that cuases the memory usage.

[Saltome]

You should probably use sprites for such graphically intensive interfaces, images are fat.
The other thing is that if you have more than one instance or copy of each image the total memory requirement inflates.
Lastly, a good point was made in the other topic... The rollback feature. I don't think it's suited for your purposes, it might be adding to the problem.