Using JSON for optimization?

[trailsiderice]

Weird question maybe, but I'm wondering if it would be worth it to store data in JSON object files as a way to minimize memory usage in my game.

See, in order for a specific gameplay function of my game to work right now, I have a pretty big class that stores quite a bit of data relating to each instance, and there are a good number of these instances. I don't really know much about how Ren'Py optimizes memory usage, but I'm worried that having this many instances of a class with that much data associated with it might be using more memory than is optimal. I haven't noticed any performance issues while testing, but I also have a fairly powerful computer, and I'm not sure how it'd perform on computers with low specs.

Most of this data doesn't need to be mutable, so I was thinking a good way to make this a little more optimized would be to store most of the data inside JSON files and access them only when specific data about a specific instance is needed.

I already know how to do this, so I don't really need advice on implementing it, I'm just wondering if it's necessary. Because doing this would require a lot of work reorganizing my data structures and the references to them, and I don't want to do it if it turns out my worries about memory are over nothing.

So my question is basically, would using JSON files to store data reduce memory usage over storing large amounts of data in class instances?

[m_from_space]

I already know how to do this, so I don't really need advice on implementing it, I'm just wondering if it's necessary. Because doing this would require a lot of work reorganizing my data structures and the references to them, and I don't want to do it if it turns out my worries about memory are over nothing.

So my question is basically, would using JSON files to store data reduce memory usage over storing large amounts of data in class instances?

If you ask me, it doesn't make much sense. But then again I don't know how much data we are talking about. Millions of data points? To be fair, still doesn't matter. Memory is cheap and easy and fast. Python handles all the memory by itself and as long as saving this data is not a problem, why would you consider changing something? If you are worried about low end computers, maybe just check it on a low end computer before changing a running system so to speak.

Putting something inside a JSON file means using more or equal memory on disk, since you also need to include the JSON structure. JSON is great because it's human readable, but other than that it still has to be loaded eventually. And I wonder if loading on the fly instead of having the data already in RAM would bother me more. Yeah probably.

When I think about Renpy projects and people wanting to optimize memory and whatnot, I always wonder if they ever played big games and wondered what kind of memory we are talking about in that projects.

So how bad could it even be?

[trailsiderice]

Putting something inside a JSON file means using more or equal memory on disk, since you also need to include the JSON structure. JSON is great because it's human readable, but other than that it still has to be loaded eventually. And I wonder if loading on the fly instead of having the data already in RAM would bother me more. Yeah probably.

When I think about Renpy projects and people wanting to optimize memory and whatnot, I always wonder if they ever played big games and wondered what kind of memory we are talking about in that projects.

So how bad could it even be?

Sure, JSON has to be loaded eventually, but when you don't need it, you can unload it. As opposed to python objects which, as far as I'm aware, sit in memory all the time, taking up space even when they're not used. If my understanding of that is incorrect, I'm open to correction, but that's how it's always been taught to me in my programming classes. Also, there are ways to "stream" JSON files (for example using the ijson python module) in such a way that allows you to only load partial sections of a JSON file at a time as a memory saving measure.

Sure, there are games that take up more memory than even the biggest Ren'Py games, but I don't think there's anything wrong with trying to minimize the memory impact of a game. Better that then not care about it at all and end up like yanderedev. And especially with regards to having mobile versions, since phones generally don't have the same amount of memory to spare.

Still, I'm willing to consider the possibility that I'm worrying over a negligible amount of memory usage. That's why i ask. There really is a lot of data that I need stored, to the level that having all of it at memory all of the time feels less than ideal, even if I haven't noticed any slowdowns.
I can't say I have any low end computers to test it on, but perhaps I can try it in a virtual machine to see if it needs better optimization.

[m_from_space]

There really is a lot of data that I need stored, to the level that having all of it at memory all of the time feels less than ideal, even if I haven't noticed any slowdowns.

Still wondering what kind of data we are talking about and how memory this data takes up right now. For a mobile variant, hundreds or thousands of images could be considered a problem, if that data has to be downloaded. But Renpy allows on-demand loading of that data in the web version anyway.

Optimization usually comes with a cost and you should always think about whether investing so much time in a small memory reduction is worth it, while in this time you could actually work on your project. My point of view is that memory is cheap and available and you should optimize for CPU time, since that will affect the player. Nobody loves stuttering or low frame rates or a battery drain on their phones.

[Ocelot]

1) Memory is cheap. Unless your profiling shows that you are consuming prohibitive amount of memory, you don't have to worry.
2) YOu can do JSON loading in two ways. One is efficient, but incompatible with RenPy saving/rollback system. That will lead to problems. Other one will work with save/rollaback system, but there is a catch. First, mutable objects take a lot more memory in RenPy, because every change in their state for many checkpoints is preserved. Second, that also meant that unloading the file will not actually free memory that was used for it, because it is still in rollback system.

In the end it is better to not fight the framework and work with what it expects.

[trailsiderice]

That's fair. I didn't think about how it might cause problems with saving and rollback.

I'm talking about 20+ instances of a class with sometimes 50+ attributes depending on which of it's methods are run. Which feels like a lot to me, but I may be overthinking it.

Even if it is a lot, it's probably not worth it to fight with Renpy's saving and rollback. Thanks for the advice.