[BUG] MultiPersistent data being wiped out

[opaopa13]

We're experiencing an issue where some users are having their MultiPersistent data wiped out. We currently have two games. Users who play the second game have noticed that data from the first game is initially being pulled in correctly (that is, READING works perfectly fine); however, when the second game attempts to save its own data to the MultiPersistent object, all the data from the first game ends up being wiped out (that is, WRITING seems to initialize the MultiPersistent object).

We've experienced three cases so far. All three cases happens to be Windows platforms, but I don't know if that's relevant. I have not being able to reproduce the bug on my Mac.

Here is only time the second game writes to the MultiPersistent object, "mp". I don't see anything suspect in the code:

Code: Select all

    python:
        # if sequel_saved_frozensets doesn't exist yet, create it
        if not mp.sequel_saved_frozensets:
            mp.sequel_saved_frozensets = set()
            
        # add the frozenset (note: a native Python type) from the latest playthrough to the sequel_saved_frozensets set
        if some_frozenset not in mp.sequel_saved_frozensets:
            mp.sequel_saved_frozensets.add(some_frozenset)
            mp.save()

All of the keys associated with the first game are prefixed with "practicerun_", so this isn't a simple naming collision. As far as I can tell, the the second game is initializing the MultiPersistent data when it attempts to save, after initially reading it properly.

Thank you for any help you may have.

[opaopa13]

An update: it's hitting more of our users than we thought. It seems like the problem only occurs when the second game is run, saves data and then is CLOSED, before being reopened. That is, when the game is run the first time, everything works perfectly as long as the app is still running. Once the app is closed and reopened, though, all the data is wiped out.

My suspicion is that something is going wrong with the pickling of the data into the MultiPersistent file. The problem doesn't appear immediately, because as long as the data is still "alive" as Python variables, it's fine. When the second game saves, it corrupts the file, but this file isn't read from again until the next time the game is run. Thus, the wipe doesn't occur until then.

So far, this has still only affected Windows users; however, that may be a byproduct of Windows being the most popular operating system, rather than a problem with the OS itself. (I was assuming it might have something do with how text or files are encoded/terminated?) We've had reports of it happening on machines from Windows 7 to Windows 10.

Edit: we had an affected user pass us a MultiPersistent file that SHOULD be corrupted, according to my theory, and we were able to load it on our machines. So this theory may be incorrect. It just fits with the other facts: of the data being read initially, only to vanish when the second game is run a second time.

[opaopa13]

We looked into it further, and my friend reports that RenPy is WRITING MultiPersistent data in binary mode, but READING MultiPersistent data in text mode. This could potentially be the problem, when it comes to newline characters in particular; that said, we haven't been able to fully reproduce the bug on our machines. Still, this seems like something that definitely should be fixed, even if it turns out not the cause of our particular problem.

[PyTom]

That seems to be right, good catch.

I'd also check that you're only including basic types into your multipersistent data. If you introduce a new class that wasn't in the first game to the second one, and stick it into multipersistent, then the first one will have problems loading the file.