[solved] Persistent dictionary disappearing upon restart :(

Discuss how to use the Ren'Py engine to create visual novels and story-based games. New releases are announced in this section.
Forum rules
This is the right place for Ren'Py help. Please ask one question per thread, use a descriptive subject like 'NotFound error in option.rpy' , and include all the relevant information - especially any relevant code and traceback messages. Use the code tag to format scripts.
Message
Author
goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

[solved] Persistent dictionary disappearing upon restart :(

#1 Post by goldo » Tue May 29, 2018 5:51 pm

Hi everyone,

I have been struggling for the past few days with a very strange issue with Ren'py not properly saving/restoring a persistent dictionary.

I was trying to implement a dynamic CG gallery to display images that are not declared as ren'py images (I won't get into the various reasons for this here, but the idea is that players can use their own image packs).

Those images are shown within specific screens, and in turn those screens track the file path for the images that have been seen as 'True' in a persistent dictionary. This more or less emulates the way renpy.seen_image() works for regular renpy images.

Everything works fine and dandy as long as Ren'py is running. Images unlock and display all right in the CG Gallery, and the dictionary works fine. A 'renpy.full_restart()' works just fine too.


However, when closing and restarting the game or doing a 'renpy.utter_restart()', all hell breaks loose.


The persistent dictionary disappears, all stored values are removed and the CG gallery is empty. I narrowed the problem down to the persistent dictionary itself disappearing: the code for the gallery seems not to matter. It's not just the individual values that are reset, the dictionary itself is gone.

On a few occasions, when only a handful of pictures were unlocked, the dictionary saved properly. But that was a very rare case. In most cases, the dict simply resets. I strongly suspect that something is messing with Ren'py's system for saving and/or restoring persistent values, because on occasion other persistent values from the game (such as the 'persistent.seen_intro' value I use to track a new game) have gone missing as well, even though they share zero code with the offending script. Those errors, if they happen, go completely silent.

It does not seem to be related to the method I use to declare the dict. I have tried using :

Code: Select all

if persistent.seen_dict is None:
	persistent.seen_dict = defaultdict(bool)
	
With exactly the same (lack of) result.

I also don't think it's because I am using a DefaultDict: I am using other DefaultDicts as persistent variables in the game (for some menu options) without any trouble. I have also tried variants with a regular dictionary, or even just a plain list: the same bug happens either way.

I should also mention that I have used the 'Delete persistent' option in the Ren'py launcher to make sure previous attempts didn't mess with the persistent data. I have also played around with the init order, but didn't notice any difference.

So, I'm at a complete loss to explain what is going on here. Things that could maybe play a role (but I am not sure how to work around them right now):
- The image paths are long: The image file paths used as keys for the dictionary are strings that can be quite long and run over, say, 60 or 70 characters. I'm not sure why that would be a problem for a modern system, but, there.
- The number of images can be large: several hundreds. Does that somehow 'overload' the dictionary with too many keys? I'm not sure, but the fact that it doesn't reset when only a few pictures are unlocked could suggest that.
- The image paths contain '/' characters: Again, I'm not sure why it would mess with Ren'py saving persistent variables, but that's the only specificity I could think of compared to the other dictionaries I use.

The code for the game itself is very complex and largely unrelated, but I have included the relevant bits below. I have double-checked the whole code, and nothing else messes up with or even accesses the dictionary in question.

If anyone can help, you are my only Hope.

Code: Select all


init -999 python:
    from collections import defaultdict
    
init -3:
    define persistent.seen_dict = defaultdict(bool)

init -2 python:

    IMGFORMATS = (".jpg", ".jpeg", ".png",".bmp", ".gif")
    
    def is_imgfile(file):
        if (file[-4:] in IMGFORMATS or file[-5:] in IMGFORMATS):
            return True
        return False
    
    ## This method checks if an image has been seen using the persistent dictionary or the regular renpy.seen_image. 
    ## It works fine as long as the game is running.
    def was_seen(pic): # Where pic is a String, either a file path or a renpy image name
        
        if is_imgfile(pic): # 
            if persistent.seen_dict[pic]:
                return True
        
        elif renpy.seen_image(pic) or persistent.seen_dict[pic]:
            return True
        
        return False

    def list_girl_packs(): # Returns a list of girl pack folders. Used for CG gallery
        
        pack_list = []
        
        for file in [f for f in renpy.list_files() if f.startswith("girls/")]:
                
        file_parts = file.split("/")
            
         if file_parts[1] not in pack_list:
                pack_list.append(file_parts[1])
                
        return pack_list


    def init_galleries(): # This is run on start-up in an init block. The bug doesn't seem to originate from the gallery code itself, even though it's unwieldy.

        global gp_gallery
        
        gp_gallery = defaultdict(bool)
        
        for pack in list_girl_packs():
            gp_gallery[pack] = Gallery()
            gp_gallery[pack].pics = []  # Not a native parameter for the Gallery object, used here for convenience
            
            gp_gallery[pack].slideshow_delay = 1.0
            gp_gallery[pack].navigation=True
            gp_gallery[pack].span_buttons=True
            gp_gallery[pack].locked_button=lock
            
            for file in [f for f in renpy.list_files() if f.startswith("girls/" + pack) and is_imgfile(f)]:
                gp_gallery[pack].button(file)
                gp_gallery[pack].condition("was_seen('" + file + "')")
                gp_gallery[pack].image(file)
                gp_gallery[pack].pics.append(file)
                
            gp_gallery[pack].renpy_images = False


#### SCREENS ####

## This screen is used to display non-renpy pictures
screen show_event(event_pic, x = None, y = None, proportional = True, bg = "#000"):
    
    tag show_event
    
    zorder 0
    
    frame:
        background bg

        if event_pic:
            $ persistent.seen_dict[event_pic.path] = True # This is how the game tracks that this particular picture has been seen.
            add event_pic.get(x, y, proportional) xalign 0.5 yalign 0.0


## The gallery screen displays the gallery attached to a particular girl pack. It is called from the main menu.
screen gallery(name="", gal=None, bg=None): # The Gallery object must have a pics variable (a list of renpy displayables or image files)
    
    tag menu
    
    default page=0
    
    key "mouseup_3" action Return()
    key "K_ESCAPE" action Return()
    
    add "#000"
    add bg xalign 0.5 yalign 0.5
    
    hbox:

        vbox spacing 10:

            text name

            frame background None:

                id "gallery"

                xsize 1.0
                ysize 0.9

                has hbox box_wrap True

                $ index = page*12

                for i in xrange(12):

                    if index+i < len(gal.pics):
                        $ pic = gal.pics[index+i]

                        frame background None xsize 250 ysize 190:
                            if gal.renpy_images:
                                add gal.make_button(pic, ProportionalScale(ImageReference(pic), 240, 180), xalign=0.5) alpha 0.8
                            else:
                                add gal.make_button(pic, ProportionalScale(pic, 240, 180), xalign=0.5, background=c_cream) alpha 0.8


            $ max_page = (len(gal.pics)-1) // 12

            text "Page " + str(page+1) + "/" + str(max_page+1) size 14

            hbox:
    #            textbutton "Start Slideshow" action gal.ToggleSlideshow()

                if page > 0:
                    key "K_LEFT" action SetScreenVariable("page", page-1)
                    key "repeat_K_LEFT" action SetScreenVariable("page", page-1)
                    textbutton "Previous" action SetScreenVariable("page", page-1)
                if page < max_page:
                    key "K_RIGHT" action SetScreenVariable("page", page+1)
                    key "repeat_K_RIGHT" action SetScreenVariable("page", page+1)
                    textbutton "Next" action SetScreenVariable("page", page+1)
                    
                key "K_HOME" action SetScreenVariable("page", 0)
                key "K_END" action SetScreenVariable("page", max_page)
                
                textbutton "Return" action Return()


## These two screens override the native ren'py screens
screen _gallery:

    if locked:
        add "#000"
        text _("Image [index] of [count] locked.") align (0.5, 0.5)
    else:
        add "#000"
        
        for d in displayables:
            add d xalign 0.5 yalign 0.0

    if gallery.slideshow:
        timer gallery.slideshow_delay action Return("next") repeat True

    key "game_menu" action gallery.Return()

    if gallery.navigation:
        use gallery_navigation
        
        
screen gallery_navigation:
    hbox:
        spacing 20

        style_group "gallery"
        align (.98, .98)
        
        key "mouseup_3" action gallery.Return()
        key "K_ESCAPE" action gallery.Return()
        key "K_LEFT" action gallery.Previous(unlocked=True)
        key "repeat_K_LEFT" action gallery.Previous(unlocked=True)
        key 'K_BACKSPACE' action gallery.Previous(unlocked=True)
        key "K_RIGHT" action gallery.Next(unlocked=True)
        key "repeat_K_RIGHT" action gallery.Next(unlocked=True)
        key 'K_SPACE' action gallery.Next(unlocked=True)
        key "K_RETURN" action gallery.ToggleSlideshow()
        key "K_SCROLLOCK" action gallery.ToggleSlideshow()
        
        textbutton _("prev") action gallery.Previous(unlocked=True)
        textbutton _("next") action gallery.Next(unlocked=True)
        textbutton _("slideshow") action gallery.ToggleSlideshow()
        textbutton _("return") action gallery.Return()

    python:
        style.gallery = Style(style.default)
        style.gallery_button.background = None
        style.gallery_button_text.color = "#666"
        style.gallery_button_text.hover_color = "#fff"
        style.gallery_button_text.selected_color = "#fff"
        style.gallery_button_text.size = 16
Last edited by goldo on Sat Jun 02, 2018 6:08 pm, edited 1 time in total.

kivik
Miko-Class Veteran
Posts: 786
Joined: Fri Jun 24, 2016 5:58 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#2 Post by kivik » Tue May 29, 2018 6:56 pm

Try

Code: Select all

    if hasattr(persistent, "seen_dict"):
	persistent.seen_dict = defaultdict(bool)
As you said, it may not be the problem, but my understanding is that you check if an attribute exists in an object by using hasattr().

I can't run your code as it without the rest of the scripts so I can't test what's going on myself unfortunately :(

User avatar
xavimat
Eileen-Class Veteran
Posts: 1433
Joined: Sat Feb 25, 2012 8:45 pm
Completed: Yeshua, Jesus Life, Cops&Robbers
Projects: Fear&Love, unknown
Organization: Pilgrim Creations
Github: xavi-mat
itch: xavimat
Location: Italy
Contact:

Re: Persistent dictionary disappearing upon restart :(

#3 Post by xavimat » Tue May 29, 2018 7:12 pm

I don't know if this will work with defaultdict, but, try using "define" for persistent variables and see if this works. Use it outside any "init" block.

Code: Select all

define persistent.seen_dict = {}
Comunidad Ren'Py en español: ¡Únete a nuestro Discord!
Cops&Robbers A two-player experiment
Fear&Love Why can’t we simply express our feelings? Why am I afraid to say ‘I love you’?
Honest Critique (Avatar made with Chibi Maker by ~gen8)

kivik
Miko-Class Veteran
Posts: 786
Joined: Fri Jun 24, 2016 5:58 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#4 Post by kivik » Tue May 29, 2018 7:19 pm

xavimat wrote:
Tue May 29, 2018 7:12 pm
I don't know if this will work with defaultdict, but, try using "define" for persistent variables and see if this works. Use it outside any "init" block.

Code: Select all

define persistent.seen_dict = {}
Would that not reset the dict each time on load? I thought we wouldn't want to do that to persistent variables?

User avatar
Ocelot
Miko-Class Veteran
Posts: 849
Joined: Tue Aug 23, 2016 10:35 am
Skype: miinipaa
Contact:

Re: Persistent dictionary disappearing upon restart :(

#5 Post by Ocelot » Wed May 30, 2018 3:33 pm

No, that would not reset it. define has a special handling for persistent objects (also persistent object has every attribute possible with value "None" by defining __getattr__ magic method)
< < insert Rick Cook quote here > >

goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

Re: Persistent dictionary disappearing upon restart :(

#6 Post by goldo » Wed May 30, 2018 4:44 pm

Hi guys! Thanks for your replies.

I am using define since it is the recommended method (see the code part in the OP):

Code: Select all

init -3:
    define persistent.seen_dict = defaultdict(bool)
Before calling for help, I did try other methods to see if the defaultdict was causing problems.

I have tried both using a regular dict like this:

Code: Select all

init -3:
    define persistent.seen_dict = {}
Or even a plain list:

Code: Select all

init -3:
    define persistent.seen_dict = []
Of course adjusting the other coding so that it would work as intended.

In both cases, the same bug happened. So it's not only the defaultdict that causes this: Any persistent variable I use for this particular task disappears in the same fashion. It's very frustrating.


Edit: Reading your post carefully, I haven't tried using define outside of an init block. I will and report here.
Edit2: Nope, same old same old. Even when defined outside of an init block, the variable still resets

philat
Eileen-Class Veteran
Posts: 1656
Joined: Wed Dec 04, 2013 12:33 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#7 Post by philat » Wed May 30, 2018 8:58 pm

At this point, probably better to share your project (or recreate the bug with placeholder assets).

goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

Re: Persistent dictionary disappearing upon restart :(

#8 Post by goldo » Thu May 31, 2018 2:05 pm

Ok, so I have made some progress diagnosing the issue (but still couldn't solve it).

The problem lies with the screen:

Code: Select all

## This screen is used to display non-renpy pictures
screen show_event(event_pic, x = None, y = None, proportional = True, bg = "#000"):
    
    tag show_event
    
    zorder 0
    
    frame:
        background bg

        if event_pic:
            $ persistent.seen_dict[event_pic.path] = True # This is how the game tracks that this particular picture has been seen.
            add event_pic.get(x, y, proportional) xalign 0.5 yalign 0.0
Specifically this line:

Code: Select all

$ persistent.seen_dict[event_pic.path] = True # This is how the game tracks that this particular picture has been seen.
It seems that setting the value within the screen like this is what kills the dictionary or list. I get the same error if I use a custom function like '$ unlock_pic()' within the screen instead of setting the value directly.

I have tried using other methods to unlock pictures and they all work fine, even for a large number of pictures. So definitely the problem is with this screen.

My problem is that the 'show_event' screen is called from a very large number of lines in the code, so if I have to add more code to unlock pictures whenever it's called it's going to be a lot of clutter. Is there a safe way to change a variable upon calling a screen?

User avatar
Ocelot
Miko-Class Veteran
Posts: 849
Joined: Tue Aug 23, 2016 10:35 am
Skype: miinipaa
Contact:

Re: Persistent dictionary disappearing upon restart :(

#9 Post by Ocelot » Thu May 31, 2018 4:05 pm

Can you share code for unlock_pic function you used? I have a suspiction that it has something to do with variable scope.
< < insert Rick Cook quote here > >

goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

Re: Persistent dictionary disappearing upon restart :(

#10 Post by goldo » Thu May 31, 2018 4:51 pm

Sure, actually I was using that function with a different version of the script (the one that uses a plain list), but the principle is the same. It looked like this:

Code: Select all

def unlock_pic(pic):
	if pic not in persistent.seen_list:
		persistent.seen_list.append(pic)

And of course, the screen code was then:

Code: Select all

## This screen is used to display non-renpy pictures
screen show_event(event_pic, x = None, y = None, proportional = True, bg = "#000"):
    
    tag show_event
    
    zorder 0
    
    frame:
        background bg

        if event_pic:
            $ unlock_pic(event_pic) # This is how the game tracks that this particular picture has been seen.
            add event_pic.get(x, y, proportional) xalign 0.5 yalign 0.0

kivik
Miko-Class Veteran
Posts: 786
Joined: Fri Jun 24, 2016 5:58 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#11 Post by kivik » Thu May 31, 2018 5:09 pm

Just to check - you've got persistent.seen_list here, but your gallery uses persisent.seen_dict - are you using the same in both codes?

goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

Re: Persistent dictionary disappearing upon restart :(

#12 Post by goldo » Thu May 31, 2018 6:26 pm

Yes, that's why I mentioned this code is from the list variant (I tried using a defaultdict, a normal dict or a list). I wanted to make sure the problem wasn't caused by the defaultdict.

I haven't tried using a function with the defaultdict variant, but I don't believe it would make a difference. I'll see if I can reproduce the bug in a test game, that would be easier to share.

kivik
Miko-Class Veteran
Posts: 786
Joined: Fri Jun 24, 2016 5:58 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#13 Post by kivik » Thu May 31, 2018 6:30 pm

Ok, I meant the attribute / variable name not the data type though, but I assume that's been factored in as well from your last reply. Sorry for not being able to help on this one!

philat
Eileen-Class Veteran
Posts: 1656
Joined: Wed Dec 04, 2013 12:33 pm
Contact:

Re: Persistent dictionary disappearing upon restart :(

#14 Post by philat » Thu May 31, 2018 9:53 pm

Besides the fact that screens aren't supposed to run code that has effects outside the screen -- Seriously. Share something that can be recreated.

goldo
Regular
Posts: 43
Joined: Mon Jan 23, 2017 8:23 am
Contact:

Re: Persistent dictionary disappearing upon restart :(

#15 Post by goldo » Fri Jun 01, 2018 4:41 pm

So, there's a simple game that goes some way into demonstrating the problem I'm having:
http://www.mediafire.com/file/ybw8pea51 ... 1.0-pc.zip

You can start it, click on the menu to display a picture, and watch the persistent variable evolve (in this case, a list).

It's pretty clear multiple items get added to the list every time, not just the picture that was clicked on. The list gets cluttered very quickly, and there are only 3 pictures. With hundreds of pictures in the actual game, I'm guessing the combinations get out of hand quickly, and maybe the list exceeds the available memory for storing persistent variables, or something.

The problem comes from this part:

Code: Select all

## This screen is used to display non-renpy pictures
screen show_pic(pic, x = None, y = None, proportional = True, bg = "#000"):
    
    tag menu
    
    modal True
    
    zorder 10
    
    add bg

    if pic:
        $ persistent.seen_list.append(pic) # This is how the game tracks that this particular picture has been seen.
        add Picture(pic).get(x, y, proportional) xalign 0.5 yalign 0.5
            
    key "mouseup_1" action Return()
If I had to guess, I'd say Ren'py predicting screens is what's causing this. I'm not 100% sure this is the whole problem, but at least it shows that my idea of handling the unlocking through the screen itself was bad.

Unfortunately, short of adding additional code every time an instance of 'show screen' is called (and that's over a hundred and counting), I cannot easily go around this (not to mention it would be easy to forget and make a mistake when writing new events).

Is there a way to safely influence a persistent variable when calling a screen?

Edit: My suspicions are confirmed: it turns out that when calling the screen with 'nopredict', the screen behaves as intended. I didn't see a noticeable performance difference but I'm sure it's there. If anyone knows a better way, I'd love to learn...

Post Reply

Who is online

Users browsing this forum: Google [Bot]