Split-second lag when playing videos

Discuss how to use the Ren'Py engine to create visual novels and story-based games. New releases are announced in this section.
Forum rules
This is the right place for Ren'Py help. Please ask one question per thread, use a descriptive subject like 'NotFound error in option.rpy' , and include all the relevant information - especially any relevant code and traceback messages. Use the code tag to format scripts.
Post Reply
Message
Author
pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Split-second lag when playing videos

#1 Post by pfranzen »

Hi all! I'm wrapping up developing on an FMV game using Ren'py, and there's an issue that's absolutely making me lose my mind.

The basic gameplay is that you watch a series of video clips, make a decision, and then watch a different series of video clip. More and more I'm noticing a lag in the way the video clips are displayed. As in, the subtitle for the next clip appears first, and then half a second later, the actual clip starts playing. Ideally, the subtitle and clip would appear simultaneously.

This is what it looks like:

Image
Image
Image

Here's how it works ("works") so far. First, I need the video to appear in a specific part of the screen, so this line defines that under an init:

Code: Select all

image movie = Movie(size=(985,560), xpos=262, ypos=29, xanchor=0, yanchor=0) 
Then, when we display a movie, it looks like this:

Code: Select all

show case 4 - defending 1 # Backdrop for the video.
show movie with None # Adds the video displayable if it's not already there.
show subtitle 1-defending # Adds a generic header, in case you have subtitles turned off
play movie "videos/case 4/case 4 - 16.webm" noloop
if persistent.captions == "true":
    show subtitle case 4-36 # Shows the actual subtitle, if you have subtitles turned on
$ narrator.add_history(kind="adv", who="Don't Stop Defending", what="Don't Stop Defending: Why?\n") # Adds the text of the subtitle to the history screen
$ renpy.pause(1.1, hard=True)
# Pauses until the video is finished

I know that looks like a lot of lines—a lot of it likely superfluous, esp. if I were a better coder/designer—and the obvious solution would be to try to trim it down. I already tried that. As a test, I trimmed it all the way down to the two problematic elements:

Code: Select all

show subtitle 1-defending 
play movie "videos/case 4/case 4 - 16.webm" noloop
And the lag is still there. No matter what, the subtitle always displays before the video. Sometimes it's even worse, too—not just a split-second, but a full second, or two seconds before the video clip plays. I've noticed this particularly when coming off of a choice menu. I just...I just want everything to display simultaneously :cry:

I don't know if it's gotten worse over the course of development, or if it's always been like this and I'm just noticing it now. It's not even that you necessarily notice the subtitle issue specifically, but the transition from clip-to-clip is extremely un-smooth and annoying to look at.

Things I have tried to fix it:

- Reducing video file size
- Converting video files to different file types
- Deleting most of the video files, to see if it's an overall file size issue
- Disabling the custom cursor (since that was causing other, separate lag issues in older versions of Ren'py)
- Removing every single line of code for when we're playing videos except "show subtitle" and "play movie"
- Rearranging all the lines of code for when we're playing videos, in every order I could think of
- Changing the Renderer
- Disabling Powersave
- Changing the Framerate
- Enabling/Disabling Tearing
- Switching to an older version of Ren'py
- Updating to the latest experimental build of Ren'py
- Limiting use of the "show movie" command
- Adding "with None" to the "show movie" command so that it doesn't look for a transition
- Breaking the script files into "Case 1", "Case 2", "Case 3" and "Case 4" files (previously, I had everything in one gigantic file, because I am bad at this)
- Changing various parameters of the "image movie" code
- Restarting my computer
- Generating a fresh build of the game, to see if it's only an issue when I'm playing it from the Ren'py launcher. (It's not.)
- Complaining to my spouse
- Switching all the pauses in the game between "hard=True" and "hard=False"
- Attempting to preload all the videos when the game initializes (not sure if I ever actually got this to work)
- Taking a screenshot of the initial frame of the video, then displaying THAT simultaneously with the subtitle

That last solution actually kind of works, but since there are over 1,000 video clips it's untenable, unless anybody knows of a command that can automatically display the first frame of a video, so it doesn't have to be done by hand. Also it's the world's clunkiest workaround.

In the future (i.e., if we make a sequel, or a similar FMV game) I think the solution would be to hard-code the subtitles directly into the videos, but I think it's too late at this point to do that. (There are over 1,000 video clips and 1,500 subtitle images. Plus, then, if I want the player to be able to enable/disable subtitles, I'd have to have two full sets of videos—one with subtitles and one without—which makes me a little concerned about the potential file size of the game.)

So...does anyone have any thoughts?

User avatar
Triority
Regular
Posts: 185
Joined: Fri Jul 20, 2018 1:28 pm
Completed: Welcome To... Chichester 0, 1,2 OVN 1, OVN 2, OVN 3, No Regrets For The Future
Projects: Welcome To... Chichester series
Organization: Triority
Tumblr: Sku-te
itch: triority
Location: England
Discord: sku_te
Contact:

Re: Split-second lag when playing videos

#2 Post by Triority »

Are your movies .webm ones ?

This has something about it all : viewtopic.php?t=34194

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#3 Post by pfranzen »

Triority wrote: Thu Jan 21, 2021 1:50 pm Are your movies .webm ones ?

This has something about it all : viewtopic.php?t=34194
Yep, they're all webm's! I've tried ogg and matroska, too, but it makes no difference.

User avatar
Imperf3kt
Lemma-Class Veteran
Posts: 3794
Joined: Mon Dec 14, 2015 5:05 am
itch: Imperf3kt
Location: Your monitor
Contact:

Re: Split-second lag when playing videos

#4 Post by Imperf3kt »

It's most likely the bitrate of your videos is too high, try compressing them more to see if that fixes the desync.
Warning: May contain trace amounts of gratuitous plot.
pro·gram·mer (noun) An organism capable of converting caffeine into code.

Current project: GGD Mentor

Twitter

User avatar
PyTom
Ren'Py Creator
Posts: 16096
Joined: Mon Feb 02, 2004 10:58 am
Completed: Moonlight Walks
Projects: Ren'Py
IRC Nick: renpytom
Github: renpytom
itch: renpytom
Location: Kings Park, NY
Contact:

Re: Split-second lag when playing videos

#5 Post by PyTom »

Which version of Ren'Py are you on? 7.4.1 has some fixes to this, but we might need something that lets you pause until at least one frame has been shown.
Supporting creators since 2004
(When was the last time you backed up your game?)
"Do good work." - Virgil Ivan "Gus" Grissom
Software > Drama • https://www.patreon.com/renpytom

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#6 Post by pfranzen »

PyTom wrote: Thu Jan 21, 2021 2:58 pm Which version of Ren'Py are you on? 7.4.1 has some fixes to this, but we might need something that lets you pause until at least one frame has been shown.
7.4.0, though I tested it with 7.4.1 and the lag seemed about the same. Trying Imperf3kt's idea about the bitrate now...

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#7 Post by pfranzen »

Imperf3kt wrote: Thu Jan 21, 2021 2:23 pm It's most likely the bitrate of your videos is too high, try compressing them more to see if that fixes the desync.
Okay, I tried converting a batch of videos to lower bitrate. It didn't make a noticeable difference, until I went to around 400 bitrate; but then the quality of the videos dropped significantly, so I'm not sure this is ideal.
Last edited by pfranzen on Mon Oct 17, 2022 5:51 pm, edited 1 time in total.

User avatar
Imperf3kt
Lemma-Class Veteran
Posts: 3794
Joined: Mon Dec 14, 2015 5:05 am
itch: Imperf3kt
Location: Your monitor
Contact:

Re: Split-second lag when playing videos

#8 Post by Imperf3kt »

pfranzen wrote: Thu Jan 21, 2021 3:38 pm
Imperf3kt wrote: Thu Jan 21, 2021 2:23 pm It's most likely the bitrate of your videos is too high, try compressing them more to see if that fixes the desync.
Okay, I tried converting a batch of videos to lower bitrate. It didn't make a noticeable difference, until I went to around 400 bitrate; but then the quality of the videos dropped significantly, so I'm not sure this is ieal.
This is about the result I expected, you have five videos on screen at once after all. It sounds like your PC doesn't meet the minimum specs to play your game.
With a more powerful computer the higher bitrate should be handled more easily.

If you don't mind, what are the specs on the PC you're currently using?
Warning: May contain trace amounts of gratuitous plot.
pro·gram·mer (noun) An organism capable of converting caffeine into code.

Current project: GGD Mentor

Twitter

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#9 Post by pfranzen »

Imperf3kt wrote: Thu Jan 21, 2021 4:33 pm This is about the result I expected, you have five videos on screen at once after all. It sounds like your PC doesn't meet the minimum specs to play your game.
With a more powerful computer the higher bitrate should be handled more easily.

If you don't mind, what are the specs on the PC you're currently using?
Sorry, I should specify—the images on the left-hand side are all static avatars; the only video is the big image on the upper-right.

I'm on a Windows 10 laptop, about 2-3 years old. 8GB ram; AMD FX-9830P Radeon R7 processor; 64-bit. External monitor connected to the laptop, too, if that makes a difference.

Woetoo
Regular
Posts: 28
Joined: Sat Feb 09, 2019 9:48 am
Contact:

Re: Split-second lag when playing videos

#10 Post by Woetoo »

You might want to take a look at play movie( {stuff} , start_image={imgname} ).

I don't know that it will solve anything for you, but I can imagine a small lag while the movie is initially being processed. You show the movie and the subtitle at the same time, but the movie is a fraction behind (because it's a movie). It's there, but it hasn't appeared yet.

That seems to be what "start_image=" is all about, putting a placeholder image on screen while the movie is being dealt with. I think the way it's intended to be used is that you create a static image with the first frame of the video and use that as your start image.
If I'm reading it right, it won't actually remove the lag - it'll just make it less visually obvious.

Maybe it's not your solution, but it shouldn't take too long to figure out if it will or won't work.

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#11 Post by pfranzen »

Woetoo wrote: Thu Jan 21, 2021 7:41 pm Maybe it's not your solution, but it shouldn't take too long to figure out if it will or won't work.
Okay, that's an interesting solution. I found a program that will let me rip the first frame from a batch of videos automatically, so that's helpful. I'm having a little trouble getting the code right, though. I've been playing around with this:

Code: Select all

play movie "videos/case 3/case 3 - 29.webm"(start_image = "images/frames/case 3/case 3 - 29 1.jpg") noloop 
But it's giving me this error message:

Image

I'm sure I'm doing something wrong with the code; I'm just not sure what the right combination/order/styling of commands is to get it to work.

strayerror
Regular
Posts: 159
Joined: Fri Jan 04, 2019 3:44 pm
Contact:

Re: Split-second lag when playing videos

#12 Post by strayerror »

Maybe something like?

Code: Select all

image movie vid_case3_29 = Movie("videos/case 3/case 3 - 29.webm", start_image="images/frames/case 3/case 3 - 29 1.jpg", loop=False, size=(985,560), xpos=262, ypos=29, xanchor=0, yanchor=0)

Code: Select all

show movie vid_case3_29
If the above works, you could potentially also do something like the code below to automate your clip definitions (assuming all your assets are named similarly):

Code: Select all

# Add some defaults to a Movie and save it as movie.
define movie = renpy.partial(Movie, loop=False, size=(985,560), xpos=262, ypos=29, xanchor=0, yanchor=0)

# Loop over the cases and clips defining each using the defaults we prepared, plus the video and first frame files.
init python hide:
    for case, clips in ((1, 3), (2, 4), (3, 29)):          # case 1 (3 clips), case 2 (4 clips), and case 3 (29 clips)
        for clip in xrange(1, clips + 1):
            renpy.image('movie vid_case{0}_{1}'.format(case, clip), movie(
                'videos/case {0}/case {0} - {1}.webm'.format(case, clip),
                start_image='images/frames/case {0}/case {0} - {1} 1.jpg'.format(case, clip)))

Code: Select all

# Now all being well, they should be ready to use! \o/
show movie vid_case1_2
pause
show movie vid_case2_3
pause
show movie vid_case3_29
EDIT:
  • Updated usage of renpy.curry to renpy.partial instead. Ooops!
  • Updated init python to init python hide to avoid any leaks.

pfranzen
Regular
Posts: 42
Joined: Fri Apr 01, 2016 2:39 pm
Completed: Cat President 1+2, Internet Court, Too Many Santas, The Awkward Steve Duology, The Pizza Delivery Boy Who Saved the World, Francy Droo & Friends
Organization: Oh, a Rock! Studios
itch: oharock
Contact:

Re: Split-second lag when playing videos

#13 Post by pfranzen »

strayerror wrote: Thu Jan 21, 2021 10:48 pm Maybe something like?
Thanks so much for all of this! It's still not quite working, unfortunately. It's just doing the same thing as before: the subtitle appears, then a split-second later the video starts playing.

Is it possible it's totally ignoring the "start_image" qualifier? I've quadruple-checked that it's pointing to the right image file (and I've tested it with other image files, too); it seems like it's not displaying the start_image at all.

Woetoo
Regular
Posts: 28
Joined: Sat Feb 09, 2019 9:48 am
Contact:

Re: Split-second lag when playing videos

#14 Post by Woetoo »

pfranzen wrote: Fri Jan 22, 2021 11:00 am Is it possible it's totally ignoring the "start_image" qualifier? I've quadruple-checked that it's pointing to the right image file (and I've tested it with other image files, too); it seems like it's not displaying the start_image at all.
Personally I'm not sure. It was a bit of a shot in the dark from me, based purely on reading the documentation description a few weeks ago. It sounds like you've already tried the thing I would have, which is to use a different image (maybe solid red or something) to see if the image flashes up, no matter how briefly. If you're not seeing at all, I can only presume that the lag you're seeing isn't a result of the time spent decoding the first frame and that the movie is starting instantaneously (at least, as far as RenPy is concerned).

I think you're back to figuring out PyTom's suggestion about somehow delaying the display of the subtitle for a brief moment, maybe using some sort of transform?

The only other thing that comes to mind... and again... purely theoretical on my part is to use some sort of on "show" action Show("subtitles_overlay"). My somewhat hopeful intent being that on "show" ... is processed after the main screen has been displayed fractions of a second later. Yeah... I'm reaching here.

Post Reply

Who is online

Users browsing this forum: Bing [Bot]