Can Lip Flap Be Synched (Sync / Synced) With Voice [solved]

[leon]

I'm using Blink And Lip Flap code which makes the character's mouth move while the text is being displayed and it stops when the text is fully displayed. For a fully voiced game, is it possible to have the character's mouth move, while the voice is playing instead?

[xela]

I'm using Blink And Lip Flap code which makes the character's mouth move while the text is being displayed and it stops when the text is fully displayed. For a fully voiced game, is it possible to have the character's mouth move, while the voice is playing instead?

I can't see why not but you're prolly looking for an advice from someone who's actually done this before

[Asceai]

I haven't done this before either, but if I'd have to guess how to modify the code, I'd probably replace while_speaking with:

Code: Select all

    def while_speaking(name, speak_d, done_d, st, at):
        if renpy.music.is_playing('voice'):
            return speak_d, .1
        else:
            return done_d, None

This might be a bit naive though- there's probably other considerations I'm forgetting.
EDIT: For example, making sure that the right character is actually speaking...

[Asceai]

Alright, here we go:

Code: Select all

init python:
  
    # This is set to the name of the character that is speaking, or
    # None if no character is currently speaking.
    speaking = None
  
    # This returns speaking if the character is speaking, and done if the
    # character is not.
    def while_speaking(name, speak_d, done_d, st, at):
        if renpy.music.is_playing('voice') and speaking == name:
            return speak_d, .1
        else:
            return done_d, None
  
    # Curried form of the above.
    curried_while_speaking = renpy.curry(while_speaking)
  
    # Displays speaking when the named character is speaking, and done otherwise.
    def WhileSpeaking(name, speaking_d, done_d=Null()):
        return DynamicDisplayable(curried_while_speaking(name, speaking_d, done_d))
  
    # This callback maintains the speaking variable.
    def speaker_callback(name, event, **kwargs):
        global speaking
        speaking = name
  
    # Curried form of the same.
    speaker = renpy.curry(speaker_callback)

[leon]

Thank you very much Asceai! It works perfectly!

[xela]

Could you rename the thread to include the word "sync" (synced or synchronized) as well? Synched does not seem to show up in searches and this can be very useful to others.

[leon]

Sure, I added these too. Synchronized makes the title too long...

[cosmo]

I think this could be a nice addition to the wiki article as well…

hmmm… How to add that now!

[ZeroQ]

This is an excellent alteration to an already excellent code. Thanks a lot, Asceai! :3

I hope that one day somebody discovers a way to implement full animation/audio synching into Ren'py. Although I don't know how many people besides myself and a few others, are interested in such a feature.
I also realize that this would be a difficult thing to accomplish and may even require the aid of third-party software, but it would be a nice little feature to have in the engine.

[Asceai]

It's really hard to do anything timing-dependent with audio in ren'py - you can play a sound but you don't know when it will start and you can't detect it. I don't know exactly why but renpy uses pygame which uses SDL1 which I recall liked to create huuuuge audio buffers.

If you give me some graphics (a character with a lip-flap animation) and a voice file I'll play around with this for you, but no promises. I've looked at the code to do this in commercial VNs so I've got a bit of an idea, just a matter of getting it to work in Ren'Py

[ZeroQ]

It's really hard to do anything timing-dependent with audio in ren'py - you can play a sound but you don't know when it will start and you can't detect it. I don't know exactly why but renpy uses pygame which uses SDL1 which I recall liked to create huuuuge audio buffers.

LOL, so I've noticed. There used to be another code for lip synching, where you would have to manually input the duration of each individual frame of animation whenever you wanted to have a character talking. It was virtually impossible to properly sync it with voice audio. What's worse is that the animation wouldn't automatically stop when you clicked to progress the story, so you would sometimes have Character #1's mouth still moving while Character #2 was talking.

If you give me some graphics (a character with a lip-flap animation) and a voice file I'll play around with this for you, but no promises. I've looked at the code to do this in commercial VNs so I've got a bit of an idea, just a matter of getting it to work in Ren'Py

I'll try and get something to you soon, then. Even if you aren't able to find a solution, I appreciate the help. Thanks.
^_^