Speaking animation while playing an audio using layeredimages

[Moshibit]

Hi. This is my first post in this forum.

I have taken an old script from the renpy wiki that does this effect and modified it to work with layeredimages and the animation happen when playing voice audio.

The script animates the lips while playing an audio or while the letters appear in the dialog box.

The script works thanks to a function named renpy.curry. I do not understand much of it, it is not documented in https://www.renpy.org/doc/html/, apparently it is a very powerful function, it would be great if someone explains it in a post or PyTom write a Patreon article.

If you do not understand about layered images there is a great post by BáiYù viewtopic.php?f=51&t=50764#p490465
Here is the script :

Code: Select all

init python:

    # This is set to the name of the character that is speaking, or
    # None if no character is currently speaking.
    speaking = None

    # This returns speaking if the character is speaking, and done if the
    # character is not.
    def while_speaking(name, speak_d, done_d, st, at):
        if renpy.music.is_playing(channel='voice') or speaking == name:
            return speak_d, .1
        else:
            return done_d, None

    # Curried form of the above.
    curried_while_speaking = renpy.curry(while_speaking)

    # Displays speaking when the named character is speaking, and done otherwise.
    def WhileSpeaking(name, speaking_d, done_d=Null()):
        return DynamicDisplayable(curried_while_speaking(name, speaking_d, done_d))

    # This callback maintains the speaking variable.
    def speaker_callback(name, event, **kwargs):
        global speaking

        if event == "show":
            speaking = name
        elif event == "slow_done":
            speaking = None
        elif event == "end":
            speaking = None

    # Curried form of the same.
    speaker = renpy.curry(speaker_callback)

Example. Add something like this after init python block:

Code: Select all

define aug = Character("Augustina", voice_tag="aug",callback=speaker("aug"))

image speaking_august:
    "august_mouth_closed"
    pause 0.2
    "august_mouth_open"
    pause 0.2
    repeat
    
layeredimage august:

    always "august_base"

    group outfit auto:
        attribute jeans default

    group eyes auto:
        attribute open default

    group eyebrows auto:
        attribute normal default

    group mouth auto:
        attribute smile default

        attribute speaking:
            WhileSpeaking("aug", "speaking_august", "august_mouth_closed")

    group glasses auto

    group emotion auto

label start:

    "START"

    show august speaking

    # voice aug_voice01 ## TODO add your voice file

    aug "Blah blah blah blah blah blah blah."

    "END"

    return

To adjust the speed of the letters change this line in options.rpy:

Code: Select all

default preferences.text_cps = 15

[Remix]

Nice

Just for info though, you could dispense with the character callback part of it by using the built-in config.speaking_attribute setting. Then just use the voice channel test to limit the time the wobble lasts.

Example of simple LayeredImage with mouth

Code: Select all

layeredimage eileen:
    # always:
    #     "images/base.png"
    group mouth:
        pos (100, 110)
        attribute speaking: # check for the attribute "speaking" which is added when the character has dialogue
            "speaking_mouth" # image to use - likely WhileSpeaking("aug", "speaking_august", "august_mouth_closed") without the 'aug' bit
        attribute mute default:
            "images/mute_mouth.png"

define config.speaking_attribute = "speaking" # setting this adds this attribute to the character image only when they have dialogue

renpy.curry is basically used to pass additional parameters into functions when the abstraction or DSL does not easily allow.
The ATL 'function' keyword is a good example to help explain:

Code: Select all

transform do_stuff( some_id='eileen' ):
    linear 2.0 xoffset 30
    function my_python_function

Written like that, Ren'Py will call the python function and internally pass it ( the_transform, the_widget_st, the_widget_at )

Now, imagine we wanted to pass in the some_id value, renpy.curry lets us do that

Code: Select all

transform do_stuff( some_id='eileen' ):
    linear 2.0 xoffset 30
    function renpy.curry(my_python_function)(some_id)

As a non keyword argument, it would be received as the first argument, so ( the_id, the_transform, the_widget_st, the_widget_at )
3 arguments automatically, plus the one we curried into it
As a keyword, we'd just use function renpy.curry(my_python_function)(the_id=some_id)

Hope that helps explain and shows where/how it is generally used

[BáiYù]

Wow hey, thanks for the shout out! May I add it to the LayeredImage Tutorial, crediting you of course? I've had a lot of people ask me how to do this and I could never figure out how to.

[Moshibit]

Nice

Just for info though, you could dispense with the character callback part of it by using the built-in config.speaking_attribute setting. Then just use the voice channel test to limit the time the wobble lasts.

Example of simple LayeredImage with mouth

Code: Select all

layeredimage eileen:
    # always:
    #     "images/base.png"
    group mouth:
        pos (100, 110)
        attribute speaking: # check for the attribute "speaking" which is added when the character has dialogue
            "speaking_mouth" # image to use - likely WhileSpeaking("aug", "speaking_august", "august_mouth_closed") without the 'aug' bit
        attribute mute default:
            "images/mute_mouth.png"

define config.speaking_attribute = "speaking" # setting this adds this attribute to the character image only when they have dialogue

This config is interesting, I couldn't make it work, but I discovered why. You lacked to indicate a modification that has to be made to the Character object.

This links the image tag with the say statement:

Code: Select all

define e = Character("Eileen", image="eileen")

I'm still not sure if I will occupy this in my project, I want to make different types of speaking animations.

[Moshibit]

Wow hey, thanks for the shout out! May I add it to the LayeredImage Tutorial, crediting you of course? I've had a lot of people ask me how to do this and I could never figure out how to.

it's ok. sorry about delayed answer.

[BáiYù]

I've been playing around with the LipFlap code and noticed that the original one posted has an issue where all characters present on the screen will animate, even if the voice line is not theirs. If you have voice acting in your project, use this code instead:

Code: Select all

## Lip-Flap Code
init python:

    # This is set to the name of the character that is speaking, or
    # None if no character is currently speaking.
    speaking = None

    # This returns speaking if the character is speaking, and done if the
    # character is not.
    def while_speaking(name, speak_d, done_d, st, at):
        if renpy.music.is_playing(channel='voice') and speaking == name:
            return speak_d, 0.1
        else:
            return done_d, None

    # Curried form of the above.
    curried_while_speaking = renpy.curry(while_speaking)

    # Displays speaking when the named character is speaking, and done otherwise.
    def WhileSpeaking(name, speaking_d, done_d=Null()):
        return DynamicDisplayable(curried_while_speaking(name, speaking_d, done_d))

    # This callback maintains the speaking variable.
    def speaker_callback(name, event, **kwargs):
        global speaking

        speaking = name

    # Curried form of the same.
    speaker = renpy.curry(speaker_callback)

The key difference being and as opposed to or in the def while_speaking block, and removing the section with if event == "show":, as it will not differentiate between which sprite is being shown.

Do note that if you do not have voice acting in your game, the original version is fine (in this version, the lipflap will never stop animating...). At the moment I don't have time to figure out how to fix that, but here is a solution for others using LipFlap and VA for now.

[Moshibit]

I've been playing around with the LipFlap code and noticed that the original one posted has an issue where all characters present on the screen will animate, even if the voice line is not theirs. If you have voice acting in your project, use this code instead:

Code: Select all

## Lip-Flap Code
init python:

    # This is set to the name of the character that is speaking, or
    # None if no character is currently speaking.
    speaking = None

    # This returns speaking if the character is speaking, and done if the
    # character is not.
    def while_speaking(name, speak_d, done_d, st, at):
        if renpy.music.is_playing(channel='voice') and speaking == name:
            return speak_d, 0.1
        else:
            return done_d, None

    # Curried form of the above.
    curried_while_speaking = renpy.curry(while_speaking)

    # Displays speaking when the named character is speaking, and done otherwise.
    def WhileSpeaking(name, speaking_d, done_d=Null()):
        return DynamicDisplayable(curried_while_speaking(name, speaking_d, done_d))

    # This callback maintains the speaking variable.
    def speaker_callback(name, event, **kwargs):
        global speaking

        speaking = name

    # Curried form of the same.
    speaker = renpy.curry(speaker_callback)

The key difference being and as opposed to or in the def while_speaking block, and removing the section with if event == "show":, as it will not differentiate between which sprite is being shown.

Do note that if you do not have voice acting in your game, the original version is fine (in this version, the lipflap will never stop animating...). At the moment I don't have time to figure out how to fix that, but here is a solution for others using LipFlap and VA for now.

I had not tried two characters at the same time and I began to investigate what was happening and I found out, This happens because the last attribute of the first character who spoke is speaking. remember that speaking has two states, one where it is without animation and the other animated. If you change the attribute before another character speaks, you will not have this problem. I tried the solution that you provide, and I only find a problem, the lips do not animate if the two conditions are not met, it may be the case that not all the characters have voice acting or that it is partial voice acting.