Ren'Py 5.1 Development Journal

[PyTom]

Okay, I'm opening up this thread, as an experiment, to give running commentary as I develop the 5.1 series of Ren'Py. The idea is to keep users up to date about the bleeding edge of Ren'Py development, and at the same time to have people let me know if some of the changes to Ren'Py are too insane.

Right now, my goal for the 5.1 release is to rewrite the sound support in Ren'Py. The reason for this is twofold, but both folds are motivated by voice support. The first reason is that the old libraries didn't let us include sounds in the archive files, so we would have to clutter up the users system with hundreds or thousands of voice sound files, which isn't very neighborly. The second goal is to support playing multiple compressed sound files at once. This is important, as it will let us play (say) ogg vorbis background music while at the same time playing (say) an ogg speex or vorbis voice file. And compressed voice files are important, as people are already worried about game sizes. (I think we are going to have to bite the bullet and just go with bigger games, if it means color pictures, voice, and in general higher-quality games.)

Anyway, progress for yesterday and today:

2005-07-31

Wrote python bindings for the SDL_sound library, a library that lets us play multiple compressed sounds at once. This lets us play up to 7 channels of any combinations of the following formats: wav, mp3, ogg vorbis, ogg speex, a whole slew of mod formats, and a number of other formats. This library will play midi on Linux using an updated version of Timidity (which hopefully fixes Icekiss's problems), but won't play midi at all on Windows or Mac.

The bindings support fading, and they also support sample-accurate concatenation of compressed files. So it will be possible to break music into a number of sections, and not have gaps between the sections.

They also support reading music out of file-like python objects. (Which is what I create when loading things from archives.) The downside of this is that the whole archived sound needs to be kept in memory while it's played... which will increase the memory usage for large sound files.

I spent the evening and into the late late night compiling on Windows... cursing as I went. The library is around 750k in size, which shrinks to under 300k when I compress it. So that's what this will cost in increased package size, unless I figure out a way to shave the stuff this duplicates off of pygame. Stay tuned.

2005-07-31

Tired today from staying up last night.

Wrote a nativemidi package, which is the same as the above but for the native midi support on Windows and (soon) Mac. So we'll have basically the same support for midi in the new version as we did in the old version. (And many of the same bugs... no support for pausing midis, and no support for accurate concatenation of midis.)

Next up is starting to integrate this into Ren'Py. Any aspiring voice actors out there want to send me a few lines of dialogue, so I have something to test with?

[Icekiss]

Thanks a lot for letting us peak over your shoulder while you work.

The library is around 750k in size, which shrinks to under 300k when I compress it. So that's what this will cost in increased package size, unless I figure out a way to shave the stuff this duplicates off of pygame.

Even if you don't: Most games will be able to save it many times over in decreased sound file size. So no worries...

So we'll have basically the same support for midi in the new version as we did in the old version. (And many of the same bugs... no support for pausing midis, and no support for accurate concatenation of midis.)

So the midi situation won't actually get this much better, will it? What happened to the plans to convert them to mods (not that I have ever used those), or, for that matter, anything else on the fly?
It irks me that there is a file format where music takes up 15KB instead of 1Meg, but no sensible way to use it...

[PyTom]

So the midi situation won't actually get this much better, will it? What happened to the plans to convert them to mods (not that I have ever used those), or, for that matter, anything else on the fly?
It irks me that there is a file format where music takes up 15KB instead of 1Meg, but no sensible way to use it...

Well, I tried converting a midi to a mod. By having to bring along the sound samples, it bloats it unacceptibly... And the converter tool I tried (midi2xm) doesn't produce results I consider acceptible. So no joy there.

On windows/mac, we use native midi, which plays using the samples that come with the computer. We could ship timidity + patches on all platforms, but the patches are unacceptibly big. Just one patch, the acoustic grand piano, is over 1.3 uncompressible megs, and there are dozens like it required to support general midi.

BTW, icekiss, if you want to make sure the new version will work on your system, grab libsdl-sound1.2, and try playing a midi with the playsound command it installs. If it plays there, it will play in the new version of Ren'Py.

[Icekiss]

Sorry, can't to.
If I tell apt-get to install that package, it replies that it will have to remove python2.3-pygame to do so.
I'll just have to wait for the new version.
Besides, I can always use oggs on my own system. I'm just sad I won't be able to make the midi version of the game a really viable option for anyone (If shizeetv2 does do what he says he wants, and we start sticking music pieces together on the fly, and overlaying them with each other).

But then again, I can't do the "overlaying with each other" part right now at all, and that alone will be a huge improvement!

Edit removed because I can't read the damn manual.

[shizeetv2]

So the midi situation won't actually get this much better, will it? What happened to the plans to convert them to mods (not that I have ever used those), or, for that matter, anything else on the fly?
It irks me that there is a file format where music takes up 15KB instead of 1Meg, but no sensible way to use it...

On windows/mac, we use native midi, which plays using the samples that come with the computer. We could ship timidity + patches on all platforms, but the patches are unacceptibly big. Just one patch, the acoustic grand piano, is over 1.3 uncompressible megs, and there are dozens like it required to support general midi.

Did you try some compression options for the samples? I remember in another board that someone had interestingly enough made a sort of GM soft-synth made use of the OGG vorbis compression for it's samples (actually, it was just continuous OGG file, and which probably only sampled keys on an octave or so basis, using pitch bending and other processes to emulate the rest). It had it's kinks and all (the sample quality was felt kind of low, and I'm not sure if it supports all the GM/GM2 features), but it definitely sounded better than the MS-GS, and was only about 5 megs or so. I wonder if you could maybe do something like that, only with some more polish - if it gets too large, you could set it as an optional feature (though there's always the risk that people may simply ignore it all together ). I realize this doesn't really solve some of the troubles of using MIDI, but still, just some ideas to get you going .

[PyTom]

Did you try some compression options for the samples? I remember in another board that someone had interestingly enough made a sort of GM soft-synth made use of the OGG vorbis compression for it's samples (actually, it was just continuous OGG file, and which probably only sampled keys on an octave or so basis, using pitch bending and other processes to emulate the rest). It had it's kinks and all (the sample quality was felt kind of low, and I'm not sure if it supports all the GM/GM2 features), but it definitely sounded better than the MS-GS, and was only about 5 megs or so. I wonder if you could maybe do something like that, only with some more polish - if it gets too large, you could set it as an optional feature (though there's always the risk that people may simply ignore it all together ). I realize this doesn't really solve some of the troubles of using MIDI, but still, just some ideas to get you going .

Hm... I'm just using the compression that's in the pat files that are found on my system. I tried gzipping them and they didn't compress much, but I don't really know enough about the patch file format to really do much with it.

Is this code for the vorbis soft-synth available, and open source? I would consider using it if it was, especially if it can be made optional.

I'm not going to write my own soft-synth... I don't have the time, nor the skills necessary for the signal processing work required. The fact that I'm basically tone-def and can't tell good synthesis from the bad just clinches it.

It's too bad fmod is so expensive for commercial licensing. If it was open source, I would have just used it and saved myself the trouble. But I do want to leave open the option of making commercial games with Ren'Py...

[PyTom]

2005-08-01

Spent today determining the strategy for volume management in Ren'Py. Originally, the plan was to simply limit the volume to whatever the user's wave device was set to... but that's icky, as it doesn't let the user raise the volume.

So, instead, I've decided to have Ren'py take control of the wave and midi volume channels on the user's computer. The idea is that each of the 8 Ren'py channels will have a volume control associated with it. We'll find the maximum volume of those channels (call it maxvol), and then set the hardware volume to it. We'll then set the software mixer volume of each channel to chanvol / maxvol. This will allow for
precise control over channel volume.

A question is: what should the volumes be initialized to? My current plan is to read the volume from the wave mixer the very first time a game is run, and store that as the initial volume for all channels in the preferences. On future loads of the game, those volumes will be restored... which could change the user's volume drastically. Does that sound like a good idea?

My plan is to maintain three volumes in the system. The 'sfx' volume will be applied to channels 0-5 of the mixer, which are intended to play sound effects. The 'voice' volume control will be applied to channel 6, which is reserved for voice. The 'music' volume control is given to channel 7, your all-music channel.

Anyway, did some coding, too... A new winmixer module, which lets us take control of the windows mixer. I'll also write an equivalent linmixer module, for Linux... Macs may not have mixer support, at least until I get access to a Mac. (Donations of OS X 10.3 or .4 compatible hardware gladly accepted.) Sound should probably work on macs... maybe, probably, testers wanted to be sure.

It looks like I'm pretty much done with the C level coding, and everything else will be in Python. A big thanks goes to the Pyrex folks, since without them I'd still be cursing having to write C to Python bindings.

[Icekiss]

On future loads of the game, those volumes will be restored... which could change the user's volume drastically. Does that sound like a good idea?

You could provide one master volume, which you always set to the system volume at game startup. Save the channel volumes (music,sfx,voice), but interpret them relatively to this master volume.
That way, the sound picture the user has sat up (e.g. loudness of voice in comparison to music) is respected, but the users choice of system volume is too.

The 'sfx' volume will be applied to channels 0-5 of the mixer, which are intended to play sound effects. The 'voice' volume control will be applied to channel 6, which is reserved for voice. The 'music' volume control is given to channel 7, your all-music channel.

If I and shizeetv2 follow through with our plans for a dynamic character piece, we will probably need the ability to play more than one music piece at once, and thus more than one music channel.

Can you raise the number of channels to provide room for it, or is 8 a hard limit in some way?

[PyTom]

If I and shizeetv2 follow through with our plans for a dynamic character piece, we will probably need the ability to play more than one music piece at once, and thus more than one music channel.

Can you raise the number of channels to provide room for it, or is 8 a hard limit in some way?

Well, we could always reallocate one of the other channels to music as well. I mean, it's unlikely that anyone will use 6 sfx channels at once. The renpy.music_* apis wouldn't work (since they assume one channel), but the audio.* apis would, and you could build your own music layer atop them.

We'd still only support 1 midi file at a time, at least on windows/mac.

That being said, it's tough to think of a use case for playing two tracks of music concurrently, as opposed to sequentially. At least, I've yet to see a musical-dice type game that plays concurrently.

[Icekiss]

Well, if channels are the scarce ressource here, why don't you provide a way to assign roles to them, the way you do with layers?

Code: Select all

audio.sfx_channels=[ channel0, channel1, channel2, channel3, channel4, channel5 ]
audio.voice_channels=[ channel6 ]
audio.music_channels=[ channel7 ]

That way anyone could decide how to allocate them themselves.

And the music API could be expanded like this (just a look alike, since you've now programmed them in C):

Code: Select all

class audio(object):
    def music_queue( filename, channel=None ):
        if channel==None: channel=self.music_channels[ 0 ]

Perfectly source code compatible with the current behavior, but much more flexible.
(And I know that audio is most likely a module and not a class, but all I want is to get the idea across, anyway)

What to use more than one music track for? Well, you know that you normally record different instruments on different tracks to still be able to change their volume individually, or to rerecord specific instruments.
And you only mix them all together into a single track at the very end.

If I understood shizeetv2 correctly, he wants to do this final mixing while the game is running:

Languages and History:
Affects instrumentation (acoustic): the greater the stat, the more acoustic accompaniment joins the piece

Since you wrote that sample-accurate concatenation is possible, I would say there is a decent chance something like this can be done.
Only other requirement as far as I can see: Ability for individual channels to have a volume relative to their base channel (which is sfx, voice, or music).

Sure playing multiple oggs at once takes up quite a bit of system ressources, but since we don't need them to display 30 frames per second and run a realtime physics engine, I'd say we can spare them. It's not as if decoding an ogg in realtime was that expensive.

[PyTom]

Hm... The one problem will be starting n channels simultaneously, so the music on them is synchronized. Right now, we have sample-accurate concatenation, but not sample-accurate multiple channel start. So it would be possible (and even likely) to start the first channel, and then not have the second channel join in until 4096 samples later.

But, hey, it's just a matter of figuring out an api to fix this. So I'll do that... probably, by adding an option to play that makes it start paused, and then an api to pause/unpause multiple channels at once.

(This is internal to pysdlsound, not exposed to the user.)

Still no luck getting SDL_sound working? What version of Debian are you using?

[Icekiss]

Debian unstable.
So I'll need to have sdl_sound1.2 installed anyway for renpy 5.1? I thought it was just to try it beforehand...

I just tried to install it again: no more conflicts, it's installing.

Trial run: No sound at all (timidity still plays them).
There is no man entry for playsound. What has to be setup for it to work?

[PyTom]

Trial run: No sound at all (timidity still plays them).
There is no man entry for playsound. What has to be setup for it to work?

Hm... It looks for timidity.cfg in a bunch of places, and expects that to point to a patch library, like freepats.

I'd run strace -eopen playsound foo.mid, and then look to see where it's expecting timidity.cfg to live, and then make sure that file is there.

(IIRC Debian may have messed with the paths in one or more of their versions of timidity. So they may not match... One may be looking for /etc/timidity/timidity.cfg, the other for /etc/timidity.cfg.)

[Icekiss]

Thanks, got it working. Now playsound plays it exactly the same way as timidity.
So hopefully I can use it in Ren'Py 5.1 without crashes.

[PyTom]

Bully!

playsound uses the exact same version of timidity as Ren'Py will, and it's a newer one than is used by SDL_mixer. So hopefully, crashes will be a thing of the past.