Database

[Kuh-Couch]

Hi everyone!

Would someone point me to information on the use of databases with renpy?
I want to create -for example- a database with a lot of different items with attributes to them. Should be easier to manage them in my game then.

Somewhere I read that database support exists but well.. i can't find it anymore.

Thanks in advance =)

[0ion9]

AFAICS this is beyond RenPy's scope. However, there are many modules in the Python 'standard library' which are suitable to use to read/write large amounts of data. Since they are in the standard library, they are available to RenPy scripts.

Exactly which libraries are available varies according to Python version. Recently, RenPy has switched over to Python 2.7* (which has, for example, the 'json' module as part of the standard library. JSON is one way to make a modest, text-editable database.).
The version that you are running is probably using Python 2.6, so that particular library is not available.

Here are some of your options, given you are probably running RenPy on Python 2.6:

• Pickling (a solution used by RenPy itself) via the pickle or cPickle modules. binary (not editable with a text editor), fairly fast and small

• XML via xml.etree.Flexible, text-editable, relatively large, somewhat slow

• One of the above in combination with ZIP via the ZipFile module to subdivide the database into individually-requestable documents. may make things somewhat slower. Easy compression of the data if needed. (alternatively, you can just put each document in its own file rather than using anything like ZIP -- this is perfectly practical if the total number of documents is small.)

I'm aware that choosing between the above may seem a bit overwhelming. Database systems (even modest ones) have a number of factors to balance, such as speed, file size, ease of editing.. so none of the options in the standard library are ones I can say unconditionally 'is suitable', without knowing your priorities.

If YAML support was available in RenPy, I would recommend using that. I feel it's a balanced solution that's fairly brief, flexible, (with the C implementation) fairly fast, and easily editable (text-editable) -- suitable for a range of simple database uses.

However, if you install YAML support systemwide in Python, you would be able to write your data in YAML format, then use a Python script to load in each of your items and dump a pickle for each one.. those pickles could later be read by your game script, which would circumvent the problem of not having YAML support in RenPy proper.

* Don't quote me on that. All I know is that I'm running the development version and it's running on Python 2.7

[Sexo Grammaticus]

SQLite is also part of the standard Python library.

[0ion9]

I was also interested in this subject.
After a bit of hacking (for some reason RenPy would not cooperate with PyYAML's relative imports, so I had to change them to absolute imports), I
managed to get YAML I/O working. It's without using the C library, so it's slower than otherwise; I still find it a fairly acceptable speed.

I've attached a zipfile of the modified PyYAML (based on PyYAML 3.10). Unzipping it in your game/ directory should do the right thing.
Then you can

Code: Select all

import yaml

and load some data from yaml files (you must open a file object first, using open().. and then you probably want to pass that file to something like

Code: Select all

yaml.safe_load()

or

Code: Select all

yaml.safe_load_all()

..
here's the code I used to test it:

Code: Select all

init:
    # how can I make this dynamic: fade in the new image using its brightness as a dissolve map?
    python:
        import yaml
        f = open (renpy.loader.transfn('./test.yaml'))
        yamldoc= yaml.safe_load (f)
        creeps = yamldoc['creeps']

label start:
    "This is Ion's test: %(creeps)r"

Here's the content of test.yaml (which I put in the game/ directory), FYI:

Code: Select all

Some : data
creeps : [suspiciously, derisively, incisively]

Other notes:
* I've noticed that the RenPy games I have access to (my 'build distribution' command doesn't work, so I can't test on my own.), don't include some modules, in particular some interesting I/O modules... like sqlite, Sexo Grammaticus. (I checked this versus the Ludum Dare 20 Linux game distribution currently linked at http://www.renpy.org/). Though I'd be quite pleased if you could refute this
* I'm happy to put this hack up on the wiki if anyone is interested.
* Personally, I really want the ability to use NumPy with RenPy. NumPy+YAML would nuke this whole problem comprehensively.

[Kuh-Couch]

0ion9:
I'm trying to get the hang of your yaml hack. I read the wiki page about yaml and now i have a question:
Is it possible to use the syntax of the "normal" yaml in this renpy-version (wiki)? I'm not very familiar with such import-modules in renpy and how they work.
Also I only have basic knowledge of databases but it should be enough for what I want to do.

Would be nice to give some hints on what to read on this topic, too. I'm willing to learn =)

[0ion9]

0ion9:
I'm trying to get the hang of your yaml hack. I read the wiki page about yaml and now i have a question:
Is it possible to use the syntax of the "normal" yaml in this renpy-version (wiki)? I'm not very familiar with such import-modules in renpy and how they work.

PyYAML is the official YAML implementation, it's correct in every respect. This version is just slower, because it doesn't have the option to use the C-Library to speed things up.

Also I only have basic knowledge of databases but it should be enough for what I want to do.

I created a test.yaml in my game/ folder with the following content:

Code: Select all

weapons: Melee
items:
 - id: w_m_001
   name: Club
   descrp: A wooden stick
   price: 10

 - id: w_m_002
   name: Dagger
   descrip: A small sagger
   price: 50

I imported the file correctly in my renpy-document and I want to print the following onto the screen:

Code: Select all

label   start:
    " %(weapons)r"

which should print

Code: Select all

Melee

if I'm right. But the output is the following:

Code: Select all

u'<weapons unbound>'

That's correct. it would be a horrendous mess if your data invaded your global script namespace and you had random values like "descrip='A small sagger'" lying around ... anyway, what happens is, yaml.safe_load()
returns an entire document, which you assign to a variable.
that is, in this case it returns the python value

Code: Select all

{
'weapons': 'Melee',
'items': [
  {'id': 'w_m_001',
  'name': 'Club',
  'descrp': 'A wooden stick',
  'price': 10},
  {'id': 'w_m_002',
   'name': 'Dagger'
   'descrip': 'A small sagger'
   'price': 50}
 ]
}

(which is a 'dictionary' in python terms -- a map between keys, like 'id' (a key) and 'w_m_002' (a value))
..
and it goes into that variable
(say you wrote

Code: Select all

myitems = yaml.safe_load(f)

then, myitems['weapons'] would access the value you want.
Because you can't access that 'part' of myitems via the default speaking mechanism, you have two options...
which I like to call the 'elegant' and 'inelegant' options. (I'll only describe the 'inelegant' option on request)

The elegant solution involves being more explicit in your say-ing,
like so:

Code: Select all

$ narrator (" %(weapons)r" % myitems)

Here, we are using Python's (not RenPy's, although it's based on the same thing) string interpolation functions.
The notable difference about Python's string interpolation is that you must specify a dictionary (in RenPy, the dictionary used is always the global namespace -- the one containing all your variable, transition, and character definitions). So! Using Python string interpolation allows us to specify the exact dictionary we want ... in this case we specify

Code: Select all

myitems

. Which means that Python looks inside 'myitems' for a key named 'weapons', and replaces '%(weapons)r' with its value (in this case, 'Melee')

This can be very useful.. for example, look at this:

Code: Select all

$ narrator (" %(name)s   [id:%(id)s\n\n%(descrp)s\nPrice: %(price)d" % myitems['items'][0])

That will describe all of the details of your 'Club' (since it's the first item in 'items', its index is 0).
if you wanted to describe the 'sagger' (lol) you could just change the '0' to a '1'. Well, you could if it weren't for the 'sagger' currently having a 'descrip' attribute rather than a 'descrp' attribute

Where's the problem? Would be nice to give some hints on what to read on this topic, too. I'm willing to learn =)

For a start,
Python String Interpolation reference

In particular, notice that above, I used the 's' (string) and 'd' (decimal -- integer, to be more exact) formatting codes.
'r' is actually lesser used.. I only used it in my example because.. well, I thought I had to use it for printing more complex things like lists. Turns out I don't, 's' will do
Anyway! There is an important difference between 's' and 'r'.. when you apply it to a string, 's' will print the string exactly as it is, without quotes.
'r' will print it with whatever type of quoting makes sense in context. So 'r' is mainly good when you want to quote stuff

(at this point, maybe you've guessed that 's' was what was suitable for the 'Melee' message, rather than 'r')

'f' is the other useful one to keep in mind (for formatting floating point numbers, aka numbers with a fractional part.. -- it gives you more control over the number's formatting than '%s' or '%r' can)

If you decide to learn more about how Python dictionaries work in a general sense, this link gives a nice intro and overview

[PyTom]

I'll note that for the sort of "database" here, using more than Python could be overkill. One pattern I like to use is to create a class with a constructor that adds instances of that class to the correct data structure(s). One can then create instances of that class in a declarative style, which is easy to read.

(This is python, so put it in an init python block if you want.)

Code: Select all

# A list of all the Enemys we know about, in the order they were declared.
enemies = [ ]

# A map from enemy tag to Enemy object.
enemy_by_tag = { }

class Enemy(object):

    def __init__(self, tag, name, description, hp):
        self.tag = tag
        self.name = name
        self.description = description
        self.hp = hp

        enemies.append(self)
        enemy_by_tag[tag] = self

Enemy(
    tag="cube",
    name="Gelatinous cube",
    description="It's hard to think of a beast more adapted for life in a graph paper environment.",
    hp=5)

Enemy(
    tag="bunny", 
    name="Killer Rabbit of Caerbannog",
    description='Has nasty big pointy teeth. Vulnerable to blessed hand grenades.",
    hp=10)

# ...

(Code not tested, but the general idea should be ok.)

The nice thing about using Python as the data language is that there's very little impedance mismatch between it and python the programming language. The integers are python integers, rather than integers-as-strings. (I don't recall if YAML integers are like that or not.) You can use parameters with defaults, or positional parameters, to reduce the verbosity. You can add methods to the created objects to make them more useful, etc.

[0ion9]

PyYAML is very polite to Python, FWIW. That's why I like it so much (aside from being very easy to write).
It's JSON which tends to do that integers-as-strings things, depending on the implementation.
Anyway, yeah.. when I think database, I think of something containing 200 or more items, rather than something with 40 or so.
I definitely agree with you Tom in that if the number of items is that small, there's no need to store the data in a separate format.

If RenPy is running on Python 2.6+, additionally you may have the option of using

Code: Select all

collections.namedtuple

, which would make that kind of 'database' even easier to make:

Code: Select all

from collections import namedtuple
_Enemy = namedtuple ('Enemy', 'tag name description hp')
_Enemy_defaults = {'description' : '(No Description)'}
enemies = []
enemy_by_tag = {}

def Enemy (*args,**kwargs):
    # use the defaults if available
    actual_kwargs = dict ()
    actual_kwargs.update(_Enemy_defaults)
    actual_kwargs.update(kwargs)
    result = _Enemy (*args, **actual_kwargs)
    enemies.append (result)
    enemy_by_tag[result.tag] = result

#(declaration of items in an identical way to Tom's go here)

(in addition, a 'named tuple' is readonly, unlike a class, which is probably a desirable property in this situation.)

[LegsWild]

That's how I use yaml:

With RenPy 8.2.0 you can use the pip yaml package: https://pypi.org/project/PyYAML/#files / 6.0.1

I copied utar://PyYAML-6.0.1/lib/yaml to my game directory.

Then I've added a simple helper function:

Code: Select all

init python:
  import yaml
  def load_yaml(filename):
    f = open (renpy.loader.transfn(filename))
    return yaml.safe_load(f)

I'm then able to do:

Code: Select all

define validAchievements = load_yaml('./local/yaml/achievements.yaml')

I've a lot of static content like definition about people, their images, their location and default properties. It's great to use a yaml file for this.