How to import python libraries?

[fluxus]

Uhm.

So I looked around for answers to this, and though I did find a thread on installing lua for Ren'Py, I'm still not certain about the exact process.

I'd like to be able to import libraries like numpy and networkx to use them for games.

Ren'Py has (I guess) it's own python interpreter so I'll probably need to target installation/compilation of the libraries I want to use for Ren'Py's interpreter somehow.

But, what kind of python is Ren'Py running? Cython?
And, do I need to compile the libraries or is it done ad-hoc?
If I need to compile them, are they needed for each platform that I want the game to target? Windows, Linux, iOs, Android, etc?

I may be able to figure out the details, but a helpful pointing in the right direction would be nice :>

[xela]

Libraries written in pure python you can import and they'll just work. You can open Ren'Py console and check for Pythons version just as you would in any other Python implementation (I think (import sys in init + sys.version)).

You can search the forum and github for similar issues, I recall a few of those... If the module you wish to import has C dependencies (like numpy), you'll (most likely) have to recompile Ren'Py, which is not as easy as one might think in the first place (guide available somewhere on github as well). I recon that by the time you figure out how to compile the engine, how to import a module will prolly no longer be a question.

(installing lua for Ren'Py... that's like blasphemy, is it not? )

[fluxus]

So that would mean.. if it's in pure python, I can just dump the library in the game directy and import? That sounds nice.

Networkx is pure python I think. Unfortunately it requires some sort of installing, too. I localized python executables in Ren'Py, one for each distribution channel, but I've had no luck setting things up so far. Networkx requires setuptools, setuptools itself seems to require some compilation.

I might figure it out, given some time, so I'll wait a little on getting into the specifics. Just thought I'd update - thanks for the suggestion


As for Lua, the question is here: viewtopic.php?f=32&t=33923

Whether it's black magic.. well, through no fault of my own, I was hornswoggled into some web development recently, and naively tried to combine rails with jquery mobile by hand, without really knowing what I was doing except that I didn't want to mess around with fine-tuning css.

Everything else looks so nicely predictable now. When things fail, they continue failing. If they succeed, they continue succeeding.
It's an underappreciated trait, really :>

[xela]

Networkx is pure python I think. Unfortunately it requires some sort of installing, too. I localized python executables in Ren'Py, one for each distribution channel, but I've had no luck setting things up so far. Networkx requires setuptools, setuptools itself seems to require some compilation.

I doubt it... it "can" be installed through egg but you never really have to, especially in Ren'Py. Ask the Good, Old Doc about installing python modules. You should be able to download networkx (not the egg, the one with the soure), throw it is the folder mentioned by the Doc (our, Ren'Py Doc) and:

Code: Select all

init python:
    import networkx

and you should be done. Doesn't look like they have any C dependencies. Maybe you'll have to take some other steps as well, I usually manage to import those modules with little effort.

Hmm... also keep stuff like this in mind:

https://github.com/renpy/renpy/issues/701

It may not be relevant to your module at all but if it is, it can save you loads of time

[fluxus]

Actually I just found out about the egg option and got all excited, but I still managed to make it 'not work'.

About the doc.. I've already searched the Ren'Py doc for information on how to import libraries (my first step, that and extensive googling).. but yes, admittedly I was skimming for 'likely sections'.
I'll have another go at it.

[fluxus]

https://github.com/renpy/renpy/issues/701

Okay thanks.. and yes, that one *would* have stumped me :]

[fluxus]

Actually, something I did worked.

I'd copied networkx into the /game directory, and that worked. Except that it failed, which let me on to the notion that I needed some sort of compilation, especially since what I'd done wasn't any sort of recommended install method.

I've managed to find a reference to game/python-packages now, too, and that one also works. And fails. Same thing happened when using pip, which -is- a recommended install method as I've found out, and it still fails.

Specifically, it complains about lacking the uuid and fractions module. I'm not really sure, but maybe it's missing access to the standard library or something.

[fluxus]

https://www.renpy.org/doc/html/python.html for the curious by the way.
Waaay down at the end, "First and Third Party Python Modules and Packages".

[xela]

Specifically, it complains about lacking the uuid and fractions module. I'm not really sure, but maybe it's missing access to the standard library or something.

Yeap, Ren'Py doesn't come with the full Python 2.7.10. You can find the module files in official distribution and copy them into renpy/your project.

[fluxus]

Yeap, Ren'Py doesn't come with the full Python 2.7.10.

Well, that explains it :]

You can find the module files in official distribution and copy them into renpy/your project.

Does this mean getting 2.7.10, installing it, then copying /module from python to renpy[platform]/lib?
I assume the c files from source won't help much alone.
Also.. this needs to be done for each platform?

Or did I miss an Obviously Easier Way to do it?

[xela]

Yes. Just the modules you need. I don't know much about other platforms, I always thought same Python was used everywhere but I never looked into it too deeply. This seems obvious and easy, if those modules have c dependencies, you're back to the same problem

Sometimes you can rewrite the module, I did that for SimPy, it had some dependency which was barely in use.

[fluxus]

I don't know how much trouble it will be or whether there's c dependencies or not - I don't know much about the internal workings of python, I just searched around for a while, found out that the standard python modules seemed to be placed in /modules, and looked what was there.

In the source version, it's a lot of files, mostly c. And it wasn't immidiately apparent what was what, so I tried comparing renpy's [platform]/lib directory with it, to see if there was overlap; a positive indicator if there was.
I didn't spot an immediate overlap, and so I began to question the magnitude of this undertaking and wanted some confirmation that I wasn't off on some tangent (which happens).

I'm sort of wondering if pip can't install the standard library to renpy somehow. It would seem I'd just need to figure out the name of 'python standard library' if it exists separate of python. It *would* make things much easier.

Interesting with SimPy. Agent-based modeling is the next thing I want - the idea is to have an autogenerated map kept in the network structure and then have agents as population to keep it alive. If it's manageable it should make for a highly immersive environment.

[xela]

Keep in mind that SimPy is based around generators and you're not allowed to allow generators to survive between Ren'Py statements if you want to use Ren'Py's saving/loading system.

Both uuid and fractions are in Python27/Lib module. I don't know if they have c dependencies, prolly not...

[fluxus]

TLDR is that I copied python 2.7.11's modules and no longer get complains about modules missing.
On the other hand it complains about unicode.
Which the guy looking to install lua also seemed to encounter.

That wasn't what I did first though.
I looked around for that module name, and couldn't find one.
It looks as if the 'standard library' comes from a series of modules and that they're only bundled up together when delivered as part of a general python distribution.
So using pip does not seem like a straighforward path.

Then I thought I should install python 2.7.10 and copy the files from there, like you said.
Only I wanted to preserve my system python and got sidetracked trying to figure out how virtualenv is supposed to be used, before finally discovering that it can only use system pythons for binaries anyway, it's only the packages it can keep contained apparantly.

So, having system-installed 2.7.11 (it can't find .10) I now get unicode-complaints.

Code: Select all

While running game code:
  File "renpy/common/00updater.rpy", line 23, in script
    init -1500 python in updater:
  File "renpy/common/00updater.rpy", line 27, in <module>
    import tarfile
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 210: invalid continuation byte

It's kind of reminiscent of the lua thread (Adding "other" libraries to Ren'Py), so I presume the issue is the same.

You need to link against a wide unicode (UCS4) build of Python, which is what Ren'Py ships with. You built against a narrow build, which is why you have this problem.

I guess I'll try compiling python 2.7.10 from source next time, specifying --enable-unicode=ucs4.

----------------------

Keep in mind that SimPy is based around generators and you're not allowed to allow generators to survive between Ren'Py statements if you want to use Ren'Py's saving/loading system.

I'm not sure what you mean by generators. Each agent-object generates another agent-object each step of the simulation?
Why is that a problem? As far as I recall, global python variables are kept, and others appears to be scoped for the scene/session one instantiates them in.
They aren't saved when saving the game, of course. Not automatically.
But there's a ren'py class somewhere to inherit from that would work? Or so I thought.

[xela]

----------------------

Keep in mind that SimPy is based around generators and you're not allowed to allow generators to survive between Ren'Py statements if you want to use Ren'Py's saving/loading system.

I'm not sure what you mean by generators. Each agent-object generates another agent-object each step of the simulation?
Why is that a problem? As far as I recall, global python variables are kept, and others appears to be scoped for the scene/session one instantiates them in.

It's fairly straight forward: https://www.python.org/dev/peps/pep-0255/

They aren't saved when saving the game, of course. Not automatically.
But there's a ren'py class somewhere to inherit from that would work? Or so I thought.

If you keep generators alive between statements, Ren'Py will try to pickle them as it saves game state which is impossible. It may be fixable somehow, I've seen people writing special methods for their classes that controlled what gets pickled and how, it's not something I've tried here. But it's not importing store to your module and trying to inherit from Ren'Py's implementation of object which you are most likely referring to by "there's a ren'py class somewhere to inherit from that would work", that will not do anything meaningful to mitigate this issue.