(Solved) Compiling Python as C -> .os / .dll, and using in Ren'Py?

Discuss how to use the Ren'Py engine to create visual novels and story-based games. New releases are announced in this section.
Forum rules
This is the right place for Ren'Py help. Please ask one question per thread, use a descriptive subject like 'NotFound error in option.rpy' , and include all the relevant information - especially any relevant code and traceback messages. Use the code tag to format scripts.
Post Reply
Message
Author
User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

(Solved) Compiling Python as C -> .os / .dll, and using in Ren'Py?

#1 Post by SypherZent » Sun Aug 23, 2020 5:24 pm

Marqin wrote:
Sat Oct 22, 2016 9:16 am

There is one half-good way for protecting game sources (but it won't protect strings, like text told by characters)

1. Write all your VN logic as python module
2. Import that and use in your VN (documentation)
3. Use Cython or Nuitka to compile that python module to C code
4. For each platform/operating system you release, compile that C into .so/.dll and package it with your game

That way it won't be easy(but still possible) to steal your scripts (eg. minigames/logic),
because for that someone will have to know assembly and python's C api calls.
Hi, I'm curious about what user Margin posted above in another thread.

I'm looking to protect source code from .rpa extractors on Github.
Nowadays, pretty much anyone without coding knowledge can just extract any Ren'Py game and see the source code.
I feel it's this type of individual who would be keen on stealing a game's source code, swapping assets, and releasing a clone.
A real hardcore hacker who knows "assembly and Python's C api calls," could easily make their own games.
So, I'm more concerned about the casuals who would be eager to steal.

Since the games I make are not Visual Novels, they are heavy in terms of mechanics.
I would like to protect the source from being stolen and reused in other games.

Note that it is much easier to protect images, and text, in terms of pursuing copyright infringement in a court of law.
They are easily recognizable even by gamers.

It's not so easy to prove that someone stole source code. Therefore it's up to us as developers to protect our source code. Unfortunately, Ren'Py doesn't provide tools for us to do this. We cannot easily choose a unique encryption key, for instance, that would render .rpa extractors on GitHub useless.

This is why Margin's post above caught my attention.

I have a few questions regarding Margin's proposed methodology:

1) For Windows: Is it as simple as using 'import (module_name)' in an init python block in a Ren'Py script, or do I need to do anything special like using 'ctypes' to load the DLL into memory once the code is in .dll format?

2) Mac OS X doesn't use DLLs. However, .so should work for both Linux and Mac. Same question as above: Do I need to do anything special in the code itself to load the protected functions into memory once the file is in .so format, or is it as simple as using 'import (module_name)' in an init python block in a Ren'Py script?

3) IF it's not as simple as using 'import (module_name)' in an init python block, then can you provide specific examples of how I can use the .so / .dll for Windows / Linux / Mac?

4) Android is packaged as a .apk. Is it as easy to unpackage a .apk file to view source code as it is to unpackage .rpa files using extractors on GitHub? If so, how do I protect a .apk using the .dll method (or equivalent of a .dll for Android, if such a thing exists)?

I'm not really interested in releasing for iPhones. However, I am interested in releasing for mac OS X, as per question #2.

Any help with this is really appreciated.

Edit: I already know that nothing is 100% hack-proof. My goal is to deter people who only know how to use Ren'Py extractors from GitHub to access the source code. I feel this would prevent at least 80% of potential casual assailants, while allowing me to protect my game through softer DRM measures, such as constant updates. If it takes a lot of work to crack it every time, this would ultimately deter someone from cracking every updated version of my product, and it would allow me to grow my playerbase before a cloned copy hits the market. I am not worried about cracked versions of my game. I am worried about someone easily stealing my source code and using it in their own game. At least if it is difficult enough that a versatile coder is needed, such a person could already make their own games and wouldn't have to steal code (wouldn't want to steal code, as generally such skilled people take pride in their skills). I hope this clarifies my intent.
Last edited by SypherZent on Mon Aug 24, 2020 8:47 pm, edited 1 time in total.

User avatar
rayminator
Miko-Class Veteran
Posts: 541
Joined: Fri Feb 09, 2018 12:05 am
Projects: Redoing games to fix spelling mistakes other companys that don't care
Location: Canada
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#2 Post by rayminator » Sun Aug 23, 2020 8:42 pm

renpy only uses rpa at the moment maybe in the next couple of updates who knows it might happen or it might not but if you really want to protect your projects then I recommend to use unity instead but hacker will always find a way
when I try to help someone I point them into 2 direction what's right and wrong so they can learn how to

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#3 Post by SypherZent » Sun Aug 23, 2020 10:03 pm

Margin's post applies to Python, he's not discussing Unity.
I already know that one can import Python modules for use in Ren'Py.
I'm looking for someone who knows this stuff before I hire a friend who knows Python like the back of his hand to modify Ren'Py source for me.
I am hoping for an easier way.

I know all games can be hacked. I'm more concerned about people who don't know how to hack, who only know how to download .rpa extractors and steal other people's code and change the image assets to release cloned games. I'm concerned about casuals who steal code, not hackers. I said this in the initial post.

If you don't know about it, you don't have to answer.
And rayminator, not for nothing but, this is the 3rd time that you've commented on my posts here with 'wrong' answers.
Every time you comment, the information is inaccurate, and I end up finding a solution where you said it was not possible, or you gave a totally wrong answer.
If you can't help, or if you know nothing about the topic, you don't have to post.
Posting a non-answer doesn't help anyone and it just wastes all of our time.
I actually feel bad for other people you may be doing this to, who might find actual answers to their problems if you didn't lead them in a wrong direction.

User avatar
uyjulian
Regular
Posts: 114
Joined: Sun Mar 08, 2015 1:40 pm
Github: uyjulian
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#4 Post by uyjulian » Sun Aug 23, 2020 11:49 pm

If you are looking for an easy way to prevent currently-released archive unpackers from unpacking your game, see the following: viewtopic.php?f=51&t=59725

If you really want to compile your Python code:
The compiled python file (.pyo) can still be decompiled with uncompyle6. However, you can convert python to C code by using Nutika.
From there, you can compile the C code to each platform. You will need to compile to different platform, so you can't share the same compiled code with different platforms.
The compiled C file (.pyd) can still be decompiled with Ghidra.

There becomes a point where it is just a waste of time to implement junk code and reduces the number of platforms the project can run on, so I suggest just looking at the link ^.
Rule of thumb is if it takes you 10000 seconds to implement the junk code, other people can take 1000 seconds to reverse the junk code.

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#5 Post by SypherZent » Mon Aug 24, 2020 12:15 am

Thank you, uyjulian, this is more like it! I remember attempting this once, a couple of years ago, and I managed to get it working only for Windows.
Does your method work for Mac OS X, Linux, and Android without problems? Honestly, I may have modified the loader.py and archive.rpy files wrong back then.

Also, Andredron replied in that thread that it is possible to make Ren'Py 'rewrite' all of the .rpy files using renpy.list_files and renpy.file.
Is it safe to remove those functions from Ren'Py source code that you know of, or are they used internally after distribution is built?

Finally, I understand that it is possible to compile the Python code to C. I also understand that the resulting C code can be compiled to .dll (for Windows) and .so (for Mac OS X and Linux). I'm curious if it would be necessary to do anything other than 'import (module_name)' inside of the .rpy file once that Python code is no longer a .py file. Any ideas there? From what I have seen (in the links below) it seems I may have to do extra coding to retrieve the functions from the .dll for usage in the .rpy files, but I'm not entirely sure if it's that complicated or if a simple 'import (module_name)' would still work.

Using a .DLL from Python (Windows): https://stackoverflow.com/questions/252 ... rom-python
Using a .so from Python (Mac OS X): https://stackoverflow.com/questions/248 ... lib-on-osx

Thanks for the link, too! At least that is one step in the right direction, IMHO. ^^
Would still like to figure out the .dll and .so method as well, if possible.

User avatar
uyjulian
Regular
Posts: 114
Joined: Sun Mar 08, 2015 1:40 pm
Github: uyjulian
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#6 Post by uyjulian » Mon Aug 24, 2020 12:21 am

Yes, it works for all platforms.
Ren'py uses list_files and file internally, so you will need to rename occurrences of those functions instead of deleting them.
You can just place the Python compiled extension in the search directory (look for where the .pyd files are placed. Put your compiled file there)
For compiling C to all supported platforms, you will need SDKs for Windows, macOS, Linux, iOS, Android, and Emscripten, which will take up multiple gigabytes.

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#7 Post by SypherZent » Mon Aug 24, 2020 1:32 am

Rename seems reasonable. Excellent, thanks.
Ok I have some SDKs, but if I need to redistribute them and raise the file size of the final product, then maybe that's not the ideal solution.
I may test it out and see. I'll leave this open in case anyone else wants to chime in before I actually start implementing this (roughly between Sept 7th and Sept 14th.)

Thanks again, uyjulian!

ichfly
Newbie
Posts: 14
Joined: Sat Aug 15, 2020 12:40 pm
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#8 Post by ichfly » Mon Aug 24, 2020 12:33 pm

There are mutable way interact with a dll or so.

First as already stated ctypes.
Second you can make a extension that can be directly loaded with import and then used like a normal python class. However the C part is harder to program https://github.com/lexygon/python-c-ext ... lculator.c <-- example of simple function like addition. I don't know if python2c compiler create the matching functions or if you have to do it yourself (Cython dose create them if I remember correct).
Third running a child processors and communicating via pipes I used it here https://github.com/ichfly/renpy-Asset-Protect (a project to protect images from being extracted out of renpy projects) however I do not recommend that.

If you want to protect the rpy file you can additionally:

Change the obfuscation key for the Archives in loader.py in "def index_archives():" around "key = int(l[25:33], 16)"
Same in the creator tools/archiver.py around "def archive(prefix, files):" "key = random.randint(0, 0x7ffffffe)" or
launcher/game/archiver.rpy "self.key = 0x42424242" "self.f.write(b"RPA-3.0 %016x %08x\n" % (indexoff, self.key))"


You can also disallow loading not packed files
renpy/config.py "force_archives = False" <-- set it to True

All the above options are easily bypassed however still better than being dump able without any adaptation.

Also if you want to protect your code from being analyzed by other you can use a python/C obfuscator (however python obfuscator are not very effective and some of them cost money)
If you compile them to C you don't use an obfuscator it is a good idea to strip the debug information from the files. Else most de-compiler could restore the names of the function if they are removed they can not. However the python2c compiler may preserve the variable name and/or the function names anyway.

For apk you need 2 tools instead of 1 to extract the data.

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#9 Post by SypherZent » Mon Aug 24, 2020 4:26 pm

Thank you ichfly.

I did some testing. I import math a lot in my scripts. I decided to look at how math is built, because I just use 'import math' and that's really simple.
I discovered that I can create my files as third-party Python packages and simply use 'import (module_name)'.
From there, it would only be a matter of compiling as C, then turning into .so or .dll files.
The math module is a .so file in darwin lib and linux lib internally, and I believe it is within a .dll in the windows lib within renpy folders.
So, I want to package my files the same way, as external libraries or as third-party Python packages.

I got 90% of the way.
However, I use Ren'Py ATL in my functions.
I do not know how to import them into the third-party Python file.
I already figured out how to import the following:

Code: Select all

from renpy.display.screen import show_screen, hide_screen
from renpy.exports import restart_interaction
from store import persistent
from store import store
^ This allows me to use information and functions that are internally built into Ren'Py.

However, I do not know how to import custom functions I have created in .rpy files.
I also do not know how to import ATL transforms.

I can get around the ATL transforms if I can learn how to import custom functions I created in .rpy files.
Since Ren'Py interprets those functions at initialization, it stores them somewhere inside and I don't know how to retrieve them within a third-party package.

So far, I like this method because I do not have to alter Ren'Py source, and even if they unpackage with a .rpa extractor, my files would remain as .dll and .so files, which are a bit harder to open. If someone 'steals' those (i.e. not copy/pasting the code, but stealing the entire files for their games), it's easier to prove in a court of law, and it's easy for a DRM like Steam to identify these copied files in other games in order to remove them from the marketplace after I deliver Cease & Desist notifications and report any potential offenders.

Hopefully they will be even more protected once I sign the files with an official Microsoft validated certificate.

In any case, I need to learn how to import custom functions. This is the last hurdle. I already tested my file as a third-party Python package and it works.
Converting the Python to C is trivial. Packaging the C as a .dll or a .so should also be simple. Testing on a Mac / Linux / Windows will take some time, but hopefully it will work just like the 'math' module (which already works on all platforms) if I can package it the same way.

So, if anyone can help me to learn how to import custom functions from .rpy files into a third-party Python package, I think that's the last help that I'll need to get this protection method set up before my game launches.

Thanks again for all of the input thus far. ^^

drKlauz
Veteran
Posts: 217
Joined: Mon Oct 12, 2015 3:04 pm
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#10 Post by drKlauz » Mon Aug 24, 2020 4:50 pm

No one will steal your code. People sometimes steal art, very rarely they even reuse it to release another game. Code is pointless to steal, if thief know how to steal he probably code it faster himself, if thief don't know how to code himself then he fail to steal/reuse properly.

Someone have to say it, sorry.

To keep on topic tho(not sure how useful it is, but still). There are ways to make reverse engineering much more problematic, but RenPy by nature is bad candidate for secure software.
With python you can play with internal bootloader, disabling/removing all import loaders except for builtins and inject your custom py/pyc loader. This will disable easy code injections.
You can rework marshal format used to save/load pyc files. This will kill most if not all automatic python decompilers.
You can recompile python (and libs used) with randomly shuffled opcodes, this likely will kill rest of [semi-]automatic decompilers.

Realistically only reasonable way to keep game/software logic safe is keep it on server and deliver to client only results of logic work. If you use short hashes instead of meaningful filenames/urls you game is safe enough.

About calling python/renpy functions from native code, unless there was very limited number of called functions i used some sort of bridge function which was called from native code, eval'ed/exec'ed supplied string in appropriate scope and returned value. Less secure, but flexible.

Generally attempts to hide/obfuscate python code is fruitless journey ending in some dead end, often with lots of issues for actual code you tried to protect. Spend time on something more useful. Play Skyrim maybe.
I may be available for hire, check my thread: viewtopic.php?f=66&t=51350

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#11 Post by SypherZent » Mon Aug 24, 2020 5:21 pm

Sorry drKlauz, but you're wrong about a thief failing to steal/reuse properly.
I've coded my game to be moddable for Steam Workshop integration.
It means anyone can just swap assets and the code will work perfectly.
I also have an A.I. that generates content in my game, so just changing the pool of images will allow it to create new content.
It's very possible for my code to be stolen. I need to protect my source files.
You really cannot make a claim like the one you've made without actually seeing the code.
I'm not creating a Visual Novel here, I'm creating a CCG Roguelike Dungeon Crawler in the vein of Slay the Spire. 90% of the content is Gameplay.
Unlike a Visual Novel, which relies on Art and Story, my game relies on Mechanics which can be easily stolen with just simple image swaps.

Back on topic: The rest of the information you've provided seems like you did not read all of the posts in the thread.
I already learned how to package my files as external Python modules (the way modules like 'math' are written in C and included with Python internally).
I want to include my files in the same manner. I need to learn how to import custom functions I've created in Ren'Py scripts to the external module.

I already ran tests and successfully imported the external module. I just do not know how to retrieve data that Ren'Py loads into itself upon initialization (i.e. the custom functions I create within .rpy scripts) in order to call them from the external module.

Basically 90% of the problem is solved. The final 10% will probably require the help of a Ren'Py developer who knows the engine very well and knows where it stores its data and how to retrieve that data from third party scripts.

If I can't get help with this, I may have to rewrite my code a bit more than anticipated, but at least I have a working solution to protect a good portion of my code. At this stage I'm trying to avoid a 30% recode, and bring it down to something like 10%, if I can figure out how to call functions from .rpy scripts inside of the Python packaged module. If I can't figure it out, I'll just rewrite my code to avoid the need to call the functions from the module (i.e. turn a two-way street into a one-way street where I have functions in .rpy that simply use the module and the module never needs to use the functions from my .rpy files).

User avatar
uyjulian
Regular
Posts: 114
Joined: Sun Mar 08, 2015 1:40 pm
Github: uyjulian
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#12 Post by uyjulian » Mon Aug 24, 2020 7:07 pm

Override renpy.file to return a buffer on the rpyc inside the C code converted by xxd.

Also, if you don't get the point…

There becomes a point where it is just a waste of time to implement junk code and reduces the number of platforms the project can run on, so I suggest just following the loader.py modification I mentioned before.
Rule of thumb is if it takes you 10 hours to implement the junk code, other people can take 2 hours to reverse the junk code.

There becomes a point where it is just a waste of time to implement junk code and reduces the number of platforms the project can run on, so I suggest just following the loader.py modification I mentioned before.
Rule of thumb is if it takes you 10 hours to implement the junk code, other people can take 2 hours to reverse the junk code.



If you want a turnkey solution that is known to work to protect modification of your code (Anti DRM cracking etc) for the first few weeks of sales, look at VMProtect, Easy Anti-Cheat, or Denovo.

User avatar
SypherZent
Veteran
Posts: 319
Joined: Fri Sep 02, 2016 3:14 am
Completed: Multiverse Heroes, Space Hamster in Turmoil
Soundcloud: Chrysopoeist
Location: Puerto Rico
Contact:

Re: Compiling Python as C -> .os / .dll, and using in Ren'Py

#13 Post by SypherZent » Mon Aug 24, 2020 8:46 pm

Thanks uyjulian. As I have mentioned before, I'm mostly concerned about people who wouldn't know how to do those 2 hours to reverse anything.

The main demographic to protect against are:
1) People who only know how to use .rpa extractors to decode existing games, who don't know how to access the files if that fails.
2) People who only know how to swap images and use other people's code that they have extracted, to release "their own" games.

Any other preventative measures I implement would be simply to help identify the files in question so that I can fully pursue copyright claims (i.e. someone stealing a .dll without modifying it, to use it in their own game, or stealing a whole code file and using it in their own game). If it is easier for Steam or Epic Games or whatever distribution outlet, to identify other games that have my source files in them, then all the better.

I hope this is clear enough, of what I want to protect against.

As for third-party protection, I don't want to force my players to have to be online when playing the game, since it is single player. Such server-based protection tends to require online connectivity for players to be able to play the game. Even though Steam is DRM, it's very light and people can still play offline (and can still boot the game from steamapps folder without using Steam itself) so I prefer using Steam. Perhaps Denuvo may work for offline titles, but I'll have to pursue that post-launch during Early Access, as I'm sure it will cost a bit more than I'm able to invest in protection at this time (money is currently tied by the initial marketing budget).

PyTom actually got in touch with me and shared some insight as to how Nintendo did copyright protection, and brought some other ideas into the mix, so I have a few resources at my disposal now to help secure the game files and to help ensure that I can pursue copyright infringement claims if my scripts do get stolen and used in other people's games. So, I will label the topic as closed since I now have more than enough answers and methods of obfuscation. I should be able to release something that at least a non-coder won't be able to tamper.

Thanks again to everyone who has shared information here! I really appreciate it.

Post Reply

Who is online

Users browsing this forum: Imperf3kt, Neyunse