[Tutorial] Expanding Mobile Functionality With Pyobjus/Pyjnius (Notifications)

A place for Ren'Py tutorials and reusable Ren'Py code.
Forum rules
Do not post questions here!

This forum is for example code you want to show other people. Ren'Py questions should be asked in the Ren'Py Questions and Announcements forum.
Post Reply
Message
Author
User avatar
storykween
Regular
Posts: 154
Joined: Mon Sep 30, 2013 1:17 pm
Completed: Serafina's Saga, Quantum Conscience, Echoes of the Fey, miraclr - Divine Dating Sim
Organization: Woodsy Studio
Location: St Louis
Contact:

[Tutorial] Expanding Mobile Functionality With Pyobjus/Pyjnius (Notifications)

#1 Post by storykween »

Our latest visual novel, miraclr: Divine Dating Sim, was designed from the ground up with mobile devices in mind. miraclr takes place in real time and missed choice opportunities can make certain endings hard or impossible to get. With those kinds of stakes, we realized we needed to--at the very least--schedule alarms or notifications to tell players when these key moments are available in game. That functionality isn't native to Ren'Py but is available in Android Studio via Java and Xcode via Objective C, so we had to figure out a way to put those functions in and call them/pass arguments to them in our Ren'Py scripts.

We used this to set notifications/alarms but theoretically it could be used to utilize all sorts of other functionality for mobile Ren'Py games--calendar, file system, facebook integration, etc--so I thought I'd write up our process for notifications to help out anyone else who might want to use more mobile functions in their games. We also used this system to create admob ads that we could hide/display through Ren'Py, which I'll try to add to this post later.
Keep in mind I'm largely self-taught so this might not be the ideal/most efficient way to make this work but it has been effective for us.

First step: import the necessary libraries

When we were just starting out, we got waaaaay ahead of ourselves with adding functions and trying to call them within Ren'Py and nothing worked because we weren't adding the necessary libraries in the right part of our script. Here's the code we ended up putting in an init python block:

Code: Select all

init python: 
    if renpy.android:  #if we're on android
        import jnius   #import jnius so we can use its functions to call to java
        PythonSDLActivity = jnius.autoclass("org.renpy.android.PythonSDLActivity")  #class of functions that will call functions in our java
    if renpy.ios: #if we're on ios
        import pyobjus #import pyobjus so we can use its functions to call to Objective C
        NotificationManager = pyobjus.autoclass(b"NotificationManager") #Define NotificationManager as the Objective C Class NotificationManager
        MyNotificationManager = NotificationManager.alloc().init() #Create the Notification Manager MyNotificationManager
As you can see, the process is a little different for iOS and Android. This is because we are going to put our Java functions in PythonSDLActivity, an already existing activity that will be created when we use the Android export function. This is the main activity for the Android version of the game and it is always created.

Your iOS version is built a bit differently and there's no good automatically created class where we can put our new functions. This is a brand new class you'll have to define in Xcode, which means we have to tell our game to create it before we can use it. We called ours NotificationManager because, well, it manages notifications. But you can replace that with whatever iOS functions you plan to use it for.

(As a result of this, if you build now your Android version will be fine but your iOS version will break because PythonSDLActivity is defined and Notification Manager is not. For that reason, I'm going to talk about the iOS version next)

Build out your project in Xcode or update your Xcode project and open it up. It’s time to create your new class (NotificationManager or whatever you want).
Once you have Xcode open, you’ll want to check online or in the documentation to figure out the lowest version of iOS your desired features will work on. Then go to your project deployment settings (click on the project name in the top bar hierarchy) and change the deployment target to that version of iOS. For alarms/local notifications, we had to change our deployment target to iOS X/10.

Next, right click over in your content browser and select “New File…”. Select “Objective-C file” and click “next”. Change the file type to “empty file” and name your new class. Your name should be identical to the pyobjus autoclave you referenced in Ren’Py above. In our case, we named it “NotificationManager”. Click next, and make sure you’re adding it to your project, then click “Create”.

A new objective-c class will show up in your content browser with the name you gave it. Click on it and you’ll find that it’s pretty barebones. It will look something like:

Code: Select all

//
//  NotificationManager.m
//  miraclr
//
//  Created by Woodsy Studio on 8/31/17.
//  Copyright © 2017 Woodsy Studio. All rights reserved.
//

#import <Foundation/Foundation.h>
With just this in place, your iOS game will run again because “NotificationManager” is now a defined class. But it can’t do anything. It’s just a blank object. Now we need to add functionality, which takes three steps

Import your headers
Depending on what you want your new class to do, you may need to import headers beyond Foundation.h, which only contains the base level of app functionality. For local notifications, we added:

Code: Select all

#import <UserNotifications/UserNotifications.h> //imports user notification functionality
Before you can even get here, you may need to import frameworks. You don’t need this for alarms/notifications, so I won’t cover that here. But when we get to Admob ads in a later post, I’ll go over that. Right now, this is the only header we need, so we move on to

Declare your object, variables, and methods

Before we can use our object, we have to declare its properties. This includes any local variables we want it to have and any methods (functions) we want to invoke. For alarms, we need three different methods. The first method gets permission from the player to set notifications in the first place. We’ll call this NotificationApprove. This only has to be called once, but it must be called so we need to create it. The second method we need creates and sets the alarm. We’ll call this NotificationText The third we need will cancel the alarm, NotificationClear.

I won’t go into the full explanation of how notifications work in Objective-C because, honestly, I’d probably bungle the detailed explanation. But I will list the variables we need to declare to make the aforementioned methods work: We need a UNMutableNotificationContent, a UNTimeIntervalNotificationTrigger, a UNNotificationRequest, and a UNUserNotificationCenter. If you’ve correctly imported the header above, Xcode will recognize these all as valid variable types. Declare them all with easy to remember names. Since they’re local variables they can be relatively simple. We called these variables “content”, “trigger”, “request”, and “center”.

Here’s the code we used to declare our variables and methods:

Code: Select all

@interface NotificationManager : NSObject //defines NotificationManager as an object
//declare our variables
@property UNMutableNotificationContent *content;
@property UNTimeIntervalNotificationTrigger *trigger;
@property UNNotificationRequest *request;
@property UNUserNotificationCenter *center;


//declare our methods
-(void) NotificationApprove;
-(void) NotificationText:(NSString*)text andNumber:(NSString*)index andTime:(int)time;
-(void) NotificationClear:(NSString*)index;

@end
A few notes:
Because of how Objective C works with methods that take multiple arguments, NotifcationText actually becomes something like NotificationText:andNumber:andTime: and this is important to remember when we go back to Ren’Py to actually call it.

Because of how notification/alarm identifiers work for iOS, we’re using an NSString instead of an integer to label the index of the notification. I know that’s confusing, but it’s a necessary evil if we want to use the same Ren’Py function for both Android and iOS. Android indexes are integers and iOS indexes are strings. It’s easier to go from int -> String so here we are.

Don’t forget the @end when you’re done declaring your methods. Simply going into the next part is not enough. Which brings us to:

Implement your methods

Above, we declared our methods. Now we have to actually make them do something. This will vary depending on what you want to do with this class, but here’s our code for notifications. I won’t go over this in full because you may be using this for something completely different and also, again, I’m probably not the best person to explain each step. If you have any questions, let me know and I’ll try to answer as best as possible.

Code: Select all

@implementation NotificationManager

//implement our methods

//Implement a method for asking the player whether they will accept notifications from the game
-(void) NotificationApprove {
    UNUserNotificationCenter* center = [UNUserNotificationCenter currentNotificationCenter];
    [center requestAuthorizationWithOptions:(UNAuthorizationOptionAlert + UNAuthorizationOptionSound)
                          completionHandler:^(BOOL granted, NSError * _Nullable error){}];
}

//Create a method that takes notification text, index, and time and set it
-(void) NotificationText:(NSString*)text andNumber:(NSString*)index andTime:(int)time {
    //initialize the content of your notification
    UNMutableNotificationContent* content = [[UNMutableNotificationContent alloc]init];
    //set notification title
    content.title = [NSString localizedUserNotificationStringForKey:@"New miraclr activity" arguments:nil];
    //set notification text to be the argument we passed to this method as "text"
    content.body = [NSString localizedUserNotificationStringForKey:text arguments:nil];
    //set sound to play when notification goes off
    content.sound = [UNNotificationSound defaultSound];
    //set the time the notification will trigger based on the argument we passed as "time", minutes from now
    UNTimeIntervalNotificationTrigger* trigger = [UNTimeIntervalNotificationTrigger triggerWithTimeInterval:time*60 repeats:NO];
    //create a notification request which will pass the content to the notification center
    UNNotificationRequest* request = [UNNotificationRequest requestWithIdentifier:index content:content trigger:trigger];
    //create the notification center and add the request
    UNUserNotificationCenter* center = [UNUserNotificationCenter currentNotificationCenter];
    [center addNotificationRequest:request withCompletionHandler:nil];
    
    
    

}

//implement a method that will clear a notification based on its already given index
-(void) NotificationClear:(NSString*)index {
    //gets the user notification center
    UNUserNotificationCenter* center = [UNUserNotificationCenter currentNotificationCenter];
    //creates an array of identifiers that includes the index we passed to it
    NSArray *array = [NSArray arrayWithObjects:index, nil];
    //removes all pending notifications that have an identifier included in the array
    [center removePendingNotificationRequestsWithIdentifiers:array];
    
}

@end
Notes:
Always begin this section with @implementation [classname] and end with @end
Make sure you implement every method you declared in your interface section
Make sure that your arguments match. If you declare NotificationText as taking a string, you have to implement it taking a string
Feel free to just copy/paste this for your notifications if I’ve totally lost you, just make sure to change “New miraclr activity” (which is the title of every notification this will create) to something more appropriate for your game!
You can set UNNotificationSound to something other than the default notification sound

Back to Ren’Py

Okay! Now we have a fully functional Objective C class that can ask for player approval, create notifications, and cancel them. But it’s locked in our Xcode project, which we can’t touch. Right? Wrong! This is where Pyobjus comes in. It lets us use all the functions we just defined in Xcode in our Ren’Py script.
That code that we stuck in MyInit all the way back at the beginning of this tutorial creates a NotificationManager class that you can call in Ren’Py which we named MyNotificationManager. All of its functions are available and conveniently translated to a format that won’t make Python unhappy. They are as follows:

Code: Select all

$ MyNotificationManager.NotificationApprove()
$ MyNotificationManager.NotificationText_andNumber_andTime_(message,textindex,minutes)
$ MyNotificationManager.NotificationClear_(textindex)
Notes:
Those underscores in the last couple of functions are important. The way Pyobjus translates messy Objective C methods sticks underscores after every argument passed to the method, even when it’s only one like NotificationClear.

NotificationApprove doesn’t have any arguments so no underscore

Run $ MyNotificationManager.NotificationApprove() ASAP in your game or none of the others will work

Put the following anywhere in your script to set a notification to go off in half an hour with the text “You have a private message from Raphael”
$ MyNotificationManager.NotificationText_andNumber_andTime_(“You have a private message from Raphael”,1,30)

Put the following to cancel that notification at any time before it has gone off
$ MyNotificationManager.NotificationClear_(1)

Again, this could be used to trigger any kinds of iOS functions you define in your Objective C class and let you pass arguments from Ren’Py to the class. Later, I’ll make a post about how to do the exact same thing to create a Display Ads and Hide Ads function.

What about Android?
Android is going to take a bit more preparation to get started, since typically Ren'Py handles the full conversion into an apk rather than into a Android Studio project.

First of all, download the latest version of Android Studio and through the Android SDK manager download the latest SDK, Google Play Services, and the Google Repository.

Now go to Ren'Py and export your apk just like you normally would. Once it's done, go to your Renpy directory and find the folder labeled “rapt”. This is basically where Ren'Py puts everything for your game right before turning it into an APK. We're going to use this to create an android studio project.

Copy the following out of the rapt folder into a new folder: android-sdk(version number), assets, extras, gen, libs, res, src, templates along with the following files: AndroidManifest.xml, build.xml, local.properties, proguard-project.txt, project.properties. Name this folder something you'll recognize as your android project; we called ours “miraclr for android”.

Now open up Android Studio and select Import Project(Eclipse ADT, Gradle,etc.). You will be asked to select your project folder. This is the folder we just created. Now you'll name your new directory and click next. Checkmark “Replace jars with dependencies”, “Replace library sources with dependencies” and “Create Gradle-style module names” and click finish. Android Studio will now process your folder into a new project.

From now on, you'll be building your apk in Android Studio instead of Ren'Py. You can do this in the drop down menu Build → Build APK or Build → Generate signed APK. For a signed APK you'll have to set up your keystore file yourself rather than have Ren'Py do it automatically. Your keystore file is in the rapt directory we visited earlier.

Now, unlike Xcode, you're not going to be able to touch your Ren'Py scripts in Android Studio. Xcode could read python and let you edit them. Android Studio can't. So whenever you change your Ren'Py scripts, you'll need to build out your apk, copy your asset folder from the rapt folder, and replace the asset folder in your android studio project. The path for this is typically C:/Users/[YourUserName]/StudioProjects/[YourProjectName]/[YourProjectName]/src/main
Yes there are nested folders that both use your project's name in that file path.

Anyway, back to Android Studio. Now that we have a full Android Studio project, we can add Java functions to our game that Ren'Py will eventually be able to call. In your asset browser on the left, find PythonSDLActivity. It is located in [YourProjectName]/src/main/java/org/renpy/android This is the main activity of your game and you will put your function here. I typically stuck them after this bit of code near the end:

Code: Select all

public PowerManager.WakeLock wakeLock = null;
Once again, you can create all kinds of functionality now that you have access to Java. For the purposes of this tutorial, again, I'm going to be setting out how to do local notifications/alarms.

Local notifications were introduced in Android SDK 16, so we're going to have to change that for our project. Go to File → Project Structure and click on the module with your project name. Then click on the “flavors” tab and set the min SDK version and Target SDK version to 16. If you don't do this, Android Studio won't recognize any of the functions we're about to put in our PythonSDLActivity.

When you click “OK”, your project will re-sync and you'll be able to begin. The good news is that, after all this, Java functions are IMO easier to write than objective-c methods.

Like before, we're going to need a function that creates a notification and schedules it, and a function to cancel that notification. Android does not require the user to actively give permission for notifications, but it does require something else: an alarm receiver. So before we can begin writing our functions, we have to create that.

Go back over to your asset browser and right click near your PythonSDLActivity. Select “New” and from the dropdown menu that appears select “New Java Class”. Give your receiver a name (we literally call ours AlarmReceiver) and leave all the other options at default.

You should now have an empty Java Class called Alarm Receiver. Paste this code after package org.renpy.android;

Code: Select all

import android.app.Notification;
import android.app.NotificationManager;
import android.content.BroadcastReceiver;
import android.content.Context;
import android.content.Intent;

public class AlarmReceiver extends BroadcastReceiver {
    public static String NOTIFICATION_ID = "notification_id";
    public static String NOTIFICATION = "notification";
    @Override
    public void onReceive(final Context context, Intent intent) {
        NotificationManager notificationManager = (NotificationManager) context.getSystemService(Context.NOTIFICATION_SERVICE);
        Notification notification = intent.getParcelableExtra(NOTIFICATION);
        int notificationId = intent.getIntExtra(NOTIFICATION_ID,0);
        notificationManager.notify(notificationId,notification);
    }
}
Once again, I'm probably not the best person to explain this step-by-step, but this is stock code for creating a class that can handle the notifications we're gonna send.

Next, we need to declare our receiver in our AndroidManifest.xml. You'll notice that there are a bunch of different files called AndroidManifest.xml over in your asset browser. You want the one inside your ProjectName module. Line 4-6 should look like this:

Code: Select all

android:installLocation="auto"
android:versionCode="138"
android:versionName="1.38">
If this (with your version code/name, which is probably lower than 138/1.38) isn't right near the top, you're in the wrong AndroidManifest.xml.
Find the </application> down near the bottom of the file and paste the following before </application> but not inside any other tags:

Code: Select all

<receiver
    android:name="org.renpy.android.AlarmReceiver"
    android:process=":remote" />
If you named your receiver anything other than AlarmReceiver you'll need to change the name.

OKAY now we can finally write our Java functions! Go back to PythonSDLActivity, find a good spot between functions near the end you can remember, and set out your notification functions. Here's what we used.

Code: Select all

public void myNotification(String text, int index, long delay){
    //the actual content of the notification
    long[] vibrateTime = {100, 400, 300, 400, 500};
    Notification.Builder mBuilder =
            new Notification.Builder(this)
            .setSmallIcon(R.drawable.icon)
            .setContentTitle("New miraclr activity")
            .setContentText(text)
            .setDefaults(Notification.DEFAULT_SOUND)
            .setVibrate(vibrateTime)
                    .setAutoCancel(true)
            .setLights(Color.MAGENTA,1000,500);
    //create an intent (action) to be performed when the notification is clicked --in this case, launch this package)
    Intent i = getBaseContext().getPackageManager()
            .getLaunchIntentForPackage(getBaseContext().getPackageName());
    //package this into a new intent we can control
    Intent intent = new Intent(i);
    //create a pending intent to schedule with an alarm
    PendingIntent activity = PendingIntent.getActivity(this,index,intent,PendingIntent.FLAG_CANCEL_CURRENT);
    //set the content intent of our notification to the pending intent we just created
    mBuilder.setContentIntent(activity);
    //build the notification
    Notification notification = mBuilder.build();
    //schedule pushing the notification to the alarm receiver
    Intent notificationIntent = new Intent(this,AlarmReceiver.class);
    notificationIntent.putExtra(AlarmReceiver.NOTIFICATION_ID,index);
    notificationIntent.putExtra(AlarmReceiver.NOTIFICATION,notification);
    PendingIntent pendingIntent = PendingIntent.getBroadcast(this,index,notificationIntent,0);
    //set the time for the alarm to actually show the notification
    long futureinMillis = (SystemClock.elapsedRealtime() + (delay*60000));
    AlarmManager alarmManager = (AlarmManager) getSystemService(ALARM_SERVICE);
    alarmManager.set(AlarmManager.ELAPSED_REALTIME_WAKEUP, futureinMillis, pendingIntent);
}
You can create a function that cancels notifications by essentially using the exact same code but replacing the final line with:

Code: Select all

alarmManager.cancel(pendingIntent);
Now, how do we call this in Ren'Py? Almost exactly like we do with the iOS notification method:
$ PythonSDLActivity.myNotification(“Your notification text”, index, minutes)

Making it easier:
If you're like us, you want to have only one Ren'Py project that you use to export all your versions of the game. This means we'd like to create a Ren'Py function that we can use in our script that will set an android notification if we're on android, and an iOS notification if we're on iOS. Here's what we stuck in our functions.rpy to do that:

Code: Select all

def setJavaAlarm(index,message, minutes):
        if renpy.android:
            PythonSDLActivity.myNotification(message,index,minutes)
        if renpy.ios:
            textindex = str(index)
	    MyNotificationManager.NotificationText_andNumber_andTime_(message,textindex,minutes)
 

    def cancelJavaAlarm(index,message, newMinutes):
        if renpy.android:
            PythonSDLActivity.clearNotification(message,index,newMinutes)
        if renpy.ios:
            textindex = str(index)
            MyNotificationManager.NotificationClear_(textindex)
As you can see, this creates a setJavaAlarm that functions with both our pyobjus defined Objective-C class and our pyjnius defined Java class.

WHEW
I know this was a lot to throw out there and it might be a bit confusing. But I hope that this points some folks in the right direction and lends some assistance to anyone in the future looking to expand the functionality of their mobile Ren'Py games. I'll follow up later this week with how we used this to set up ads on iOS and Android that we could control from inside Ren'Py. In the mean time, if you have any questions feel free to ask!
Image
My website: woodsy-studio.com

User avatar
Arowana
Miko-Class Veteran
Posts: 531
Joined: Thu May 31, 2012 11:17 pm
Completed: a2 ~a due~
Projects: AXIOM.01, The Pirate Mermaid
Organization: Variable X, Navigame
Tumblr: navigame-media
itch: navigame
Contact:

Re: [Tutorial] Expanding Mobile Functionality With Pyobjus/Pyjnius (Notifications)

#2 Post by Arowana »

Wow, this is super cool - I had no idea you could do this for Ren'Py games! Thanks so much for sharing and taking the time to write it all up. :D Are you planning to do more tutorials on other mobile functions?
Complete: a2 ~a due~ (music, language, love)
In progress: The Pirate Mermaid (fairytale otome)
On hold: AXIOM.01 (girl detective game)

Image

User avatar
Andredron
Miko-Class Veteran
Posts: 700
Joined: Thu Dec 28, 2017 2:37 pm
Location: Russia
Contact:

Re: [Tutorial] Expanding Mobile Functionality With Pyobjus/Pyjnius (Notifications)

#3 Post by Andredron »

storykween wrote: Tue Oct 17, 2017 3:15 pm
Question. Who made a notification on Android? I was able to assemble Apk. Did everything according to the instructions, but something did not go.
Gives

Code: Select all

AttributeError: type object 'org.renpy.android.PythonSDLActivity' has no attribute 'myNotification

User avatar
Imperf3kt
Lemma-Class Veteran
Posts: 3785
Joined: Mon Dec 14, 2015 5:05 am
itch: Imperf3kt
Location: Your monitor
Contact:

Re: [Tutorial] Expanding Mobile Functionality With Pyobjus/Pyjnius (Notifications)

#4 Post by Imperf3kt »

I'm interested in utilising this feature you've documented, however I feel obligated to attribute this feature to you in my credits, I assume your forum name and the feature you've implemented would be sufficient or would you prefer a different way of being credited?
Warning: May contain trace amounts of gratuitous plot.
pro·gram·mer (noun) An organism capable of converting caffeine into code.

Current project: GGD Mentor

Twitter

Post Reply

Who is online

Users browsing this forum: No registered users