6.12.2: Hardware Survey

[Aleema]

I do appreciate you bringing this forward before just implementing it, don't mistake me. You've asked the community (and me, for being in the community), our thoughts. I've said them, and then you quoted the hell out of me with terminology issues. If we're both too defensive here, then we should both apologize.

I dispute that this is ethically necessary for impersonal information. I think it's ethically necessary for sending personal information, but for impersonal information, a disclosure is sufficient.

I dispute shoving something in a text file that is not actually required to be read as actual disclosure. And I meant "silent" as in not giving the play the choice to opt in or out.

I had a paragraph suggesting you take a look at others' attempts at data collection than pursuing it yourself, but you seemed to have figured that out. After dissecting my post, but yay for progress.

I apologize for the strength of my opinion in this matter.

[PyTom]

Sorry, due to the order in which questions came in, I didn't see this one.

Then I have a question. What's the reason that you don't want to ask permission to users?

It's largely because asking would be relatively intrusive, as it requires people to make a technical/legal decision whenever they start a game. How annoying would it be if google analytics asked you to make a decision each time you visited a website?

So there's no answer on my question?
Anyway if it is used for someone's interest - in this case, making better version of renpy, but it's not related with players' interest - without "explicit" permission, people wil definitely hate that. Because they think that they are taken in.

I'd argue that a better Ren'Py is in players' interests. If it's not, why do I bother?

I dispute shoving something in a text file that is not actually required to be read as actual disclosure.

I'll note that's the way Unity does it - there's a link to the text file, but nothing requiring the user to read it.

[jack_norton]

I think opt-in is the best solution. Unity Web is different, because... is a web application, and for average gamer joe is normal that a web app connects to a server Imagine if people playing any commercial Ren'Py game see Norton or any other firewall program sending out "WARNING! Application blabla is trying to connect to... IP address."

/sneaky mode on
On the other hand, there could be a simple solution to all of this: implement an autoupdater for the games and then collect the data silently during the update process. Nobody will ever notice (since they already set the firewall permission to allow program XYZ.exe to access internet)

[Aleema]

I'll note that's the way Unity does it - there's a link to the text file, but nothing requiring the user to read it.

Unity didn't ask me for my opinion, nor is it similar to Ren'Py at all. And I'll repeat this in case it was overlooked:

For one, from what I can tell that is not an engine that allows users to create and redistribute content (whereby the creator infects all consumers of their content with data collection). It is something that YOU personally have agreed to when you downloaded. You're not making a commitment to collect data on other people. I have no right to give others' rights away.

Second, the data collecting seems to be such a prominent and known feature about it that it's literally there next to the download button: http://unity3d.com/webplayer/
You can't guarantee that developers are going to alert people before they download Ren'Py based games about data collection, you can only put things into your license/EULA that people are not forced to read, unlike other installation GUI programs that force you to agree to the terms before installation. You just unzip Ren'Py games and play.

If you want to do this, I suggest you create a self-installer that requires viewing of the EULA.

You have my opinion on the matter, and now I duck out in a cloud of ninja smoke.

[PyTom]

What do people think about asking the user to send statistics on quit?

I'm thinking that perhaps we could change the quit screen to read something like:

Code: Select all

                    Are you sure you want to quit?

                  [ yes ]                     [ no ]

[x] Send anonymous data to improve Gamename and Ren'Py (privacy policy)

I'm thinking the checkbox could be checked by default, but the user would always have time to uncheck it before he or she quits. After it's changed, the setting would be remembered. (Perhaps even across multiple games.)

Since the user is opting to send the data, we could even send slightly more sensitive data than I'd be comfortable sending otherwise - like the game name and version, the play time, and perhaps some measurement of user activity, like the number of times he or she clicked.

EDIT: I could even see making some sort of API available to game-makers. Something like this could be useful for tracking how many times each ending was reached, to help tune the difficulty of a game.

[Camille]

I think I'm one of the few that doesn't mind this at all... Websites collect your information all the time without asking you, and it's not like you're collecting any incredibly sensitive information... But I do think it'd be good to inform people since most people aren't as laid-back as I am.

Asking the user to send statistics on quit sounds like a good idea! That way the user isn't bombarded first thing with the request upon opening the game, but it's still brought to their attention in a way that they can't miss. It's kind of like when a program crashes and a window pops up asking if you'd like the crash data sent to the developers.

[jack_norton]

Presented in that way, I really think should be fine by anyone. On quit seems good, beside I suppose sending that info, even on a slow connection would take 1-2 seconds max?

[PyTom]

Yes. If it wasn't done in 5 seconds, I'd give up.

[Taleweaver]

What do people think about asking the user to send statistics on quit?

That sounds not only feasible but fair as well. I wouldn't object to a game that asks me that question _after_ I've played it.

[Jake]

I'm definitely 100% against collecting information without telling anyone. For a variety of reasons, many of which have been mentioned already, and just a few of which are:

- Ren'Py is not a program, it is a tool to make programs, which means you're actually intending to make Ren'Py game developers complicit in your data-collection. I don't want to have to carefully read the Data Protection Act every time I distribute a Ren'Py game, and I seriously doubt you're well enough informed about data protection laws in every jurisdiction worldwide to be able to confidently say it would be perfectly legal for everyone.

- However harmless you think the data is, you're still holding people's data. As the old adage goes, take only what you need, and keep only what you absolutely definitely need. And in the case of Ren'Py, it doesn't need any of that information to perform its primary function, which is "running VNs".

- Precedent != justification. Starforce (or something like it, I forget which exactly) disabled my ability to backup my own data with my own CD writer ten years ago, just because I had the temerity to pay a publisher money for a videogame - that doesn't mean it was remotely justified then, and it doesn't mean that you'd be justified in doing the same thing today. It's an extreme example, sure, but the point is that "other people do it" isn't a good enough reason to think it's OK for you to do it.

- It will decrease the user's faith in the program, and probably in other programs made with the same engine. People, when taken as a whole, are not rational or objective enough to say "hey, look, it's only my OGL version and platform, it's no problem" - the reaction of a significant proportion of people who notice will be "Ren'Py is spyware", regardless of whether you've mentioned it in the readme or license. Worse, it'll also probably be "Developer X's game is spyware", which gives them a bad name for no good reason - they may not even be aware Ren'Py is doing this, you know that nobody reads the documentation unless they absolutely have to. Such a move is likely to put people off downloading Ren'Py-made games and possibly tarnish the name of innocent game-makers.

- Just because you aren't using it for evil doesn't mean other people won't. Ren'Py is open-source and pretty easy for a skilled developer to modify in a lot of ways, and if it did come to pass that people get used to the idea that Ren'Py games connect to the Internet and do mysterious data-exchange stuff, then this also makes people more willing to ignore Ren'Py games doing such things, leaving the possibility open for malicious people to suck up web-browsing history and personal info and random files stored on the user's hard disk and passwords accidentally typed into the Ren'Py window and whatever while the user is playing their cunningly-disguised Ren'Py-game-which-is-actually-spyware. Not very likely, I know, but if you make people used to the idea of saying "yes, ignore this traffic" on their firewall, that also opens the door for people to exploit it.




As it goes, you probably already know this, but since you brought it up - the Unity logging isn't so notable or similar as it might at first seem, because the user has to voluntarily send most of that data already just to be able to view web pages; the HTTP request your browser makes already sends a lot of information about the computer it's running on. The other reason it gets away with it is most likely simply because the user is already connecting to the Internet with the "Firefox.exe" application (or whatever) in order to get to the Unity web player in the first place, so their firewall quite probably doesn't even mention the new connection to the Unity servers from the same app... even a technically-aware user will have no idea that the transfer goes on unless they specifically look for it. That's simply not the case for a game which doesn't have to access the network at all for its normal mode of use.

What do people think about asking the user to send statistics on quit?
...
[x] Send anonymous data to improve Gamename and Ren'Py (privacy policy)
...
I'm thinking the checkbox could be checked by default, but the user would always have time to uncheck it before he or she quits. After it's changed, the setting would be remembered. (Perhaps even across multiple games.)

On one hand, asking the user explicitly on game-exit is a much better approach than just doing it without telling them.

On the other hand, I would strongly advise the following:

- Actually ask the user a direct question which is obviously different from the regular "do you want to quit yes/no?" question, so it's actually clear to the user what's going on and they can't accidentally click "yes, send all my information to some random guy" button when they really meant "yes, of course I want to quit, that's why I clicked the 'quit' button".

- (One way you can do the above is by popping a new obviously-different window up after the user has already hit 'Yes', and never doing this on their first use or two of the software, so they have a chance to get used to what the 'normal' quit flow is like.)

- Remember that a significant proportion of the user base doesn't have English as a first language, and even some English-speakers ignore bits of text that they don't think they absolutely have to read. That doesn't mean these people are going to be happy to find out that Ren'Py is sending their details into the aether.

- Give the developer an option to disable it. Make sure that this is prominent in the options.rpy, and preferably defaults to off. The developer should not be drawn into your plans unwittingly, and you can bet that a significant number of potential Ren'Py developers aren't reading this thread and may well remain unaware of this whole idea.

- You probably have sufficient grounding in stats already, but of course: realise that just by giving the user an option to not send their details, you're already biasing your population and ensuring that you don't necessarily get a representative sample of users. You can bet that there's a very significant demographic skew on the kind of user who's likely to uncheck that checkbox, say 'no', or whatever... and it's quite possibly the kind of demographic that also has a strong correlation to particular ranges of hardware. Is it still so much better than a voluntary survey of users?

Since the user is opting to send the data, we could even send slightly more sensitive data than I'd be comfortable sending otherwise - like the game name and version, the play time, and perhaps some measurement of user activity, like the number of times he or she clicked.

... Why on Earth would you want to know how many times they clicked?! If you can't think of a really useful reason to have that data, then don't collect it. I can't, myself.

EDIT: I could even see making some sort of API available to game-makers. Something like this could be useful for tracking how many times each ending was reached, to help tune the difficulty of a game.

This sounds even more potentially-problematic, to me - how are you going to make the information available to the game-maker? Does it go to their server (most game-makers probably don't have or couldn't configure a server with software to receive the data)? Does it go to your server and you allow them access? How do you ensure that the data isn't viewed by anyone other than the people the various players have authorised to see it? Insert dramatic ellipses here.




I understand that it's useful to have some idea of the hardware that Ren'Py's end-users are running on, but it's also useful to have a clean nose and a good reputation, and not piss off players or get developers into trouble. I'm not a tinfoil hat, and I don't mind reasonable data being sent off to software vendors a lot of the time, but bear in mind that we're living in an age where Internet identity theft is front-page newspaper news and people call up their techy friends afraid that their own AV software is malicious internet-delivered spyware. I work in the same room as a guy who refuses to block ad-bots on Skype because he doesn't want any record on any remote server to connect him to them, even in such a negative way as a block!


(Also, data collection requires central servers, and everyone knows centralisation is basically communism! :P)

[jack_norton]

Play time can be useful to see player behaviour on demos and see what technique works better.
The problem here is letting the player know. If you tell him "are you ok to send anonymous statistics" and player answer YES, then there's no problem (it's his decision). Companies like BFG and Zynga are doing this since years - and most MMOs too. Even Google ask you to send data when you use Chrome. I have replied NO, but maybe some others replied yes

[Jake]

Play time can be useful to see player behaviour on demos and see what technique works better.

Play time/click count/etc. can be useful to game-makers to see player behaviour; what I don't see is why PyTom wants it. It starts to look like he's getting carried away and just suggesting any piece of information he thinks may possibly be interesting to someone somewhere sometime, which is too-often the end result of justifying to oneself the quiet collection of "small, insignificant" pieces of data.

(It seems to me that offering collected data to game-makers is then fraught with an extra layer of potential headache.)

[jack_norton]

Well, in the end though I never considered using that system I know of developer friends who change/finetune the demo based on collected data but honestly I just try to make a good game and that's all

[Counter Arts]

I would say that asking on the quit screen is a very good idea.

My suggestions earlier were made because I noticed that there are still new computers being sold that are mini-pc's/eepcs 1.5-2.0 GHz with the atom processors and no less than 1GB of memory. And some of those things looks like they only support open gl 1.5 at most looking at wikipedia.

My guess is that you'll find a lot of 1.x capable graphics chips and not too much over 2.0 open gl capable computers.

[PyTom]

- Ren'Py is not a program, it is a tool to make programs, which means you're actually intending to make Ren'Py game developers complicit in your data-collection. I don't want to have to carefully read the Data Protection Act every time I distribute a Ren'Py game, and I seriously doubt you're well enough informed about data protection laws in every jurisdiction worldwide to be able to confidently say it would be perfectly legal for everyone.

I think that I'll say that there's now no chance that I will collect data without getting their consent.

- Actually ask the user a direct question which is obviously different from the regular "do you want to quit yes/no?" question, so it's actually clear to the user what's going on and they can't accidentally click "yes, send all my information to some random guy" button when they really meant "yes, of course I want to quit, that's why I clicked the 'quit' button".

There are a couple things that I want this system to satisfy:

1) I don't want to make it more difficult to quit than it is now. I'd really prefer if there wasn't a second click, in the common case.
2) I'd like the status of the system to be really obvious, each time the game quits.
3) Data should be sent on the first quit. That's because people with marginal systems (which I'm most interested in) may only start a game once.

The checkbox seems like a reasonable approach to that, as long as it's clear enough to the user what's going on. It's relatively non-intrusive, conveys what's happening, and it's pretty much the standard interface for this.

- Remember that a significant proportion of the user base doesn't have English as a first language, and even some English-speakers ignore bits of text that they don't think they absolutely have to read. That doesn't mean these people are going to be happy to find out that Ren'Py is sending their details into the aether.

- Give the developer an option to disable it. Make sure that this is prominent in the options.rpy, and preferably defaults to off. The developer should not be drawn into your plans unwittingly, and you can bet that a significant number of potential Ren'Py developers aren't reading this thread and may well remain unaware of this whole idea.

My plan is to have it enabled in newly-created games, and disabled in upgraded games. Since presumably the developer would quit his game at least once during development, and since the survey functionality would be presented to the user each time the user went to quit, the developer would see this function, and translate or disable it. I think this constant presentation is an advantage of the checkbox.

- You probably have sufficient grounding in stats already, but of course: realise that just by giving the user an option to not send their details, you're already biasing your population and ensuring that you don't necessarily get a representative sample of users. You can bet that there's a very significant demographic skew on the kind of user who's likely to uncheck that checkbox, say 'no', or whatever... and it's quite possibly the kind of demographic that also has a strong correlation to particular ranges of hardware. Is it still so much better than a voluntary survey of users?

I think so. I have access to a lot of statistics for the various sites I run, and the number of people who download games is many times the population of this forum. There have been at least a hundred thousand downloads of Moonlight Walks. I don't know how to reach those "silent users" without putting code in Ren'Py - and in, Ren'Py games, rather than the SDK.

I don't see how participation/nonparticipation could correlate with hardware. (Embarrasment over low specs, maybe?)

... Why on Earth would you want to know how many times they clicked?! If you can't think of a really useful reason to have that data, then don't collect it. I can't, myself.

The idea would be to come up with some sort of metric of how "involved" people are with the games, that could be used on games.renpy.org.

EDIT: I could even see making some sort of API available to game-makers. Something like this could be useful for tracking how many times each ending was reached, to help tune the difficulty of a game.

This sounds even more potentially-problematic, to me - how are you going to make the information available to the game-maker? Does it go to their server (most game-makers probably don't have or couldn't configure a server with software to receive the data)? Does it go to your server and you allow them access? How do you ensure that the data isn't viewed by anyone other than the people the various players have authorised to see it? Insert dramatic ellipses here.

I was thinking that we could make all the (aggregated) data public. This way, there's no special relationship between me and the game-maker. We couldn't send sensitive data that way - but that's a good thing, no?

I understand that it's useful to have some idea of the hardware that Ren'Py's end-users are running on, but it's also useful to have a clean nose and a good reputation, and not piss off players or get developers into trouble. I'm not a tinfoil hat, and I don't mind reasonable data being sent off to software vendors a lot of the time, but bear in mind that we're living in an age where Internet identity theft is front-page newspaper news and people call up their techy friends afraid that their own AV software is malicious internet-delivered spyware. I work in the same room as a guy who refuses to block ad-bots on Skype because he doesn't want any record on any remote server to connect him to them, even in such a negative way as a block!

Yeah. I'm kind of hoping I've hit upon a way of doing this that makes players aware of what's going on, makes developers aware of what's going on, doesn't harm people's privacy, doesn't have people with pitchforks showing up on my doorstep, and so on...

My suggestions earlier were made because I noticed that there are still new computers being sold that are mini-pc's/eepcs 1.5-2.0 GHz with the atom processors and no less than 1GB of memory. And some of those things looks like they only support open gl 1.5 at most looking at wikipedia.

It's possible that these computers will support Open GL 1.5, but also shaders. (My mac is like that.) I really want to determine if there's a significant number of users without shader support. (And also to try and figure out what fraction of people are in software mode, why, and to hopefully fix that.)