Opinions on Player Data Tracking

[Jackkel Dragon]

AKA Please convince me to stop being so data-crazy...

I've noticed that, over the past year since my first commercial game was released, I've been getting more and more tempted by the use of analytic data to compensate for not getting the kind of feedback I'd like. While I'd not have any information on why players made certain decisions or played certain sections, I'd at least be able to say with certainty things like "this was the most-played route". So far I've been able to get some semblance of this data through the use of Steam Achievements, but not all of my games are on Steam and I don't want to make my games look like the achievement-spam games that flooded the market for a while.

After trying (and so far, failing) to get the Ren'Py analytics through Google Analytics to work, I started to wonder what people thought of this kind of data being collected. Putting aside the legal concerns (the EU requires consent for this kind of data collection), I'm curious what players think about games basically telling the publisher exactly what players do. As a developer, I'm very much interested in gobbling up as much information as possible so I know what people want from games. But I bet people who like single-player shooter games were unhappy when EA/DICE and Activision cut the single-player modes of some of their big games because of lack of play time, and a vocal minority managed to shame EA into putting a single-player campaign into EA Battlefront II despite the lack of one in the first game.

So, what do you think? Does anyone here use analytics, or else specifically avoid them?

[parttimestorier]

I can definitely see how it would be interesting and tempting to collect as much player data as possible, but I think you'd be missing out on a lot if you started relying on just the data and your assumptions about what it means in order to make decisions about your future projects. For instance, you mention finding out that a certain route was the most played, so I can imagine that with data like that, you might start thinking, "This character was the most popular, so I'll make more like them." But you can't actually be sure if that's the reason more people played their route than the others. Maybe there was something in the way your choices were set up that made a lot of players just kind of end up on that route without thinking about it. Maybe some of them were actually a lot more interested in some other character, but couldn't figure out how to get their route at first, and then ended up getting distracted and playing some other game instead of figuring it out, and so the achievement data doesn't reflect their preferences at all. As hard as it can be to get people on the internet to participate and comment on stuff, I think you'd be much better off finding a way to encourage your audience to fill out a survey or something than just trying to figure out what they want from their data.

Also, as important as it can be to try to appeal to an audience if you're making commercial projects, I always kind of worry that anyone who tries too hard at that won't end up writing a very interesting story. Maybe some people can do it well, but I can't imagine a writer having much fun just trying to slot the popular character types into the popular setting to make something that they think people will like. And I think that sometimes when you read stuff like that, you can tell that it wasn't fun to write, and it makes it less fun to read. I would personally much rather read something that's weird and unique and really reflects someone's artistic vision, even if it's nothing like what I would have expected or thought that I wanted.

[Nighten]

Some really good point have been made in the comment above, but I want to add that, as an alternative to data collection, you could do some playtest! When I did it I was really surprise by how much I learned by watching someone playing my game in real time : What option did they choose, was something not clear enough, what was their feedback at the end of the session... Having a 1 to 1 conversation is way more clear that some achievement or reviews online.
You can also contact a player by private message to ask them questions and get more in depth feedback than "It was great !".

This are just ideas, but I think it can help you to gather useful informations!

[parttimestorier]

You can also contact a player by private message to ask them questions and get more in depth feedback than "It was great !".

Personally, I would recommend against that method. I really dislike getting DMs from strangers and I'm sure I'm not the only one. I actually once had something like this happen - I left a short comment on itch.io saying that I liked a game, and the developer contacted me privately to ask for more detail - and it made me feel really uncomfortable. I didn't want to get into a whole conversation with someone I didn't know about my in-depth opinions on their art. That sounds stressful and awkward to me. But I agree with you that playtesting is really useful, and if you have people who are volunteering to help or that you already have a friendly relationship with, you can probably ask for more some more detailed feedback from them.

[Elsa Kisiel]

You can also contact a player by private message to ask them questions and get more in depth feedback than "It was great !".

Personally, I would recommend against that method. I really dislike getting DMs from strangers and I'm sure I'm not the only one. I actually once had something like this happen - I left a short comment on itch.io saying that I liked a game, and the developer contacted me privately to ask for more detail - and it made me feel really uncomfortable. I didn't want to get into a whole conversation with someone I didn't know about my in-depth opinions on their art. That sounds stressful and awkward to me. But I agree with you that playtesting is really useful, and if you have people who are volunteering to help or that you already have a friendly relationship with, you can probably ask for more some more detailed feedback from them.

It didn't happen to me, but I sure would have feel uncomfortable too to be asked for a more detailed opinion in DM. °v°'

[ISAWHIM]

If you make it part of the game, like "Telltale games" has done... It can be 100% acceptable.

If you are unaware, you can see how others have responded to certain path situations... "How many saved the girl", "How many left Kenny behind", "How many stole the watch". (But I am sure they collected other bits of info too, which were just not shared... How long before someone picked an answer. How many times they "replayed", the game. Etc...)

Provided that you, first, let them know that this information is being used, and for what purpose. There are legal reasons, especially related to the abuse of "data-collection of minors". (Since they can not be "bound by contract", they can not legally "accept the terms". Thus, any data-collections would be "illegally collected", in those situations. Pointing to recent and not-so-recent issues that facebook, microsoft and steam have gone through, related to games and users.)

In all fairness, the data they were collecting was "Knowingly" from minors, and "Knowingly", being "sold for money", to third-parties. (Not just being used for internal diagnosis or "for fun". Like displaying high-scores.)

However, instead of making it a "blind collection", you can make it a pop-up option... "Would you like to upload your scores to compare to others?" At which point you provide any required disclaimers, or provide them with something like a "COPY/PASTE" code to enter manually, into a website. (Thus, your program is not, "explicitly", harvesting data, of any kind.)