Gunbad Poll Results

[Rubius]

Hi everyone,

Recently we ran a Survey in-game which asked players for their thoughts on the updated Gunbad re-work, and today the Dungeons Team would like to share the results of that survey publicly, as well as offer any comments from our side.

We had hoped to get at least 50-100 responses back to have a good pool for averages, and we were blown away when we checked the final tally and saw that over 550 of you had sent in responses! So first, a big thank you to everyone for taking the time to send your feedback!

To add their thoughts, a player needed to have at least run the left wing of Gunbad, but did not need to have completed the Dungeon. It asked five questions related to Dungeon difficulty, duration, and the new Dungeon Group Buffs. The answers on most questions gave scores from a 1-5, with 1 being "too easy", a 5 being "too difficult", and a 3 being perfect. Shown visually, this is represented as:

1 = Too easy / short
2 = A little easy / short
3 = Perfect
4 = A little difficult / long
5 = Too difficult / long

Next, here's where we netted out (averages of over 550 responses):

Question 1: How would you rate the difficulty of the updated Mount Gunbad Bosses?

Player Response Average: 3.3

This was closer to the 3.0 bullseye than we were expecting, but we were happy to see it. Almost all players surveyed felt the updated boss' difficulty was close to perfect, with a few feeling that some engages could be a little easier.

Question 2: How would you rate the increased gold payout for 1-handed classes in Mount Gunbad?

Player Response: Overwhelming Support (~95% supporting)

This one was as expected. (Everyone likes getting more gold!) Bringing 1-Hander classes up to the same payouts that 2-Hander classes had for years was a nice win for many players.

Question 3: If you fought the new Hard Mode bosses, how would you rate their challenge?

Player Response Average: 3.2

This question also had the option to opt-out if a player didn't try the Hard Modes yet. Those who voted rated the Hard Mode bosses even closer to perfect than the regular fights, making this our closest bullseye of the re-work. We're glad to see that players are enjoying the extra challenge, and our focus group has given us some good feedback on ideas to further reward players who want to keep challenging those hard modes regularly. (You'll see some of these already in the new Bastion Stair Hard modes, which reward an additional Champion's Mark when clearing a Hard Mode fight five or ten times.)

Question 4: How do you feel the current total time spent per run is for Mount Gunbad?

Player Response Average: 3.7

Here, a 4 meant "a little too long" and 5 "too long", so players on average feel that the run time is good, but should be a little shorter. The healing in the Left Wing's Garithex' fight was noted in comments as a key pain point for overall run time, and we're looking at some small adjustments here to give players options to make this faster.

Question 5: How do you feel about the new Group Buffs that can be purchased for gold?

Player Response: Overwhelming Support (~95% supporting)

Players voted overwhelmingly in favor of the new Dungeon Group Buffs, which offer randomized group-wide bonuses in exchange for Gold. Seeing the strong reception, we've moved these out of their trial phase and have now implemented them in Bastion Stair as well.

Overall thoughts: We're very glad to see that players are generally quite happy with the updated tuning of the bosses, and are even more happy with the Hard Mode difficulties. We had predicted the dungeon's run-time score would land around a 4.0 (a little too long), so the 3.7 here is better than expected, and gives us some room to adjust a fight or two so they become a bit faster. The Dungeon Group Buffs landing as much support as they did was also encouraging, and has cemented them firmly as a permanent addition in both Gunbad and Bastion Stair moving forward.

A few final adjustments based on this survey will go live in the next patch, and at that point we'll be putting the bow on the finished Gunbad re-work. We appreciate everyone's feedback across both phases of the update, and all of the time and comments you shared with us as we continued to adjust it in the right direction.

As always, thanks for reading and sharing your feedback!

[gisborne]

I got a survey after doing Bastion Stair. It's great that you are able to implement surveys like this as I think it a) shows that you are interested in our feedback and b) get the feedback from everyone running the dungeon rather than the forums.

[leftayparxoun]

It is great that you are implementing in-game surveys like this. I wasn't active during the period they were sent out, but it is really good tool to have.

I want to only raise caution on 3 important statistical fallacies when interpretting the results of surveys such as this.

1. Outlier exclusion:
This survey is well designed in that there are certain questions whose positive and negative scale is inversed (e.g. Likeability of mechanics has a max positive of 5 and max negative of 1, while current length of the dungeon has a max positive of 1 and max negative of 5). That way it is quite easy to figure out 0 effort outliers since they will have most likely used either the max positive rating of the 1st question for all questions or the max negative of it (and would not have bothered reading the followup questions).
It would be interesting to see how many of those outlier answers were submitted (I imagine they are not part of the 550 total answers), but what I want to point out lies elsewhere. While it is proper to remove all those outliers (both positive and negative ones), it is a mistake to ignore the negative outliers completely. After all, from your wording that you expected 50-100 answers we can infer that the players had the option to refuse to answer the survey.
As such, the people that chose to fully reply to the survey with completely negative answers clearly cared enough to do so and used the survey as an avenue to express their disaproval of the survey's subject as a whole. What I'm getting at is that if that number is significant of "votes of no confidence" is significant, it should in my opinion be taken into account.

2. Non-replies:
As established above, the survey participation was optional. Where the falacy lies here, is in the fact that people who are disappointed by the topic of the survey are more likely to not engage with that survey.
To bring a real life example. Imagine that you are hired to poll people outside a new Clothing store that just opened. If there are 5000 people visiting that shop that day and at the end of that day you have gotten 300 votes with an average of 4.5/5, would you still claim that the result is statistically significant? After all, more than 90% of the people refused to take that survey. You could use these 300 to formulate an opinion, but inconvenienced customers are way more likely to have skipped your questionare and as such your poll could be heavily skewed towards positive opinions.
Same thing happens here. How many people were sent the survey for 550 to provide an answer? This is a very significant question so as to ensure that the takeaways you have from your survey actually represent the whole playerbase and not just the ones positively inclined towards the subject.
Of course there is the arguement that if the rest of the people wanted their opinions represented then they should have replied to the poll, but even in reality it does not work that way. Game Devs are supposed to struggle and manipulate player(human) behavior, and is this one more case of this.

3. Standard Deviation:
In the original post the answers for each question were represented in the form of their average value from the 550 participants. However, it is equally important to also provide information about the standard deviation of those answers.
In simple terms, standard deviation represents the "spread" of the answers in numeric form, where "spread" refers to how dissimilar the opinions were.
In a real life example:
If I ask 100 people coming off a restaurant about how the food was and I get 10 answers saying it was Bad, 80 it was Good and 10 it was Great, then on average I can conclude that the restaurant was good on average.
Now if 40 people had answered Bad, 20 Good and 40 Great, I would still have the same average rating, however, their opinions would be very varied and I (at least personally) would be quite more sceptical about visiting that place to eat in the 2nd case rather than the 1st.
In the above example both polls resulted in the same average but would have vastly different degrees of opinion "spread" which can be numerically expressed via the standard deviation metric.
Coming back to the present case, what was the degree of agreement in the answers? Could you potentially also provide us with the standard deviation for them? Regardless, it is very important to calculate and consider the stand deviation of your answers, both to gauge the degree of agreeability and to check which questions/topic might be truly divisive. Otherwise you once again risk alienating a part of your audience.


Sorry if I rambled for too long, just wanted to give some general feedback in case this polling method continues going forward (which would be great).
Thanks again for sharing those results with us.

[borc1]

when youre going to add surveys for class changes? game been on a slow death spiral over the years and pve wont save it