An Attempt of an Objective Look at Scenario Balance in RoR

[Mez]

Can someone solve this equation?

TLDR = X.

[MasterYoda]

Happy to see that the data analysis is largely well received.

I am quite busy at the moment, though seeing that dalen even encourages scraping of the killboard and will fix skipping issues with it do I think I will repeat the data analysis with more data at the beginning of next month. More text for people to not go through and enjoy their fruitful effort of commenting duh, tldr

But to address some comments in particular:

Try to use the median value instead of the average value. It's more robust and a better representation of the "typical" value.

I'd like to see what the median total damage, mean damage, healing, protection, kills and deaths for every class is.

A class distribution would be also interesting. How often do you have class x in low tier, midtier and high tier SCs, as well as cities.

EDIT: For classes that can play two roles like heal or dps and tank or dps you should do a split up. If damage > 2x heal or damage > 2x protection then it is a damage dealer, when damage and heal/protection roughly the same then it is a hybrid and when protection/heal > 2x damage then it is a tank or healer.

All good points. I think if I do the data analysis again I will actually take some time for it it and will make it a properly configurable CLI script so one can use any statistical measure to summarize the data. However I do think that the average is just as an important measure to consider and I'm fine with having chosen it as the metric for the first approach.
Chosing a multiplier to configure as to when a tank or healer is considered a damage dealer is a good idea. I'll include it in the next iteration.

Love your work here, great analysis - I've got one I'd love you to do though;

Compare win rates in a scenario by healer differential. For example if both teams have one healer, or +1 healers over the other team, +2, etc etc. It's no secret that the vast majority of pug scenarios when levelling and at max level *feel* lost just by looking at the rosters of both teams at the start of a scenario and seeing that the other team has 3 more healers than you, but I'd be interested to see if it reflects reality. Can't account for player skill difference of course but hopefully there's a decent enough sample size that some conclusions can be drawn

Great point also. I will include a flag for a filter for scenarios next time to only consider scenarios where at least e.g. in 12man scenarios 2 classes from each archetype have to be present for both realms in order for the scenario stats to be counted. Also thank you for the kind words!

Apologies if I missed this in your post, but do you have - or can you provide - any insight on the overall role composition of winning scenarios? E.g. how many tanks, how many DPS, and how many healers (in heal spec) were on the team, just to see if having a 2-2-2 setup for each group (would need to assume 4-4-4 was spread equally over 2 groups) greatly increases the chances of winning? The combat in this game is heavily balanced around 2 tank for guard/challenges, 2 DPS, and 2 heals per group.

A comparison of the win rates when just one side is 2-2-2 and the other side is not 2-2-2 would be valuable too.

One absolutely vital piece missing from the stats (and the killboard), is distinguising between premades and solo queue in a scenario. From 100% anecotal evidence, I have seen far more complaints about premades (order or destro) dominating mid-tier scenarios as people form groups to level 6 characters up together as quick as possible, which would greatly distort the win rate of whichever classes/factions are currently being levelled.

I have not considered the group compositions when analyzing the data, however as I do see it as a valid point as will I include such a filter flag should I do a next iteration of this analysis as I pointed out in my answer to Minisynn. There however is a differentiation between premades and pug groups but only to the extend that there are T4/mid-tier/T1 standard and pug scenarios to look at. The data does not allow to see if regular players solo joined a standard scenario or if they joined a standard scenario in a group. Should I do a next iteration of the data analysis will I give an insight into what data exactly is available to me.

The 'engagement-intensity' term is both an appropriate description for where the most relevant data would manifest itself and valid factor to consider. Again, considering the provision stated I have yet to work through your entire dissertation, can and/or did you consider the effects of a DPS class dealing damage over time while benefiting from a dedicated guard as opposed to not benefiting from such?

Specifically, any DPS class can have their damage contribution numbers affected adversely simply when accounting for the time they may be out of an SC due to respawn and return to the action timers. Given the (most often) thirty second respawn timer combined with the time it may take to return to the battle area, a time which can often exceed one minute of SC game time, this time factor can certainly impact the overall numbers.

As outlines above have I not considered if a DD was guarded or not. The data simply didn't allow for that. You can imagine the amount of data I got to be the same as the data you get at the end of a scenario in the scenario summary window. There is not even a total amount of incoming damage shared with guard which I might have a look at. However as outlined above will I include a team composition filter should I do this data anlysis the next time.

This is interesting stuff. Was kind of hoping more community members would do more interesting stuff with the killboard data.

I will try to fix the skipping issue in the API so you can fetch all of them.

But regarding the scenario winners, try using the `winner` attribute, especially for city sieges that should be more accurate than looking at the points score.

I will start using the 'winner' attribute in the future if you don't set 'winner' to 0 when there is a draw, same as when order is winning
But seriously though, thanks for the kind words and fixing the API.



In general would I like to say thanks to everyone appreciating the post and having read my attempt of summarizing the killboard data into somewhat interpretable metrics for scenario balance. I will take your feedback into account and hope to find the time to post another one!

Cheers

[LetsYouDie]

Can someone solve this equation?

TLDR = X.

r a m p a g e

[zulnam]

Very interesting data, nothing too shocking; as anyone with two eyes and a healthy attention span can notice.

- Healers and tanks have higher win rate since their presence can make or break a match.
- range dps have lower chance of winning T4 scenarios;
- slayer towers over all in terms of dps; no such difference on destro side

Some things i didn't expect:
- destro losing so many cities. 29.4% win rate. uff.
- blackguard outprotecting BOs and chosens; wat?
- WH underperforming by so much compared to WE. 5% less wins in standard T4; lower kill damage; lower deathblows ratio;
- WP and DoK being at the bottom of the chain in terms of healing output.

Good job. This kind of data is very useful imo. I would argue these should be grouped and stored as a snapshot of the current period.

[normanis]

that tabula showing kobs is lowest tank and squig herder is better as shadow warrior.
p.s 28% winrate for destro in city. how that possible. they win 9 to 1 in all cities what happened last 2 years. not mention win in row all lotd. (i dont complain i will gera up my destros)
p.s sad lotd dont happen every 4th day.

[tazdingo]

- blackguard outprotecting BOs and chosens; wat?

there is an odd phenomenon of hardcap toughness max avoidance snb bgs in t4 scs. sometimes i notice 3/4 on a single team and streamers tend to point them out a lot too. my guess is they end up xguarding each other or just eating a lot of guard dmg in general as there's very little else they can do outside of punting, and they're probably constantly getting good challenges into the order backline too as they're always in it. god bless them for keeping the dream alive

[rejndjer]

now we have statistical proof that SW is worst class in game

[Cyrylius]

Blackguard endgame is small-scale, therefore more experienced class users join small-scale content than, say, black Ork or chosen. And if average sc blackguard has better gear and RR he will outperform other tanks. Xguarding might be a thing too, cause BGs are indeed insanely tough with 2H spec so they can hold the frontline longer. Anyway, I don't think it's caused by any still, as shield of rage is easily calculated in terms of how much it changed: it provides a total of double the self protection of bg-class has no other way of protecting itself, except for warlord proc and bellow.

[Aethilmar]

That was much better than I thought it was going to be based on topic title.

I suspect the SC data is sufficiently large to support general conclusions of some sort about balance in that particular mode (but not in general).

The city data is not even close to being a big enough sample especially given the skew in cities from alliance clustering (i.e. who is playing what, when). I think you can make good comparisons within within a faction though.

Anyway, good job. Can't wait to see more.

[mryay]

I just want to /salute you for your effort and contribution to the community.

1. Out of curiosity, are you working on the dev team?
2. I suggest considering also healers' combination (e.g Archmage/WP, RP/WP, double WP) in one of your future iterations
3. In the scope of SC only, devs are doing a pretty good job, there are not that many discrepancies, even though there is room for improvement in terms of team synergies (for example WH with other classes)
4. Actually, I would also check the win rates with the combination of SM+SW or SM+WL or SM+AM because their damage output and healing output are significantly enhanced by CD reduction.

Cheers, Swizz.