Performance measurement - quantifiable metrics? - Downloaded from the Wizards Community, all coding removed by Solauren

Performance measurement - quantifiable metrics?

Post/Author/DateTime Post
#1
mitok
Aug 10, 2010 15:54:36 I am normally the DM of our group, but one of our players is trying to put together a spreadsheet to use as a sort of statistical analysis tool (he says his first love is baseball, and loves tracking statistics). He's looking for quantifiable measures he could use to track the performance of all the players, so that everyone can look back at prior encounters and perhaps determine where they need to make improvements.

I am thinking that certain quantifiable metrics are obvious, like damage per round, but not necessarily applicable or as important to all roles.

Are there other metrics that make sense for other roles besides strikers in this regard? My initial thinking is that there are too many variables (group tactics, buffs, use of terrain, combat advantage scenarios, etc.) for this to be a worthwhile effort, and that the players should just look at prior combat encounters as a whole and judge from just a general impression. Did strikers hit very often? Did defenders do a good job keeping the monsters on them? Did leaders keep players in the fight? Did controllers ... control?

Thanks for any advice.
#2
mobar2008
Aug 10, 2010 17:22:45 You might track how much damage a defender soaked for the team....this isn't an easy question because most of the outcomes are based on something completely out of the players control, the dice. A series of bad rolls can kill a good plan, just as a series of awesome rolls can make a poor plan shine.

Then theres the varible of the encounter itself. Fighting the big bad is going to be tougher than the first encounter of an adventure. I'm thinking that with all the things that is OUT of the players control meaningful stats are impossible to come up with.
#3
geoffa
Aug 11, 2010 8:49:27 Let me start by agreeing with the last several posters. DnD is complicated and keeping stats is not going to give an exact measurement of how valuable each player is, and how successful the party is. I think it would also be quite time consuming and distract from playing the game unless you had some way to quickly record stats and count them up later (probably involving a computer, or maybe an ipod app).

Having said that, I am a math teacher, so I am all about trying to quantify things. Here's a couple ideas of stats you might want to keep for different characters.

All Characters

Hit percentage - that seems pretty valuable for all characters. Strikers should be hitting a lot and dealing out damage. Controllers should be hitting a lot and throwing out hindering effects. Even for leaders, this would be interesting to look at because if a leader has low hit percentage then perhaps it's a sign they should concentrate on other leader activities instead of attacking.

Hit percentage of encounter/daily powers - do you actually hit with the attacks that matter?

Strikers

Damage per round - this seems to be a pretty standard measure of striker effectiveness.

Opponents killed - this is a group effort, but you would expect the striker to be doing this more than most characters.

Defenders

Attacks drawn percentage - how often does the enemy attack the defender compared to everyone else. I don't think you actually want this to be TOO high, but it should definitely be more than average.

Number of times squishy characters get bloodied - this is kind of like using number of QB sacks as a measure of how good the offensive line is.

Controllers

Minions killed - controllers should be good at hitting groups of minions.

Opponents hindered - number of times the controller manages to deal out dazed, slowed, immobilized, blinded, etc. You could get all complicated and assign some of those effects to be worth 2 or 3 points but I'd keep it simple and count them all the same.

Leaders

Number of times another character falls unconscious - this seems to be a good measure of healing effectiveness. Characters can get bloodied, no problem, but if they fall unconscious then they are missing turns when they could be doing something useful.

Assist - number of times another character succeeds in a roll that would have failed without a leader boost. This might be kind of hard to measure.
#4
mitok
Aug 11, 2010 9:17:18 Thanks for the responses, guys. It pretty much sums up what I was thinking. I happen to own the Player's Strategy Guide (the one with Penny Arcade art on the cover), and there are two side-bars that pretty much say the same thing, and give an idea of how one should really measure success in this game. The first sidebar, "Defining Success" on page 111, and the second is "Know the Numbers" on page 112.

"Defining Success" says you should judge encounters by how many resources were expended that can only be regained via an extended rest, such as healing surges, daily powers, and so forth. "A successful encounter, then, is one that expends as few of your most valuable resources as possible."
"Know the Numbers" says that the game is designed such that monsters should be hitting characters about 50% of the time on average, and characters should hit monsters roughly 60 to 65% of the time. It even has a handy little table that shows what the average attack and defense values should be, all other things being equal. For example, Attack vs. AC should be around 6 + level for a character to hit roughly 60 to 65% of the time. Of course, other factors (monster is a soldier vs. an artillery, for example) come into play that affect these, but they're just guideline figures.

I like these general guidelines rather than trying to measure things with hard figures, because you could get led astray with the hard figures if the underlying premises end up being false. Garbage in, garbage out, and you lose sight of the big picture.

Post/Author/DateTime	Post
#1 mitok Aug 10, 2010 15:54:36	I am normally the DM of our group, but one of our players is trying to put together a spreadsheet to use as a sort of statistical analysis tool (he says his first love is baseball, and loves tracking statistics). He's looking for quantifiable measures he could use to track the performance of all the players, so that everyone can look back at prior encounters and perhaps determine where they need to make improvements. I am thinking that certain quantifiable metrics are obvious, like damage per round, but not necessarily applicable or as important to all roles. Are there other metrics that make sense for other roles besides strikers in this regard? My initial thinking is that there are too many variables (group tactics, buffs, use of terrain, combat advantage scenarios, etc.) for this to be a worthwhile effort, and that the players should just look at prior combat encounters as a whole and judge from just a general impression. Did strikers hit very often? Did defenders do a good job keeping the monsters on them? Did leaders keep players in the fight? Did controllers ... control? Thanks for any advice.
#2 mobar2008 Aug 10, 2010 17:22:45	You might track how much damage a defender soaked for the team....this isn't an easy question because most of the outcomes are based on something completely out of the players control, the dice. A series of bad rolls can kill a good plan, just as a series of awesome rolls can make a poor plan shine. Then theres the varible of the encounter itself. Fighting the big bad is going to be tougher than the first encounter of an adventure. I'm thinking that with all the things that is OUT of the players control meaningful stats are impossible to come up with.
#3 geoffa Aug 11, 2010 8:49:27	Let me start by agreeing with the last several posters. DnD is complicated and keeping stats is not going to give an exact measurement of how valuable each player is, and how successful the party is. I think it would also be quite time consuming and distract from playing the game unless you had some way to quickly record stats and count them up later (probably involving a computer, or maybe an ipod app). Having said that, I am a math teacher, so I am all about trying to quantify things. Here's a couple ideas of stats you might want to keep for different characters. All Characters Hit percentage - that seems pretty valuable for all characters. Strikers should be hitting a lot and dealing out damage. Controllers should be hitting a lot and throwing out hindering effects. Even for leaders, this would be interesting to look at because if a leader has low hit percentage then perhaps it's a sign they should concentrate on other leader activities instead of attacking. Hit percentage of encounter/daily powers - do you actually hit with the attacks that matter? Strikers Damage per round - this seems to be a pretty standard measure of striker effectiveness. Opponents killed - this is a group effort, but you would expect the striker to be doing this more than most characters. Defenders Attacks drawn percentage - how often does the enemy attack the defender compared to everyone else. I don't think you actually want this to be TOO high, but it should definitely be more than average. Number of times squishy characters get bloodied - this is kind of like using number of QB sacks as a measure of how good the offensive line is. Controllers Minions killed - controllers should be good at hitting groups of minions. Opponents hindered - number of times the controller manages to deal out dazed, slowed, immobilized, blinded, etc. You could get all complicated and assign some of those effects to be worth 2 or 3 points but I'd keep it simple and count them all the same. Leaders Number of times another character falls unconscious - this seems to be a good measure of healing effectiveness. Characters can get bloodied, no problem, but if they fall unconscious then they are missing turns when they could be doing something useful. Assist - number of times another character succeeds in a roll that would have failed without a leader boost. This might be kind of hard to measure.
#4 mitok Aug 11, 2010 9:17:18	Thanks for the responses, guys. It pretty much sums up what I was thinking. I happen to own the Player's Strategy Guide (the one with Penny Arcade art on the cover), and there are two side-bars that pretty much say the same thing, and give an idea of how one should really measure success in this game. The first sidebar, "Defining Success" on page 111, and the second is "Know the Numbers" on page 112. "Defining Success" says you should judge encounters by how many resources were expended that can only be regained via an extended rest, such as healing surges, daily powers, and so forth. "A successful encounter, then, is one that expends as few of your most valuable resources as possible." "Know the Numbers" says that the game is designed such that monsters should be hitting characters about 50% of the time on average, and characters should hit monsters roughly 60 to 65% of the time. It even has a handy little table that shows what the average attack and defense values should be, all other things being equal. For example, Attack vs. AC should be around 6 + level for a character to hit roughly 60 to 65% of the time. Of course, other factors (monster is a soldier vs. an artillery, for example) come into play that affect these, but they're just guideline figures. I like these general guidelines rather than trying to measure things with hard figures, because you could get led astray with the hard figures if the underlying premises end up being false. Garbage in, garbage out, and you lose sight of the big picture.