uDG08 Voting Results and Methods

⌘-R in Chief
Posts: 1,247
Joined: 2002.05
Post: #31
Quote:I don't personally feel that median votes are somehow more fair or representative of what a game deserves. But that's just my opinion.

It's not an opion, it's a fact. If game A has ten scores as 8, and ten scores as 7, and game B has twenty scores as 8, obviously they are not tied.

And no, using the mean of other categories to determine tie breaking in a specific category is not a good idea. You do not choose the winner of the audio category based on how good someone's story was. That's just silly.

The median is not an acceptable means of determining rank, but it is useful for determining outliers in a distribution. Instead of arbitrarily picking a top and bottom percentage to ignore, you could ignore only scores that are "too far" from the median - the outliers. "Too far" is a subjective, but for example, you could ignore results more than 3 away from the median. If the median is 7, ignore anything but scores of 4, 5, 6, 7, 8, 9, and 10.



--


Also, we could increase the range of values. Instead of just 1 to 10 (which is ok with a small number of entries), in order to let voters have more flexibility with ranking the games they played we should probably double the range. IOW, there were two games I liked almost exactly
as much as each other, but I thought one was a bit better. Therefore I want to give it a higher score, but a whole point is a pretty big advantage when there are such few votes.
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #32
Quote:It's not an opion, it's a fact. If game A has ten scores as 8, and ten scores as 7, and game B has twenty scores as 8, obviously they are not tied.
That was a special case and I had already stated that Game A would have a 7.5 if this were the case.

Quote:And no, using the mean of other categories to determine tie breaking in a specific category is not a good idea. You do not choose the winner of the audio category based on how good someone's story was. That's just silly.
I don't think it's silly. If two games are scored the same in graphics then the better game should get the vote. I'm not singling out specific categories either, like in your example.

Quote:The median is not an acceptable means of determining rank, but it is useful for determining outliers in a distribution. Instead of arbitrarily picking a top and bottom percentage to ignore, you could ignore only scores that are "too far" from the median - the outliers. "Too far" is a subjective, but for example, you could ignore results more than 3 away from the median. If the median is 7, ignore anything but scores of 4, 5, 6, 7, 8, 9, and 10.
I still don't see why we don't just use the median.

Quote:Also, we could increase the range of values. Instead of just 1 to 10 (which is ok with a small number of entries), in order to let voters have more flexibility with ranking the games they played we should probably double the range. IOW, there were two games I liked almost exactly
as much as each other, but I thought one was a bit better. Therefore I want to give it a higher score, but a whole point is a pretty big advantage when there are such few votes.
I agree with this, maybe we could adjust the voting interface to allow the user to score it by typing. With standard radio buttons 1-10 and then an "other" radio button that would allow the voter to type in a 1.0-10.0 entry.
Quote this message in a reply
Member
Posts: 26
Joined: 2008.04
Post: #33
Quote:That was a special case and I had already stated that Game A would have a 7.5 if this were the case.

Then assume that Game A had five 7's and six 8's, and that Game B had eleven 8's. The median would give both of these titles an 8, would it not?
Quote this message in a reply
⌘-R in Chief
Posts: 1,247
Joined: 2002.05
Post: #34
If the scoring range was 1 to 1000 there wouldn't even be median.

*Every* vote should count*. Not just the ones where you're part of a plurality.





Quote:If two games are scored the same in graphics then the better game should get the vote.

The whole point of the categories is to rank games on how good they are in that category alone. It should have absolutely nothing to do with any other category. Ever.

Add on top of that the fact that nearly every game would be tied with at least one other, and that just makes the ranking in individual categories absolutely useless.




* (except the ones believed to likely be biased and therefore are being rejected by some scheme)
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #35
Quote:Then assume that Game A had five 7's and six 8's, and that Game B had eleven 8's. The median would give both of these titles an 8, would it not?

Yes, but I think if the median range is between two numbers, then it should be (7+8)/2. That goes along with the special case. Suppose if these two titles tied, the eleven 8's should win.

if the median range is greater than 2, then the votes were more than likely nerfed. I can't see 19 judges, half of them giving the same entry a 4, and the other half a 7. If this were the case though, then that entrant should receive the mean. (it would be nerfed, but at least to no extreme advantage or disadvantage.)
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #36
Quote:If the scoring range was 1 to 1000 there wouldn't even be median.
It's not between 1 and 1000, and even if it were, there would still be a median. Because there would still be a middle value, or if there are an even number of votes, two middle values that could be added and divided by two.

Quote:The whole point of the categories is to rank games on how good they are in that category alone. It should have absolutely nothing to do with any other category. Ever.
I agree, but there should also be tie breakers.

Quote:Add on top of that the fact that nearly every game would be tied with at least one other, and that just makes the ranking in individual categories absolutely useless.
How would nearly every game tie? if two games receive a 9 for audio, its obvious that the sound quality was really good for both. I say the better overall game gets the win though (with the other 2nd to it), because a greater effort was put into making it. I really don't see how this makes individual categories useless.
Quote this message in a reply
⌘-R in Chief
Posts: 1,247
Joined: 2002.05
Post: #37
Game 1: 17, 17, 17, 17, 17, 17, 18, 18, 19, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 15, 16, 16, 17, 17, 17, 18, 18, 18, 18, 18, 18 = 16.7


Far and away Game 1 wins. more people liked it than liked Game 2, yet according to the median Game 1 gets a 17, and Game 2 gets an 18?

How is that fair?

Median is NOT a relevant way to rank anything.
Quote this message in a reply
Member
Posts: 37
Joined: 2009.02
Post: #38
DavidGuy Wrote:Just a quick note about the prizes column:

My entry (Maneuv'It!) is not eligible for 3 prizes but only 2, because my two 3rd-place-finishes are in the same "prize-round" and the rules state that more than one choice per round is not allowed. So I get to pick once in the 3rd round, and once in the 4th round.

As another example Laserface is not eligible for 6 prizes, but rather 3. One pick in each the 1st (for best overall), 2nd (for its four 1st place finishes) and 4th (for its one 3rd place finish) rounds.

This comment by David got me wondering as I had not paid much attention to this part of the rules before. I went back and looked at the rules and was surprised to find that the rules state something even more extreme! To quote:
***
Each round consists of the following steps:

1. First place in the “Best Overall Game” category will select one prize from the pool.
2. Second place in the “Best Overall Game” category will select one prize from the pool.
3. Third place in the “Best Overall Game” category will select one prize from the pool.
4. The first place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
5. The second place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
6. The third place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.

No winner may select a prize from the pool more than once per “round.”

***
So by the definition in the rules, the "round" is the entire array of winners, from first to third place in all categories. All of the steps are part of the round. Which means that Laserface is not eligible for 6 or even 3 prizes but only one prize each time the loop is gone through - the same number as every other winning entry.
That comes as a big surprise but mainly because I didn't bother to read the rules thoroughly before. So is this stated incorrectly or is this the actually system? Carlos?
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #39
Quote:Game 1: 17, 17, 17, 17, 17, 17, 18, 18, 19, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 15, 16, 16, 17, 17, 17, 18, 18, 18, 18, 18, 18 = 16.7


Far and away Game 1 wins. more people liked it than liked Game 2, yet according to the median Game 1 gets a 17, and Game 2 gets an 18?

How is that fair?

Median is NOT a relevant way to rank anything.

That's not so, the median of Game 1 is 18, the median of Game 2 is 17. I think you think I mean the mode, when I mean the median and not the mean. Although now I'm willing to settle with the Interquartile mean, I just thought the median would be easier to do.
Quote this message in a reply
⌘-R in Chief
Posts: 1,247
Joined: 2002.05
Post: #40
I just realized I was thinking mode myself. But reading everything else mode made sense where everyone was saying median. I'll have to rethink this...........


Edit: Ok:

Game 1: 17, 17, 17, 17, 17, 17, 17, 18, 20, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 14, 15, 16, 17, 18, 18, 18, 18, 18, 18, 18, 18 = 16.7

Median (not mode Rolleyes ) says that Game 2 wins with 18 versus Game 1's 17. Clearly the mean is better here.
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #41
Quote:Game 1: 17, 17, 17, 17, 17, 17, 17, 18, 20, 20, 20, 20, 20, 20 = 18.3
Game 2: 14, 14, 14, 15, 16, 17, 18, 18, 18, 18, 18, 18, 18, 18 = 16.7

Median (not mode ) says that Game 2 wins with 18 versus Game 1's 17. Clearly the mean is better here.

ok, so we will call this an example of the new system. (seeing as how everything is doubled)
Game 1 would've gotten a 17.5
Game 2 would've gotten an 18 you are right on that, but is the mean really better here?
Didn't game 2 get nerfed by those 14's, how is it that three devs felt it deserved a 14 and eight thought it got an 18, yet it scored a 16.7

Maybe the interquartile mean would work best, 25% off the top and bottom, then averaged. Like someone mentioned earlier that's how it's done in sport competitions.
Quote this message in a reply
Member
Posts: 749
Joined: 2003.01
Post: #42
Obviously the mean is better sometimes and the median other.

©h€ck øut µy stuƒƒ åt ragdollsoft.com
New game in development Rubber Ninjas - Mac Games Downloads
Quote this message in a reply
⌘-R in Chief
Posts: 1,247
Joined: 2002.05
Post: #43
tcIgnatius Wrote:Didn't game 2 get nerfed by those 14's, how is it that three devs felt it deserved a 14 and eight thought it got an 18, yet it scored a 16.7

What gives you the right to say someone got nerfed for giving their opinion? If it's 1 when the mean is 17, that's a different story. Just because someone didn't like it exactly as much as everyone else doesn't mean their vote shouldn't count.

Some people like gore. Some don't. The ones that do will like a game with gore more than the people that don't like gore.

You're going to ignore the votes from people who don't like gore just because there aren't as many of them? That's illogical.

The ideal is to ignore obviously biased votes, not silence people who simply don't agree with the majority.
Quote this message in a reply
Member
Posts: 65
Joined: 2008.09
Post: #44
Quote:What gives you the right to say someone got nerfed for giving their opinion? If it's 1 when the mean is 17, that's a different story. Just because someone didn't like it exactly as much as everyone else doesn't mean their vote shouldn't count.
I agree, their opinion should count, it's just that mathematically, having a mean makes the minority opinion weigh more than the majority.

Quote:The ideal is to ignore obviously biased votes, not silence people who simply don't agree with the majority.
Yes, the median does not "silence" those who disagree with the majority, it actually makes the weight of their choice equal to each individual voting.
Quote this message in a reply
Apprentice
Posts: 10
Joined: 2009.02
Post: #45
pinguoren Wrote:This comment by David got me wondering as I had not paid much attention to this part of the rules before. I went back and looked at the rules and was surprised to find that the rules state something even more extreme! To quote:
***
Each round consists of the following steps:

1. First place in the “Best Overall Game” category will select one prize from the pool.
2. Second place in the “Best Overall Game” category will select one prize from the pool.
3. Third place in the “Best Overall Game” category will select one prize from the pool.
4. The first place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
5. The second place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.
6. The third place winners in each of the five categories will then each select one prize from the pool, in order of descending overall score.

No winner may select a prize from the pool more than once per “round.”

***
So by the definition in the rules, the "round" is the entire array of winners, from first to third place in all categories. All of the steps are part of the round. Which means that Laserface is not eligible for 6 or even 3 prizes but only one prize each time the loop is gone through - the same number as every other winning entry.
That comes as a big surprise but mainly because I didn't bother to read the rules thoroughly before. So is this stated incorrectly or is this the actually system? Carlos?

Hmmm, ... your right. Blush

My (it seems incorrect) reading of the "Prize Distribution" rule was that there were only 6 rounds, but the rule does state each round consists of 6 steps and the rounds repeat "until the prize pool has been exhausted".

So, yea', ... it seems the main benefit of placing high is not that you get more picks, but you get to pick sooner in each round.

Sorry for the confusion ...
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  Congratulations to all uDG08 Participants! AnotherJake 7 5,169 Mar 7, 2009 06:04 PM
Last Post: PowerMacX
  uDG voting DoG 11 5,749 Mar 7, 2009 09:57 AM
Last Post: FlamingHairball
  Questions regarding voting? FlamingHairball 20 9,177 Mar 6, 2009 10:00 AM
Last Post: FlamingHairball
  How will public voting work? pinguoren 7 5,039 Mar 2, 2009 11:22 AM
Last Post: Carlos Camacho
  Networking Minicontest "Voting" aarku 26 10,833 May 9, 2008 08:36 PM
Last Post: PowerMacX