Polish Voting by Judges

Moderator
Posts: 592
Joined: 2002.12
Post: #16
Maybe the best solution is that if a judge cannot run the game due to performance issues then they just do not vote on that game. That way the score for the game is not affected due to the developer being rich and the judge poor.

A side note: no idea where you get the idea that developers have the latest kit - I've never worked in any development job where the computers have been state of the art. As for my home computers, I've only ever twice had a brand new just released model (Amiga 1200 when the magazines still had it classed as only a rumour, and iMac G4 about a month after it was released.) However neither of these computers were top of the range models.
Quote this message in a reply
Moderator
Posts: 522
Joined: 2002.04
Post: #17
Am I the only one who read the thread title to be Polish (people from Poland) Voting by Judges? Blink

That aside, I think at the minimum a handful of developers spent a great deal of time testing their games on as many different Macs as they could. I personally spent a great deal of time getting things running on an old Rage 128, and trying to figure out why sometimes fullscreen would switch resolutions on a handful of people's machines. I also spent time testing other entries on really weird systems to help them out with compatibility. (Beige 266mhz G3 Desktop with Mac OS X 10.2.0 and a Radeon 7500 PCI, anyone?)

It may be impossible for some people's games to run on old systems, however. I do not know how those should be judged, but I just wanted to point out that it seems strange that developers who took time away from adding bells and whistles to their game to work on compatibility get treated the same in the polish category as an entry that focused on said bells and whistles. Does that make sense?

-Jon
Quote this message in a reply
Member
Posts: 715
Joined: 2003.04
Post: #18
The best solution was presented on the judging page
Quote:The judges will use the following criteria to guide them:
The game was straight-forward to install.
The game's icons were graphicly pleasing and visible.
The game's introduction or splash screen was presented professionally.
The game's menu and setup screens/windows were intuative.
The game felt like it belonged on a Mac (it uses established conventions for Macintosh GUI applications).
The game ran smoothly.
The game quit gracefully and caused no problems afterwards

7 items worth 0.7142857142857143 points for each YES answer, left up to the hundreds of people who voted.

This would have avoided this entire last minute fiasco.

Also sending feedback to the dev through the voting page would have helped, sending them feedback 120 days after the contest starts is fairly useless, many devs will not
continue to work on their games after contest. So what good is that delayed feeback?

Also my suggestion at the beginning of this contest for feedback on the udevgames site, intstead of splitting things up, would have benefitted the developers. Its ludicrous to think people will sign up at a game developer site just to post feedback when they are not interested in game development themselves.

There is no link to iDevGames from the download page.
There is no link to iDevGames from the voting page.
There is no link to iDevGames from the udevgames home page.
etc etc etc etc
How are people downloading these games going to know where to send feedback?
The iDevGames read me? About .0001% of downloaders will bother to read the game read me, .0001% of those will bother to read the iDevGames read me.


skyhawk Wrote:best beta testing, releasing it to the IMG forums and say "hey guys, bitch at me". seriously, they are great for feedback
I think the Gokuban thread would prove you wrong. Request for feedback posted 10-28-04, total replies= 5, total devs who participated in feedback=1.
Also feedback from idevgames does not guarantee that the game won't run on some judges machine. Do you see a crash report in the "Dead Days" thread? No you don't.
It took over 3,000 public downloads to get the first crash report.

Also udevgames competitors can't be trusted:
observe this comment left at a game site.
Dr. Light Wrote:Sucks big time
If all (32 competitors x number team members), left comments like that around the web about each UDG entry...you know what this contest would look like to the eys of the public?

Camacho Wrote:They are after all, people who work fulltime
And what about the people who participate in this contest?
Do you see them all as jobless slobs?

Rerturning to arekkusu's comments about renting a lab, 98% of us don't live within pissing distance of Apple computer. The labs don't exist for the majority of the devs.
Nor is the expense of renting one even feasible in this contest. Nor is the ability for many people to tell their bosses : "Hey I'm taking a day off work to test my freeware"
Or their school "My education is not as important as this contest".
I can see Tobi telling the old people he takes care of "Fend for yourself feeble ones, I've got beta testing to do!"

If one is releasing shareware or commerical product, you get ca$h money from each person who pays for the software when it works on their machine, and people who want to pay for the software send in information regarding problems because they have a need for the software.
Your suggestions are really your own "high ideals" and not reality, because, even Apple can't release software that works on all its computers, and they have hundreds of millions of dollars in resources, and the man power. This can easily be illustrated if you read macfixit, and Apple's support pages. The most recent 10.3.6 update is causing trouble with hundreds of thousands of dollars worth of hard drive equipment and openGL slowdowns from 30fps to 1fps on Blizzard games, more issues are likely to develop since the upgrade is only 7 days old.

Camacho Wrote:Sorry if that's not really the reply everyone is looking for

Likewise
Quote this message in a reply
Moderator
Posts: 434
Joined: 2002.09
Post: #19
I mostly agree with igame3d's points; our resources are limited and we don't always get the feedback we need. It's kind of annoying to be told otherwise.

But here's how I look at it: being judged is always a crapshoot. Good judges try to be fair but ultimately it's a subjective process. If my game runs on 99.9% of the machines out there, but it happens to crash on one of the judge's machines because I never was able to test on that particular setup, then that's just bad luck. I'd better get over it. I can improve my odds by putting up betas early, sticking to lowest-common-denominator OpenGL functionality, etc. But nothing is guaranteed.

In my opinion if a game advertises itself as not meeting a judge's specs then the judge probably shouldn't vote on that one. If it doesn't have minimum specs written out, or it says that it should run on their machine, but it runs badly or crashes, then polish will be very low from that judge. It might not even be the developer's fault, it might be something very strange about that judge's computer, or due to a random electrical surge. But that's just life! When a skater falls down in the Olympics their scores are low, even if it was because their laces broke.

This isn't just a theoretical topic for me; I had a bug that rendered an important part of my game invisible on some machines. I managed to fix it a few days later but I'm still not sure what the problem was, never reproduced it on my machine. For all I know the judges all downloaded the version with the bug. Oh well!

Just my opinion.

Measure twice, cut once, curse three or four times.
Quote this message in a reply
Moderator
Posts: 335
Joined: 2002.04
Post: #20
As far as the testing vs resources situation is concerned, uDev is like a scrunched version of a typical project. There's no way you'll be able to test on all the unusual frankemacs out there and there's bound to be an issue on one or two of them after you release a game, no matter how much you test.

As far as my judging went, the only project that I couldn't play was one that wasn't on the downloads list, the rest all at least started up and showed screens. I'd like to have waited until the last possible second to test them, but there's this great big "things to do and bugs from earlier releases to fix" list I'm working on Wink
Quote this message in a reply
Founder
Posts: 1,138
Joined: 2002.04
Post: #21
>Am I the only one who read the thread title to be Polish (people from Poland) Voting by >Judges?
Actually, a distinguished judge replied to me, "I wonder why Carlos would think I was from Poland." So, you are not alone.

>Do you see them all as jobless slobs?
No, not what I meant.

Anyways, here is what I shall go with. I've asked judges to report any games that they were not able to test due to (a) download being offline (b) their machine not meeting required specs © game not running at all. I will take their list of untested games, and remove their votes. Thus, they only vote on games that they were able to properly fully test.

Carlos A. Camacho,
Founder
iDevGames
Quote this message in a reply
Moderator
Posts: 771
Joined: 2003.04
Post: #22
I know this is probably asking too much but, assuming the Judges didn't erase the games after downloading & testing them, would it be possible to know which version/release they judged? (At least for games in which the version is either part of the compressed download filename or in plain sight in the ReadMe)

...or, just when did they download the games, so every developer can figure out the version by themselves...
Quote this message in a reply
Founder
Posts: 1,138
Joined: 2002.04
Post: #23
>I know this is probably asking too much but
A bit too much for this year's rushed contest. But next year, I will leave a space next to each entry so they can enter in the version # reviewed. Good suggestion. Remind me in a few months!

Carlos A. Camacho,
Founder
iDevGames
Quote this message in a reply
Moderator
Posts: 771
Joined: 2003.04
Post: #24
Carlos Camacho Wrote:Remind me in a few months!
Will do Smile
Quote this message in a reply
Post Reply 

Possibly Related Threads...
Thread: Author Replies: Views: Last Post
  uDG08 Voting Results and Methods AnotherJake 111 47,046 Mar 20, 2009 05:15 PM
Last Post: pinguoren
  uDG voting DoG 11 5,730 Mar 7, 2009 09:57 AM
Last Post: FlamingHairball
  Questions regarding voting? FlamingHairball 20 9,114 Mar 6, 2009 10:00 AM
Last Post: FlamingHairball
  How will public voting work? pinguoren 7 5,022 Mar 2, 2009 11:22 AM
Last Post: Carlos Camacho
  Networking Minicontest "Voting" aarku 26 10,786 May 9, 2008 08:36 PM
Last Post: PowerMacX