|
| Thread: Recruitment of judges to the second ICTC contest |  This thread is pages long: 1 2 3 4 · «PREV / NEXT» |
|
Gnoll_Mage
Responsible
Supreme Hero
|
posted January 10, 2007 05:09 PM |
|
|
Thanks, here's the link,
http://h.1asphost.com/GnollMage/marking.xls
How it works: you put a "y" in the "Present" column to show that the person included that particular item.
Then in the "Marks given" column, you give them a mark out of the relevant number (shown in "marks available" column).
Things that are more important have a higher mark total available.
Also, if something that should definitely be included isn't, they are penalised; similarly if they do something 'extra' such as music or campaign stories, they get additional points.
At the moment there is a sample entry. They did everything they had to but nothing extra, and got full marks in every section they attempted... 
Will, I'm on ToH at the moment if you want to comment that way. I've IMed you the password.
____________
|
|
alcibiades
Honorable
Undefeatable Hero
of Gold Dragons
|
|
posted January 10, 2007 05:20 PM |
|
|
Ehm ... the Spread Sheet is a bit - eh, actually very - confusing! I think it could definitely need a graphical overhaul to make it more easy for users to understand which numbers they should put where! If you want to, I would be willing to help you with this - just send me a PM if you want to talk about it.
____________
What will happen now?
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 10, 2007 05:25 PM |
|
|
I think that's a good way of describing it! Sorry, I forgot to say as I did to GenieLord that it's in draft format at the moment. You're welcome to tidy it up and add a little Alcibiades magic Plus I've sent you an IM
____________
|
|
Ted
Promising
Supreme Hero
Peanut Exterminator
|
|
posted January 10, 2007 07:48 PM |
|
|
i am now resigning from being a judge, i am now a contestent
____________
Visit my Site!
|
|
GenieLord
Honorable
Legendary Hero
|
|
posted January 10, 2007 09:37 PM |
|
|
Quote:
Thanks, here's the link,
http://h.1asphost.com/GnollMage/marking.xls
How it works: you put a "y" in the "Present" column to show that the person included that particular item.
Then in the "Marks given" column, you give them a mark out of the relevant number (shown in "marks available" column).
Things that are more important have a higher mark total available.
Also, if something that should definitely be included isn't, they are penalised; similarly if they do something 'extra' such as music or campaign stories, they get additional points.
At the moment there is a sample entry. They did everything they had to but nothing extra, and got full marks in every section they attempted...
Will, I'm on ToH at the moment if you want to comment that way. I've IMed you the password.
I think it's good.
We can use it to do the full list.
My suggest of categories:
Race
History
Creatures
Town
Balance
Originalilty
Class skill
And small things like emblem can't be in the final judgment.
But, as I said, if a competitor doesn't do them, every judge should notice and take him some points from the judge's rating.
Every judge has to do that, not just the chief of judges.
S/He gives him a minus of point,according to how much s/he thinks it misses:
Emblem: 1-3
Symbul(s): 1-3
Town's description: 3-5
Creatures' statuses: 4-6
Creatures' abilities: 4-6
Etc.
About bonuses to extra work:
If a competitor did something extra, like campaign, If you like it, you can give him bonus of points to the final score.
If you don't think it's good, don't give the bonus.
The maximum of point that you can give as a bonus is 5.
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 10, 2007 11:24 PM |
|
|
That sounds pretty much like what the sheet does, so I think we're alright there. Points are deducted if something is missing, added if something extra is done. But the actual numbers and flexibility are slightly different.
Hmm, it seems it's going to be difficult to get this to everyone's liking. I could add variable bonuses, but it seems a little pointless since the score from a section that receives a bonus can be changed by modifying the content score. Similarly, it would be possible to allow the judge to vary penalties too - but if you want to penalise more, it would probably be easier for you to just give it less content marks.
Opinions on GL's / my ideas?
Oh,
Quote:
And small things like emblem can't be in the final judgment.
That's one of the things that gets a bonus at the moment. Do you not agree with this then? It is currently worth 1-3 marks.
PS prettier version here. Still uses the same maths. Other than any adjustments people want, it still needs to check that marks given are in the acceptable range, and also it should really allow the judge to add their own modifier after scoring if they feel that the mark has come out slightly wrong (after all it's silly to think that this will generate completely `sensible` results each time, so we should trust the judges' opinions on small modifications to the score...).
GM
____________
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 10, 2007 11:46 PM |
|
|
 Sorry, very silly of me to think people can easily suggest adjustments when I've left the sheet in such a mess. Basically you need to look at the Marks Available column to see my marks allocation. The Present column tells you: if negative, how many marks are lost for NOT including the section; if positive, how many marks are gained for ADDING the section in as an `extra`. If you think my relative importances - mark allocations - are wrong (which they probably are), suggest some improvements, with justification, for a better selection. Ditto for the bonuses and penalties - should they be bigger, smaller, edited etc.? Finally, any other ideas? Are the sections chosen for marking in need of change? Is this sheet ultimately completely useless or rubbish?
____________
|
|
alcibiades
Honorable
Undefeatable Hero
of Gold Dragons
|
|
posted January 11, 2007 09:46 AM |
|
|
I would like to add something:
I like the idea of a Spreadsheet to type into, and which will make the summation.
I do not like the idea of the spreadsheet distributing the points - like detracting a fixed number if this is missing, or adding a fixed number if that is included. If we go for that, we don't need judges! If we want to use that approach, we just have to develop one spreadsheet that everybody agree on, and we can fill in the info for each faction to calculate the winner.
I think the judging should be subjective. There can be guidelines - like, you should look for this or that, and these things should be included - but it should be up to the judge to put down the number for his evaluation. Therefore, I would prefer to have a much simpler spreadsheet accompagned by a judging guide (like a text document). The spreadsheet should simply look like:
Race Score [ ] (type number from 0 to 10)
Creature Score [ ] (type number from 0 to 10)
History Score [ ] (type number from 0 to 10)
etc.
And then the spreadsheet will calculate the final score (0.3 * Race Score + 0.3 * Creature Score + 0.2 * History Score + ... or whatever wheights we decide on) which will come out as a number between 0 and 10.
____________
What will happen now?
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 11, 2007 07:38 PM |
|
|
Okay Alc, that means we need to agree on categories and weights. What do you think is the easiest way of doing this?
(Really though, my and your ideas are very similar - both give some overarching categories with subsections within them, both weight the sections differently, and my sheet does allow for plenty of judge input - deciding what the scores are. I don't see the sheet reduces the need for a judge. If you think too many points are fixed (i.e. the bonuses and penalties, which I feel should be there in some fashion), they can be minimised, but ultimately I think there should be fixed penalties for obvious sections, and small bonuses for extras. The slight difference I suppose comes in
Yours:
Hero info /10:
Skills
Stats
Bios
History /10:
ETC
Mine:
Hero skills / 4
Hero stats /3
Hero Bios /3
ETC
Which are fairly similar, aren't they? You basically undergo the below process in doing your method, but in your head. And if you want you can always with the second `bleed the categories` a little and put a few extra points in one category because you felt the relevant one wasn't quite large enough weight-wise.
)
Very sorry if I appear whiny, I just don't quite understand your position. But it seems your idea is preferred, so we probably now need to decide on actual sections as I said.
____________
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 11, 2007 07:48 PM |
|
|
Quote:
I do not like the idea of the spreadsheet distributing the points - like detracting a fixed number if this is missing, or adding a fixed number if that is included.
I can see where you're coming from, but my sheet awards the majority of marks (86% I think) for content quality, and the remaining marks are for bonus items (36% of the remaining marks are for sections very rarely seen in ICTC that I think should be credited). The penalties are even less noticeable since the sections they apply to will basically always be in anyway.
Quote:
If we go for that, we don't need judges!
I think that's a little extreme! There is still ample room for `subjection` I feel. And figures can be changed / categories merged to create yet more subjective judging. That wouldn't be a problem.
Quote:
If we want to use that approach, we just have to develop one spreadsheet that everybody agree on
That will be tricky won't it? As I said before, how best could this be done? Could we use the chatroom again, or is getting most/all online too infeasible?
Finally as I said before there will be a judge's factor so that they can tailor the final mark subjectively - after all, I also think that the slightly robust spreadsheet won't necessarily generate the `best` possible mark each time. Although I do think it's the best way to do it.
____________
|
|
GenieLord
Honorable
Legendary Hero
|
|
posted January 11, 2007 08:32 PM |
|
|
Let's do a summary:
The categories are:
Race
History
Creatures
Heroes
Town
Originality
Class skill
With that everyone agrees, right? And Alc will you do the document you talked about because is sounds good to me.
About the bonuses, it's a matter if we want the opinion of all the judges. I want. Do you want?
If we will approve it, the maximum bonus that a judge can give to a competitor is 5, for all the extra things together.
And anyway, emblem has to be. This is the rule.
Is it fine with everyone meanwhile?
____________
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 11, 2007 08:57 PM |
|
|
Here is a far better version of my sheet that forms a compromise between the old one and Alc's idea. There are less structed marks for sections, and less bonuses and penalties. It does still need the judge's factor. This is my `ultimate choice` I think.
It fits extremely closely to GL's categories so I would appreciate it if he allows use of my sheet. Obviously it needs his adjustments added (emblem a must - something I disagree with, but hey; and less bonuses available).
Questions - are we okay with using my sheet now?
It fits with Alc's idea since there are only a few categories, which are weighted, and bonuses and penalties are smaller (especially bonuses). If he wants, it can be cut down further. Also, it can be made to fit with GL's ideas.
GL, could you explain why you think the emblem is necessary? (I do mean drawing it here, not just describing it.) Also, you cannot really enforce the five rule on my sheet (after all 5 counts for very little when the total is 170 or whatever), but I'm happy to cut bonuses down, as I said above re. Alc's ideas.
____________
|
|
VokialBG
Honorable
Legendary Hero
First in line
|
|
posted January 11, 2007 09:18 PM |
|
|
Quote:
Let's do a summary:
The categories are:
Race
History
Creatures
Heroes
Town
Originality
Class skill
Quote:
We need category for small details, and "Others" like Town specialization, maybe extra points for images, and town music... like extra points for unobligatory parts from the fractions
____________
|
|
GenieLord
Honorable
Legendary Hero
|
|
posted January 11, 2007 09:24 PM |
|
|
Quote:
Here is a far better version of my sheet that forms a compromise between the old one and Alc's idea. There are less structed marks for sections, and less bonuses and penalties. It does still need the judge's factor. This is my `ultimate choice` I think.
It fits extremely closely to GL's categories so I would appreciate it if he allows use of my sheet. Obviously it needs his adjustments added (emblem a must - something I disagree with, but hey; and less bonuses available).
Questions - are we okay with using my sheet now?
It fits with Alc's idea since there are only a few categories, which are weighted, and bonuses and penalties are smaller (especially bonuses). If he wants, it can be cut down further. Also, it can be made to fit with GL's ideas.
GL, could you explain why you think the emblem is necessary? (I do mean drawing it here, not just describing it.) Also, you cannot really enforce the five rule on my sheet (after all 5 counts for very little when the total is 170 or whatever), but I'm happy to cut bonuses down, as I said above re. Alc's ideas.
Better now. It fits most of our opinions, which is good.
For me, things like miscellaneous you can add to bonus, and remember only if you like it.
The reason is that if a competitor just did a fast and silly thing just for getting the bonus, he won't get the bonus.
S/he has to put effort in it and do good thing.
Emblem is one of the basic demands when you create a race. I don't think anyone has a problem to open Paint and draw it.
And if a competitor didn't do something like the whole town, you don't take point from him, you give him rating 0 in the category "Town".
This far I understood GM agrees, right?
Now all what we need is Alc's (important) opinion.
____________
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 11, 2007 09:24 PM |
|
|
Vokial: That you can find in my excel sheet at the bottom.
GL - of course, if someone did something silly, then don't give them the bonus on my sheet by simply not putting a y in the present column.
____________
|
|
GenieLord
Honorable
Legendary Hero
|
|
posted January 11, 2007 09:27 PM |
|
|
Quote:
Quote:
Let's do a summary:
The categories are:
Race
History
Creatures
Heroes
Town
Originality
Class skill
Quote:
We need category for small details, and "Others" like Town specialization, maybe extra points for images, and town music... like extra points for unobligatory parts from the fractions
This why we have bonus points. 
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 11, 2007 11:56 PM |
|
|
I've made a new version. Will post it up as soon as possible (at the moment the hosting service is down).
____________
|
|
alcibiades
Honorable
Undefeatable Hero
of Gold Dragons
|
|
posted January 12, 2007 01:28 AM |
|
|
It's deffinitely better, Gnoll Mage. Of course, some things hold to much weight to my taste:
- Heroes: Heroes descriptions etc. is for me completely waste of time! If somebody want to write them, fine - and you can throw in a couple of bonus points for that - but making Hero history is *completely* unimportant for the quality of the faction. Same for starting skills etc. - anybody can do that, it's just a matter of time, and for me holds no importance for the quality of the faction - and therefore be omitted or at best be granted only a *very* few points (therefore: go for a bonus point section).
- Campaign Storyline: Same as above. People writing more or less crappy dialogues for Campaigns offers me absolutely nothing - and I will not rate a Faction better or worse because someone wrote a storyline for a Campaign. Again, throw in bonus if you want, but you should not be detracted for not having this.
- Images: Be very carefull about putting too much weight on Images (that being creatures, buildings, or other things). Most of us don't develop these things ourselves (in fact, most of us aren't skilled enough to do that, with notable exceptions!) - and therefore, you should not get penalties for not spending as much time on googling up a nice image. Of course, nice Images will add to the overall impression, and thus on the overall score - and that's fine - and might even throw in some bonus points to compensate for extra nice images.
I would weigh the categories something like:
Basics: 10 Points [Warship, Magics, Philosophy and Motto, Geography, Emblem]
History: 10 Points
Race: 20 Points
Creatures: 25 Points
Town: 10 Points
Skills and Abilities: 15 Points [Class Skill + Ability 10 points, others 5 Points]
Miscelaneous: 10 Points
Miscelaneous should include Campaign Storylines, Hero descriptions, plus various unrelated for. Apart from the above, I agree with your subdevisions.
Plus, new Spreadsheet is much easier to understand!!! 
____________
What will happen now?
|
|
Gnoll_Mage
Responsible
Supreme Hero
|
|
posted January 12, 2007 06:44 PM |
|
|
Good.
Quote:
and therefore, you should not get penalties for not spending as much time on googling up a nice image.
You would not of course be penalised for no images.
Great to have your opinion Alc. I'd really appreciate it if you could express precisely what you want changed - this number to that number, this category to be removed etc. - so that I have a comprehensive view of your opinions, and then we can see what others think.
Quote:
Plus, new Spreadsheet is much easier to understand!!!
Thanks, I'm quite proud of the conditional formatting! (Try putting something other than y or n in the present column, or putting in the mark column a mark bigger than the possible total / a negative value / some letters...)
New file (tidied but not yet edited).
Alc, I am online on ToH at the moment if you want me.
____________
|
|
VokialBG
Honorable
Legendary Hero
First in line
|
|
posted January 12, 2007 08:12 PM |
|
|
Gnoll_Mage, just
And a question:
We all will write, a table like this for all races, or it will be one blank for one town, from all judges?
____________
|
| |
|
|
