Judging Sites

Okay, so you've taken the plunge and created your awards program. Your criteria are defined and well written, your purpose is understood and you've promoted the heck out of your program. Now you've started getting applications - and the real work begins.

This is the part of awards programs that is, well, both interesting and extraordinarily dull. The creative work has all been done. What remains is to examine each site that applies for your award and determine if they are good enough to deserve to be recognized.

So how does this work? It's pretty simple, really. It all starts from your criteria.

Let's use an example of a pretty standard set of criteria:

  • Reasonable download time
  • Understandable navigation
  • Readable text
  • Not under construction
  • No broken links
  • Good quality content
  • Way to turn off music
  • Good HTML
  • Viewable in all screen resolutions
  • Does not violate copyright or steal bandwidth
  • No sites which promote illegal activities, pornography, hacking or warez.

What I like to do is set aside a few hours a week to go through my list of sites. Then you visit each and every site and compare it to the criteria that you have created for your award. You should also keep in mind whether the sites must meet ALL criteria or just most of them!

This is so important that I will repeat it again: compare the applying site against your criteria and only your criteria. In fact, you should only compare it against the criteria that you had posted at the time the site was submitted.

So let's check a site against the criteria. You surf to it and find that it seems to download very slowly. This, at first glance, appears to violate your criteria - but spend a couple more seconds to make a judgment call: is this the result of the site design or something beyond the webmasters control (such as slow server)? If the page has 500kb of graphics, then the site flunks immediately - go on to the next one. If the page looks reasonable (and it will not take long to figure this out), then continue with it.

Okay, what's next on the criteria? Understandable navigation. That's an easy one - can you figure out quickly how to navigate the site? Hmm, suppose a site has a JavaScript menu and you hate JavaScript? well, if you used these criteria, then you cannot flunk the site for this - but you can update your criteria so the next series of submissions cannot have JavaScript menus.

Readable text is next on the list. Can you read the text? It does not matter whether the characters are pink and green - can you read them? If so, the site passes - if not, it doesn't.

The criteria of "not under construction" is a little vague and could give you some trouble. All good sites are ALWAYS under construction. The web is a dynamic place and static, unchanging sites quickly die. Generally, this is interpreted to mean, "it's not obvious that your site is under construction". Signs which read "under construction" are virtually always an immediate flunk.

Be careful with the criteria of "no broken links". Do you mean your going to check every single link on every page? That could be quite a task! Or do you mean only internal links - still, check them all could be a huge undertaking. Perhaps you just mean "everything that I click on works."

And that's how it works ... you look through the site and compare it to each and every one of your criteria. Keep a checklist and mark passes and flunks (unless one flunk means "no award", in which case you don't need a list). If the site has too many flunks, it does not get the award.

All right, what do you do if you run across a site which meets your criteria but which you find morally reprehensible? It you subscribe to the theory of ethical awards programs, you grant the award - and then you update your criteria. You could even adjust your criteria to say "no sites which are morally reprehensible to me" if you want. Or you could get more specific and say "No sites which promote animal cruelty". 

Remember to think of your criteria as a sort of contract with people who submit their sites for awards. You are explaining the them what you are looking for. Presumably, they read your criteria and only submit their sites if they believe that they pass. (This is often not true - most award submitters really never read the criteria, but lack of ethics on one person's part does not imply that lack of ethics is okay).

Should you notify the people who do not win your award? NEVER. Let me be fully and completely clear about this - NEVER UNDER ANY CONDITIONS, WHETHER REQUESTED OR NOT, SEND BACK CRITICAL COMMENTS TO ANYONE WHO HAS APPLIED FOR YOUR AWARDS PROGRAM. NEVER. After all, if you apply your criteria to the letter it should be obvious why the site didn't win the award - it didn't meet the criteria.

There is nothing more devastating to a webmaster than getting back critical comments, especially when those comments are not anticipated. It's one thing to be in a classroom environment and receive feedback, it's entirely a different matter to have a professional webmaster tell you your site is horrible or even that the "navigation needs work".

Additional Information