Let me weight in on this discussion with some specifi and random thoughts. I am a National Quilting Association Certified Judge and have been for many years. Here are some thoughts.
First, all the comments above are valid.
Judges in any situation must do their job according to the rules and wishes of the organization sponsoring the event. Some organizations have certain requirements and objectives as to what they want in the final results. This is not demanding results but rather how those results will effect the aims of the organiation and it's quilters.
Fairs and quilts shows are in my opinion different The judges can be experienced/certified or not. I have noticed over the past few years that more and more organizations are going to the NQA Certified Judges List to select their judges as they know we have had experience in a wide variety of situations.
There are two method for judging - Elimination System and Point System. I favor the first and believe the results can often be the same. Some groups feel multiple judges ae better. I prefer to judge alone but have worked with a wide group of judges. It is tru that any judge has his or her focus. Some will look at deign first, other workmanship.
Usually the critique sheet will determine the order of consideration. There is no master check list with rating points which we use. we may have our own. Remember in the pint system, any judge will rank a factor within his or her own beief about the factors. Years ago one large show had three judges of very wide interets. One would rank workmanship high while another would rank it low. It is my opinion that if you have a situation where one ranks high, another low and a third in the middle - then to average the scores does not give a tru picture of the results.
Some groups want no comments - only a check list while other allow for specific comments.
I believe if a judge makes a constructive comment (negative) then it is only fair to the quilter to tell what needs improvement.
Keep in mind that if there are, say ove 75 quilts to be judged and we have only one day to do the work then each quilt can get only about 2 minutes for critique.
Some groups want all ribbons given while others will allow the judge to skip the blue ribbon if no quilt in the group merits the award, while other want a ranking within the group. This often can be misleading a the blue ribbon may or may not be of that quality. I have found that small fairs want this. I always have a problem with this as giving the top prize to a less than good quilt to be misleading to the quiltmaker.
It is true that the majority of shows do not announce the judges. The would not prevent a quilter to ask and if they know the person then enter accordingly.
An interesting case about how judging can be different is several years ago at a large show the quilt that had won in Paducah (with the prize of $18,000)(In Paducah the quiltmaker may or may not take the money as if they do they must give up the quilt for the museum). the make entered the same quilt in another show and didn't win and it had nothing to do with the judges. The winning quilt was definitely better.
As to measuring - we don't have time to do this. The only thing we are able to do is to make sure the quilt is square.
Every quilt entered usually gets the same amount of time. Often I spend more time on a quilt that is less than good because I want to give comments that will benefit the quilter in his/her workmanship and word those comments in a positive way.
I encourage every quilter to enter. It can be a positive experience and even if you do get some negative comments the person making the comment was seeing the quilt with fresh eyes. We tend to see the quilt with our prejudiced eyes.
Just some thoughts.