My sister and I were judges @ two annual quilt shows a few years ago, and ckcowl hit most of the points. There are quilt judging forms to check/review for each quilt submitted.
I definitely agree with the "is it square" thing first. We would simple fold it over from corner to corner, diagonally, to check that fast. Mitered corners sewn down was a biggie, and binding full. Quilting effect and stitches even. Theme to fit the category was a biggie, as some did not fit the category they entered.
Yes, when there are many to judge in same category, it makes it more difficult. We would set 6-8 aside in that category that would be "finalists" then go onto another category, and then come back to the finalists for a different look & examination.
Just enter your quilts...you'll learn more that way, and it's so fun to see your quilt hanging in a show.