Truthfully, I'm a new user to your website, and I LOVE searching there by color. It's also how I search at HoP and at Craft Connection most of the time, my two main online sources of fabric. If I'm looking for a specific fabric, I search the web for it, not a particular retailer.
My suggestions for the 'shop by color' section would be:
-just include TOTs, blenders, the basics (SSI Compliments for example), and the like
-put all the marbles together under collections instead of color, same with solids
-divide your colors by the 12 step wheel (red, red-orange, orange, orange-yellow, yellow, yellow-green, etc)
-also have neutrals (tans, beiges, etc), and then, depending on the quantity of each, black (include gray), white (include ecru) and "black, white, and..." for the mixes
Your sense of color is excellent and I know the suggested 'go-withs' is a feature that is uniquely you and is very useful to your customers. Go with your instincts. ;-)