I do it by color and then maybe pattern. But I do have a tub labeled Big enough for a back. So I've got all the color groups: red, blue, yellow/gold, browns/beige, pink.....then also floral where there were too many colors. Also a group of "I really like this one", Halloween, Christmas, Food themed.
I do not sort by size at all. But what ever works for you.
Marcia