I happen to be doing this right now.
I'm putting small fabric pieces together by colour.
My stash isn't huge, either, and my yardages are separated into specialty (Christmas, flannel) and then colour.
I don't have enough fat quarters to worry about sorting, just put them together into a FQ carrier.