NO you're not crazy ... people come in ALL different types and you are just a PRACTICAL thinking quilter
I used to be very deliberate in purchases years ago, when I had to watch every penny. It was easier then too, fabrics were limited. When I started quilting it was CALICO, FLORAL OR SOLIDS !! Now the sky's the limit for fabrics & ITS SO GREAT !!!
I have a dear quilt buddy who NEVER buys fabric except for each planned project. She doesn't want fabric "lying around" and doesn't like scraps or scrap quilts. She is a very pragmatic, controlled type of quilter. She also finishes every quilt before starting a new one ! WEIRD ... !
She is content with her approach ... but you sound like you are not ? So start small, no pressure, buy just 1 yd of several coord. fabrics YOU REALLY LIKE. You'll be on your way to fear-less fabric buying ! :)