How much do you buy when you add a piece to your "stash". I'm starting with baby sized quilts.

I am new to quilting so don't have a "stash" yet. I am starting a tradition of making a baby quilt for the new babies in the family. Knowing how much fabric to buy for the tops is the hard part for me. If I have a pattern planned I try and figure out how much for each color I need. But I want to keep an eye out for cute boy/girl fabrics. Is there a general "amount" 1/2 yd, 1 yd ect that will be enough for piecing a top of a baby sized quilt. If I find a piece of fabric I like with out having a actual pattern planned out yet?