I sew my binding on the same way. I love to hand stitch it, but usually I just don't have the time. So far I haven't had this problem, or if I did I just didn't really notice it. I don't care for a really tight look or feel; I like the quilts to be a little more fluffy and soft, so maybe that's why I never noticed. Prewashing would probably make a difference, but I'm one of those (gasp) that doesn't prewash my fabric.