It's sort of a physics thing isn't it? Like how people can lay on a bed of nails without being punctured? It completely has to do with distribution. It's very similar...you will notice weaker areas of your quilt will wear quicker...but just think of all those antique quilts that are still around - from way before sergers.
You'll be fine. Lol* So will your quilts!