I, too, love muslin - both the unbleached and the bleached. However, as with most things, there are different qualities of the fabric. I always go for the top quality - usually means higher thread count. Some muslin is so wimpy that you can see through it. I also prefer what is called "permanent press. All cotton wrinkles, but if it's not perma-press, you just can't iron those wrinkles out :)
My Dear Jane is a scrappy quilt & I have used muslin as the background fabric. I really like muslin for backings to my quilts, especially wall hangings. Walls don't have eyes and I'd rather spend my fabric $ on the front.
I think whomever said there was adownside to muslin must have gotten hold of the low thread count, non-permanent press stuff. Continue to use it and don't worry. Out ancestors used lots of it!