Not at all. I think we often forget that we purchased two sides to fabric. If the wrong side is perfect for your needs, it's clearly the "right" side!
It should not have any affect on the quality of the quilt, and I would encourage you to do it. Many board members have used the wrong sides either knowingly or unknowingly and I suspect their quilts are just fine. There is no wrong side with solids, and we never have problems with those either, so I think you're fine.
Please don't forget to show us a picture of what you make