I agree with trying to hand quilt a small sandwich using the backing fabric or just do a very small part of your quilt and see how you like it.
Sounds like it could be a polished cotton, maybe? If so, they usually soften up fine when washed.
If it is really bothering you, then just take it off now and switch it out. Otherwise, you will always wonder. (If you are like me. If only, if only....)