If the fabric washes and is colorfast, I say go for it! I have used sheets as backs with great results, and have used linen and decorator fabrics in quilts. Different fabrics can add texture and interest to a quilt. I have seen some great ones will wool and corduroy too.