I would think it would be fine, but it is a little stretchy so you will have to be careful. It may not be as easy to quilt when you have it all put together. I think if you are tying the quilt that would be the easiest. Hopefully someone here has experience with fleece as a backing and can give you some tips on how to make it work.
Good Luck!!