If you decide not to interface all of the fleece at least do the parts that will be on the edge under the binding. Over time, and multiple washings, it will get much thinner. But, honestly, lightly interfacing all the fleece will only help in the quilting process by making it more maneable and less likely to stretch out of shape over time.
I have a fleece quilt on my bed that is flannel on top and fleece on the bottom. The fleece is also the border. It has stretched on the edges. I should have at least interfaced the edges.