I agree, use the fleece for the back of the quilt. I would piece the fleece and make it look like a crazy quilt.
If you need to piece fleece, one trick that I learned while making fleece jackets is to open the seam and top stitch on either side of the seam. The seam lies beautifully flat with the extra stitching. For clothing with a 5/8 inch seam, I would top stitch about 3/8 inch away from the seam on both sides. You can do a small sample and test it. Takes extra time but pays in looks.