Sorry everyone, I should have specified.
I don't know a lot about fleece, but this is the second quilt I'm making with it. The first one appears from the descriptions above to be polar fleece, the same on both sides, I chose the side making sure the pattern was right side up.
This one is more like the sweatshirt fleece. There's the side we usually see outside, and the side that feels great til you wash it. The way I have it loaded right now is with the smooth side out, when it pills up and starts to feel weird, it will be inside and no one will be able to tell.
I see the comments about having the other side out because it's more cuddly, and I think that's why I was second guessing myself... does it really not matter?