No reason why you can't use it at all - I couldn't find any Pellon when I did my first one, and I used Heavyweight vylene on both outside and inside. It wasn't adhesive, so I quilted both sides, but it was hard to turn inside out. The second two I did, I still used the vylene on the outside, and poly batting on the lining, which made it a lot easier to turn. I happen to like the rather more structured look that the vylene gave.
I finally managed to get some Pellon, because I was curious to see what it was like, and I don't intend to use it in a bag, because it seems so very soft to me after doing the firmer versions.