I too like the bag to be a bit firmer. I'm using Pellon heavy fusible fleece for the body of the bag and the Pellon 978F for the rest. I did try to use all heavy with a bag but it got to the point it was way too thick to sew. I couldn't get the handle under my presser foot even using a walking foot. Try using the heavy just for the body and see if you like that. I'm able to get the thick stuff from Wal-Mart for about $4.00 a yard.
HTH
Kristina