I have two panels of blue foam covered in one layer of flannel. The flannel is secured on the back with duct tape. Then to attach to the wall, I nailed it down with 8 penny large nails.
This was moved from another apartment that I rented out. I nailed it to the wall in the apartment and it stayed much better than it is now in the old house with not too much to nail it to.
Interesting comment about moving the boards to the floor, but my floor space is very limited now.