I never would have thought about using boxer shorts....but why not??? LOL
I would use a light weight interfacing like k3n suggested, too.
Cut off the elastic and make a block out of the fronts of them?
Do any of them have sayings on the back or animals on them?
If so you could put them in a snowball block, sash them, use them as a center of a modified log cabin block...or parts of them to go around the centers for the logs...