You know it's your quilt so I would do what you want. Now that being said I've never made a quilt from sweaters! I think if you felt them, use a lightweight iron on stabilizer, serger, 1/2 inch seam allowance you should be good to go. I can't see why you can't use his shirts as well. If it were me I would add interfacing or stabilizer to the shirts so all your materials are the same weight. It's so cool your excited about this project. This is what quilting is all about. I hope you keep us posted.