I made a series of denim rag quilts several years ago. I ask at the Thrift Shop for jeans they could not sell. Some had spandex, most were plain denim, I used all in the quilts. I did not use a middle layer, did not sew an X in the blocks. I did not treat the spandex denim any differently than the regular and the quilts looked great.
If you open up the seam that is not flatfelled, you can use the top stitched seam in the blocks for a decorative accent. Same goes for the patch pockets.