I always use 3 layers of flannel, all cut the same size, which eliminates the need for quilting an X through each block and gives more loft to the ragging. I also like to occasionally use some corduroy in the top of the quilt to give an added texture. The quilts are not at all heavy. I think what would make them heavier would be to make the blocks smaller, which would mean more seams, therefore more fabric and a heavier quilt.
Eta: if you're doing a rag quilt you do want the seams to be larger, from 1/2" to 1". I usually do 1/2" seams.