But I think it's a BRILLIANT idea to use them as the cake layers. BRILLIANT!
Typically, as you'll see in those links I gave you, folks cut from 1" to 1.5" off because they're collecting.
Now, I don't collect, and never specifically cut my selvedges. They get chopped off in the cutting of pieces for blocks. The only time I do it deliberately is if I'm sewing together large WOW pieces for backing (and even then I trim after the sewing).
Please post some pictures of your cakes when you're finished.