I do remove my selvedges and often put a pieced strip into the center so it is not as obvious that I had to piece two lengths together to get the fit. That strip is usually right over center horizontally as I turn my fabric to get the 40 inch height (selvedge to selvedge) both above and below it.... Most of my quilts are close to 80 in length and around 65-70 in width, so 4 yards works really well for backing.
There is a way to piece so that there is no center seam, but if I remember correctly, you end up with two offset seams. I've never done it that way, becuase the middle seam/pieced part is now like a signature on my quilts!