Both.
If piecing the back, I use one piece of fabric at its regular width, then rip the other down the center and sew them to either side of the wider one. No seams in the middle.
I piece the back if I can't find something just right or decide to use one of the fabrics from the front.
It's all in what you're going for in the finished product.