So far (just a handful of quilts) I've always sewn to the front first. But it makes sense to sew to the back first if you're not matching a pieced border to the front of the quilt.
One thing I have done a couple times is cut either the front or the back of the quilt bigger all the way around, and then wrapped it to the other side to serve as the border. Then you don't have to worry about staying exactly along another seamline or catching the other side when you sew.