I used to work in a mill so I know how tight and solid the selvages can be. I remove the selvages before I wash my fabric. If it's a fabric that might ravel quite a bit I run it under the serger first. Selvages may look nice and straight but actually they aren't.
It depends on the fabric as to how I remove the selvages. If it is cotton or a straight woven fabric I nip about 1 inch in from the selvage and tear it. Then I prewash my fabrics.