I would think corduroy would be fine for sashing and borders on a man's quilt. It's not all that heavy (comes in many different weights actually) and I've mixed it with cottons in quilts with no problems at all. I love corduroy quilts!
Whichever direction you choose for the wales, it would probably look best if they all go in the same direction (mine were vertical). You also need to make sure the nap goes in the same direction on each piece, otherwise they will look like different colors because of the way light hits them.
If you decide against the corduroy, take a look at cotton broadcloth, like Kona cotton solids. It has a heavier hand and more pronounced weave than cotton prints, thus adding texture, and would calm the look of your other fabrics.