i'd stay away from anything floral - actually i'd stay away from all novelty prints but thats just me - i see them as feminine regardless of the design.
if you have to use a pattern i'd go for a paisley. all colors are good except the pastels. red is a nice color thats pleasing to both men and women.
i'd opt for solids, blenders, tone on tone, etc - these fabrics are gender neutral - batiks would be nice.
then for a pattern i'd do a D9P - its a lovely geometric design, is to put together, and is gender neutral.