I only wear fragrance for dressing up, and only a little bit then. I wear the kind you dab on, rather than spray, so it's hard to go overboard. My fiance never wears cologne, and I'm fine with that.
Most people who wear it wear waaaaaay too much, and then I just imagine that they haven't showered in days and they're trying to cover up their BO. Kind of ruins the image for me. ;)