I thought I was being really picky but I see it is a common problem!
I now kick his socks and underwear under the bed. When he has no more socks and underwear in his drawer he asks about it. "I only wash what's in the hamper."
He has improved immensely since I started doing this.