I buy almost all my clothing at re-sale shops. You just have to have the time to search through the items to find the gems. It's amazing what people give away. I've got a closet full of designer labels.
OTOH, I give away any clothes that I'm not going to wear or are too big/small for me as well. So it goes both ways.
I do find that the prices in the consignment shops have gone up and they are getting really picky about what they will take. The problem there is that a lot of the people who buy the high end stuff seem to be smaller sizes and then the consignment store ends up with fewer larges and extra larges to choose from because they got a little too snobby.
Watson