I like that- life is too short for bad blocks.
I did the black & white swap. Got some awesome blocks- some had floral, some had animals. So it set into different piles. I even had safety pins- i put that with a fat quarter I have with sewing machines on it. It will be saved for something special for me.
So, just use them for different purposes.