I give myself permission to toss without guilt. I also think of the cost of my time. We don't have the option to get our time spent back in a refund. The hour or day is gone. Do I want to spend my time with it?
There is always more fabric available. Hopefully, the piece isn't the last skosh of fabric needed to finish the project.