This is my response when this happens to me (yes, and I've been quilting for 50 years):
1. If I need that piece of fabric I un-stitch and remove it, then restitch everything.
2. If I don't need the fabric piece and don't want it on the back I cut along side the stitching, about 1/8" on both sides. Then I can pick the small threads out of there and it is gone.
3. If I like the way it looks, I do as the others have said--turn it into an accent piece, a quilt label, etc.
Good luck. The worst thing you can do is be hard on yourself for this type of thing. It happens to all of us.
Grammahunt