I did this once in a quilt in which I surrounded squares of color with white on white fabric....that in fact looks very like yours, except mine was pure white. I realized half way through that some of it was right side up, some not. I didn't rip out a single seam! I considered it a "learning" mistake and today it doesn't bother me at all, even though I have mild OCD.
Only you know whether or not it will haunt you. If not, I'd say leave it alone. If you can't live with it, then rip it out and recut those pieces. By the way, I have some of that swirls on burgundy fabric, too! I think it's a fat quarter. We must shop in the same chain.