Okay, time for a design decision. Make the pinwheels and then frame them with a co-ordinating (or contrasting) fabric, to bring them back to the size of the 4 patch. (or set them as a square in square or goodness knows how many other ways they could be put together) If that will not actually work with the original pattern, well, you could recut everything, hoping you have enough fabric, or just totally change the whole thing and make it all your own.
This is an opportunity NOT a mistake.