If you took out the four blocks in the center so you left the cross it would take away the stripes in opposite directions that are side by side.
I'd replace them with on block of the circle stars that are in the corners, facing out from the middle.
Or, you could replace the striped fabric with something less busy, or just non-directional.