I like it the way it is, but if it makes you cringe it makes you cringe.
My recommendation would be to rotate all of the outside borders counter-clockwise one time. That way all of the borders would be perpendicular to the same fabric in the blocks. There's something about the yellow being on top that I don't like. The balance isn't right. I think the red on top might work better.
I don't like the balance of the yellow and blue borders being opposite each other nor the balance of the yellow border next to the green border. What about swapping the blue with the yellow? Or swapping the yellow and the red?
I also was wondering if you took apart the green border and swapped the ends. The longer piece looks like it might be lighter on the top. If so, you could try sewing that lighter end to the shorter piece.
Good luck with it! Love the quilt and how colorful and cute it is!