I love the bright colors.
If you are just wanting to "tone it down"... you could possibly take the "big" blocks apart and add white sashing to each color block to put a little space in between them.
I think it would really look super and also it would increase your size without looking like you just slapped a border around it.