You realize you will get a different opinion from each of us that post a response!
I am finding the side bright blocks jarring. My first thought would be to continue the browns in those slots with much smaller pieces of the red and green. BUT, I continued to consider if those same colors are taken to the next round that would integrate them into the total design a bit more. SO, perhaps, you just need to continue and add more of those colors to make them seem like they do belong in this quilt.
I agree that the center is a bit white, hows about tea dying it to tone it down a tad?
Go ahead and put your blocks together, if you don't use them in this quilt I'm sure they'll fit into another. Have fun while you're playing and remember there is no right or wrong answer here. Do what pleases you!