Lovely quilt!
My vote is for without the brown, as it's not in the quilt anywhere (that I can see), so it looks like a misfit.
I totally agree, the purple would be perfect! And if by chance you don't have anymore to do it with, your quilt will be nice the way it is.