I was never fond of browns, beige and "neutral" colors (probably because I think they're boring and they are used everywhere to not "offend" anyone when trying to sell a house). However, I was forced to use them in some round robins, and I rather like them now. There are colors I don't work with much or at all as dominant colors in a quilt, but I can and have worked with all the colors of the rainbow and the mud and rocks, and I'm OK with that.
The quilt tells me what color it wants to be. It's not like painting a room or something.