I would not enjoy the olive and mustard colors you're contending with, but at least they are close to the greens and yellows that are common in nature. Maybe you could think of them as parts of trees and other plants and find some fabrics with animal characters that would coordinate with the "jungle" colors? That way a little sky blue could be justified, too.
My own least liked colors are the dusty rose/mauve and grayed out orange and blue that were popular in the 1980's. I've just about purged those from my stash because every time I used one in a quilt I regretted it.