The creams and browns were put with the blacks and greys because they coordinate. I would be tempted to at least do a trial of adding them. At worst, maybe you'll end up with a black and grey wall hanging and a cream and brown table runner!
But I think it's worth trying to intersperse the brown and cream.
But I tend to try things A LOT....not always with success. But I figure it's more about having fun.