I don't think I could give up coffee easily. I use just plain old half and half. For the small amount that's used I see no harm in using that small an amount. If I use sugar it's Stevia or one of those natural plant sweetners. Raw sugar works well. Don't stress, just relax and enjoy it.
Ellie