I've done it both ways and haven't noticed much difference. My DH doesn't like cake and I do so this works just perfectly for me. I, too, usually top it with some kind of fruit and a spritz of whipped cream......Yummy and so easy....
. And I can whip it up any time of the day for whenever....
I generally just soak the container I make it in with a little Dawn.....washes out with no sweat!