My mother said that saying someone means well is an insult. She also taught me that if you don't like someone who is going away, you can still say,"I'll miss you," because there are many ways to miss people.
My grandfather was a minister. Whenever someone showed him a new baby, he always said, "That's a real baby," no matter what the baby looked like.