Don't be disheartened. He sounds like a jealous man who doesn't want you to be happy. You may want to suggest counseling. There may be some other underlying problems that you need to be aware of. Sometimes people who are envious use these types of tactics to put down those around them to bolster themselves up. I know...I went through it for a long time till I got a divorce.