From the terrified stand-point....
We live in a guest house right now. Cramped (espeically with two yound kiddos) is just one way to describe it. We have been working on paying stuff up, now have to build up the savings, but could actually look at getting a house in the spring, and that terrifies me.
I may not be 'happy' with our current situation, but it is what I know. Even though we are looking at a house payment that we can easily afford (ie the total of the monthly bills we have paid off) it still scares me. Not a big fan of change.
I can see where your husband isn't very keen on the idea. To be honest, my hubby had to basically drag me kicking and screaming into the first builders office. Amazingly they didn't bite, and we didn't have to sign anything to just look. It may be a fight, but maybe that is what you need to do to get the ball rolling.