I remember well those hour long drives to work, and then another hour home after working all day. That leaves little time for anything else during the work week. The weekend's were full of laundry, house cleaning, grocery shopping, etc. About the only time I could sew was on vacation time or long weekends. Those were my special cherished times. With all that said, buying fabric now is a good idea with costs constantly rising. You will use it in the future as you have more time available. You can always turn around and resell fabric later if you don't use it yourself.
Just tell hubby you are planning for your future when your time schedule permits you to do what you'd like to do without rushing anything.